Preparing a Knowledge Base for AI Chatbots

The AI support plan defines the topics the chatbot can answer. Product descriptions, delivery, membership, internal procedures and personal account lookups require different sources. Collect genuine user questions and map each to an approved answer source. Define an explanation and handoff route for topics outside the scope.

In a RAG approach, the system can retrieve relevant source passages and provide them as context for answer generation. This does not mean that every document upload retrains model weights. Retrieval, generation and model training are separate concepts. Adding sources does not guarantee correct or faithful answers; relevant information must be found and used appropriately.

In an AI agent workflow, the source provides the basis for a decision. Record title, owner, language, product or process scope, version and review state for each document. Creation date alone does not establish currency. Resolve competing guidance from different teams before indexing it.

For example, an old PDF says delivery takes 30 days while a webpage says seven. The business owner resolves the applicable product and conditions, then archives the obsolete source. Averaging the numbers or selecting the most recently uploaded file does not resolve company policy.

Separate public guidance, internal procedures and customer records. Changing stock or order status needs a separately authorized live lookup. The example records below show what source owners should complete before ingestion.

During content preparation, preserve headings, definitions, conditions and exceptions. Microsoft’s RAG preparation guidance examines document structure, columns and tables separately. Carry context with references such as “the table above”; keep product names, units and footnotes attached to the answer passage.

Inspect extracted content from PDFs and images. Columns can become mixed, a table price can attach to the wrong product, or scanned text can be incomplete. Successful upload does not establish successful extraction. The technical team should expose indexed text so an editor can review representative passages.

No chunk size fits every source. Test alternatives with representative short conditions and long procedures. Check product names, abbreviations and local conditions in Turkish and English. Translation must not make an obsolete policy appear current in another language.

A customer messaging chatbot should use only information the person may access. Uploading private material and adding “do not reveal secrets” to a prompt is not an adequate access-control design. Apply relevant identity and document permissions in source selection and retrieval. Personal order data and general delivery guidance require different access decisions.

Microsoft’s access-control documentation describes query-time document filtering; implement the required controls explicitly. Instructions inside documents cannot grant roles or tools. One customer’s query must not retrieve another customer’s contract. Test permission changes, deleted documents and earlier conversation caches together. Exclude credentials from the source collection.

For absent, conflicting or unauthorized evidence, clarify or hand off. Check whether a cited passage supports the actual conditions stated. A source link alone does not establish answer correctness.

In the automation acceptance plan, inspect retrieval and generation separately. Microsoft’s evaluation guide distinguishes groundedness, completeness, relevance and correctness. Record the question, expected document, required exception and unacceptable claim. When an exception disappears, examine the source, retrieved passage and final answer as separate stages.

For example, if “Will this arrive tomorrow?” retrieves regional and working-day exceptions, an unconditional delivery promise should fail. Include typos, unanswerable questions, other-customer records and obsolete sources. Compare automated scores with human review; fluent output or one high score does not measure every risk.

The AI support plan should include the source inventory, prepared content, access map, test questions and update owner. Set freshness expectations by source type. When policy changes, verify removal of obsolete passages from the index, retrieval results and answer caches.

Rerun questions linked to the changed document. Suspend affected answers if permission or stale-source failures remain. Send the question, missing evidence and checked version to the responsible person. Reevaluate the repaired content before restoring it.