How to Set Up an AI Knowledge Assistant: A Step-by-Step Guide

Your team asks the same questions every week. Where is the latest price list? What is the process for a refund? Which form does a new starter fill in? The answers exist somewhere in shared drives, old emails and one person's head. An AI knowledge assistant lets staff ask in plain English and get an answer from your own documents.

The short answer: to set up an AI knowledge assistant, pick one area of knowledge, list the questions people ask most, clean up the documents that answer them, decide who can see what, connect the documents to an assistant, test it against your question list, then run a small pilot before opening it to everyone. A first version for one area usually takes weeks, not months.

Before you start

If you are still deciding whether your business needs one, read our guide to the AI knowledge assistant for business first. It explains how these assistants work and the signs that one will pay off. This article is the practical next step: how to set one up.

You need three things before any setup work begins:

• An owner, the person who knows the subject well and will check answers. Not the person who sets it up technically.
• A clear area, such as HR policies, product information or operating procedures.
• Agreement on what stays out, for example salaries, client contracts or management files.

Step 1: List the questions people really ask

Ask the people who answer questions all day: the office manager, the team lead, the senior technician. Write down 20 to 30 questions they hear most often, in the words staff actually use. "Can I carry over vacation days?" is more useful than "annual leave carry-over policy".

Next to each question, write the correct answer and the document it comes from. This list becomes your test set. It tells you which documents matter and lets you check the assistant fairly later.

Step 2: Clean up the source documents

An assistant can only be as accurate as what it reads. If three versions of the refund process exist, it may quote the wrong one with full confidence.

• Keep one current version of each document and archive the rest.
• Fill the gaps your question list exposed. If an answer only lives in someone's head, write it down.
• Use clear titles and headings, such as "Refund process, updated March 2026", so answers can point to the right section.
• Turn scanned images into text where possible. Photos of documents are harder to read reliably.

This step often takes longer than the technical setup. It is also where much of the value comes from, because the team ends up with documents they can trust even without AI.

Step 3: Decide who can see what

An assistant should never show someone an answer drawn from a document they could not open themselves. Decide access by role before anything is connected: everyone, managers only, finance only. The simplest rule is to mirror the access people already have in your shared drive, then fix any folders that were too open to begin with.

Write these rules into your AI usage policy so staff know what the assistant can and cannot see.

Step 4: Choose the tool and connect the documents

There are two routes. Many workplace suites now include an assistant that searches your files, which suits businesses whose documents already sit in one well-organized place. A custom assistant suits businesses whose knowledge is spread across several systems, such as a shared drive, a CRM and a job management tool, or who need tighter control over permissions and answers.

Whichever route you take, check four things:

1. Does it respect the access rules from Step 3?
2. Does it show which document each answer came from?
3. Does it say it does not know, instead of guessing, when the documents have no answer?
4. Where is your data stored, and is it used to train anyone else's model?

If you are weighing up the options, our guide to build vs buy software walks through the same decision.

Step 5: Test it against your question list

Ask the assistant every question from Step 1 and mark each answer: correct, partly correct, wrong, or no answer. Wrong answers matter most. Usually the cause is an old document, a missing document or a vague heading, not the AI itself. Fix the source and test again.

Also try questions it should refuse, such as asking about a manager-only file from a staff account. If it answers, fix the access before going further.

Step 6: Run a small pilot

Give it to five to ten people for a few weeks. Ask them to use it before asking a colleague, and to flag any answer that looks wrong. The owner reviews those flags each week and updates the documents.

Watch three simple measures: how many questions it gets, how many answers are flagged, and whether the people who used to field questions all day notice fewer interruptions. When flagged answers are rare and the team keeps using it, widen access or add the next area.

Step 7: Keep it up to date

An assistant goes stale when the documents do. Make updating it part of normal work: when a process changes, the document changes the same day. Each month, the owner looks at the questions it could not answer. These show exactly which documents to write next.

Setup checklist

• One owner and one knowledge area agreed
• 20 to 30 real questions with correct answers and sources
• One current version of each source document
• Access rules set by role, and sensitive files kept out
• Tool shows its sources and admits when it does not know
• Test set passed, including questions it should refuse
• Pilot group, a way to flag wrong answers, and a weekly review

Frequently asked questions

Start small, then grow what works

The businesses that get value from a knowledge assistant start with one area, real questions and clean documents. The AI is the easy part. The habits around it, owning the answers and keeping documents current, are what make staff trust it.