Knowledge Bases (RAG)

Required Access Level

Admin, Modeler, Manager or Super Admin

The Knowledge Base module allows administrators to upload company documents (PDF, Word, Excel, PowerPoint, OpenDocument, RTF, HTML, CSV, TXT and Markdown — and scanned PDFs, when the installation has OCR) to give context to the AI Agents using Retrieval-Augmented Generation (RAG) technology.

The screen sits under Process design for the Admin and the Modeller, and under Work for the Manager — a knowledge base is content that feeds the agent, not platform configuration.

The role is enforced on the server

The barrier is not only the menu: the tenant knowledge-base endpoints require Admin, Modeler or Manager (or Super Admin) on the server itself, and anyone calling the API directly without that role gets a 403 — on reading the listing as much as on the upload, the deletion and the link to a process. The global base keeps its own requirement: only a Super Admin creates or changes a global base.

The same holds for the per-process base, the one the catalogue's RAG tab feeds at /a/definitions/{key}/knowledge: uploading and deleting a document, wiping the whole base and asking for a reindex all require the same roles, and so does reading the listing. It is the same capability reached by another route, so it takes the same role.

When attaching a document to a Knowledge Base, Flowi Agentic performs:

1. Text Extraction: Secure reading of the original content.
2. Chunking: Splitting the text into fixed-size passages, measured in tokens rather than paragraphs — a passage can start mid-sentence, which is why the guidance below is to write blocks that stand on their own.
3. Embeddings: Converting the text into mathematical vectors, saved in the database (pgvector).

Creating a Base

1. Access the Process design → Knowledge Bases menu (a Manager finds the same entry under Work → Knowledge Bases).
2. Click on New Base.
3. Define a name and a clear description for the scope of this base.

A base created here belongs to your tenant and only your tenant sees it.

Global bases

Besides your own bases, the listing also shows the global bases — provided by the platform and read by every tenant, usually travelling with a global process template. They are flagged as Global and are read-only for a tenant administrator: they cannot be edited or deleted, and the delete button is disabled with the reason.

Global bases are created and maintained by the platform administrator, under Global catalogue → Global Knowledge Bases. The split is deliberate: each role configures only the scope it answers for, and the tenant screen offers no switch to mark a base as global.

Anything global is visible to everyone

Every tenant reads a global base. Content one customer must not see does not belong in the global scope — load it into the tenant's own base.

Writing for vector search

Search does not read the whole document: it splits the text into chunks and sends the agent only the closest chunks to that step's instruction — 3 by default, 1 to 20 as the agent or the step sets it (see Hybrid search, below). Everything else stays out. Writing for that cut is what separates a base that helps from one that gets in the way.

Write chunks that stand on their own. A rule spread across five sections of a manual does not arrive whole — at most three pieces of it do, possibly without the opening premise. Prefer a short block that states the complete rule, carrying its context inside it, over a long exposition that only makes sense read from the beginning.

A small, focused base beats a complete one. With no floor configured, the nearest chunks are always sent, even when none of them is good — the agent can set a minimum similarity, but it only cuts; it does not improve what the base holds. A base that mixes subjects is not neutral — it fills the space with irrelevant material and pushes out what mattered. Prefer several lean bases, each linked to the process where it belongs, over a single base holding everything.

Speak the language of the agent's instruction. The search is driven by the instruction configured on the step, not by a user question. If the instruction says "delivery deadline" and the document says "lead time", proximity drops and the right chunk may not make the cut. Review both sides together and use the same terms.

Prefer text over scans. OCR recovers the text of a scanned PDF, but not its structure: headings and tables become running text. A file with real text always locates better — and in an installation without OCR the scan stays as Needs OCR (see below).

In practice

The same content, written both ways:

<!-- ❌ the chunk cannot stand alone: it depends on what came before -->
As described in the previous section, the limit mentioned applies in
full. In all other cases, the preceding paragraph governs, save for the
exceptions already noted.

If that piece is one of those selected, the agent receives "the limit mentioned" without knowing which limit, which section, or which exceptions.

<!-- ✅ the chunk answers on its own -->
## Approval limit — travel reimbursement

Travel expenses up to R$ 2,000 are approved by the direct manager.
Above R$ 2,000 they require approval from the area director.

Applies to every cost centre, from 01/2026 onwards.
Exception: international travel always requires director approval,
regardless of amount.

A heading that names the subject, the complete rule, explicit value bands, the effective date and the exception — all inside one block. Even if this is the only chunk retrieved, the agent still decides correctly.

What must NOT go into a Knowledge Base

Code and identifier tables — product catalogues, cost centres, municipality codes, job title tables, price lists. They look like documents but do not work by similarity: neighbouring codes have nearly identical descriptions, and thousands of rows repeat generic terms like "Other". Asking for a specific code returns, with full confidence, an unrelated row.

That kind of data belongs in the CMS, where lookups are exact. The rule of thumb is: exact identifier → CMS; text that must be interpreted → Knowledge Base.

Hybrid search: keyword + vector

Vector search finds chunks of similar meaning, and misses exactly what people type when precision matters: an NCM code (8471.30.12), art. 37-A, a SKU, a clause number. That is what hybrid mode is for: besides the search by meaning, the platform looks for the question's words and codes literally and merges the two lists (Reciprocal Rank Fusion — it counts each chunk's position in each list, not its score).

• The mode is chosen on the agent (Knowledge base search: Vector or Hybrid, chunks per search and minimum similarity) and can be overridden on the step — see Agents.
• An existing agent keeps the mode it had (Vector, 3 chunks, no floor, unless someone changed it). An agent created from now on starts Hybrid, 3 chunks, no floor. To decide the mode of an older agent, measure the base with Reference questions (below).
• Keyword search calls no model: it spends no token. And each search now embeds the question once for the tenant's base and the global one together.
• Keywords help most on the agent's search tool, where the model types the code as it appears in the document. On the automatic search, the query is the step's whole instruction.

Keyword search language

On the base's Search tab you choose how words are compared:

Language	What it does
Simple — no stemming (default)	Codes and numbers stay intact; plural and singular count as different words
Portuguese, English, Spanish	Reduces words to that language's stem (policy ↔ policies); codes stay intact

Changing the language recomputes only this base's keyword index, in the background — no document is reprocessed and no token is spent. While it runs the tab shows a notice and search keeps working. The tab also shows Retrievable chunks and Chunks in the keyword index: if the second is zero while the first is full, hybrid search on this base is using the vector side only.

Matching does not ignore accents: "licitacao" does not find "licitação" by keyword (the search by meaning still does).

Measuring search: reference questions

The base's Reference questions tab tells, with a number, whether search brings the right document — before an agent answers wrongly because it never saw the chunk it needed.

1. Write the questions the way a person or an agent would, and say where the answer is: 1 to 5 documents of the base, picked from the document list (searchable by name). A page and a snippet that must appear are optional and make a hit stricter. Up to 100 questions per base.
2. Evaluate search opens the choice of configurations — Current (vector, 3), Hybrid 3, Vector 5, Hybrid 5 or one of your own (mode, chunks, floor), up to 6 at a time. Estimate shows how many questions run, the approximate tokens and the quota left; only Confirm and run starts it. Change the choice and you estimate again.
3. Each question is embedded once and scored in every configuration, by the same search the agent uses. No model answers: only search is measured. 50 questions cost about 1 to 2 thousand tokens, counted in the AI quota as RETRIEVAL_EVALUATION.

How to read the result — one column per configuration:

Metric	What it says
hit@k	in what share of the questions at least one expected document appeared among the k chunks
recall@k	how many of the expected documents appeared (differs from hit@k only for questions with more than one target)
MRR	how high the first hit came: 1 is always first, 0.5 second on average, 0 never appeared

Next to each number is the difference from the previous run with the same configuration — that is how a language change, a new document or hybrid mode shows as better or worse. The grid below shows, per question, the rank of the first hit in each configuration; a click opens the list of what came back (file, page, vector and keyword rank) and the chunk's text.

• Whoever runs it sees only what they reach. The evaluation searches at your access level: a document above it never enters the ranking, and a question whose expected documents are all above your level is Above your level — it is not counted as a search miss. Someone viewing a result with a lower level sees the rank of restricted chunks, not their name.
• Out-of-date result. When the base's chunks or the embedding model change after the last run, the tab says so and offers Run again. Nothing runs on its own.
• One evaluation at a time per base (starting another while one is queued or running is refused), one running at a time per tenant, and a short pause between questions: the evaluation does not compete with the agents' search for the embedding provider's rate limit. A failure shows with a translated explanation; the technical detail stays in the server log.
• The base keeps its 20 most recent runs. Global base: only the platform administrator writes questions and runs evaluations; the tenant sees the results.

Accepted formats and how each one is cut

The file type is recognised from its content, not from the extension or from what the browser declares: an executable renamed to .docx is refused, and the message names the detected type. The upload area shows the list this installation accepts and the maximum size (50 MB by default).

Family	Formats	How it becomes chunks	How a chunk is located
PDF	.pdf	one unit per page	page
Word processing	.docx, .doc, .odt, .rtf	one unit per section, cut at headings (Heading 1 to 3)	heading path
Spreadsheet	.xlsx, .xls, .ods, .csv	blocks of rows that repeat the header, never splitting a row	sheet and row range
Presentation	.pptx, .ppt, .odp	one slide per unit, with title, body and speaker notes	slide number and title
Web and text	.html, .md, .txt	HTML and Markdown by section; TXT whole	section, when there is one
Image	.png, .jpg, .tiff	only with OCR on	page (TIFF frame)

Every chunk reaches the agent starting with the file name and the locator — for example Daily rates.xlsx › Planilha: 2026 · linhas 2–41 — so a piece cut in the middle still says where it came from.

Before its text, each chunk gets a reference line with a label — for example [F2] Daily rates.xlsx · v1 · planilha "2026", linhas 2–41 · base Norms — and the agent is told to cite the document and the place (or the label) when it uses the chunk. The reference carries no internal identifier. What each step received is recorded on the instance and shows as Sources in the activity history (see Instances), with the document version that answered. None of this makes an extra model call.

The excerpt saved on the instance (up to 300 characters) and the file name stay while the document can still be restored. When the deleted document is purged — after the orphan window, together with its vectors — that excerpt, the name and the place are erased too: the source then shows as Source removed, with only its label. Deleting the document does not, by itself, erase the saved excerpt.

Word cites the section, not the page

A .docx has no page number: pagination is an effect of printing, not of the file. So a Word document is located by its section heading. Use real headings (Heading 1, 2 and 3 styles), not just bold text: without them the whole document becomes a single section.

Formulas come in through their saved calculated value, empty rows are skipped and hidden sheets are read — the screen says it read a hidden sheet. A sheet with more than 5,000 rows carries a warning: it is probably a code table, and exact lookups belong in a CMS collection (see the warning below).

Scanned PDFs and OCR

A page with fewer than 20 characters of text is treated as a page without a text layer. What happens depends on whether the installation has OCR:

• With OCR — the page is rendered and read by Tesseract inside the installation: the document goes to no external service and no tokens are spent. Up to 200 pages per document go through OCR; the rest is left out and the version shows as Partial.
• Without OCR, no page with text — the version shows as Needs OCR, with no chunks and no embedding billed.
• Without OCR, some pages with text — the pages with text are ingested, the version shows as Partial and the screen lists the skipped pages.

Once OCR is on, the Re-ingest button processes again the version that needed it. The upload area says whether this installation has OCR.

Limits

A file above 50 MB, a PDF or deck with more than 2,000 pages/slides, sheets adding up to more than 200,000 rows, more than 5 million extracted characters or an extraction running past 15 minutes are refused before any embedding — nothing is billed for a document that does not fit. A password-protected file is refused too, with the reason on screen.

The status of each document

The list shows, for each document, the status of its latest version, what extraction read (for example, 12 of 14 pages read · 2 by OCR), the skipped pages, who uploaded it and the current version. While any document is in progress, the list refreshes itself every few seconds.

Status	What it means
Queued, Extracting, Indexing	ingestion is in progress
Ready	all the content is searchable
Partial	searchable, but some pages were left out (listed on screen)
Needs OCR	a scanned PDF in an installation without OCR; nothing was indexed
Failed	the reason shows on the row: password protected, above the limits, no readable text, embedding model error…
Unchanged	the uploaded file has the same text as the current version
Superseded	an older version, kept in the history

An ingestion interrupted by a server restart goes back to the queue on its own; after three attempts without finishing, the version shows as Failed with the reason interrupted.

Versions: uploading the same document again

Uploading a file with the same name to the same base again duplicates nothing:

• same file, byte for byte — the screen answers already in the base, unchanged, and nothing is stored or billed;
• different file, same text (a re-saved DOCX, for example) — the version is recorded as Unchanged and the file is discarded;
• new text — it becomes the next version (v2, v3…). While it is processed, the previous version keeps answering; the switch happens at once when the new one is ready, and if it fails the previous one stays in use. Search never sees both at the same time.

The same content under another name in the same base is refused, naming the document that already holds it: the two would compete for the same three search slots. Across bases, it is allowed.

Each document's versions button opens its history: status, extraction numbers, author, embedding tokens spent and which version each one was restored from.

Restoring an older version

Restore creates a new version with the content of a superseded one — the history only grows. Within the orphan retention window (30 days by default), the old version's vectors are still kept and the restore is immediate and spends no tokens. After the window, the file is processed again and the embedding is billed again; the screen warns before you confirm and shows how many tokens the version spent last time.

The restore becomes the newest version: an earlier upload still being processed ends as Superseded when it finishes, instead of taking the restored version's place.

Who may read each document

Every document in a base has a level: Public (the default), Secret or Top secret — the same levels as CMS collections. A search only returns excerpts of documents up to the searcher's level:

Who searches	Level used
AI step of a process or case	the agent's knowledge base access (default Public)
Execution copilot	the logged-in user's level (User and Modeler → Public, Manager → Secret, Admin → Top secret)
Agent playground	the lower of your level and the agent's
Any other path (MCP, for instance)	Public

The filter runs inside the database query itself: a denied excerpt never reaches the application server, let alone the model. Reclassifying applies on the next search, with no reprocessing of the document and no tokens spent.

When an excerpt classified above the level would have been among the results, the search tool tells the agent that access was denied — how many excerpts and at which level, never the document name — so that it reports a missing permission instead of concluding the information does not exist.

On the base screen:

• The Level column shows each document's level; the Classification button changes it, offering only the levels you reach.
• Default level sets the level new documents start at. Tick Also apply to the documents already in the base to reclassify everything at once. A new version of a document keeps the document's level.
• You can only change, re-upload, restore, re-ingest, download or delete a document up to your own level. A Modeler curates public material only; a Manager reaches Secret; Top secret needs an Admin. A document above your level is listed with Above your access level and no actions.
• The Admin sees Access to restricted material: every search that returned or denied a classified excerpt, with its origin (step, automatic context, copilot or playground), the level used and the instance. Searches over public material only are not recorded. The record is kept for 365 days.

Documents attached directly to a process (outside a base) stay public. Deleting a whole base requires reaching its most restricted document, and a base's files only leave through the base screen — the generic attachments API does not serve them.

The level bounds what reaches the model, not what the model writes. An agent with Secret access may summarise a Secret document into a variable a User will see on the next task. Granting the level to the agent means accepting that.

Syncing with SharePoint and OneDrive

Instead of uploading files by hand, a base can have sources: a folder of a SharePoint library or of a OneDrive for Business drive, kept up to date on its own. Every file of the folder goes through the same path as a manual upload — type detected from the content, 50 MB limit, OCR or "Needs OCR", versions — and its embedding is counted in AI usage like any other.

Before you start (tenant administrator):

1. The installation operator turns sync on (app.knowledge.sources.providers=microsoft, see Deployment). Without it, the Sources tab says the feature is off.
2. In your directory's Entra ID, register an app with the Sites.Selected application permission and ask the Microsoft 365 administrator to grant only the sites Flowi should read. Files.Read.All also works, but it reads every drive in the directory.
3. Keep the app secret in a secret variable under Variables and secrets (the value, or a vault:// reference). The connection screen has no secret field: it only lets you pick the variable.
4. Under Integrations and automation › Knowledge connections, create the connection (directory, client ID, cloud, variable) and click Test connection. The test names the host that failed: "this installation cannot reach login.microsoftonline.com" is a firewall or proxy matter.

Adding the source (the base's Sources tab): pick the connection, browse to the folder — with Sites.Selected the site list does not appear, so type the site address or the OneDrive owner's e-mail —, decide whether subfolders are included, the documents' level and the interval (60 min by default, 15 at least). Before the first sync the screen shows how many files and MB the folder holds and a ceiling of embedding tokens; a large library takes hours and is processed in rounds of 500 files.

What happens next:

• A new or changed file becomes a document or a new version; an identical file costs nothing. Renaming or moving inside the folder only changes the name, with no new embedding.
• A deleted file, or one moved out, leaves the base at once. If one round would delete more than 30 % of the source's documents (and at least 20), it pauses without deleting anything and tells the base's curators: Confirm deletion deletes; Keep documents keeps them as regular documents of the base, no longer synced.
• A synced document carries the Synced from … badge and cannot be deleted by hand (it would come back next round): delete it at the source or remove the source. Removing a source asks whether its documents stay.
• SharePoint/OneDrive permissions are not copied. Who sees the documents in Flowi is the level chosen on the source. You may raise one document by hand, and the sync does not lower it.
• The source card shows the state (Up to date, Partial, Credential refused, Host unreachable, Paused…) with the reason in words, the last complete sync, the next one and the files that failed — type not accepted, above the ceiling, scanned PDF with no text, repeated content. When a source enters a failing state, ADMIN, MANAGER and MODELER who reach its level get one notification.
• A failed download is retried on its own, with a growing interval (5 min, 10 min, 20 min… up to a day), without the file having to change at the source. After 10 attempts the failure stays as Retries ran out until the file changes or someone clicks Try again under Details. A file deleted and put back with the same content stays in the base.
• Raising the base's default level also raises the sources that were below it.
• A source is visible and editable only to whoever reaches its level; creating or editing a source also requires reaching the base's default level.
• A refused credential or a revoked site grant never deletes documents: the source stays failing until someone fixes it.

Global bases have no sources in this version. Google Drive comes later.

External bases (Elasticsearch / OpenSearch)

A base can be an external index instead of documents uploaded to Flowi. It keeps nothing here: on every retrieval of the step the question goes to the index at the source — an Elasticsearch 8.x or OpenSearch 2.x the company already runs, often inside its own network — and the passages that come back are cited like a document's ([F2] Travel policy · page 4 · base Standards).

Creating. In New Base, the tenant Administrator picks Type: External index and fills in:

Field	What it is
Backend, Address, Index	the cluster's https://host:port and the index name (no wildcard, no comma)
Query mode	Words (BM25) works with any index. Platform vector only when the index was built with this installation's embedding model — Test connection checks the vector size and refuses a mismatch
Field mapping	which index fields are the content (required), title, link, page and id. Without a title the base name is used; without a page the citation has none
Static filter	a fixed JSON (for example {"term": {"status": "published"}}) applied to every query. Only the clauses term, terms (with a list of values), range, match, match_phrase, exists, prefix, ids, bool and nested are accepted, with plain field and value — no script, wrapper, query_string, regexp or lookup in another index. The agent never writes the filter: it only supplies the question text
Passages, time limit, characters	how many passages to ask for (1–20), how long to wait (3 s by default) and where to cut each passage
When the base does not answer	Continue (the agent is told the source did not answer) or Fail the step — the step fails before the model is called, the job retries and, once exhausted, shows in the monitor
Authentication	API key, user and password, or none. The credential is a tenant secret variable (or a vault reference) picked by name — no field accepts the value

Managers and Modelers see the base, link it to processes and use Test a query; only the Administrator changes the destination and the credential.

Allowed destinations. On the same screen the Administrator keeps the list of hosts the tenant's external bases may query (*.company.com covers one level of subdomain). Without a list the installation policy applies — by default no external base is queried until there is a list. An internal network address (10.x, 192.168.x…) is reachable only when the installation operator lists the host — a tenant screen never opens the internal network. https is required; redirects are not followed.

Test a query. The tab shows the passages exactly as the agent would receive them, at your access level, with the outcome and the time the query took.

Access. An external base has a single level, the base's (Classification of the base). Whoever retrieves below it does not query the index at all — the question never leaves the platform — and the agent is told access is missing. The index's own per-user access control is not used.

When the source fails. Unavailable is not empty: if the index does not answer, times out, refuses the credential or the destination is not allowed, the agent gets a line saying which source was missing, the step records the source under Sources in the instance history, and the other sources keep answering. After 5 failures in a row within 60 s the base is not called for 30 s on that server.

What reaches the model. Local and external passages are mixed by the position each source returned them in, not by score — a BM25 score does not compare with a vector similarity. The total is still the step's topK.

Cost. Querying the index spends no token: it is counted, not priced, under AI Governance → External knowledge base queries. In vector mode the question vector is the one the local search computed — no extra embedding call.

In this version: Elasticsearch and OpenSearch. Azure AI Search, an HTTP contract for an internal RAG, Bedrock and Vertex come later.

Linking to Processes

After populating the knowledge base with corporate documents, you link it to a process definition, not to an agent. The agent reaches the base because it runs inside that process — if the same base serves more than one process, each one needs its own link. When the agent runs, the AI automatically searches the linked bases before responding, so the answers stay grounded in your company's guidelines and the risk of hallucinations drops.

The process link defines the set within reach; inside it, each step of the flow can narrow which bases it consults, or declare that it consults none — see Knowledge bases for this step.

Deleting a base

Deleting a base removes the record immediately, but the vectors built from its documents do not disappear at the same moment: they are marked as orphaned and removed by a daily cleanup after a retention window — 30 days by default, configurable through RAG_ORPHAN_VECTOR_RETENTION_DAYS.

This is deliberate. Rebuilding a large index costs hours of processing plus the price of the embeddings, so the window exists to give you time to recover a base deleted by mistake. During that window the vectors show up in no search — retrieval only considers bases linked to a process, and a deleted base is linked to nothing.

Deleting a document follows the same rule: it leaves the list and the search at once, all its versions go with it, and the vectors are stamped as orphans for the same window before the cleanup.

A base still linked to a process is not deleted: the platform refuses and answers listing the processes that still use it. Unlink each one and repeat the deletion.

Reloading a whole base

To replace all the content of a base, create a new base, load into it, link it to the process, and only then unlink the old one. Uploading a file with the same name again creates a new version and takes the old one out of search (see Versions); a file with a new name, however, is one more document, and the old one keeps answering until it is deleted.