WikiTeq
WikiTeq's self hosted AI chat now answers from its documents, citing every source
WikiTeq ran OpenWebUI as its self hosted AI chat and wanted it to answer from its own documents. We kept retrieval in its own LlamaIndex service, connected it through OpenWebUI's own filter pipeline, and made sure the chat keeps working if retrieval fails.
B2B SoftwareRAG API for a chatAI added to a live product
The challenge
WikiTeq wanted its OpenWebUI chat to answer from the firm's documents through a separate retrieval service, with the source shown for every answer and no changes to OpenWebUI itself.
What we did
We built a LlamaIndex RAG API over the documents in Amazon S3 and a thin OpenWebUI filter that feeds the top passages into each question and shows where they came from.
The outcome
The chat answers from the firm's own documents, shows file, version and relevance for each source, and carries on as before if retrieval is down. The client rated the work 5 stars.
- What shipped
- LlamaIndex RAG API and an OpenWebUI filter
- For users
- Source shown with every answer, with relevance score
- Platform changes
- None: connected through OpenWebUI Pipelines
- Rating
- 5 star client review
The starting point
WikiTeq, a software services firm, used OpenWebUI, an open source self hosted chat interface. Its team wanted the chat to answer from the firm's own documents stored in Amazon S3.
What had to change
The client wanted retrieval to run as its own service, not through OpenWebUI's built in retrieval. Users also needed to see where each answer came from.
What could not break
- OpenWebUI itself stays unchanged; the integration uses its Pipelines filter model
- Retrieval runs as a separate service, not OpenWebUI's built in retrieval
- Users see the source behind every answer
- A retrieval error never stops the chat
- Fixed scope, agreed in scoping calls before work began
The calls we made
We drew the scope at the integration point: one retrieval service and one thin filter, with nothing changed inside the chat platform.
- 01
Retrieval as a separate service
A standalone LlamaIndex API can be improved and tested on its own, and the chat needs only one HTTP call to use it.
Ruled outOpenWebUI's built in retrieval, which the client asked us not to use.
- 02
Answers bounded by the documents
The retrieved passages go in just before the user's question, labeled with their source, and the model is told to say so when they do not hold the answer.
- 03
Fail safe by design
If the retrieval service errors or times out, the filter returns no passages and the chat continues normally, so the new part can never take down the tool people already use.
- 04
Two response formats accepted
The filter reads both the current and the legacy response format of the service, so the service and the chat side could change on their own schedules.
How the system fits together
What we built
LlamaIndex RAG API
A retrieval service over the client's documents in S3 that returns the most relevant passages with file key, checksum, version and format, protected by an optional bearer key.
OpenWebUI filter pipeline
A filter, now at version 2.2, that sends the user's latest question to the service, places the top five passages just before the question, and shows each source in the chat with its relevance score.
How it works
Ask
A user asks a question in the OpenWebUI chat as usual.
Retrieve
The filter sends the question to the RAG service, which returns the five most relevant passages.
Ground
The passages go in just before the question, with an instruction to say so when they do not hold the answer.
Cite
The chat shows the file, version and relevance score behind each answer.
A closer look
{
"query": "How do we rotate the staging API keys?",
"references": [
{
"score": 0.86,
"text": "Staging keys are rotated by the on call engineer ...",
"metadata": {
"file_key": "docs/runbooks/staging-keys.md",
"checksum": "9f2c41ab",
"version": "4",
"format": "md"
}
},
{
"score": 0.71,
"text": "After rotation, update the secrets in the deploy job ...",
"metadata": {
"file_key": "docs/runbooks/deploys.md",
"checksum": "c07e9d12",
"version": "2",
"format": "md"
}
}
]
}What changed
The client's chat can now answer from its own documents with the source shown for every answer, and it keeps working if retrieval is down. The contract closed with a 5 star review.
“Persistent and focused. Thanks!”
Under the hood
- Datics role
- AI added to a live product
- Integration
- OpenWebUI Pipelines filter interface, calling the RAG service over HTTP with an optional bearer key and a request timeout.
- Data
- Documents in Amazon S3, with file key, checksum, version and format kept on every passage.
- Configuration
- Service URL, API key, timeout and an enable flag set through environment variables in the client's deployment.
Cloud and infrastructure
Apps and APIs
Also in the stack
How it shipped
Scoping calls
Calls with the client set the boundary before work began: a retrieval service and a filter, with no changes to OpenWebUI.
Retrieval service
We built the LlamaIndex API over the client's documents and revised it through internal review rounds.
Filter pipeline
The OpenWebUI filter reached version 2.2, with context injection, source citations, both response formats and fail safe handling.
Scoped with the client first
Scoping calls with the client set the boundary before any work: one retrieval service and one filter, with nothing changed in OpenWebUI.
Configured for their deployment
The filter is switched on and pointed at the service through environment variables in the client's own OpenWebUI deployment.
Review before delivery
Our technical lead reviewed the service and sent it back for revisions before the filter pipeline reached version 2.2.
What this unlocked
Because retrieval runs as its own service, the client can improve or reuse it without touching the chat.
Related work
Have a product with the same kind of constraints?
Tell us what already exists and what you want it to do. We will tell you what it takes.