In-product surfaces
The value is rarely a chat panel: it is a drafted reply in the field someone was already typing into, or a summary pinned to a record with an edit path.
Citation-backed UI componentsAI & data
Ship generative features into a product with users
We add retrieval, copilot and generation features to software already in production — using the permission model, API and release pipeline you have, in the screens your users already work in.
- Fixed scope
- Ships behind a flag
- Runs on your data
- Full IP transfer
Retrofit, not greenfield
A generative feature demos well on day three and stalls for six months after. The model is rarely what stalls it. Your product already has tenants, roles, an audit log, a release train and a support queue, and a feature that answers in natural language has to satisfy all five before a customer sees it. This is retrofit work, so grounding reuses the permission model you already have, tool calls go through the endpoints your web app already uses, and prompts ship through your pipeline, reviewed like code.
Most of the effort lands in retrieval, not prompting: ingestion that survives your actual document formats, chunking that respects section boundaries, hybrid search because customers paste order numbers as often as sentences, and a re-index path that fires when a record changes at 2am. Prompts get tuned last, because they are the cheapest thing to change. Then a scored test set gates every later change, traces link each answer to its retrieved chunks and its cost, and wrong answers come back as test cases rather than folklore.
| What is true of this work | What it forces | What we do about it |
|---|---|---|
| Existing tenants, roles and audit log | Retrieval must filter before generation | ACLs mirrored into chunk metadata |
| Source documents are messy and varied | Ingestion, not prompting, is the work | Per-format parsers, scheduled re-index |
| Model calls are slow and bursty | Response paths must stream or defer | Separate service, own scaling profile |
| Answers can be confidently wrong | The unsure state needs designing first | Citations, support check, designed refusal |
| Token spend scales with traffic | Cost becomes a runtime variable | Per-tenant ceiling, cache, kill switch |
In scope
- Retrieval, grounding and citation plumbing
- In-product surfaces inside your existing UI
- Tool schemas over endpoints you already ship
- Scored eval set wired into CI
Not in scope
- Multi-step autonomous agents that act unsupervised
- Rebuilding the systems your documents come from
- Ongoing on-call once the feature ships
Handled by
- AI Agent DevelopmentWhen the feature has to plan and act, not just answer.
- Data ModernizationWhen the answers are not in the source systems yet.
What we build
Retrieval behind your auth
Your product already knows who may see what. We mirror ACLs into chunk metadata at ingestion, filter at the vector query, and re-mirror on permission-change events rather than a nightly rebuild.
Permission-filtered retrieval layerYour API as tools
We generate tool schemas from endpoints you already ship, validate arguments on both sides, and require an idempotency key on writes. Reads stream; writes render a diff the user approves.
Validated tool schemasFeedback that becomes tests
Every thumbs-down and every correction typed over a generated draft is captured with its trace and its chunks. Real failures graduate into the scored set, so a complaint fixes it once.
Graded test setPrompts in your release train
Prompts, retrieval settings and tool schemas live in your repository and ship through your existing pipeline, not a vendor console. Each change opens a pull request and posts a per-case score delta.
Prompt repositoryPer-tenant rollout controls
The feature turns on per tenant, per plan or per cohort, each with a spend ceiling and a kill switch. Support can see exactly what a customer was shown.
Flags, budgets, kill switchHow the work runs
- 01Week 1
Answerability audit
We take fifty real user questions and trace whether the answer exists in your systems, who may see it, and which screen the user was on. We design the unsure state here too.
You getAnswerability map and interaction spec
- 02Weeks 2–3
Slice behind a flag
We build one path inside your codebase: ingest a real document set, index it behind your existing permission model, and render an answer with citations in your own UI. Staging, flagged off, by day fourteen.
You getFlagged staging feature on real data
- 03Weeks 4–6
Grade and gate
We run the slice in shadow on real traffic, generating answers nobody sees, then grade case by case with the people who own them: correct, unsupported, wrong, should have refused. That becomes the CI check.
You getScored test set wired into CI
- 04Weeks 7–10
Tenant by tenant
We enable the flag for one cohort, read traces and corrections daily, and widen only while the scores hold. Handover is the prompt repository, the scored set, the trace view and a runbook.
You getRollout plan, runbook, trained team
What you are handed
- Ingestion and re-index pipeline
- Permission-filtered retrieval layer
- Tool schemas for your endpoints
- In-product UI components with citations
- Prompt repository with review workflow
- Graded test set and CI gate
- Model gateway with fallback routing
- Per-tenant flags, budgets and kill switch
- Support-facing trace view and runbook
Typical stack
Retrieval
Postgres + pgvectorQdrantOpenSearch BM25Docling, UnstructuredCross-encoder rerankers
Product surface
TypeScript, Vercel AI SDKReact streaming componentsServer-sent eventsZod tool schemasOpenAPI-derived tools
Evaluation
Langfuse datasetsRagas, DeepEvalpromptfoo in CIPytest score gatesTrace-sampled online checks
Gateway and ops
OpenRouter gatewayRedis response and semantic cacheOpenTelemetry GenAI tracesGrafana budget alertsTerraform, Docker
The calls we make, and why
Our answers are wrong on our internal jargon — should we fine-tune?
Fix retrieval first; fine-tuning rarely fixes facts.
Wrong on jargon is almost always a retrieval problem: the glossary is not in the index, or the embedding cannot match an acronym to its expansion. A synonym layer and hybrid search cost days and stay correct when the jargon changes.
We’d choose otherwise whenthe gap is output shape or house voice, not facts, and prompting keeps drifting.
Do we ship a chat box or put AI into the existing screens?
Put it in the screens people already use.
A chat box moves the burden onto the user: they must know the feature exists, what to ask, and how to phrase it — three chances to drop out before any value lands. A draft in the field they were already filling needs none of that.
We’d choose otherwise whenqueries are genuinely open-ended over a large corpus — internal knowledge search is the honest case.
Should retrieval read our production database directly or a separate index?
A separate index, kept current by a change feed.
Reading production directly looks fresher and cheaper until a badly shaped retrieval query lands on the box that serves checkout. The index also holds embeddings, ACL copies and chunk metadata that have no business in your transactional schema.
We’d choose otherwise whenthe table is small and volatile — pricing or stock — where staleness hurts more than coupling.
Can we skip the evaluation set and just have the team spot-check?
No. Spot-checking cannot catch a regression it never looked at.
It holds until the first prompt change nobody can reason about; after that you are trading one fix for an unnoticed regression elsewhere. The scored set is a real line in the quote, and it is what makes every later change cheap.
We’d choose otherwise whenthe tool is internal, has a handful of users, and every output is reversible.
This fits if
- You have a live product with real users and permissions
- Your team ships features but nobody has owned evaluation
- A demo impressed someone internally and now must survive customers
- Support answers the same document question dozens of times weekly
- Legal needs citations and an audit trail before launch
Look elsewhere if
- You want a marketing-site chatbot — a widget is cheaper
- You need the same answer every time — write rules
- Nothing in production yet — a PoC engagement fits better
EngagementFixed-scope integration
Typical length6–10 weeks
How it startsSend fifty real user questions and grant read access to one document source; we scope from there.
Questions we get asked
What does a generative AI integration actually cost?
Two bills, and buyers usually budget for one. The build is fixed price after discovery, and its size is driven by how messy your source documents are rather than by anything about the AI. The second is the running model bill, which scales with context and traffic; we forecast it during the slice and cap it per tenant.
Who owns the prompts, the eval set and the retrieval code?
You do — everything we write transfers to you, including the prompt repository, the graded evaluation set, the ingestion pipeline and the infrastructure definitions. The eval set matters most: it is the asset that lets your team change a prompt or swap a provider later without guessing. It ships in your repository, not a vendor console.
How do you stop it inventing answers about our product?
Constrain it to retrieved passages, show the citations, and design a refusal it can fall back to. An output check compares the answer against the passages it was given and downgrades to a refusal when support is weak, which is why the unsure state gets designed in week one. Anything that slips through becomes a graded case.
Will our data be used to train someone else’s model?
Not on the configurations we deploy. We use enterprise endpoints with zero-retention terms and no training on inputs, and we confirm that in writing per provider before the first byte of your data moves. Where the classification permits no third-party call at all, the alternative is a self-hosted model in your own environment, priced honestly as the expensive option.
What if the feature turns out not to work well enough?
You find out at the slice, not at the end. Day fourteen runs on your real data, so if retrieval cannot reach the answers, that is visible before the main build is committed. Sometimes the honest verdict is that the corpus does not contain what people are asking for, and the right next project is a data one.
Often bought alongside
Tell us the requirement.
Thirty minutes with the engineers who would build it. You leave with a scope, a timeline and a fixed price — or an honest no, and the reason why.