LLM Integration Services
Language models in production, with the output checked before anything uses it.
What is LLM integration?
LLM integration is putting a large language model to work on a defined task inside your systems, with the instructions under version control, the output validated before anything acts on it, and tests that catch changes in behaviour.
Getting a language model to do something impressive takes an afternoon. Getting it to do the same thing reliably, every time, on inputs you did not anticipate, is the actual engineering.
Two things separate a demonstration from a production system. Output is validated before anything downstream uses it, and there is a test suite that tells you when behaviour has changed.
That second one matters more than people expect. Prompts are code that produces different results without being edited, because the model behind them can change underneath you.
What we insist on before shipping
Duara integrates language models from Nairobi into products and internal systems, on tasks narrow enough to be tested rather than open ended enough to be impressive.
We keep prompts under version control and build an evaluation set before anything ships. Prompts edited directly in production by whoever was available is the normal state of these projects, and it is why they degrade quietly over months without anybody being able to say what changed.
Who this is for
- A model prototype worked and production did not
- Prompts are edited live by whoever is available
- Nobody can tell when model behaviour changes
- Output feeds a system that cannot handle surprises
Industries we serve
- Software companies
- Financial services
- Customer operations
- Media and publishing
- Legal and professional services
- Healthcare technology
Problems this solves
These describe language model work that has left the prototype stage without acquiring any engineering around it.
- Prototype quality in production. Something that worked in a demonstration fails on real inputs nobody anticipated.
- Prompts edited in production. Instructions are changed live by whoever was available, with no record of what changed or why.
- Silent behaviour drift. The model behind the prompt changes and results shift, and nobody notices until a complaint arrives.
- Unvalidated output used directly. Model output feeds another system without being checked, so a malformed response breaks something downstream.
What we deliver
Narrow defined tasks, prompts under version control, output validated against an expected structure, an evaluation set that catches regressions, and fallback when validation fails.
Language model features on defined tasks
Prompts under version control with change history
Output validated before anything uses it
An evaluation set that catches behaviour changes
Fallback when validation fails
Cost per request measured and reported
Features
- Prompts treated as code. Version controlled, reviewed and deployed like anything else, not edited live by whoever was available.
- Output validated before use. Structure and content are checked, so a malformed response is caught rather than passed downstream.
- An evaluation set from the start. Tests that tell you when behaviour has changed, which is the only defence against silent drift.
- Narrow tasks by preference. A model doing one testable thing beats one doing many untestable things, and we scope accordingly.
- Failure handled explicitly. What happens when validation fails is defined rather than discovered in production.
Benefits
- Production behaves like the prototype. Validation and testing close the gap that makes these projects disappoint.
- Changes stop being invisible. An evaluation set tells you a model update shifted behaviour before your customers do.
- Downstream systems stay safe. Validated output means a bad response is contained rather than propagated.
- You can change what a prompt does safely. Version control plus tests make editing instructions an ordinary change rather than a risk.
Technologies we use
- Task scoping. Narrowing the job to something specific enough to be tested reliably.
- Prompt versioning. Instructions held in source control with review and deployment like any other code.
- Output validation. Checking structure and content before downstream systems act on it.
- Evaluation harness. A set of cases with expected results, run on every change and on model updates.
- Provider abstraction. Keeping the option to change model or provider without rewriting the application.
Prompts are code that can change behaviour without being edited, because the model underneath moves. That is the single most unusual property of this work and the reason evaluation sets are not optional.
How the process works, step by step
An LLM integration usually takes four to eight weeks, and the evaluation set is built before the feature ships rather than after the first problem.
-
Step 1
Free task review
We establish whether the task is narrow enough to test, since untestable scope is where these fail.
-
Step 2
Evaluation set built
Cases with expected results are agreed before shipping, because you cannot detect drift without them.
-
Step 3
Prompts under version control
Instructions go into source control from the start rather than being tidied later.
-
Step 4
Validation and fallback
Output checking and failure behaviour are built alongside the feature, not after an incident.
-
Step 5
Live with monitoring
Cost per request and evaluation results are reported so drift and spend stay visible.
Quality assurance and how we work
The commitments attached to running language models in production.
- Prompts are version controlled. Instructions edited live by whoever was available is why these systems degrade quietly with nobody able to say what changed.
- An evaluation set exists before launch. Without one you cannot detect that a model update shifted behaviour, and your customers will find out first.
- Output is validated before anything uses it. A malformed response should break at the boundary rather than propagate into a system that trusted it.
- We scope narrow rather than impressive. A model doing one testable thing is worth more in production than one doing many untestable things.
- We report cost per request. Measured on your real usage, because language model spend scales with success and that surprises people.
Frequently asked questions about LLM integration
Why do prototypes fail in production?
Because they meet inputs nobody anticipated. Validation and an evaluation set are what close that gap.
What is an evaluation set?
Cases with expected results, run on every change. Without one you cannot tell that a model update shifted behaviour.
Why treat prompts as code?
Because they are. Edited live by whoever was available is how these systems degrade with nobody able to say what changed.
Can behaviour change without us editing anything?
Yes, when the model underneath is updated. That is the most unusual property of this work and the reason tests matter.
What happens if output is malformed?
Validation catches it at the boundary and the defined fallback runs, rather than a bad response propagating downstream.
Will it stop hallucinations?
No. Validation contains them rather than eliminating them, and any supplier claiming otherwise is overstating what is possible.
How narrow should the task be?
Narrow enough to test. A model doing one testable thing is worth more in production than one doing many impressive things.
How long before the feature is in production?
Four to eight weeks, with the evaluation set built before it ships rather than after the first problem.
Which models do you use?
Whichever suits the task and budget, with provider abstraction so changing later does not mean rewriting the application.
Can we self host a model?
Sometimes worth it at volume or where data cannot leave. We compare honestly rather than assuming hosted is right.
How do you control cost?
Measured per request on your real usage, plus caching and routing simpler work to cheaper models.
Why does cost surprise people?
Because language model spend scales with success. The better the feature performs, the more it is used and the more it costs.
Can it work with our own documents?
That is retrieval rather than integration alone, and it is usually the right approach when the model needs your facts.
How well do models handle Swahili?
Adequately for many tasks and poorly for some. We test on your actual content rather than assuming parity with English.
What if a provider raises prices?
Provider abstraction means switching is a configuration decision. That is a real risk worth designing for rather than hoping about.
Can our developers maintain it?
Yes. Prompts in source control and a documented evaluation harness are what make that possible.
How do we know if it degrades?
The evaluation set runs on a schedule and on model updates, so drift shows up as a failing test rather than a complaint.
Can it make decisions automatically?
Only where a wrong decision is recoverable. Where it is not, output goes to a person and we would insist on that.
Is our prompt and data confidential?
Yes. Prompts and content stay yours, and we configure provider settings so they are not used for training.
What does an LLM integration cost?
Task complexity and expected volume determine it, quoted after the task review.
Glossary of terms
Language model terms in plain language.
- Prompt
- The instructions given to a model, which should be treated as code rather than configuration.
- Evaluation set
- Cases with expected results, run on every change to detect behaviour drift.
- Output validation
- Checking a response is well formed and plausible before anything acts on it.
- Drift
- Behaviour changing over time, often because the underlying model was updated.
- Context window
- How much text a model can consider at once, which limits what you can send it.
- Hallucination
- Confident output that is simply wrong, which validation contains rather than eliminates.
Related services in AI Development
Talk to us about LLM integration
If you have a prompt in production that nobody has version controlled, that is the first thing worth fixing.
Start your project