015 articles
LLM Foundations
Learn how tokens, context windows, prompting, and hallucinations shape LLM behavior—and when retrieval, validation, or control logic is needed.
- Next-token generation
- Prompting and context windows
OptiVerse Engineering Journal · AI Systems
AI systems engineering, from first principles to production.
Practical guidance on LLM fundamentals, RAG, AI agents, document intelligence, multimodal systems, and inference infrastructure—including implementation details and production trade-offs.
- topic series
- 4
- technical articles
- 21
- topic connections
- 36
Recommended starting point
LLM Foundations
What Large Language Models Actually Do
FoundationStart reading
Latest
Recently published
Read the latest articles on AI systems design and engineering practice.
All articles
Browse every article by series.
Articles follow the recommended reading order and show the learning stage and estimated reading time.
LLM Foundations
Learn how tokens, context windows, prompting, and hallucinations shape LLM behavior—and when retrieval, validation, or control logic is needed.
5 articles
LLM Foundations
Learn how tokens, context windows, prompting, and hallucinations shape LLM behavior—and when retrieval, validation, or control logic is needed.
- 01Next-token generationWhat Large Language Models Actually Do
- 02Prompting and context windowsPrompts, Context Windows, and How You Talk to an LLM
- 03Hallucinations and groundingWhy LLMs Need Help — Hallucinations, Grounding, and the Case for Systems
- 04AI autonomy spectrumAI Assistants, AI Agents, and Everything In Between
- 05Intelligent customer supportAI-Powered Customer Support — From Chatbot to Intelligent System
Building AI Systems
Design production AI systems that combine models with retrieval, tools, control logic, validation, human review, and audit trails.
7 articles
Building AI Systems
Design production AI systems that combine models with retrieval, tools, control logic, validation, human review, and audit trails.
- 01Compound AI systemsFrom Models to Compound AI Systems
- 02Reliable pipelines and control logicReliable LLM Pipelines and Control Logic
- 03Retrieval-augmented groundingGrounding with RAG: How AI Systems Retrieve Evidence Before They Answer
- 04Memory, state, and knowledgeMemory, State, and Knowledge: Stop Calling Everything "Memory"
- 05Assistants, workflows, and agentsAssistants, Workflows, and Agents: Designing for the Right Level of Autonomy
- 06ReAct agent loopsAgent Loops in Practice: ReAct, Tools, and Failure Modes
- 07Hybrid retrieval and working memoryWhen RAG Is Not Enough: CAG, Hybrid Retrieval, and Working Memory
Document & Multimodal Intelligence
Extract structured, traceable evidence from documents and images using layout analysis, table reconstruction, vision-language models, and multimodal retrieval.
3 articles
Document & Multimodal Intelligence
Extract structured, traceable evidence from documents and images using layout analysis, table reconstruction, vision-language models, and multimodal retrieval.
- 01Document evidence reconstructionDocument Intelligence Beyond OCR: Layout, Tables, and Evidence Reconstruction
- 02Multimodal evidence retrievalMultimodal Evidence Systems: VLMs, Figure Grounding, and Cross-Modal Retrieval
- 03Auditable copilot architectureBuilding the Travel Copilot: End-to-End Architecture, Approval Gates, and Auditability
LLM Inference Infrastructure
Understand the serving layer behind an LLM API, including continuous batching, paged KV cache, prefill–decode disaggregation, prefix-aware routing, and MoE sharding.
6 articles
LLM Inference Infrastructure
Understand the serving layer behind an LLM API, including continuous batching, paged KV cache, prefill–decode disaggregation, prefix-aware routing, and MoE sharding.
- 01LLM inference pipelineWhat Happens After You Call the API
- 02Continuous batchingContinuous Batching: Serving Many Requests on One GPU
- 03Paged KV cachePaged KV Cache: GPU Memory Management for LLM Serving
- 04Prefill-decode disaggregationPrefill-Decode Disaggregation: Splitting the Two Stages of Inference
- 05Prefix-aware routingPrefix-Aware Routing: Cache-Conscious Request Distribution
- 06Mixture-of-experts shardingMoE Sharding: Parallelism Strategies for Mixture-of-Experts Models
Topic series
Choose a series for the problem you are solving.
Each series follows a recommended sequence based on prerequisites and difficulty. Start with the first article or jump directly to the article you need.
Swipe to browse topic series
027 articles
Building AI Systems
Design production AI systems that combine models with retrieval, tools, control logic, validation, human review, and audit trails.
- Compound AI systems
- Reliable pipelines and control logic
033 articles
Document & Multimodal Intelligence
Extract structured, traceable evidence from documents and images using layout analysis, table reconstruction, vision-language models, and multimodal retrieval.
- Document evidence reconstruction
- Multimodal evidence retrieval
046 articles
LLM Inference Infrastructure
Understand the serving layer behind an LLM API, including continuous batching, paged KV cache, prefill–decode disaggregation, prefix-aware routing, and MoE sharding.
- LLM inference pipeline
- Continuous batching
Topic map
Review prerequisites and related topics.
Pick a series to browse its topics, grouped by how deep they sit. Every topic shows what to read first and where to continue — the connections are one click away.
- topics
- 21
- topic connections
- 36
Open the topic map21 topics · 36 topic connections
Foundation
4 topics · What the rest of the series assumes you already know.
- 01Next-token generation
You type a sentence into an AI application. Seconds later, it returns several paragraphs of fluent, well-organized text that reads like it was written by a knowledgeable human. That experience is now routine. What is not routine — and what matters if you plan to build anything on top of these sys...
Continue with - 02Prompting and context windows
In the previous post, we sent a single line to an LLM — "Plan a trip to Helsinki" — and got back an itinerary full of specific-sounding details: restaurant names, transit directions, day-trip logistics. It was fluent and plausible, but several of those details turned out to be wrong. The model wa...
Read firstContinue with - 03Hallucinations and grounding
Large language models produce fluent, confident text. That confidence is the problem. A model can sound authoritative about a property listing that no longer exists, a tax rate that changed last quarter, or a school rating from three years ago. It has no mechanism to check. It was not designed to...
Read firstContinue with - 04AI autonomy spectrum
A useful AI system is defined less by whether it is called an assistant or an agent than by how much control it has over the next step.
Read firstContinue with
Applied
1 topic · Worked builds that put the core ideas together.
- 05Intelligent customer support
Customer support is a useful capstone example because one message can require retrieval, tool use, memory, routing, and approval boundaries at the same time.
Read first
Core
5 topics · The main body of the series.
- 01Compound AI systems
Most failures in real AI products do not come from the model suddenly becoming unintelligent. They come from asking a model to do work that actually belongs to a larger system: fetch the right data, interpret a messy document, check a schema, track state across steps, and show evidence for the an...
Read firstContinue with - 02Reliable pipelines and control logic
Useful AI systems usually fail for ordinary software reasons before they fail for exotic model reasons. A prototype looks impressive when a single prompt produces a plausible answer, but production systems do not consume plausibility. They consume records, decisions, and actions that need to be r...
Read firstContinue with - 03Retrieval-augmented grounding
Large language models are useful because they can synthesize, explain, and transform information in fluent language. They are unreliable when we ask them to know something current, something private, or something that needs verifiable support. A model may have seen similar material during trainin...
Read firstContinue with - 04Memory, state, and knowledge
A travel-planning copilot for a mid-size agency is asked a straightforward question: "Did this hotel fail an accessibility review before for wheelchair users?"
Read firstContinue with - 05Assistants, workflows, and agents
Agent has become one of the most overloaded terms in AI. Product teams use it to describe everything from a chat interface with retrieval to a long-running process that can plan, call tools, and take actions on its own. That vocabulary drift creates a practical problem: teams start arguing about ...
Read firstContinue with
Applied
1 topic · Worked builds that put the core ideas together.
- 06ReAct agent loops
Most engineering discussions about agents start too early with the word agent and too late with the operational loop. In practice, the important design question is simpler: once a model can take more than one step, how does the system decide what to do next, what tools it may call, what it is all...
Read firstContinue with
Advanced
1 topic · Deeper material to read once the core is familiar.
- 07Hybrid retrieval and working memory
Basic retrieval-augmented generation, or RAG, is still the default grounding pattern for most production systems. If you have a large corpus, frequent updates, and a need to show where an answer came from, retrieval remains the cleanest starting point. But there is a practical limit case where si...
Read firstContinue with
Core
1 topic · The main body of the series.
- 01Document evidence reconstruction
Most teams first meet document processing through OCR. The problem seems straightforward: convert pages into text, index the text, and let retrieval or an LLM answer questions from it.
Read firstContinue with
Advanced
2 topics · Deeper material to read once the core is familiar.
- 02Multimodal evidence retrieval
Text-only systems break as soon as the evidence stops being mostly text, which is exactly what happens in accessible travel planning when photos, floor plans, route maps, captions, and measurements all shape the answer.
Read firstContinue with - 03Auditable copilot architecture
By the time a team reaches an advanced AI travel copilot, the hard question is no longer "Which model should we use?" It is "What has to happen, in what order, with what evidence, with what state, and under whose approval before this system can be trusted in production?" That is an architectural ...
Read first
Foundation
1 topic · What the rest of the series assumes you already know.
- 01LLM inference pipeline
You have built a travel copilot. A user types a query, your application sends it to an LLM provider's API, and a few seconds later a response streams back. From the application developer's perspective, that is one function call. From the infrastructure's perspective, that function call triggers a...
Read firstContinue with
Core
2 topics · The main body of the series.
- 02Continuous batching
Post I-00 traced a single request through the inference pipeline: prefill processed all input tokens in parallel, decode generated output tokens one at a time, and the KV cache grew with every step. At the end of that trace, we noted that 49 other agents were submitting queries at roughly the sam...
Read firstContinue with - 03Paged KV cache
In Post I-00, we traced a single API call through the inference pipeline and introduced the KV cache: the data structure that stores attention key-value vectors so the model does not recompute them at every decode step. The KV cache grows with every generated token, and it must reside in GPU memo...
Read firstContinue with
Applied
1 topic · Worked builds that put the core ideas together.
- 05Prefix-aware routing
In Post I-02, we saw that PagedAttention enables different requests to share physical KV cache blocks on the same replica. Two requests with the same system prompt can point to the same physical blocks rather than storing duplicate copies. That sharing mechanism is real and it works -- but only i...
Read first
Advanced
2 topics · Deeper material to read once the core is familiar.
- 04Prefill-decode disaggregation
Post I-00 established that LLM inference has two phases with fundamentally different resource profiles. Prefill processes all input tokens in parallel and is compute-bound -- the GPU's arithmetic units are the bottleneck. Decode generates tokens one at a time and is memory-bandwidth-bound -- the ...
Read first - 06Mixture-of-experts sharding
In Post I-00, we listed five ways that LLM inference differs from conventional model serving. The first four -- variable-length computation, two-phase resource profiles, growing memory requirements, and cache-aware routing -- have each received a full post in this track. The fifth was stated in a...
Read first