The same prompt can be sent to a model in four ways: typed into a web chat, handed to a console agent, called from code via an SDK, or assembled as a raw HTTP request. The response will look similar. The system built around that response will differ wildly in latency, cost, predictability, and security — and that difference lives not in the model, but in the layer you use to reach it.
Do not choose this by convenience. "I prefer the chat interface" and "this service needs an SDK" are statements from different planes. The first is about how you want to work tonight. The second is about whether your production pipeline will be reproducible, cheap, and observable six months from now. When engineers confuse these two questions, they get a system that works in a demo and leaks under load: unpredictable response formats, tokens paid for that could have been cached, and no way to tell which step of generation went wrong.
Here is a breakdown of all four layers: how each works internally, how the ecosystems (Anthropic/Claude, OpenAI, Google/Gemini) differ, and which layer actually fits which engineering task. I walk this staircase every day: my own content pipeline calls models over HTTP with a routing rule — simple goes to the cheap model, complex goes to the expensive one. I look at "chat vs. API" from operations, not from documentation.
Web Chat Is a Product, Not a Model
Start with what everyone has seen. A web chat is not "access to a neural network." It is a finished product application sitting on top of the model. Between your text and the model stands an orchestrator that handles a whole load of tasks you never think about:
- Indexing and RAG. Upload a document, and the interface automatically splits it into chunks, runs embeddings, and stores them in a temporary vector store for contextual retrieval.
- Context window management. As the dialogue grows, the orchestrator summarizes or truncates old messages on the fly to avoid hitting the model's context limit. You do not see this.
- Specialized rendering. The text stream is separated from structured entities. In Claude AI, these are Artifacts — separate rendering processes for code, SVG, Mermaid diagrams, or entire HTML applications. In ChatGPT, the Canvas editor plays the same role.
- Preset system instructions. Modes like Projects (Claude) or Custom GPTs (OpenAI) are not "a different model." They are a mechanism that injects your system prompt into every outgoing HTTP request. Attached files are either placed into context wholesale or pulled via retrieval (RAG) depending on their size.
Chat is a convenient wrapper that takes over all the life support around the model. That is exactly what makes it good for some tasks and useless for others.
Where chat fits well: ad-hoc exploration, prototyping, working with unstructured documentation, one-off scripts. From there, the ecosystems diverge:
- Claude AI (Claude.ai) — long texts, analyzing large codebases (through the deep integration of Projects and Artifacts), complex technical documentation.
- ChatGPT — when you need out-of-the-box multimodality (image generation, voice) or fast web search for current data.
- Gemini — analyzing ultra-long media (hours of video, long audio) and tight integration with Google Drive.
The key property of chat to carry forward: context management here is automatic and hidden, and determinism is low. You do not control what exactly went into the model or what was cut from history. For exploration, that is a plus. For a system that must be reproducible, it is a disqualifier.
CLI: The Model Leaves the Chat and Enters the OS
The next layer changes the very role of the model. In chat, it is an interlocutor in a sandbox. In CLI, it becomes an autonomous agent living in your OS: reading and writing files, running shell commands, accessing external services.
┌─────────────────────────────────────────────────────────┐
│ CLI Agent │
│ ┌──────────────────┐ ┌─────────────────────────────┐ │
│ │ Read/Write Files │ │ Execute Shell Commands │ │
│ └─────────┬────────┘ └──────────────┬──────────────┘ │
└────────────┼──────────────────────────┼─────────────────┘
│ │
▼ ▼
┌─────────────────────────────────────────────────────────┐
│ Model Context Protocol (MCP) │
│ (Database, Git, Internal Services, Tools) │
└─────────────────────────────────────────────────────────┘Internally, everything rests on the Agent Loop:
- The model receives a task, looks at the current context and local environment.
- It returns a response with a function call (
tool_use) — for example, "read filesrc/main.py" or "runnpm test." - The CLI client intercepts this response, executes the command locally in the OS, and sends the result (
tool_result) back to the model. - The cycle repeats until the task is closed.
The difference between systems here is the degree of protocol integration. Claude Code and the Anthropic ecosystem rely on the Model Context Protocol (MCP) — an open standard that lets a CLI agent uniformly connect to local and remote sources: Git repositories, databases, bug trackers. Additionally, Claude models in CLI emit thinking structures (thinking tokens) — showing intermediate logic steps before modifying code. For an engineer, this is the ability to intercept a wrong intent before it turns into a file edit.
Outside the Claude ecosystem, the picture is different: engineers typically use third-party provider-agnostic agents (Aider, OpenHands — which also work with Claude) or a local runtime for open-source models (Ollama).
Where CLI fits the task: automated refactoring and bug hunting across an entire repository, generating and running database migrations, automating CI/CD, and writing integration tests. Context management here is hybrid: partly automatic, partly yours. Determinism is medium. It is the workhorse of local development, but not yet what you should build a backend on.
SDK: Where Production Begins
Here is the boundary between "playing with a model" and "embedding a model into a service." An SDK is an abstraction over the network protocol, providing strictly typed interfaces for Python, TypeScript, Go, or Java. Context management is now fully manual: what you put in the request is what goes out — no hidden orchestrator. And it is at this level that providers diverge on three crucial mechanics.
Prompt Caching. In the Anthropic SDK, the developer manually marks static blocks — a massive system instruction, a specification, a chunk of a codebase — with the cache_control: {"type": "ephemeral"} structure. On repeated calls, this cuts token cost by up to 90% and latency by up to 85% on long prompts. In the OpenAI SDK, a similar mechanism works mostly automatically on the server side (implicit caching), without explicit markup in the code.
# Example of explicit Prompt Caching markup in the Anthropic SDK
import anthropic
client = anthropic.Anthropic()
response = client.messages.create(
model="claude-3-7-sonnet-20250219",
max_tokens=1024,
system=[
{
"type": "text",
"text": "Large system context or API specification...",
"cache_control": {"type": "ephemeral"} # Caching instruction
}
],
messages=[{"role": "user", "content": "Generate a controller based on the specification"}]
)Reasoning / Extended Thinking. On reasoning-class models, the SDK provides parameters to control how much the model thinks before answering. It is telling how quickly this knob evolves: on the Claude 3.7 Sonnet generation, the thinking budget was set strictly and explicitly — thinking: {"type": "enabled", "budget_tokens": 2048}. On current Anthropic generations, this form is already displaced: budget_tokens is rejected, and depth is set by adaptive thinking (thinking: {"type": "adaptive"}) paired with an effort level (effort) — the model decides when and how much to think. The intent remains the same — balancing response time against quality — but the engineer now controls it declaratively rather than with a fixed token count.
Structured Outputs. The OpenAI SDK integrates natively with Pydantic and guarantees JSON schema compliance at the token decoding level (Guaranteed Structured Outputs). Anthropic today has a native output schema at the API level (output_config.format and/or strict strict: true on a tool); historically, the same result was achieved indirectly — through strict tool use (tool_use) and response parsing. Both providers arrived at guaranteed schemas, but by different paths.
Put this together, and it is clear why the SDK becomes the primary tool for production systems: microservices, backend logic, specialized RAG, autonomous business agents. Manual context, explicit cache, managed reasoning, and a predictable response format — this is exactly the set that chat does not give you by design. When I say "for production — SDK or HTTP, not chat," it is not a matter of taste: determinism, cache, and explicit routing between models either exist at this level or they do not exist anywhere above it.
Raw HTTP: Removing the Last Dependency
The lowest layer is direct calls over HTTP/HTTPS without any libraries. Here you speak to the provider's API in the language of headers and JSON bodies, and format differences surface into plain view.
[ HTTP REST API ]
├── Anthropic API: POST /v1/messages
│ ├── Header: anthropic-version: 2023-06-01
│ ├── Header: anthropic-beta: prompt-caching-2024-07-25
│ └── JSON Body: { "system": "...", "messages": [...] }
│
└── OpenAI API Standard: POST /v1/chat/completions
└── JSON Body: { "messages": [ {"role": "system", ...}, {"role": "user", ...} ] }JSON payload structure. In the Anthropic Messages API (/v1/messages), the system prompt is extracted into a separate top-level system parameter, and the messages array contains only user and assistant roles. In OpenAI (/v1/chat/completions), the system instruction is a regular element of the messages array with the system role ({"role": "system", "content": "..."}). The OpenAI format has become the de facto industry standard: local servers (vLLM, Ollama) and most open-source model providers (Groq, Together AI, DeepSeek) replicate it. In practice, this means the "OpenAI format + custom router" combination lets you keep different vendors behind a single interface — making this scheme work for cost balancing.
Versioning and experimental features. Anthropic uses a hard anthropic-version header and an anthropic-beta header to enable capabilities while they are in beta. The example below shows the beta header format on a historical feature: anthropic-beta: prompt-caching-2024-07-25. An important caveat for today: prompt caching has long been in GA, and this beta header is no longer needed for it (just use cache_control in the body). anthropic-beta remains the mechanism for features that are not yet in GA:
curl https://api.anthropic.com/v1/messages \
--header "x-api-key: $ANTHROPIC_API_KEY" \
--header "anthropic-version: 2023-06-01" \
--header "anthropic-beta: prompt-caching-2024-07-25" \
--header "content-type: application/json" \
--data '{
"model": "claude-3-7-sonnet-20250219",
"max_tokens": 1024,
"system": "You are a professional system architect.",
"messages": [{"role": "user", "content": "Analyze system design."}]
}'Server-Sent Events. With stream: true, Anthropic returns a strictly decomposed stream of SSE events:
message_start— initializes the response structure and counts input tokens.content_block_start— begins a text block or a thinking block.content_block_delta— incremental chunks (text_deltaorthinking_delta).content_block_stop— closes the current block.message_delta— final message metadata:stop_reasonand theusageaggregate (including output tokens).message_stop— end-of-stream marker.
This granularity lets you see exactly what stage generation is at: reasoning, forming the answer, or calling an external tool. For an API gateway that logs and proxies third-party requests, this is not decoration; it is necessary observability.
Where HTTP fits the task: development in languages without official SDKs (Rust, C++, Elixir, Swift), building high-performance API gateways where you need to strip the overhead of external dependencies, and fine-tuning load balancing, proxying, and network request logging. Context management is fully manual, and control over the request is absolute: not a single hidden decision stands between you and the model. The engineering caveat is honest: "full control" refers to the request and pipeline reproducibility, not the model's output. The LLM response itself is nondeterministic even at temperature=0 (batching and hardware cause drift), and at the payload level, the SDK and raw HTTP send identical requests — the difference between them is dependencies and overhead, not "predictability" of generation. This is both the price and the meaning of the layer: you are responsible for everything yourself.
The Difference Is the Layer, Not the Model
Let us assemble the four levels into one table — this is the map for choosing:
| Criterion | Web Chat (UI) | Console (CLI) | SDK | Raw HTTP API |
|---|---|---|---|---|
| Abstraction level | Highest (finished product) | High (agent environment) | Medium (programmatic) | Low (network protocol) |
| Context management | Automatic (hidden) | Automatic/manual | Fully manual | Fully manual |
| Request control / pipeline reproducibility | Low | Medium | Full | Full |
| Primary use case | Exploration, ad-hoc tasks | Local dev, CI/CD | Production backend services | API gateways, rare stacks |
| Key Claude focus | Artifacts, Projects | Claude Code, MCP, Thinking | Manual Prompt Caching | Headers (anthropic-beta), /v1/messages |
Read the table along one axis: left to right, control over the request increases, and magic decreases. Chat takes over context, cache, and rendering — you get speed of entry at the cost of control. HTTP takes over nothing — you get full control at the cost of all life support now being your problem. The SDK is the sensible middle for most production tasks: manual context and explicit cache without the need to manually assemble headers and parse SSE.
The choice is:, and it is about matching the layer to the task:
- Exploration, prototype, one-off document analysis — chat. Do not build an SDK for what a browser tab solves.
- Local development, repository refactoring, CI/CD — CLI with an agent loop and MCP.
- Backend service, RAG, autonomous agent under load — SDK: manual context, prompt caching, managed reasoning, predictable format.
- Gateway, exotic stack, maximum control over network and cost — raw HTTP.
When a pipeline operates at volume, the model stops being an "interlocutor" and becomes a conveyor component called from code: mass runs go through a cheap, fast model; quality-critical steps go through a stronger, more expensive one. This "cheap / expensive" routing and balancing between vendors lives only on the SDK and HTTP layers — where context, cache, and model selection are in your hands, not hidden in a chat orchestrator. Choose the access layer first.
https://www.dobryakov.net/lead-magnets/llm-interaction-layers.html?utm_source=None&utm_medium=None&utm_campaign=llm-interaction-layers