Category: LLMOps

FinOps & Resilience: How to Run AI in Production Without Going Bankrupt or Falling Over

A gateway with budgets, semantic caching, and fallback is the only thing standing between your AI workload and a six-figure overnight bill or a cross-tenant data leak.

A B2B SaaS team turned on semantic caching with a similarity threshold of 0.88 and a global namespace to save on tokens. The cache returned a Tier-1 customer's cancellation summary during a Tier-3 customer's session. Two different requests were "similar enough" by embedding distance — and a cost optimization became a cross-tenant data leak. The team rolled back to exact-match caching.

Continue reading “FinOps & Resilience: How to Run AI in Production Without Going Bankrupt or Falling Over”

Manifesto: AI Governance as a control plane, not a PDF policy

Governance that can’t be bypassed technically is not a document — it’s a control plane. Every model call physically passes through PII masking, access control, guardrails, and audit.

There are two ways to do AI governance. The first is to write a policy. A forty-page document: responsible AI principles, a committee, a role matrix, a requirement to "use AI ethically." It gets signed, filed in SharePoint, and revisited at the next audit. The second way is to build governance into the request path: every model call is physically unable to bypass PII masking, access control, guardrails, limits, and audit, because they stand in the way of the packet, not in the developer's head.

This is about the second way. The thesis is simple and blunt: governance that can't be bypassed technically is not a document — it's a control plane.

Continue reading “Manifesto: AI Governance as a control plane, not a PDF policy”