Search
Posts
Noise-Robust LLM-Judge Evals: Don't Sign a Coin Flip
An un-seeded LLM judge is a coin flip even at temperature 0. How I made signed eval verdicts replayable instead of attesting noise.
ReadEvery Safety Gate Has a Failure Direction
Safety gates fail in opposite directions: one crashed fail-closed, another failed open. Both from a swallowed error. A shell-script control-failure case study.
ReadThe Relevance Score That Broke Our Cite-or-Refuse Gate
A retrieval test asserted every relevance score sits in [0,1] — it failed on a live -1.83 silently corrupting a cite-or-refuse RAG safety gate.
ReadThe Moat Is the Trust Layer: Turning a Local-RAG App into a BYOK Document-Intelligence Platform
Naive local-RAG app became Intent NEXUS: BYOK document-intelligence platform. The moat is enforced trust; adversarial review caught the evals lying.
ReadShipping gpt-5.4 as One Config Line
Deploy gpt-5.4 to production as a one-line config with eval-gated rollout. Key finding: report word-ceilings came from the model, not the prompt.
ReadBacklog Zero: Burning Down the Estate with a Triage Machine
How a wave-based triage machine burned an estate backlog from 973 to 531 open issues using deterministic gates and verified close manifests.
ReadSurviving CoreWeave: the GPU failures that burn your hours
A practitioner field guide to CoreWeave's real GPU failure modes: Xid triage, silent RDMA fallback, MPIJob launch, LOTA storage, and building your own cost view.
ReadRun the Readiness Audit Before You Flip DNS
An adversarial readiness audit found a payment-path bug that would have charged every customer, then returned HTTP 500 — before the irreversible DNS flip.
ReadRent the Agent, Own the Proof
Anthropic's Claude Tag is the best agentic teammate you can rent. The question it doesn't answer: who owns the memory, and who can verify the audit log? Here's what customer-owned, verifiable agent governance looks like in code.
ReadGate the Statement, Not the Tool Name
When one MCP tool carries every SQL verb, allowlisting tool names is theater. The safety boundary has to read the statement — here's how that gate was built.
Read