Topic archive
AI Engineering
AI tools, prompt engineering, and practical AI usage guides.
-
Caching AI Model Inference: What a Hit Costs, When the Answer Goes Stale, and How a Semantic Cache Lies
A reproducible NumPy lab that puts an exact-match cache, a TTL, a model redeploy, a thread stampede and a cosine-similarity semantic cache in front of a deterministic model, measures hit rate, model calls and p50/p95 on a synthetic request stream, and shows the one number a semantic cache must report that a hit rate hides:…

-
Compressing a Model for the Edge: What int8 and Pruning Actually Cost, Measured
A reproducible NumPy and onnxruntime lab that quantizes a small MLP to int8 per-tensor and per-channel, prunes it to 98 percent, and measures bytes, held-out accuracy and latency at every step, including the one weight rewrite that leaves the float model identical and drops int8 accuracy from 82 to 12 percent, and the reason a…
