· 2 min read
Streaming is a UX decision you keep making by accident
Token streaming makes chat feel alive and dashboards feel broken. Where streaming earns its complexity and where buffering is the adult choice.
#streaming#ux#api
Tagged: api
3 articles
· 2 min read
Prompt caching is the only discount you control
Cache reads cost a tenth of normal input tokens. The math on when caching pays, when it doesn't, and the mistake that quietly doubles your write costs.
#cost#caching#api
· 1 min read
Structured outputs are API contracts, act like it
Schema-enforced output killed the regex parser. It did not kill the need to version, validate and monitor what your model returns.
#structured-outputs#api#reliability