· 1 min read
The batch API is 50% off and your cron jobs don't know
Both major providers sell the same tokens at half price for anything that can wait a few hours. The three-question test for what belongs there.
#cost#batch#api
Tagged: api
4 articles
· 2 min read
Streaming is a UX decision you keep making by accident
Token streaming makes chat feel alive and dashboards feel broken. Where streaming earns its complexity and where buffering is the adult choice.
#streaming#ux#api
· 2 min read
Prompt caching is the only discount you control
Cache reads cost a tenth of normal input tokens. The math on when caching pays, when it doesn't, and the mistake that quietly doubles your write costs.
#cost#caching#api
· 1 min read
Structured outputs are API contracts, act like it
Schema-enforced output killed the regex parser. It did not kill the need to version, validate and monitor what your model returns.
#structured-outputs#api#reliability