Skip to main content
pcollins.tech
Back to all posts
The cheaper model was the one behind the paywall
3 min read
Technology

The cheaper model was the one behind the paywall

#llm#cost#ai-gateway#debugging

Share this post

I wanted to shave the running cost of a WhatsApp assistant I'm building. It answers simple questions for small food producers ("what's this customer's usual order?"), and it was running on Sonnet 5. For that kind of lookup, a smaller model should do fine, so I went to swap it down to Haiku 4.5.

On paper the saving is real: Haiku 4.5 is about three times cheaper, $1/$5 per million input/output tokens against Sonnet 5's $3/$15. The change itself was one line. Swap the model slug, typecheck, commit. Easy win.

Then the first live reply came back as my own catch-block error message.

The instinct that wasted my time

My immediate assumption, with all the confidence of someone who has fat-fingered a hundred config strings, was that I'd got the model name wrong. So I sat there re-reading a single string. anthropic/claude-haiku-4.5. It looked right. It kept looking right, because it was right.

Eventually I stopped staring at my own code and asked the provider directly. Three probes against the gateway, using the same key the app uses.

  • claude-sonnet-5: 200 OK.
  • claude-3-haiku (the old 2024 one): 200 OK.
  • claude-haiku-4.5 (the one I actually wanted): 403, no_providers_available, "Free tier users do not have access to this model."

The slug was correct the whole time. The gateway even lists the model in its catalogue. The block was my account tier, not my code.

The counterintuitive bit

Read those three results again. The expensive model flows fine on the free tier. The old cheap model flows fine. The one that's gated is the new, cheaper one I was trying to switch to.

That is exactly backwards from the mental model I walked in with, where "cheaper" quietly implies "more available". Here, the newer efficient model is the one the provider puts behind a paywall, presumably because it's the one everybody wants to move to.

What I did about it

I reverted to Sonnet 5 and deleted the feature branch, because the net change was zero. I considered dropping to the old claude-3-haiku instead, since it's reachable on the free tier and cheap, but it's a 2024 model and noticeably weaker at tool-calling. For an assistant whose whole job is looking up orders and invoices, worse tool use is not a saving, it's a regression wearing a discount.

The real fix is adding paid credits to the gateway, which unlocks Haiku 4.5. That's a billing decision for later, not a debugging one for now.

The lesson I keep relearning

When an LLM call fails, the error often points at the wrong thing. A model that "isn't available" reads like a typo in the slug. It was actually a policy on the account.

So the rule I'm writing down: probe the provider directly before you assume the bug is in your code. One raw request to the gateway would have told me in ten seconds what I spent far longer suspecting my own keyboard for. The failure that looks like a mistake in your code is sometimes just a line item you haven't paid for yet.

More to come as I build this out. If you've been caught by a tier gate dressed up as a code bug, I'd like to hear it.

Found this helpful? Share it!

Enjoyed this post? Subscribe to my newsletter for more insights on web development, career growth, and tech innovations.

Subscribe to Newsletter