Choosing a gateway or a third-party model
"Keep the client, swap only the model" is the smallest of the three routes: one environment variable across, the same one back. The precondition is a gateway that holds up — pick badly and it is worse than not switching.
Separate the three supplier classes first: official reseller-style resale, an API aggregator with published terms, and a private relay. The risk difference is not price but what credential pays for your requests — its own third-party quota is commercial, an official subscription credential passes the exposure to you. Six checks filter out most unreliable gateways.
Separate the three supplier classes
Mix the classes up and every risk judgement after this point is invalid.
| Class | Typical form | What it pays with | Risk tier |
|---|---|---|---|
| Official reseller-style resale | Model calls on a cloud platform you opened | Your own account and quota | Low — you hold the credential |
| API aggregator | Public terms, metered billing | Your own prepaid balance | Moderate — public terms, your balance |
| Self-hosted proxy or relay | Small "API relay" services, one person or a team | Possibly its own subscription credential | High — quota-abuse risk reaches you |
One sentence carries the point: if the provider asks the official model for you with its own subscription credential, the quota-abuse risk it carries propagates to your requests. You pay money; you carry account-level uncertainty. If it merely forwards you to a third-party model provider, the terms at stake are that provider's.
Before paying, ask one question: where does your quota come from? A vague answer is a reason to walk away.
Risk tiers: five dimensions
Five questions separate a service from an exposure, in any of the three classes.
| Dimension | Question to ask | Warning sign |
|---|---|---|
| Credential source | Where does the quota come from? | No direct answer, or "stable channels" |
| Log retention | How long are requests stored? | "We don't log", no privacy terms |
| Request rewriting | Will my prompts be altered? | A refusal, or "we optimise it" |
| Continuity | How long has it operated? Takes company payment? | Personal transfers, no invoice, new domain |
| Billing transparency | Can a charge be reconciled to one request? | A total, no line items |
The rewriting row gets skipped and deserves the most suspicion: any intermediary can read your full prompt — code, keys, business logic. If it also rewrites content, you cannot know what was sent.
Where compatibility breaks
After the backend changes, the usual disappointment is not "the answers got dumber". It is the five items below — where a gateway shows up when it has only half-implemented the Anthropic Messages API, so tool calls, streaming, prompt caching and token accounting stop behaving as expected. None of it appears on a spec sheet.
- Tool-call stability. Whether a multi-step task finishes in sequence depends on consistent parsing of the emitted tool-call format. Most likely to break, and it most affects how the client feels.
- Prompt-cache hit rate. Without an equivalent cache on the backend, the same long context is billed again and again.
- Thinking-block handling. Some models' reasoning does not match the client's expected format and arrives truncated, or as ordinary answer text.
- Usable share of a long context. An identical nominal window hides how much of the middle is dropped; only a test tells you.
- Concurrency and rate limits. One person and a batch workload need different things, and rate-limit policy is often undocumented.
Six checks before you pay
Do not pick a gateway from benchmark scores; measure it. Each check has a result that should end the conversation; run all six on a small project first.
- Ask it to identify itself. Which model, and who provides it. If the answer contradicts the marketing, routing is being substituted — walk away.
- Test a compound tool call. Three consecutive calls with a conditional in the middle. If it changes course instead of finishing, the loop is not stable — walk away.
- Test long context. Push in a large file, then ask about a detail at the end. A wrong answer means silent truncation — walk away.
- Test format consistency. The same request ten times, compared. A gateway that jitters badly is painful in multi-step work — walk away.
- Audit the billing detail. Token counts must reconcile on your side. If they cannot, you are the weaker party in every dispute — walk away.
- Read the privacy terms. Retention, training use, jurisdiction. No terms at all means assume the worst — walk away.
Why popularity makes a relay riskier
A 2026 public analysis reported that the client carries a built-in list of roughly 147 domains matched against ANTHROPIC_BASE_URL, described as containing many third-party API relay services. How traces leak covers the mechanism.
So the relay everyone uses is the one most likely to be listed already. "Find a site that opened last week" is not safer: it trades being listed for vanishing, retained logs and rewritten prompts — and the second set is more probable. Treat it as reported, not confirmed; the rest of this page is reasoning.
Terms exposure, and why "grey" is honest
Three sets of terms can apply at once: yours with the client vendor, the gateway's with you, and the upstream provider's with the gateway. Only the middle one can be read before paying, and it decides whether prompts are logged or altered.
What terms rarely settle is routing. A relay can promise "official model, no logs" while the path is shared quota, a cross-region hop or a substituted backend. That is why the honest label is grey, not clean: you are buying an assurance about a path you cannot inspect, and the party carrying the consequence is usually the one whose requests sit in the log.
Read the client's documentation on where requests may go. No clear answer means the route is unresolved, not approved.
Three principles and a decision table
- Prefer your own quota. Official resale or an aggregator: the credential stays in your hands and can be switched away at any time.
- Ask where the credential comes from before paying. The only question separating a commercial service from transferred risk.
- Test, then migrate. The six checks take half an hour — cheaper than migrating for two weeks and then finding tool calls unstable.
| Your situation | First move |
|---|---|
| You want the least account-level uncertainty | Your own quota at a cloud platform, or a prepaid balance |
| A relay is the only option that fits | Ask the credential source, then test |
| Tool calls break after you switch | Run the tool-call and format checks first |
| You would rather leave the dependency behind | The migration guide, not another gateway |
Leaving the dependency behind is a different question: read what you would give up first, then the migration guide.
Can a gateway get my account banned too?
It depends on the class. If it forwards your requests to the official model and pays with a subscription credential, the risk passes straight to you. If it only forwards to a third-party model provider, the terms at stake are that provider's.
What is the biggest drop after switching models?
Usually not answer quality but tool-call stability and cache hit rate. The first decides whether a multi-step task finishes; the second decides cost. Neither can be read off a spec sheet.
Are cheaper gateways more dangerous?
A low price usually means cost was cut somewhere: shared quota, retained logs, cross-region forwarding. The bigger point is popularity — the better known a relay domain, the more likely it is already on the client's built-in list.
How long do the six checks take?
About half an hour with the cheapest model, which is the right way to run them: you are testing the gateway's engineering, not the model's ability. Doing it before a migration beats finding an unstable tool loop two weeks after one.