Build with AI
Amazon Cut One AI Bill From $22,200 to $500 Per Million Tickets. Your Product Is Paying for Text It Never Needed (2026)
Amazon swapped Claude Sonnet 4.6 for Strands Decider, a small open-source decision model, and its ticket labeling bill fell from about $22,200 to $500 per million tickets. Most AI features in a SaaS product are the same kind of multiple-choice question. Here is the cost math at 10k, 100k and 1M events, the rollout Amazon used, and when to keep the LLM. SaaSCity, the launch directory publishing this, has no stake in either model.

Contents (15)
- The short version
- What did Amazon replace, and what did it save?
- What is Strands Decider, and how is a decision model different from an LLM?
- Where did the 98% come from?
- The audit: find the classification calls hiding in your product
- How much does LLM classification cost per million requests?
- Decision model vs LLM vs classic classifier: which should you use?
- Which Decider size should you pick: E2B, E4B, 12B or 26B-A4B?
- Is a cheap classifier as accurate as a frontier model for tagging and routing?
- How do you migrate an LLM classification job without breaking it?
- Three gotchas that will bite you
- What Hacker News got right
- When should you keep the LLM instead of a decision model?
- Questions founders ask about decision models
- Your move
Every time a ticket landed in one of Amazon's largest internal queues, Claude Sonnet 4.6 wrote about 170 tokens of JSON so that a Lambda function could read one label out of it.
Quick answer: Strands Decider is an open-source decision model from AWS's Strands Agents team that picks from fixed options instead of writing text. On November 9, 2026, two Amazon engineers reported that replacing Claude Sonnet 4.6 with Strands Decider E4B cut ticket labeling from about $22,200 to $500 per million tickets, a 98% drop, with equal or better labels.

Amazon and AWS teams had leaned on that setup for a while. Then Carr Onstott and Mehul Jaiswal, two engineers who work on the ticketing system, replaced Sonnet with a model that can't write a sentence, and the bill fell by a factor of 44. Their write-up on the Strands blog reached the Hacker News front page the next morning.
For anyone running a SaaS, the useful part is smaller than the headline. Plenty of the AI in your product is a multiple-choice question, and you're paying essay prices for every answer.
You're reading this on a directory's blog. SaaSCity is a gamified startup directory with a live city map and human editorial review. We don't sell models or inference, and nobody paid for this post. We care because the products that launch with us live or die on unit economics, and an AI feature running at a 40% gross margin can sink a small company slowly.
Last updated: November 10, 2026. Every figure links to the page it came from, opened on November 10. Where we did the arithmetic ourselves, we say so and show the inputs. Screenshots were captured in Chrome the same day.
The short version
- Amazon's ticket labeler went from 4,215 ms to 897 ms per call and from 6,724 to 2,010 tokens per ticket when it moved from Claude Sonnet 4.6 on Amazon Bedrock to Strands Decider E4B on Amazon SageMaker (Strands blog, November 9, 2026).
- Cost per million tickets fell from about $22,200 to about $500, roughly 44 times cheaper, and would be about $250 at full GPU utilisation.
- By our arithmetic, about three quarters of that saving comes from the shorter prompt and the one-token answer. Amazon never tested Sonnet on the short prompt.
- One GPU instance costs about $1,028 a month at any traffic level, so it only beats Sonnet 4.6 above roughly 46,000 classifications a month, and Claude Haiku 5.5 above roughly 1.4 million.
- A decision model gives you a calibrated confidence with every answer and output that can't fail to parse, and its weights can run inside your own network. The Decider README says frontier LLM APIs expose nothing like that confidence score.
- It suits closed-choice jobs only. Labels that need explanations, catalogs that change weekly, or inputs longer than about 4,096 tokens still belong on an LLM.
What did Amazon replace, and what did it save?
Amazon replaced a Claude Sonnet 4.6 call that sorted every incoming ticket into one of 20 categories with a single call to Strands Decider E4B, and the cost per ticket fell from $0.0222 to about $0.0005. The categories included customer escalations, capacity requests, automated alarms and security findings, per the November 9 case study.
The old path sent a long system prompt to Sonnet 4.6 through Amazon Bedrock and got JSON back. The new path sends the same ticket data to a decision model on a SageMaker AI endpoint and gets back one choice. Here are the measured numbers from the post:
| Metric | Claude Sonnet 4.6 (Bedrock) | Strands Decider E4B (SageMaker) | Change |
|---|---|---|---|
| Average time per call | 4,215 ms | 897 ms | 78% faster |
| Input tokens per ticket | 6,554 | 2,009 | |
| Output tokens per ticket | 170 | 1 | |
| Total tokens per ticket | 6,724 | 2,010 | 70% fewer |
| Price basis | $3.00 / $15.00 per million input / output tokens | $1.408 per instance-hour, ml.g5.xlarge | |
| Cost per ticket | $0.0222 | about $0.0005 | |
| Cost per million tickets | about $22,200 | about $500 | about 44x cheaper |

The single output token is the decision itself. The Lambda function around the call got cheaper as well: its average execution duration dropped 74% on the day the team switched Sonnet off, according to a chart in the post.
Serving it takes one modest GPU. An ml.g5.xlarge carries one NVIDIA A10G with 24 GB of memory, per AWS's G5 instance page. E4B uses about 19 GiB of it, and a request takes about 640 ms on the instance, so one instance handles about 5,600 tickets an hour. At full utilisation that's about $250 per million tickets. Amazon keeps headroom for bursts, which puts the real figure near $500.
Everything around the model is standard AWS. Ticket events land in Amazon SQS and trigger a Lambda, which reads the ticket and calls the SageMaker endpoint. AWS AppConfig holds the rollout switch, Amazon CloudWatch carries logs, metrics and alarms, and the stack is AWS CDK with Beta, Gamma and Prod stages. The endpoint autoscales, has no outbound network access and copies its weights in from S3 at start-up. Only the internal ticketing API is Amazon-specific.
The authors say Decider labelled tickets "just as well" as Sonnet 4.6, and that the change "will save millions of dollars of token costs in a year."
What is Strands Decider, and how is a decision model different from an LLM?
Strands Decider is a small model that answers typed questions about a piece of text, such as "which of these options?" or "how likely is this true?", and never generates prose. Marc Brooker, Mike Chambers and Fabio Nonato de Paula introduced it on October 1, 2026 as one of a new class of "decision models or system one models."
A decision model, in their description, is a language model whose text-writing head has been swapped for one that can only score the options it's given, so every answer is one of those options plus a probability for each.
The first release took the body of a pre-trained LLM, Qwen3.5-2B, removed the head that writes text and attached a pointer head of just over a million parameters that scores each offered option. The rest was fine-tuned with a rank-16 LoRA adapter. The result has roughly 2 billion parameters and answers in a single parallel pass. That makes it fast, and the launch post admits it also makes it "significantly worse at solving complex problems than reasoning models."
You can ask it three kinds of question, per the inference docs:
noulreturns the probability that a statement is true.choicepicks one option and returns a probability for every option plus a confidence.scorereturns an expected value on an ordered scale, with the distribution and a confidence.
The confidence is what you can't get from a frontier API. It's meant to be calibrated, which means that when the model says 0.9, it should be right about nine times in ten. The docs define it as the top probability rescaled so that an even split reads 0 and a certain answer reads 1. The README says that on short classification tasks the model has never seen, answers at 0.9 confidence or above are right about 95% of the time, and that frontier LLM inference APIs "do not expose anything equivalent."

The code is on GitHub under Apache 2.0 at strands-labs/strands-decider, which had 891 stars when we first captured it on November 10 and 906 a few hours later. Weights, training data and training scripts sit in the StrandsAgents organisation on Hugging Face. It runs on CPUs, NVIDIA GPUs and Apple silicon, where the README says the MLX backend is 1.4 to 1.6 times faster than MPS. For the cloud, the repo has a CDK stack for a SageMaker real-time endpoint and an AWS Lambda image that runs the 2B model on arm64 CPUs, and the deploy notes warn that Lambda requests "take seconds."
To try it, run the example from the post that added builds based on Google's Gemma 4. The README says the Gemma 4 builds aren't on PyPI yet, so install from GitHub:
pip install "strands-decider @ git+https://github.com/strands-labs/strands-decider"
strands-decider ask StrandsAgents/strands-decider-E4B-gemma4-v1-2610 \
--state "Help! My payouts have been failing for 3 days!" \
--choice "Which team should handle this?=billing,sales,retail" \
--noul "Does this convey urgency?"
The built-in HTTP server binds to localhost with no authentication, and the docs say it's for local experiments only. In production, use the SageMaker stack or put your own auth in front of it.

Decider has company. TypeSafe AI's Jev, which InfoQ covered on October 1, 2026, is a hosted model of the same kind that answers choice, score and yes-or-no questions with probabilities. InfoQ lists it at $0.042 per million input tokens, with free output and a 32,000-token window. The Strands launch post says decision models started drawing attention after Jev launched.
Where did the 98% come from?
Most of Amazon's 98% saving came from shrinking the prompt. Of Sonnet's $0.0222 per ticket, 88.5% paid for input, 6,554 tokens at $3 per million, and the 170-token JSON answer cost the other 11.5%.
To fit Decider's 4,096-token window, the team rewrote the label catalog. Production's label descriptions alone ran longer than the whole window, so each of the 20 labels became a one- or two-sentence criterion, sent as the options of a single choice question along with a "none" option. On long tickets, Decider drops the oldest comments to fit.
Suppose you sent Sonnet 4.6 that same 2,009-token prompt and asked for a bare label of about 10 tokens instead of JSON:
- 2,009 input tokens at $3 per million: $0.0060
- 10 output tokens at $15 per million: $0.00015
- Total: about $0.0062 per ticket, or about $6,200 per million
That's 72% off the original bill with the same model. Decider takes it the rest of the way to $500, so of the roughly $21,700 saved per million tickets, about $16,000 comes from the prompt diet and the missing JSON, on paper.
Amazon never ran Sonnet on the short prompt, though, so nobody knows whether its labels would have held up. The long catalog may have been doing real work. If you run an LLM classifier today, the first saving could take an afternoon: shorten the label descriptions, drop the JSON wrapper, and measure accuracy before and after. For calls that stay on an LLM, our guide to cutting LLM token costs 60 to 95 percent with Headroom covers prompt compression.
The audit: find the classification calls hiding in your product
Any model call whose output gets parsed into a label, a boolean or a number on a fixed scale is a classification call, however chatty the UI around it looks. In a typical SaaS product that covers:
- tagging and routing support tickets or inbound email
- scoring leads, or flagging churn risk from account notes
- deciding whether a review, comment or signup is spam
- picking a category, plan tier or intent for a free-text input
- checking whether an agent's tool call is safe to run, or whether its answer is grounded in the source
Search your codebase for every model call and see what happens to the response. If the next line parses JSON and switches on one field, you've found one. Pull a week of logs for each and note calls per month, input tokens per call and output tokens per call. Multiply by your model's list price and you have cost per 1,000 events.
Take a hypothetical customer on a $49 monthly plan who triggers 2,000 classifications a month. At Amazon's Sonnet cost of $0.0222 each, that's $44 of inference against $49 of revenue. At Decider's fleet rate it's about $1. A flat seat price hides the difference until a few heavy users arrive.
How much does LLM classification cost per million requests?
At Amazon's prompt size, Claude Sonnet 4.6 costs about $22,200 per million classifications, Claude Haiku 5.5 about $740, and a self-hosted Strands Decider anywhere from $250 to over $1,000 depending on how busy you keep the GPU. Your volume decides which one is cheapest.
Self-hosting has a floor. An ml.g5.xlarge real-time endpoint bills $1.408 per instance-hour in us-east-1, or $1,027.84 for a 730-hour month, per CloudBurn's SageMaker pricing breakdown, which reads the AWS price list file (updated September 16, 2026). We couldn't read the figure off AWS's own pricing page, so we checked CloudBurn against the case study, and the two agree. SageMaker Serverless Inference scales to zero, but AWS's documentation says it doesn't support GPUs.
Monthly cost by volume, using Amazon's token counts. Rows marked "our math" are arithmetic on published prices, not measurements:
| Option | 10k events / month | 100k events / month | 1M events / month |
|---|---|---|---|
| Sonnet 4.6, Amazon's prompt (measured tokens) | $222 | $2,221 | $22,212 |
| Sonnet 4.6, label catalog cached (our math, 4,500 cached tokens assumed) | $101 | $1,006 | $10,062 |
| Sonnet 4.6, trimmed 2,009-token prompt (our math) | $62 | $618 | $6,177 |
| Haiku 5.5, Amazon's prompt (our math) | $7 | $74 | $740 |
| Decider E4B, one always-on ml.g5.xlarge | $1,028 | $1,028 | $1,028 |
| Jev, hosted, 2,010 input tokens per call (our math, InfoQ price) | $0.84 | $8 | $84 |
Prices: Sonnet 4.6 at $3 input, $15 output and $0.30 for cache reads per million tokens, and Haiku 5.5 at $0.10 input and $0.50 output for prompts up to 100,000 tokens, from Anthropic's pricing page on November 10, 2026. We ignored cache writes. Our Claude Haiku 5.5 breakdown found that its newer tokenizer produces roughly 30% more tokens for the same text, which would push the Haiku row toward $960 per million.

By SaaSCity's arithmetic, one always-on ml.g5.xlarge running Strands Decider breaks even against Claude Sonnet 4.6 at about 46,000 classifications a month and against Claude Haiku 5.5 at about 1.4 million. Below those volumes the GPU sits idle most of the day and you pay for it anyway. One instance can serve about 4 million a month at 640 ms per request, so above the Haiku line it pays off. Amazon's $500 per million is a fleet number. You only reach it by keeping instances busy, which takes Amazon-sized traffic.
So why would a small team bother? Cost is one reason out of four. The others are latency, a confidence score you can route on, and data that stays inside your VPC. If none of those matter to you and you're under a million events a month, a cheaper LLM with a trimmed prompt is the honest answer. Our piece on model fatigue has a sane way to pick one without re-running the whole evaluation every quarter.
Decision model vs LLM vs classic classifier: which should you use?
Use an LLM when the label set changes often or each label needs an explanation, a decision model when the choice is closed and volume or latency matters, and a trained classifier when you already have plenty of labelled examples and the categories hold still.
| Option | Cost per 1M events | Confidence you can route on | Needs labelled data | Wins when | Loses when |
|---|---|---|---|---|---|
| Claude Sonnet 4.6 (Bedrock or API) | about $22,200 at Amazon's prompt | No | No | The catalog changes often, or you need a written reason | Volume is high or latency matters |
| Claude Haiku 5.5 (API) | about $740 at Amazon's prompt (our math) | No | No | Low to mid volume, quick to ship | Hard label sets it hasn't been tested on; data leaves your network |
| Strands Decider E4B (self-hosted) | about $250 to $500 at fleet scale; $1,028 a month floor per GPU | Yes, calibrated | Only to test and set thresholds | Closed choice, high volume, private data | Context over about 4,096 tokens; labels that need explanations |
| Strands Decider 26B-A4B (self-hosted) | Not published; about 51 GB of weights needs an 80 GB-class GPU | Yes, best calibrated of the four | Only to test | Harder decisions where E4B misses | Memory and latency roughly double |
| TypeSafe AI Jev (hosted API) | about $84 at 2,010 tokens (our math) | Yes | No | You want a decision model without running a GPU | Third-party data handling; vendor-reported benchmarks |
| Fine-tuned encoder such as ModernBERT | Not published; small enough for a CPU | Only if you calibrate it | Yes, for every label | Stable labels and a long labelled history | Every catalog change means retraining |
| Embeddings plus logistic regression | about $40 in embeddings at $0.02 per 1M tokens (our math) | Probabilities, which need calibrating | Yes | A cheap baseline you can train in minutes | Subtle or overlapping labels |
Sources for the bottom rows: Answer.AI and LightOn released ModernBERT on December 19, 2024 in 149M and 395M-parameter sizes with an 8,192-token context, and its authors say to fine-tune it for classification "following standard BERT fine-tuning recipes." OpenAI's model page lists text-embedding-3-small at $0.02 per million tokens. Hacker News commenters made the case for those last two rows, covered below.
Which Decider size should you pick: E2B, E4B, 12B or 26B-A4B?
Start with E2B, try E4B and 12B if it misses your target, and only reach for 26B-A4B after that. That's the order the Strands team gives in its October 15, 2026 post adding four builds on Google's Gemma 4 models. Amazon did roughly the same: the team started on the original 2B model, found that the E4B build "got clearly better results" on its ticket test set, and shipped it. All four sizes take the same three question types, so switching sizes doesn't mean rewriting questions.

| Model | JevBench v1 accuracy (231 tasks) | Brier score (lower is better) | H100 median latency | bf16 GPU weights | Fits a 24 GB A10G? (our read) |
|---|---|---|---|---|---|
| E2B | 180/231 (77.9%) | 0.313 | 102 ms | about 9 GB | Yes |
| E4B | 187/231 (81.0%) | 0.243 | 139 ms | about 15 GB | Yes; Amazon's uses about 19 GiB |
| 12B | 200/231 (86.6%) | 0.173 | 136 ms | about 23 GB | Too tight |
| 26B-A4B | 207/231 (89.6%) | 0.162 | 245 ms | about 51 GB | No |
Accuracy, Brier score and latency come from the Strands post, measured on one NVIDIA H100 80 GB with a 4,096-token window. JevBench v1 is the set of 231 public decision tasks the Strands team scores against, and the Brier score measures how far the model's probabilities sit from what actually happened, so it rewards honest confidence as well as right answers. Weight sizes come from the repo README. The 26B-A4B is a mixture-of-experts model that activates about 4 billion of its 26 billion parameters per inference.
The 12B runs as fast as the E4B on an H100 (136 ms against 139 ms) and scores almost six points higher, but it won't fit on the cheap GPU Amazon used. The step from E2B to E4B is mostly about calibration: the Brier score falls from 0.313 to 0.243, and if you plan to route on confidence, that drop matters more than three points of accuracy.
Is a cheap classifier as accurate as a frontier model for tagging and routing?
For a closed label set it can be, but the public evidence is thinner than the headline. Amazon reports that E4B beat production Sonnet 4.6 "on a separate set of our tickets", and gives no accuracy figure, sample size or metric for that comparison.
- The JevBench scores above rank the sizes against each other on 231 public tasks and say nothing about your tickets. Parallel's explainer also notes that several unrelated projects use the JevBench name, so check which one a number comes from.
- The calibration claim, that answers at 0.9 confidence or above are right about 95% of the time, comes from short, unseen classification tasks. The Gemma post adds that the calibration temperatures were fitted on short held-out tasks, "so test confidence thresholds on your own traffic."
- Amazon's process is the most convincing part. The team ran both models on live traffic and judged every disagreement: "Running both on real traffic, then judging every disagreement, told us more than any offline eval would have."
So a decision model can match a frontier LLM on a job like Amazon's, and you won't know whether it matches on yours until both run on your traffic. Dual mode makes that safe, because the LLM's answer is still the one applied. The price is the LLM bill you already pay plus one GPU for a few weeks.
How do you migrate an LLM classification job without breaking it?
Run the new model next to the old one on live traffic, behind a switch you can flip back in seconds, and cut over only once you understand the disagreements. The steps below adapt Amazon's write-up and the Decider docs:
- List every model call that returns a fixed answer. Search the codebase for each LLM call and mark the ones whose response is parsed into a label, a yes or no, or a score. Those are your candidates. Anything that returns prose for a person to read stays on the LLM.
- Price each one per 1,000 events. Pull a week of logs, note calls per month and input and output tokens per call, and multiply by your model's list price. Work through the candidates in order of monthly cost.
- Rewrite the label catalog as short criteria. Give each label one or two sentences, add a none option, and keep the whole prompt inside a 4,096-token window, which is what Amazon did with its 20 ticket labels.
- Build a labelled test set from real history. Use past outputs that a person checked or corrected. Synthetic examples can fill gaps, but they do not match the distribution of real traffic, so never test on synthetic data alone.
- Start small and size up. Test E2B first, then E4B and 12B, then 26B-A4B, on your own serving stack. Record accuracy, confidence and end-to-end latency for each size, and stop at the smallest one that hits your target.
- Canonicalise every input. Sort JSON keys, fix the field order, strip noise such as timestamps, and drop the oldest content first when a record is too long for the window.
- Ship behind a three-way switch and run both. Add an LLM-only mode, a dual mode and a decision-model-only mode, read on every request. Run dual mode on live traffic, apply the LLM's answer, and log every disagreement for review.
- Set the threshold, cut over and keep watching. Pick the confidence level where disagreements become acceptable, send anything below it to the LLM or a person, flip to the decision model, and re-test whenever the label catalog changes.
Amazon's version of step 7 read a three-way AppConfig value on every message. BEDROCK meant Sonnet labelled everything and Decider was never called. DUAL ran both in parallel, applied Sonnet's label and compared the two. DECIDER handed everything to Decider without calling Bedrock. Going back was "a 45-second config change," and the authors say plainly what they'd keep: "The part we'd copy regardless of model is the three-way switch."
Three gotchas that will bite you
The three things most likely to break a decision-model rollout are inputs that change order, a confidence threshold nobody tuned, and a label catalog that drifts.
JSON key order changes the answer
Sending the same content with keys in a different order moved Decider's output probabilities by up to 0.19, the Amazon team found, which is enough to flip a label near your threshold. They now sort keys on every request. In Python that's one argument:
state = json.dumps(ticket, sort_keys=True, separators=(",", ":"))
Do the same for field order, whitespace and timestamps, and do it before comparing two runs, or you'll be measuring serialization noise.
The confidence threshold is yours to set
The README's 0.9-means-95% figure was measured on short benchmark tasks, so treat it as a starting point. The right threshold depends on what a wrong label costs. A mis-tagged ticket that a person re-routes costs a minute, while a spam filter that hides a paying customer's review costs far more. Amazon tuned how confident the model had to be before a label was applied and didn't publish the number. Use your dual-mode logs to plot agreement with the old model against Decider's confidence, and pick the point where the error rate becomes acceptable. Anything below it goes to the LLM or a person.
Label catalogs drift
A decision model can only pick from the options you give it, so a new product line or a renamed plan leaves the catalog stale. The Amazon authors name a catalog that changes weekly as a sign a decision model won't work. Keep a "none" option, as Amazon did, and alert when its share climbs, since that usually means tickets that fit no label. Re-run your labelled test set whenever the catalog changes. The Gemma post adds that the models "may return the same answer when you change the question but keep the state and options fixed," so when answers look stuck, edit the options; rewording the question may not move them.
What Hacker News got right
Hacker News commenters made three fair points: a frontier LLM was a sane choice when Amazon picked it, most teams don't have the labelled data Amazon had, and a classic classifier can be cheaper still.

The Hacker News thread reached 486 points and 176 comments within about nine hours of being posted on November 10, 2026, and those objections deserve straight answers.
"It's baffling to me that a company like Amazon would choose to use a LLM for a task as simple as ticket classification," wrote Otterly99. atestu had a fair answer in a neighbouring thread: "At the time of Sonnet 4.6 it was probably the best choice. Haiku was pretty dumb back then." An LLM needs no training data on day one, and as cryptonym put it, a "1000x more expensive run can be overall cheaper. You avoid upfront training/tuning cost."
fumeux_fume objected to the framing: "They keep referring to the solution as an agent, but it's actually just a single call to a model." That's accurate, and it describes most AI features in a SaaS product too: one call, one answer.
The labelled-data objection is the strongest. "It seems you had enough labels to train, that's not always the case," wrote rgavuliak. Amazon had months of Sonnet labels to test against, and a new SaaS might have none. evolabs warned against filling the gap with generated examples: "synthetic data != real data. You need to be quite careful about the distribution." With no history, let the LLM run first to build your test set, and have a person check a sample of its labels.
ghm2180 went further: "So technically once you have training data via an expensive classifier on NL data, there is no reason one cannot reduce the footprint of the OP classifier to a very small one," and suggested an SVM with automated feature selection. armcat had built exactly that six years earlier, and it "took a couple of minutes to train, and the ticket classification was just a few milliseconds." IanCal's counterweight was that an SVM on TF-IDF features is "super fast but will make very bad classifications at times." Both are right: with stable labels and enough data, a classic classifier costs even less, while a decision model needs no training and returns a calibrated confidence from the first request.
parthdesai raised audit requirements: "you also want audit trail." A decision model logs a probability for each option and no written reason, so if someone needs to read why a ticket went where it went, you still need text.
When should you keep the LLM instead of a decision model?
Keep the LLM when a person has to read the output, when the catalog changes faster than you can test it, or when the decision depends on more context than fits in a few thousand tokens. Amazon's authors list the same three: labels that need explanation text, catalogs that change weekly, and useful context that won't fit. Low volume is a fourth reason. Under about 46,000 events a month, a dedicated GPU costs more than Sonnet 4.6 at Amazon's prompt size, and under about 1.4 million it costs more than Haiku 5.5. Multi-step reasoning is a fifth: the Gemma post says the models struggle with long documents that need several reasoning steps.
You can also run both for good: the decision model handles everything above your confidence threshold and the LLM takes the rest. Amazon's switch already has the plumbing, since the LLM path is one config value away. For the agent side of the same bill, our earlier look at agent harness costs on AWS Strands and DigitalOcean covers how the wrapper around a model moves the invoice.
Questions founders ask about decision models
Why did JSON key order change Strands Decider's answers? The model reads tokens, so reordered keys are a different input. Amazon saw output probabilities move by up to 0.19 and now sorts keys on every request.
Is Strands Decider cheaper than Claude Haiku 5.5 for a small SaaS? Usually not on cost alone. One always-on ml.g5.xlarge costs about $1,028 a month, and Haiku 5.5 would cost about $740 per million of Amazon's tickets, so the GPU wins only above roughly 1.4 million calls a month.
What's the difference between Strands Decider and TypeSafe AI's Jev? Both are decision models. Decider is open source and self-hosted. Jev is TypeSafe AI's hosted API, which InfoQ listed on October 1, 2026 at $0.042 per million input tokens with free output.
Can Strands Decider run without a GPU? Yes, slowly. It supports CPUs and Apple silicon, and the Lambda deploy option runs the 2B model on arm64 CPUs, where requests take seconds. Real-time traffic needs a GPU.
Why did Amazon use a frontier LLM for classification in the first place? It worked on day one with no training data. The case study calls Sonnet 4.6 the goldilocks choice between intelligence and cost, and HN commenters pointed out that the cheap models of that time were weaker.
Your move
Open your logs and find the model call that ends in a JSON parse and a switch statement. Count its tokens and multiply by your monthly volume. Amazon paid for 170 output tokens per ticket to read one label, so find out what your number is. If it's bigger than you'd like, you have three ways to shrink it, in rising order of effort: trim the prompt, try a cheaper model, and put a decision model behind a three-way switch.
Cheaper inference won't find you customers, though. SaaSCity gives every reviewed product a free listing page and a building on the live city map. Add the SaaSCity badge to your site and you get a dofollow backlink from a domain rated DR 71 at our latest Ahrefs refresh (November 11, 2026), plus a Monday launch slot. Quick Pass ($19.99) goes live within 24 hours, Premium ($39.99) adds a written launch post with three dofollow links, and Max ($99.99) adds 30 days on the billboard, the ticker and a homepage pin.
Get your SaaS in front of founders
List your product on SaaSCity for $19.99, one-time: a permanent listing, a dofollow backlink and a building on the live city map.


