ModelsPublished ExplainedAll news from that day
Ongoing case: Typed decision models and APIs 8 stories
How a confidence score lets code act on Gemini Decisions API answers
Gemini Decision 1 returns a probability and a confidence with every answer; Google's docs show where a threshold sends a case to a person and where it falls short.
Check our sources · 10 facts from 2 sources
Image: Google
Key points
- Confidence is the estimated probability, from 0 to 1, that the top answer is right; for yes/no questions it is the larger of the yes probability and one minus it.
- The docs' example sends a message to its department when confidence is at least 0.7, otherwise to human review, and says to set thresholds from your own labeled traffic.
- An input fitting no option still gets a top option, so check confidence or add a yes/no scope check; user text can also argue for its own classification.
What happened
Google's Gemini Decisions API, released on 11 November, returns a confidence score with every answer, so that code can decide whether to act on the answer or pass the case to a person.
In this API a confidence score is the estimated probability, from 0 to 1, that the top answer is right. For a choice question it equals the top option's probability. For a yes/no question it is the larger of the yes probability and one minus it.
Each call sends one shared input and a set of questions, and gets back one typed answer per question. A yes/no question returns a probability from 0.01 to 0.99. A choice question returns the winning option and a probability for every option. A rating question returns an expected score and a probability for each rubric level. Questions run in parallel and cannot see each other.
Google's quickstart shows the shape. For the message "The blue mug I ordered on Monday arrived cracked on Wednesday" and the question "Did the item arrive damaged?", the sample response gives a yes probability of 0.99 and a confidence of 0.99.
The decision then moves to ordinary code. The docs' example routes a message to the chosen department when confidence is at least 0.7 and sends it to human review otherwise. The docs say to set such thresholds using labeled examples from your own traffic, and to weigh the cost of a false positive against a false negative.
The docs list where the score alone does not protect you. An input that fits none of the options still gets a top option, so check confidence before acting, or pair the question with a yes/no check on whether the message is in scope.
Rating scales should be linear, because the score averages the level positions and an exponential scale skews the expected value. And input is untrusted: user text can argue for its own classification, so keep a human or second check on high-stakes actions such as payouts.
Google says the model has the lowest calibration error in Google's own comparison, but the post's text does not name the benchmarks and the result is Google's. The docs' own advice is the practical check: measure the score against labeled cases from your traffic before you trust a threshold.
Underlined sentences link to the facts we checked. Dotted words open a short explainer.
Fact 6 of 10 · from Google
input="The blue mug I ordered on Monday arrived cracked on Wednesday." ... "instructions": "Did the item arrive damaged?" ... "is_damaged": { "type": "noul", "noul": 0.99, "confidence": 0.99 } }, "usage": { "total_input_tokens": 100, "input_tokens_by_modality": [{ "modality": "text", "tokens": 100 }], "total_output_tokens": 0
What it means for you
Our viewA confidence score needs a threshold. The docs say to set thresholds from labeled examples of your own traffic. They also say to weigh the cost of a false positive against a false negative, and to keep a human or a second check on high-stakes actions such as payouts.
Google's claim of the lowest calibration error is its own, as given in the post's text. For each question type you use, decide which confidence level sends a case to a person, and write that setting down before launch.
It adds no new facts.
Your reaction
We count reactions per story and day, never who reacted. The counts help us choose what goes in the monthly issue. If you are signed in, your own page shows yours too.
We count reactions per story and day, never who reacted. They help choose what goes in the monthly issue.
Check our sources
Every sentence above is checked against these 2 sources.
1 Gemini Decisions API: Fast, calibrated multimodal decisions (11 Nov 2026)
Open the source archived copy-
Google's blog post of 11 November 2026 says: "Every answer includes a confidence score (from 0 to 1) so your code can route ambiguous cases to a fallback or human reviewer".
citeEvery answer includes a confidence score (from 0 to 1) so your code can route ambiguous cases to a fallback or human reviewer
-
Google says: "On benchmarks spanning agent routing, reasoning, ranking, and multimodal understanding, Gemini Decision 1 leads across the suite, with the lowest calibration error." The post text does not name the benchmarks; the comparison is Google's own.
citeOn benchmarks spanning agent routing, reasoning, ranking, and multimodal understanding, Gemini Decision 1 leads across the suite, with the lowest calibration error.
2 Decisions API guide, Gemini API docs
Open the source archived copy-
The Decisions API guide says: "Confidence: Estimated probability (from 0 to 1) that the top answer is right. Equals the top outcome's probability, or max(noul, 1 - noul) for yes/no questions."
cite**Confidence:** Estimated probability (from 0 to 1) that the top answer is right. Equals the top outcome's probability, or `max(noul, 1 - noul)` for yes/no questions.
-
The guide lists three question types: yes/no (noul), returning noul (0.01 to 0.99) and confidence; choice, returning the choice, probabilities and confidence; and score, returning score, probabilities, legend and confidence.
cite| Question type | Goal | Returns | | :--- | :--- | :--- | | [Yes/no (`noul`)](/docs/decisions-api#noul) | Check whether a statement holds | `noul` (0.01 to 0.99), `confidence` | | [Choice (`choice`)](/docs/decisions-api#choice) | Pick one option from a fixed list | `choice`, `probabilities`, `confidence` | | [Score (`score`)](/docs/decisions-api#score) | Rate input against ordered rubric levels | `score`, `probabilities`, `legend`, `confidence` |
-
The guide says every question runs against the shared input in one forward pass: "Questions can't see each other, and latency stays nearly flat when you ask multiple questions."
citeEvery question runs against the shared input in one forward pass. Questions can't see each other, and latency stays nearly flat when you [ask multiple questions]
-
The guide's quickstart for the input "The blue mug I ordered on Monday arrived cracked on Wednesday." and the question "Did the item arrive damaged?" shows a sample response of "noul": 0.99 and "confidence": 0.99, with total_output_tokens 0.
citeinput="The blue mug I ordered on Monday arrived cracked on Wednesday." ... "instructions": "Did the item arrive damaged?" ... "is_damaged": { "type": "noul", "noul": 0.99, "confidence": 0.99 } }, "usage": { "total_input_tokens": 100, "input_tokens_by_modality": [{ "modality": "text", "tokens": 100 }], "total_output_tokens": 0
-
The guide's example code reads: if answer.confidence >= 0.7: route_to(answer.choice), else route_to("human_review"). It adds: "Set thresholds using labeled examples from your own traffic. Weigh the cost of a false positive against a false negative."
citeif answer.confidence >= 0.7: route_to(answer.choice) else: route_to("human_review") ... Set thresholds using labeled examples from your own traffic. Weigh the cost of a false positive against a false negative.
-
The guide's best practices say: "Plan for inputs that fit no choice option. Check confidence before acting on the top option, or pair the choice question with a yes/no scope check such as 'Is this message about an order?'."
citePlan for inputs that fit no choice option. Check confidence before acting on the top option, or pair the choice question with a yes/no scope check such as "Is this message about an order?".
-
The guide says: "Because score averages array indices (0, 1, 2, and up), exponential scales like 1, 10, 100, and 1000 skew the expected value." It tells developers to keep score scales linear.
citekeep score scales linear. Because score averages array indices (0, 1, 2, and up), exponential scales like 1, 10, 100, and 1000 skew the expected value.
-
The guide says: "Treat input as untrusted. User-supplied text can argue for its own classification. Write explicit criteria, test with adversarial examples, and keep a human or a second check on high-stakes actions such as payouts."
citeTreat input as untrusted. User-supplied text can argue for its own classification. Write explicit criteria, test with adversarial examples, and keep a human or a second check on high-stakes actions such as payouts.
Every sentence above is checked against these 2 sources. Each fact has its own address you can cite.
Source 1 · Google · 11 Nov 2026
Gemini Decisions API: Fast, calibrated multimodal decisions (11 Nov 2026)
Google's blog post of 11 November 2026 says: "Every answer includes a confidence score (from 0 to 1) so your code can route ambiguous cases to a fallback or human reviewer".
Every answer includes a confidence score (from 0 to 1) so your code can route ambiguous cases to a fallback or human reviewer
Google says: "On benchmarks spanning agent routing, reasoning, ranking, and multimodal understanding, Gemini Decision 1 leads across the suite, with the lowest calibration error." The post text does not name the benchmarks; the comparison is Google's own.
On benchmarks spanning agent routing, reasoning, ranking, and multimodal understanding, Gemini Decision 1 leads across the suite, with the lowest calibration error.
Source 2 · Google
Decisions API guide, Gemini API docs
The Decisions API guide says: "Confidence: Estimated probability (from 0 to 1) that the top answer is right. Equals the top outcome's probability, or max(noul, 1 - noul) for yes/no questions."
**Confidence:** Estimated probability (from 0 to 1) that the top answer is right. Equals the top outcome's probability, or `max(noul, 1 - noul)` for yes/no questions.
The guide lists three question types: yes/no (noul), returning noul (0.01 to 0.99) and confidence; choice, returning the choice, probabilities and confidence; and score, returning score, probabilities, legend and confidence.
| Question type | Goal | Returns | | :--- | :--- | :--- | | [Yes/no (`noul`)](/docs/decisions-api#noul) | Check whether a statement holds | `noul` (0.01 to 0.99), `confidence` | | [Choice (`choice`)](/docs/decisions-api#choice) | Pick one option from a fixed list | `choice`, `probabilities`, `confidence` | | [Score (`score`)](/docs/decisions-api#score) | Rate input against ordered rubric levels | `score`, `probabilities`, `legend`, `confidence` |
The guide says every question runs against the shared input in one forward pass: "Questions can't see each other, and latency stays nearly flat when you ask multiple questions."
Every question runs against the shared input in one forward pass. Questions can't see each other, and latency stays nearly flat when you [ask multiple questions]
Show all 8 factsShow fewer facts
The guide's quickstart for the input "The blue mug I ordered on Monday arrived cracked on Wednesday." and the question "Did the item arrive damaged?" shows a sample response of "noul": 0.99 and "confidence": 0.99, with total_output_tokens 0.
input="The blue mug I ordered on Monday arrived cracked on Wednesday." ... "instructions": "Did the item arrive damaged?" ... "is_damaged": { "type": "noul", "noul": 0.99, "confidence": 0.99 } }, "usage": { "total_input_tokens": 100, "input_tokens_by_modality": [{ "modality": "text", "tokens": 100 }], "total_output_tokens": 0
The guide's example code reads: if answer.confidence >= 0.7: route_to(answer.choice), else route_to("human_review"). It adds: "Set thresholds using labeled examples from your own traffic. Weigh the cost of a false positive against a false negative."
if answer.confidence >= 0.7: route_to(answer.choice) else: route_to("human_review") ... Set thresholds using labeled examples from your own traffic. Weigh the cost of a false positive against a false negative.
The guide's best practices say: "Plan for inputs that fit no choice option. Check confidence before acting on the top option, or pair the choice question with a yes/no scope check such as 'Is this message about an order?'."
Plan for inputs that fit no choice option. Check confidence before acting on the top option, or pair the choice question with a yes/no scope check such as "Is this message about an order?".
The guide says: "Because score averages array indices (0, 1, 2, and up), exponential scales like 1, 10, 100, and 1000 skew the expected value." It tells developers to keep score scales linear.
keep score scales linear. Because score averages array indices (0, 1, 2, and up), exponential scales like 1, 10, 100, and 1000 skew the expected value.
The guide says: "Treat input as untrusted. User-supplied text can argue for its own classification. Write explicit criteria, test with adversarial examples, and keep a human or a second check on high-stakes actions such as payouts."
Treat input as untrusted. User-supplied text can argue for its own classification. Write explicit criteria, test with adversarial examples, and keep a human or a second check on high-stakes actions such as payouts.
Topics
The morning email
On the mornings we publish: the three top stories and up to four short ones. Free.
We email you a link to confirm. An issue may include one sponsor, always labelled Sponsored · Advertisement. Our emails count opens and clicks, not who made them. Unsubscribe in one click. What we keep