Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
248 changes: 248 additions & 0 deletions aigw/integrations/llms/azure-foundry.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -604,6 +604,254 @@ The native `/v1/ocr` endpoint requires gateway version **2.18.0** or higher for

---

## Decisions (Microsoft-Decision-1)

`Microsoft-Decision-1` is a decision model available in Microsoft Foundry. It does not generate text. It reads a piece of text or JSON, answers typed questions about it, and returns probabilities that your application can threshold or rank. Use it for classification, routing, ranking, grading, and yes/no checks.

The AI Gateway routes these calls through `/v1/decisions`, a separate endpoint from chat completions. You still get the gateway's full feature set: authentication, routing, retries, circuit breaking, caching, logging, and cost tracking. See the [Decisions API](/aigw/product/ai-gateway/decisions-api) for the full request and response reference.

<Note>
`/v1/decisions` does not support streaming. Only before-request [guardrails](/aigw/product/guardrails/decisions-guardrails) apply, scanning the content to judge only (`state`).
</Note>

<Info>
This section covers how to call Microsoft-Decision-1 through the AI Gateway. For model details, deployment steps in Azure, regional availability, and responsible-use guidance, see Microsoft's official guide: [Deploy and use Microsoft-Decision-1 in Microsoft Foundry](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision).
</Info>

### Model at a glance

These values come from [Microsoft's model overview](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision#microsoft-decision-1-at-a-glance).

| Model name | Model version | Deployment type | API type |
|:---|:---|:---|:---|
| `Microsoft-Decision-1` | 1 | `DataZoneStandard` in selected regions, `GlobalStandard` | Decision |

### Request model vs response model

This is the most common source of errors with this model, so read it before you send a request.

| Field | Value | Example |
|:---|:---|:---|
| `model` in your **request** | The **deployment name** you chose in Azure AI Foundry | `@azure-foundry/decision-1` |
| `model` in the **response** | The underlying model | `microsoft-decision-1` |

<Warning>
Do not hardcode `microsoft-decision-1` as the request `model`. Azure looks up a **deployment** with that exact name. If your deployment has a different name, the request fails with `The API deployment microsoft-decision-1 does not exist`. Use the deployment name you see in Foundry. Microsoft describes this in the [Call the decision API](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision) section of its guide.
</Warning>

For example, if you deploy the model with the deployment name `decision-1`, send `"model": "@azure-foundry/decision-1"`. If you name the deployment `pi-decision-1`, send `"model": "@azure-foundry/pi-decision-1"`. In both cases the response reports `"model": "microsoft-decision-1"`.

The AI Gateway logs and prices the call using the response `model`, so cost tracking works the same no matter what you name your deployment.

### Setup

1. Deploy `Microsoft-Decision-1` in your Foundry project by following [Microsoft's deployment steps](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision). From the deployment details page, copy the **resource endpoint**, the **deployment name**, and the **API key**.
2. Create your Azure AI Foundry provider by following [Creating Your Azure AI Foundry Provider](#creating-your-azure-ai-foundry-provider) above. Use the URL as shown in Foundry for your resource, for example `https://your-resource-name.services.ai.azure.com/models`.
3. If you manage models in **Model Provisioning**, add your deployment name as the model slug (for example `decision-1`), exactly as it appears in Foundry. See [Adding Multiple Models](#adding-multiple-models-to-your-azure-ai-foundry-provider).

<Note>
For decisions requests, The AI Gateway sends the call to the resource root at `/providers/microsoft/v1/systemone`. Any path in the saved Foundry URL, including a `/deployments/<name>` suffix, is dropped. No `api-version` is sent. The deployment is selected by the `model` field in the request body.
</Note>

Decisions requests work with the same authentication methods as the rest of this integration: API key, Azure Entra ID, and Azure Managed Identity. With an API key, The AI Gateway sends it in the `api-key` header, which is what the Microsoft decision endpoint expects.

### Send a decisions request

Microsoft-Decision-1 uses the Jev request shape: `state` for the content to judge, and a `questions` map. Set `model` to `@provider-slug/deployment-name`. The provider holds the Azure credentials, so no provider headers are needed.

```bash cURL
curl --request POST \
--url https://aigw.portkey.ai/v1/decisions \
--header 'Content-Type: application/json' \
--header "Authorization: Bearer $PORTKEY_API_KEY" \
--data '{
"model": "@azure-foundry/decision-1",
"state": {
"current_ask": "hello there, how is your day going?"
},
"questions": {
"greeting": {
"type": "choice",
"instructions": "Is this a greeting?",
"criteria": {
"yes": "a greeting",
"no": "anything else"
}
}
}
}'
```

Response (values are illustrative):

```json
{
"model": "microsoft-decision-1",
"answers": {
"greeting": {
"type": "choice",
"choice": "yes",
"probabilities": { "no": 0.0067, "yes": 0.9933 },
"confidence": 0.9866
}
},
"usage": { "input_tokens": 46, "output_tokens": 1 },
"provider": "azure-ai"
}
```

The OpenAI request shape (`input` and a `questions` array) also works. See [Request Shapes](/aigw/product/ai-gateway/decisions-api#request-shapes).

### Question types

Each entry in `questions` sets a `type`, which determines the shape of its answer. The `state` can be text or JSON.

| Type | Question it answers | Answer shape |
|:---|:---|:---|
| `noul` | Is this true? | `{ noul: <0 to 1 probability> }` |
| `choice` | Which option is it? | `{ choice, probabilities, confidence }` |
| `score` | How much? | `{ score, legend, probabilities, confidence }` |

The response `answers` object is keyed by the question names in your request.

#### `noul`: gates and filters

Use for yes/no conditions, such as whether a reply promises a refund or a document contains a prompt injection. `criteria` is optional. Define the true and false conditions to make borderline questions more precise.

```json
{
"model": "@azure-foundry/decision-1",
"state": "I refunded your $200. You don't need to contact billing.",
"questions": {
"promises_refund": {
"type": "noul",
"instructions": "Does this reply promise the customer a refund?",
"criteria": {
"true": "Commits to returning money",
"false": "Makes no commitment about money"
}
}
}
}
```

#### `choice`: classification and routing

Use to pick exactly one label from a fixed set, such as routing tickets or classifying intent. Describe each option clearly. The response includes a probability for every option, so a low top probability is a good signal to escalate to a person or another model.

```json
{
"model": "@azure-foundry/decision-1",
"state": "The API returns 500 on every call.",
"questions": {
"team": {
"type": "choice",
"instructions": "Which team should handle this ticket?",
"criteria": {
"billing": "Charges, invoices, and refunds",
"engineering": "Bugs, errors, and outages",
"support": "How-to and account questions"
}
}
}
}
```

#### `score`: ranking and grading

Use to place an item on an ordered scale, such as incident severity. List the levels from lowest to highest. The returned score is the probability-weighted average of the level indexes, so a four-level scale runs from `0` through `3` and the result can fall between levels. Prefer scores for relative ordering and thresholds, not as absolute ratings.

```json
{
"model": "@azure-foundry/decision-1",
"state": "Checkout is down for all EU customers since 09:00.",
"questions": {
"severity": {
"type": "score",
"instructions": "How severe is this incident?",
"criteria": ["Cosmetic", "Minor", "Major", "Critical"]
}
}
}
```

#### Combine questions in one request

Questions that share the same `state` can go in one request. This reduces request count and keeps the context consistent.

```json
{
"model": "@azure-foundry/decision-1",
"state": {
"message": "Checkout is down again. This is my third request.",
"customer_contact_count": 3
},
"questions": {
"team": {
"type": "choice",
"instructions": "Which team should handle this request?",
"criteria": {
"billing": "Charges, invoices, and refunds",
"engineering": "Bugs, errors, and outages",
"support": "How-to and account questions"
}
},
"severity": {
"type": "score",
"instructions": "How severe is this issue?",
"criteria": ["Cosmetic", "Minor", "Major", "Critical"]
},
"repeat_contact": {
"type": "noul",
"instructions": "Has the customer contacted support before?"
}
}
}
```

### Pricing and usage

Usage is reported in the response as `usage.input_tokens` and `usage.output_tokens`. The pricing entry for `microsoft-decision-1` charges for input tokens only. Output tokens, which hold the typed judgment itself, are priced at zero.

### Decisions troubleshooting

| Status | Likely cause | What to do |
|:---|:---|:---|
| `400` | Invalid question type or shape | Check `state`, `questions`, `type`, and `criteria` |
| `401` | Missing, invalid, or expired credential | Check the API key or Entra credentials on the provider |
| `403` | The identity has no access to the deployment | Check role assignments and deployment access in Azure |
| `404` | Wrong endpoint, or the deployment name does not exist | Confirm the `model` value matches your deployment name exactly. See [Request model vs response model](#request-model-vs-response-model) |
| `422` | Unsupported deployment name or question definition | Check the `model` value and each question definition |
| `429` | The deployment rate limit was exceeded | Retry with backoff, or request more quota in Azure. You can also add retries or fallbacks in a [gateway config](/aigw/product/ai-gateway/configs) |

Upstream errors use the standard error shape, with the message prefixed by `azure-ai`:

```json
{
"error": {
"message": "azure-ai error: The API deployment microsoft-decision-1 does not exist. If you created the deployment within the last 5 minutes, please wait a moment and try again.",
"type": null,
"param": null,
"code": null
},
"provider": "azure-ai"
}
```

### Limitations and responsible use

From Microsoft's guidance for this model:

- **No explanations.** The model returns probabilities, not a written rationale.
- **Wording sensitivity.** Scores can change with how you phrase or order questions and options.
- **Bias.** The model might reflect biases from its base model and training data. Do not use its scores as the sole basis for decisions about individuals.
- **Safety filtering.** If you use it as a safety filter, it can miss subtle harmful content or flag benign content.
- **Validate on your data.** Test thresholds against labeled examples from your own workload before you automate a gate.

See [Known limitations and best practices](https://learn.microsoft.com/en-us/azure/foundry/foundry-models/how-to/use-foundry-models-microsoft-decision) in Microsoft's guide for the full list.

---

## Next Steps

<CardGroup cols={2}>
Expand Down
1 change: 1 addition & 0 deletions aigw/product/ai-gateway/decisions-api.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ Either shape works with any provider. See [Switching Shapes](#switching-shapes).
| [OpenRouter](/aigw/integrations/llms/openrouter) | `openrouter` | Jev | Routes to Jev models |
| [Workers AI](/aigw/integrations/llms/workers-ai) | `workers-ai` | Jev | Cloudflare model ID, for example `@your-cf-integration-slug/model` |
| [OpenAI](/aigw/integrations/llms/openai) | `openai` | OpenAI | `gpt-6-luna` |
| [Microsoft Decision 1](/aigw/integrations/llms/azure-foundry#decisions-microsoft-decision-1) | `azure-ai` | Jev | Azure AI Foundry. `model` is your deployment name |

Any provider accepts either request shape. The AI Gateway converts the request to the provider's native shape and converts the answers back. See [Switching Shapes](#switching-shapes).

Expand Down
Loading
Loading