> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prisme.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Model Governance

> Control AI model access, usage policies, and routing strategies

Model Governance lets you control which AI models are available to your organization, set usage policies, configure routing strategies, and monitor consumption. Access these features from **Agents Controls** in the sidebar.

## Overview

The Agents Controls section has four tabs:

| Tab                  | Description                           |
| -------------------- | ------------------------------------- |
| **Models**           | Configure allowed models and policies |
| **Usage**            | Monitor consumption against quotas    |
| **Service Accounts** | Machine-to-machine authentication     |
| **Agents**           | Per-agent model restrictions          |

## Models Configuration

### Allowed Models

By default, organizations can use all models enabled by the platform administrator. You can restrict this to a specific list.

1. Go to **Agents Controls > Models**
2. Toggle **Restrict Models**
3. Select which models to allow
4. Click **Save**

### Default Models

Set the default models used when agents don't specify one:

| Setting                      | Description                       |
| ---------------------------- | --------------------------------- |
| **Default Completion Model** | Used for chat and text generation |
| **Default Embedding Model**  | Used for vector embeddings        |

### Quota Policy

Configure what happens when quota limits are reached:

| Policy             | Behavior                                |
| ------------------ | --------------------------------------- |
| **Hard Block**     | Requests fail with quota exceeded error |
| **Soft Downgrade** | Fall back to a cheaper model            |

### Downgrade Mapping

When using soft downgrade, configure which models to substitute:

```
claude-3-opus → claude-3-sonnet
gpt-4 → gpt-3.5-turbo
```

This ensures continuity when expensive models hit quota limits.

### Failover Mapping

Configure automatic failover when a model is unavailable:

```
claude-3-sonnet → claude-3-haiku
gpt-4 → gpt-4-turbo
```

Failover activates when the primary model returns errors, not for quota limits.

**Failover behavior:**

* **5xx errors**: switches to the failover model from the mapping, or falls back to the default completion model (with linear backoff 1s, 2s, 3s...)
* **429 rate limits**: retries the same model after a 5s backoff
* **Other 4xx errors**: returned immediately without retry
* Up to 3 attempts (configurable, hard cap of 10)
* Failover models are validated against governance access controls before use
* Failover only applies to **non-streaming** requests

## Model Routing

The LLM Gateway supports intelligent model routing — selecting the best model for a request based on configurable strategies. Use `model: "auto"` in API calls to trigger routing.

### Routing Strategies

| Strategy           | Description                                                                                     |
| ------------------ | ----------------------------------------------------------------------------------------------- |
| **Disabled**       | No automatic routing; use specified model                                                       |
| **Rules**          | Rule-based routing — first matching rule wins                                                   |
| **LLM Classifier** | Use a cheap LLM to classify the request and map category to model                               |
| **Capabilities**   | Query model catalog for enabled models matching required tags                                   |
| **Cost Optimized** | Same as capabilities but iterates cost tiers (`low` → `medium` → `high`) to pick cheapest match |
| **Hybrid**         | Try rules first, fall back to LLM classifier if no rule matches                                 |

### Rule-Based Routing

Define conditions to route requests to different models:

```yaml theme={null}
routing:
  strategy: rules
  rules:
    - condition: messages_count
      operator: "<"
      threshold: 5
      model: claude-3-haiku
    - condition: messages_count
      operator: ">="
      threshold: 5
      model: claude-3-sonnet
```

The only supported condition type is `messages_count` with operators `<`, `>`, `<=`, `>=`, `=`. If the selected model is blocked by governance, routing falls back to the default.

## Usage Monitoring

The **Usage** tab shows consumption against your subscription quotas.

### Tracked Metrics

| Metric               | Type       | Description             |
| -------------------- | ---------- | ----------------------- |
| `llm.requests.rpm`   | Rate       | Requests per minute     |
| `llm.requests.daily` | Rate       | Requests per day        |
| `llm.tokens.monthly` | Cumulative | Total tokens this month |
| `llm.cost.monthly`   | Cumulative | Total cost this month   |

### Understanding Quotas

Quotas are defined in your subscription:

* **Rate limits** reset after the time window (minute, hour, day)
* **Cumulative limits** accumulate until the billing period resets

### Usage Display

Each metric shows:

* Current value vs. limit
* Percentage consumed
* Visual progress bar (yellow at 80%, red at 95%)

## Per-Agent Model Restrictions

The **Agents** tab lets you restrict which models specific agents can use.

### Why Restrict Agents?

* **Cost control**: Limit expensive model usage to specific agents
* **Compliance**: Ensure sensitive agents only use approved models
* **Testing**: Restrict test agents to cheaper models

### Configuring Agent Models

1. Go to **Agents Controls > Agents**
2. Find the agent to configure
3. Click **Configure Models**
4. Select allowed models (or leave empty for org defaults)
5. Save changes

## Service Accounts

Service accounts provide machine-to-machine authentication. See [Identity & Access](./identity-access#service-accounts) for details.

In the context of Model Governance:

* Link service accounts to specific agents
* Track which service accounts consume LLM resources
* Control model access per service account

## Model Lifecycle: Hidden vs Disabled

A platform administrator can manage a model's availability at three levels of severity. Each level has a different effect on what the LLM Gateway accepts, what surfaces in UI pickers, and whether the model is auto-pruned from organization allowlists.\
"LLM Gateway" designates the product responsible for llm calls execution and authorization, whether through API calls or chat front-end.

### State matrix

| Flag                                | Set by         | LLM Gateway                                  | Org "Allowed models" panel                            | Agent-level pickers | Auto-pruned from `org.allowed_models` |
| ----------------------------------- | -------------- | -------------------------------------------- | ----------------------------------------------------- | ------------------- | ------------------------------------- |
| **Visible & enabled** (default)     | n/a            | ✅ Accepted                                   | ✅ Shown                                               | ✅ Shown             | n/a                                   |
| **Hidden** (`display.hidden: true`) | Platform admin | ✅ Still accepted via the org allowlist entry | ✅ Still shown — admin can decide to keep or remove it | ❌ Filtered out      | ❌ Stays                               |
| **Disabled** (`enabled: false`)     | Platform admin | ❌ Rejected (`MODEL_NOT_ALLOWED`)             | ❌ Filtered out                                        | ❌ Filtered out      | ✅ Removed                             |
| **Deleted** (removed from catalog)  | Platform admin | ❌ Rejected                                   | ❌ Gone                                                | ❌ Gone              | ✅ Removed                             |

### Why hidden is not the same as disabled

The `hidden` flag exists specifically to **deprecate a model from UI selection without breaking direct API usage or existing agents that already point to it**. Saving the organization's `Allowed models` panel without a hidden model would effectively forbid it, so hidden models are left in the list — only filtered out of agent-level dropdowns. An agent that had selected a model before it became hidden keeps using it, and API consumers calling `/chat/completions` with that `model_id` keep working.

If a model truly must be retired, use **Disabled** (or delete it from the catalog) to prevent any agent from using it.

### Warnings surfaced in the Agent Creator

When you open an agent whose currently-configured model is no longer fully usable, the model picker surfaces one of two messages:

* 🔴 **Error — model unusable**: the model has been deleted, disabled at platform level, or removed from the org / agent allowlist. The agent will fail to process any request until you select another model.
* 🟡 **Warning — model hidden**: the model has only been hidden at platform level. The agent still works for now, but you're no longer expected to pick this model — please switch to another one.

The current model is always shown even when filtered out so you can always see what is configured, but does not appear in models picker to force you choose another one.

## Model Access Control

When a request arrives, the LLM Gateway validates the requested model against three sequential allowlists:

1. **Org allowlist** — Is the model in the organization's allowed models? (if the list exists and has entries)
2. **Agent allowlist** — Is the model in the agent's allowed models? (passed by agent-factory)
3. **API key scopes** — Is the model allowed by the API key's scopes?

Each check is skipped if the corresponding list is empty or absent. If any check fails, the behavior depends on the quota policy:

* **Soft Downgrade**: silently swaps to the default completion model
* **Hard Block** (default): returns a `403` error with `MODEL_NOT_ALLOWED`

## Carbon Footprint Tracking

Every LLM call includes an estimated environmental impact in the response:

```json theme={null}
{
  "usage": {
    "carbon": {
      "energy": { "value": 0.000012, "unit": "kWh" },
      "gwp": { "value": 0.0000057, "unit": "kgCO2eq" }
    }
  }
}
```

The calculation considers GPU energy per token, server overhead, datacenter PUE (Power Usage Effectiveness), and regional emission factors. Results include uncertainty margins (+/- 20%).

| PUE Profile       | Multiplier |
| ----------------- | ---------- |
| Efficient         | 1.1        |
| Average (default) | 1.58       |
| Inefficient       | 2.0        |

Carbon data is included in analytics events and powers the observability dashboards.

## Supported Providers

The LLM Gateway abstracts multiple providers behind a unified API:

| Provider          | Models                                                  | Notes                                                                                                                     |
| ----------------- | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
| OpenAI            | GPT-5, GPT-4o, o3-mini, embeddings, DALL-E 3            | Direct API                                                                                                                |
| Azure OpenAI      | GPT-5, GPT-4o, embeddings, Claude (via Azure AI)        | Multiple resource configs                                                                                                 |
| OpenAI-compatible | Gemini, DeepSeek, Mistral, Cerebras, OVH, Linagora      | Via `openailike` provider type                                                                                            |
| Anthropic         | Claude Sonnet 4.5, Claude 3.7 Sonnet, Claude 3.5 Sonnet | Native API; optional custom `endpoint` + selectable `auth_scheme` for self-hosted / compatible / non-standard Azure hosts |
| Google Vertex AI  | Gemini 2.5/3, Imagen 4.0, text-embedding-005            | Via model aliases + service account                                                                                       |
| AWS Bedrock       | Claude, Titan, Cohere, Nova, Llama                      | Multiple region/credential sets                                                                                           |

All providers are normalized to the OpenAI API format for chat completions and streaming (SSE).

### Provider Options

Every provider block in the LLM Gateway config exposes a uniform `options` object that lets you reshape the request just before it hits the upstream endpoint, without changing application code. Three options are supported, all consumed by the same path inside `fetchLLM`:

| Option                      | What it does                                                                                                           | Applies to    |
| --------------------------- | ---------------------------------------------------------------------------------------------------------------------- | ------------- |
| `options.parameterAliases`  | Renames request body keys (e.g. `max_tokens` → `max_completion_tokens` for providers that adopted the new OpenAI name) | All providers |
| `options.excludeParameters` | Drops request body keys the upstream rejects (e.g. `top_p` for models that don't support it)                           | All providers |
| `options.headers`           | Attaches arbitrary HTTP headers (tenant tags, gateway subscription keys, correlation IDs…)                             | All providers |

The same three options are also exposed in the **Agents Controls → Providers** UI for every provider type. Backwards-compatible: any option you omit is a no-op.

<Note>
  Provider options are configured **per-provider** in the workspace that owns the provider config — **LLM Gateway** (`providers.<provider>.options.*`) for the current LLM Gateway architecture, or **AI Knowledge** (`llm.<provider>.options.*`) for the legacy AI Knowledge stack. The OpenAI app instance is a pass-through and is **not** the configuration surface; editing it directly there has no effect on traffic that flows through the gateway.
</Note>

#### `parameterAliases`

Each entry is a `{ from, to }` pair. When a key matching `from` is present in the request body, it is renamed to `to` (the value is preserved, the old key is dropped). Typical use: a provider exposes the OpenAI Chat Completions surface but renamed `max_tokens` to `max_completion_tokens`.

```yaml theme={null}
options:
  parameterAliases:
    - from: max_tokens
      to: max_completion_tokens
```

#### `excludeParameters`

Each entry is a body key name to delete before the request is sent. Typical use: a model rejects `top_p` or `presence_penalty` and you want to keep your app code provider-agnostic instead of conditionally stripping.

```yaml theme={null}
options:
  excludeParameters:
    - top_p
    - presence_penalty
```

#### `headers`

A map of `name → value` HTTP header pairs attached to every upstream request. Typical uses: tenant-aware gateways (e.g. Azure API Management in front of Bedrock / Anthropic / Vertex), custom auth schemes, request tagging for cost attribution, or correlating upstream provider logs with Prisme.ai audit trails.

**Static values** are sent verbatim:

| Header name     | Value       | Sent as                   |
| --------------- | ----------- | ------------------------- |
| `X-Tenant-Id`   | `acme-corp` | `X-Tenant-Id: acme-corp`  |
| `X-Cost-Center` | `R&D-EMEA`  | `X-Cost-Center: R&D-EMEA` |

**Runtime placeholders** inject context from the calling Prisme.ai request. The value of the header must be **exactly** one of the patterns below:

| Placeholder value | Replaced with                                                                                                                                                                                          |
| ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `{correlationId}` | The Prisme.ai correlation ID of the current request — propagates trace context to the LLM provider's logs                                                                                              |
| `{user.<path>}`   | Any field of the calling user, at arbitrary depth. `{user.id}` for the Prisme.ai user ID, `{user.email}` for the email, `{user.authData.entraid.firstname}` for a nested provider-specific claim, etc. |
| `{userId}`        | Legacy alias of `{user.id}`, kept for backwards compatibility — prefer `{user.id}` in new configurations                                                                                               |

Resolution rules for `{user.<path>}`:

* If the path doesn't exist on the user object, the header value resolves to an empty string.
* If the resolved value is not a string (object, array, number), it is JSON-stringified before being sent.

<Warning>
  Substitution is **full-string only**. The value must match the placeholder pattern exactly — composite values such as `prefix-{user.id}` or `req-{correlationId}-v2` are sent as-is, without substitution. Unknown placeholders that don't match any supported pattern (e.g. `{foo.bar}`) are also sent verbatim.
</Warning>

**Examples**

OpenAI-compatible (`openailike`) — typical case of routing through a custom gateway:

```yaml theme={null}
- api_key: '{{secret.myProviderApiKey}}'
  endpoint: https://my-openai-like-provider.example.com/v1/
  models:
    - my-model
  options:
    headers:
      x-correlation-id: '{correlationId}'
      x-user-id: '{user.id}'
      x-user-email: '{user.email}'
      x-user-firstname: '{user.authData.entraid.firstname}'
      x-tenant: 'acme-corp'
```

Anthropic — injecting an Azure API Management gateway's subscription key in front of a Claude-compatible endpoint:

```yaml theme={null}
anthropic:
  api_key: '{{secret.anthropicApiKey}}'
  anthropic_version: '2023-06-01'
  endpoint: https://my-apim.example.net/ai/anthropic/v1/messages
  models:
    - claude-sonnet-4-5
  options:
    headers:
      x-api-key: '{{secret.anthropicApiKey}}'
      Ocp-Apim-Subscription-Key: '{{secret.apimSubscriptionKey}}'
      x-correlation-id: '{correlationId}'
      x-user-email: '{user.email}'
```

**Notes on headers**:

* Custom headers are **merged on top** of the provider's default headers (auth + content-type + provider-specific headers like `anthropic-version`). Reusing a default header name will override it — useful for `Authorization` on `openailike` when the upstream requires a non-Bearer scheme, or for `x-api-key` on `anthropic` when fronting Anthropic with an API Management gateway whose key differs from the upstream key.
* For multipart endpoints (image edits, `openailike` only), the `Content-Type` is dropped automatically so the runtime can set the multipart boundary.
* On Bedrock and Vertex, the standard provider auth (AWS Signature v4 / Google service account token) is **always added by the gateway**, regardless of `options.headers` — you cannot disable it through this option.

### Custom Endpoint (Anthropic)

The native **Anthropic** provider supports an optional `endpoint` field. When set, it is used **verbatim** as the request URL instead of the default `https://api.anthropic.com/v1/messages`. This lets the Anthropic provider target any Anthropic-compatible host — self-hosted Claude deployments, Anthropic-compatible gateways, or **Azure AI Foundry** endpoints whose URL doesn't fit the standard `{resource}.{domain}/anthropic/v1/messages` scheme.

<Note>
  Like the [provider options](#provider-options) above, this is configured in the workspace that owns the provider config — **LLM Gateway** (`config.value.providers.anthropic.endpoint`). The OpenAI app instance is a pass-through and is **not** the configuration surface.
</Note>

Example LLM Gateway config (`workspaces/llm-gateway/index.yml`, under `config.value.providers.anthropic`):

```yaml theme={null}
anthropic:
  api_key: '{{secret.anthropicClientKey}}'
  anthropic_version: '2023-06-01'
  endpoint: https://my-foundry.example.net/custom/path/v1/messages   # used verbatim
  auth_scheme: bearer        # x-api-key (default) | api-key | bearer
  models:
    - claude-opus-4-6
    - claude-sonnet-4-5
```

**Notes**:

* **Additive** — omit `endpoint` and the provider behaves exactly as before, calling `https://api.anthropic.com/v1/messages`.
* **Selectable auth scheme** — set `auth_scheme` to control how the key is sent: `x-api-key` (default, the standard Anthropic header), `api-key`, or `bearer` (sends `Authorization: Bearer <api_key>` and omits `x-api-key`). The key or token always goes in `api_key`. Omit `auth_scheme` to keep the default `x-api-key` behavior. For richer needs (multiple auth headers, gateway subscription keys, request tagging), use [`options.headers`](#headers) — it overrides anything set by `auth_scheme`.
* **Include the full path** — the value is taken as-is; append `/v1/messages` (or whatever path the host exposes).
* **vs Azure OpenAI `format: anthropic`** — if the URL fits `https://{resource}.{domain}/anthropic/v1/messages`, configure it as an `azure_openai.resources` entry with `format: anthropic` (you get URL forging and can co-locate GPT + Claude on one resource). Use `anthropic.endpoint` when the URL does **not** fit that scheme.

## Best Practices

<CardGroup cols={2}>
  <Card title="Start Restrictive" icon="shield">
    Begin with a limited model list and expand based on need
  </Card>

  <Card title="Use Soft Downgrade" icon="arrow-down">
    Prefer soft downgrade to maintain service during quota limits
  </Card>

  <Card title="Monitor Usage" icon="chart-line">
    Set alerts before hitting quota limits
  </Card>

  <Card title="Configure Failover" icon="rotate">
    Ensure critical workflows have failover models
  </Card>
</CardGroup>

## Common Scenarios

<Tabs>
  <Tab title="Cost Control">
    To minimize costs while maintaining quality:

    1. Enable **Soft Downgrade** policy
    2. Configure downgrade mapping:
       * `claude-3-opus` → `claude-3-sonnet`
       * `gpt-4` → `gpt-3.5-turbo`
    3. Set conservative monthly token limits
    4. Use rule-based routing to prefer cheaper models for simple requests
  </Tab>

  <Tab title="Compliance">
    To restrict models for compliance:

    1. Enable **Restrict Models**
    2. Allow only approved models (e.g., models hosted in specific regions)
    3. Use **Hard Block** policy to prevent unauthorized usage
    4. Configure per-agent restrictions for sensitive workflows
  </Tab>

  <Tab title="High Availability">
    To maximize uptime:

    1. Configure **Failover Mapping** for all critical models
    2. Ensure failover targets are from different providers
    3. Use **Soft Downgrade** to handle quota limits gracefully
    4. Monitor error rates in Observability
  </Tab>
</Tabs>

## Next Steps

<CardGroup cols={2}>
  <Card title="Capabilities" icon="puzzle-piece" href="./capabilities">
    Manage tools, MCP servers, and guardrails
  </Card>

  <Card title="Observability" icon="chart-line" href="./observability">
    Monitor model costs and performance
  </Card>
</CardGroup>