Model Governance

Control AI model access, usage policies, and routing strategies

Model Governance lets you control which AI models are available to your organization, set usage policies, configure routing strategies, and monitor consumption. Access these features from Agents Controls in the sidebar.

Overview

The Agents Controls section has four tabs:

Tab	Description
Models	Configure allowed models and policies
Usage	Monitor consumption against quotas
Service Accounts	Machine-to-machine authentication
Agents	Per-agent model restrictions

Models Configuration

Allowed Models

By default, organizations can use all models enabled by the platform administrator. You can restrict this to a specific list.

1. Go to Agents Controls > Models
2. Toggle Restrict Models
3. Select which models to allow
4. Click Save

Default Models

Set the default models used when agents don’t specify one:

Setting	Description
Default Completion Model	Used for chat and text generation
Default Embedding Model	Used for vector embeddings

Quota Policy

Configure what happens when quota limits are reached:

Policy	Behavior
Hard Block	Requests fail with quota exceeded error
Soft Downgrade	Fall back to a cheaper model

Downgrade Mapping

When using soft downgrade, configure which models to substitute:

claude-3-opus → claude-3-sonnet
gpt-4 → gpt-3.5-turbo

This ensures continuity when expensive models hit quota limits.

Failover Mapping

Configure automatic failover when a model is unavailable:

claude-3-sonnet → claude-3-haiku
gpt-4 → gpt-4-turbo

Failover activates when the primary model returns errors, not for quota limits.

Failover behavior:

• 5xx errors: switches to the failover model from the mapping, or falls back to the default completion model (with linear backoff 1s, 2s, 3s…)
• 429 rate limits: retries the same model after a 5s backoff
• Other 4xx errors: returned immediately without retry
• Up to 3 attempts (configurable, hard cap of 10)
• Failover models are validated against governance access controls before use
• Failover only applies to non-streaming requests

Model Routing

The LLM Gateway supports intelligent model routing — selecting the best model for a request based on configurable strategies. Use model: "auto" in API calls to trigger routing.

Routing Strategies

Strategy	Description
Disabled	No automatic routing; use specified model
Rules	Rule-based routing — first matching rule wins
LLM Classifier	Use a cheap LLM to classify the request and map category to model
Capabilities	Query model catalog for enabled models matching required tags
Cost Optimized	Same as capabilities but iterates cost tiers (low → medium → high) to pick cheapest match
Hybrid	Try rules first, fall back to LLM classifier if no rule matches

Rule-Based Routing

Define conditions to route requests to different models:

routing:
  strategy: rules
  rules:
    - condition: messages_count
      operator: "<"
      threshold: 5
      model: claude-3-haiku
    - condition: messages_count
      operator: ">="
      threshold: 5
      model: claude-3-sonnet

The only supported condition type is messages_count with operators <, >, <=, >=, =. If the selected model is blocked by governance, routing falls back to the default.

Usage Monitoring

The Usage tab shows consumption against your subscription quotas.

Tracked Metrics

Metric	Type	Description
llm.requests.rpm	Rate	Requests per minute
llm.requests.daily	Rate	Requests per day
llm.tokens.monthly	Cumulative	Total tokens this month
llm.cost.monthly	Cumulative	Total cost this month

Understanding Quotas

Quotas are defined in your subscription:

• Rate limits reset after the time window (minute, hour, day)
• Cumulative limits accumulate until the billing period resets

Usage Display

Each metric shows:

• Current value vs. limit
• Percentage consumed
• Visual progress bar (yellow at 80%, red at 95%)

Per-Agent Model Restrictions

The Agents tab lets you restrict which models specific agents can use.

Why Restrict Agents?

• Cost control: Limit expensive model usage to specific agents
• Compliance: Ensure sensitive agents only use approved models
• Testing: Restrict test agents to cheaper models

Configuring Agent Models

1. Go to Agents Controls > Agents
2. Find the agent to configure
3. Click Configure Models
4. Select allowed models (or leave empty for org defaults)
5. Save changes

Service Accounts

Service accounts provide machine-to-machine authentication. See Identity & Access for details.

In the context of Model Governance:

• Link service accounts to specific agents
• Track which service accounts consume LLM resources
• Control model access per service account

Model Lifecycle: Hidden vs Disabled

A platform administrator can manage a model’s availability at three levels of severity. Each level has a different effect on what the LLM Gateway accepts, what surfaces in UI pickers, and whether the model is auto-pruned from organization allowlists.
”LLM Gateway” designates the product responsible for llm calls execution and authorization, whether through API calls or chat front-end.

State matrix

Flag	Set by	LLM Gateway	Org “Allowed models” panel	Agent-level pickers	Auto-pruned from org.allowed_models
Visible & enabled (default)	n/a	✅ Accepted	✅ Shown	✅ Shown	n/a
Hidden (display.hidden: true)	Platform admin	✅ Still accepted via the org allowlist entry	✅ Still shown — admin can decide to keep or remove it	❌ Filtered out	❌ Stays
Disabled (enabled: false)	Platform admin	❌ Rejected (MODEL_NOT_ALLOWED)	❌ Filtered out	❌ Filtered out	✅ Removed
Deleted (removed from catalog)	Platform admin	❌ Rejected	❌ Gone	❌ Gone	✅ Removed

Why hidden is not the same as disabled

The hidden flag exists specifically to deprecate a model from UI selection without breaking direct API usage or existing agents that already point to it. Saving the organization’s Allowed models panel without a hidden model would effectively forbid it, so hidden models are left in the list — only filtered out of agent-level dropdowns. An agent that had selected a model before it became hidden keeps using it, and API consumers calling /chat/completions with that model_id keep working.

If a model truly must be retired, use Disabled (or delete it from the catalog) to prevent any agent from using it.

Warnings surfaced in the Agent Creator

When you open an agent whose currently-configured model is no longer fully usable, the model picker surfaces one of two messages:

• 🔴 Error — model unusable: the model has been deleted, disabled at platform level, or removed from the org / agent allowlist. The agent will fail to process any request until you select another model.
• 🟡 Warning — model hidden: the model has only been hidden at platform level. The agent still works for now, but you’re no longer expected to pick this model — please switch to another one.

The current model is always shown even when filtered out so you can always see what is configured, but does not appear in models picker to force you choose another one.

Model Access Control

When a request arrives, the LLM Gateway validates the requested model against three sequential allowlists:

1. Org allowlist — Is the model in the organization’s allowed models? (if the list exists and has entries)
2. Agent allowlist — Is the model in the agent’s allowed models? (passed by agent-factory)
3. API key scopes — Is the model allowed by the API key’s scopes?

Each check is skipped if the corresponding list is empty or absent. If any check fails, the behavior depends on the quota policy:

• Soft Downgrade: silently swaps to the default completion model
• Hard Block (default): returns a 403 error with MODEL_NOT_ALLOWED

Carbon Footprint Tracking

Every LLM call includes an estimated environmental impact in the response:

{
  "usage": {
    "carbon": {
      "energy": { "value": 0.000012, "unit": "kWh" },
      "gwp": { "value": 0.0000057, "unit": "kgCO2eq" }
    }
  }
}

The calculation considers GPU energy per token, server overhead, datacenter PUE (Power Usage Effectiveness), and regional emission factors. Results include uncertainty margins (+/- 20%).

PUE Profile	Multiplier
Efficient	1.1
Average (default)	1.58
Inefficient	2.0

Carbon data is included in analytics events and powers the observability dashboards.

Supported Providers

The LLM Gateway abstracts multiple providers behind a unified API:

Provider	Models	Notes
OpenAI	GPT-5, GPT-4o, o3-mini, embeddings, DALL-E 3	Direct API
Azure OpenAI	GPT-5, GPT-4o, embeddings, Claude (via Azure AI)	Multiple resource configs
OpenAI-compatible	Gemini, DeepSeek, Mistral, Cerebras, OVH, Linagora	Via openailike provider type
Anthropic	Claude Sonnet 4.5, Claude 3.7 Sonnet, Claude 3.5 Sonnet	Native API; optional custom endpoint + selectable auth_scheme for self-hosted / compatible / non-standard Azure hosts
Google Vertex AI	Gemini 2.5/3, Imagen 4.0, text-embedding-005	Via model aliases + service account
AWS Bedrock	Claude, Titan, Cohere, Nova, Llama	Multiple region/credential sets

All providers are normalized to the OpenAI API format for chat completions and streaming (SSE).

Provider Options

Every provider block in the LLM Gateway config exposes a uniform options object that lets you reshape the request just before it hits the upstream endpoint, without changing application code. Three options are supported, all consumed by the same path inside fetchLLM:

Option	What it does	Applies to
options.parameterAliases	Renames request body keys (e.g. max_tokens → max_completion_tokens for providers that adopted the new OpenAI name)	All providers
options.excludeParameters	Drops request body keys the upstream rejects (e.g. top_p for models that don’t support it)	All providers
options.headers	Attaches arbitrary HTTP headers (tenant tags, gateway subscription keys, correlation IDs…)	All providers

The same three options are also exposed in the Agents Controls → Providers UI for every provider type. Backwards-compatible: any option you omit is a no-op.

Provider options are configured per-provider in the workspace that owns the provider config — LLM Gateway (providers.<provider>.options.*) for the current LLM Gateway architecture, or AI Knowledge (llm.<provider>.options.*) for the legacy AI Knowledge stack. The OpenAI app instance is a pass-through and is not the configuration surface; editing it directly there has no effect on traffic that flows through the gateway.

parameterAliases

Each entry is a { from, to } pair. When a key matching from is present in the request body, it is renamed to to (the value is preserved, the old key is dropped). Typical use: a provider exposes the OpenAI Chat Completions surface but renamed max_tokens to max_completion_tokens.

options:
  parameterAliases:
    - from: max_tokens
      to: max_completion_tokens

excludeParameters

Each entry is a body key name to delete before the request is sent. Typical use: a model rejects top_p or presence_penalty and you want to keep your app code provider-agnostic instead of conditionally stripping.

options:
  excludeParameters:
    - top_p
    - presence_penalty

headers

A map of name → value HTTP header pairs attached to every upstream request. Typical uses: tenant-aware gateways (e.g. Azure API Management in front of Bedrock / Anthropic / Vertex), custom auth schemes, request tagging for cost attribution, or correlating upstream provider logs with Prisme.ai audit trails.

Static values are sent verbatim:

Header name	Value	Sent as
X-Tenant-Id	acme-corp	X-Tenant-Id: acme-corp
X-Cost-Center	R&D-EMEA	X-Cost-Center: R&D-EMEA

Runtime placeholders inject context from the calling Prisme.ai request. The value of the header must be exactly one of the patterns below:

Placeholder value	Replaced with
{correlationId}	The Prisme.ai correlation ID of the current request — propagates trace context to the LLM provider’s logs
{user.<path>}	Any field of the calling user, at arbitrary depth. {user.id} for the Prisme.ai user ID, {user.email} for the email, {user.authData.entraid.firstname} for a nested provider-specific claim, etc.
{userId}	Legacy alias of {user.id}, kept for backwards compatibility — prefer {user.id} in new configurations

Resolution rules for {user.<path>}:

• If the path doesn’t exist on the user object, the header value resolves to an empty string.
• If the resolved value is not a string (object, array, number), it is JSON-stringified before being sent.

Substitution is full-string only. The value must match the placeholder pattern exactly — composite values such as prefix-{user.id} or req-{correlationId}-v2 are sent as-is, without substitution. Unknown placeholders that don’t match any supported pattern (e.g. {foo.bar}) are also sent verbatim.

Examples

OpenAI-compatible (openailike) — typical case of routing through a custom gateway:

- api_key: '{{secret.myProviderApiKey}}'
  endpoint: https://my-openai-like-provider.example.com/v1/
  models:
    - my-model
  options:
    headers:
      x-correlation-id: '{correlationId}'
      x-user-id: '{user.id}'
      x-user-email: '{user.email}'
      x-user-firstname: '{user.authData.entraid.firstname}'
      x-tenant: 'acme-corp'

Anthropic — injecting an Azure API Management gateway’s subscription key in front of a Claude-compatible endpoint:

anthropic:
  api_key: '{{secret.anthropicApiKey}}'
  anthropic_version: '2023-06-01'
  endpoint: https://my-apim.example.net/ai/anthropic/v1/messages
  models:
    - claude-sonnet-4-5
  options:
    headers:
      x-api-key: '{{secret.anthropicApiKey}}'
      Ocp-Apim-Subscription-Key: '{{secret.apimSubscriptionKey}}'
      x-correlation-id: '{correlationId}'
      x-user-email: '{user.email}'

Notes on headers:

• Custom headers are merged on top of the provider’s default headers (auth + content-type + provider-specific headers like anthropic-version). Reusing a default header name will override it — useful for Authorization on openailike when the upstream requires a non-Bearer scheme, or for x-api-key on anthropic when fronting Anthropic with an API Management gateway whose key differs from the upstream key.
• For multipart endpoints (image edits, openailike only), the Content-Type is dropped automatically so the runtime can set the multipart boundary.
• On Bedrock and Vertex, the standard provider auth (AWS Signature v4 / Google service account token) is always added by the gateway, regardless of options.headers — you cannot disable it through this option.

Custom Endpoint (Anthropic)

The native Anthropic provider supports an optional endpoint field. When set, it is used verbatim as the request URL instead of the default https://api.anthropic.com/v1/messages. This lets the Anthropic provider target any Anthropic-compatible host — self-hosted Claude deployments, Anthropic-compatible gateways, or Azure AI Foundry endpoints whose URL doesn’t fit the standard {resource}.{domain}/anthropic/v1/messages scheme.

Like the provider options above, this is configured in the workspace that owns the provider config — LLM Gateway (config.value.providers.anthropic.endpoint). The OpenAI app instance is a pass-through and is not the configuration surface.

Example LLM Gateway config (workspaces/llm-gateway/index.yml, under config.value.providers.anthropic):

anthropic:
  api_key: '{{secret.anthropicClientKey}}'
  anthropic_version: '2023-06-01'
  endpoint: https://my-foundry.example.net/custom/path/v1/messages   # used verbatim
  auth_scheme: bearer        # x-api-key (default) | api-key | bearer
  models:
    - claude-opus-4-6
    - claude-sonnet-4-5

Notes:

• Additive — omit endpoint and the provider behaves exactly as before, calling https://api.anthropic.com/v1/messages.
• Selectable auth scheme — set auth_scheme to control how the key is sent: x-api-key (default, the standard Anthropic header), api-key, or bearer (sends Authorization: Bearer <api_key> and omits x-api-key). The key or token always goes in api_key. Omit auth_scheme to keep the default x-api-key behavior. For richer needs (multiple auth headers, gateway subscription keys, request tagging), use options.headers — it overrides anything set by auth_scheme.
• Include the full path — the value is taken as-is; append /v1/messages (or whatever path the host exposes).
• vs Azure OpenAI format: anthropic — if the URL fits https://{resource}.{domain}/anthropic/v1/messages, configure it as an azure_openai.resources entry with format: anthropic (you get URL forging and can co-locate GPT + Claude on one resource). Use anthropic.endpoint when the URL does not fit that scheme.

Best Practices

Start Restrictive

Begin with a limited model list and expand based on need

Use Soft Downgrade

Prefer soft downgrade to maintain service during quota limits

Monitor Usage

Set alerts before hitting quota limits

Configure Failover

Ensure critical workflows have failover models

Common Scenarios

Cost Control

To minimize costs while maintaining quality:

1. Enable Soft Downgrade policy
2. Configure downgrade mapping:
   • claude-3-opus → claude-3-sonnet
   • gpt-4 → gpt-3.5-turbo
3. Set conservative monthly token limits
4. Use rule-based routing to prefer cheaper models for simple requests

Compliance

To restrict models for compliance:

1. Enable Restrict Models
2. Allow only approved models (e.g., models hosted in specific regions)
3. Use Hard Block policy to prevent unauthorized usage
4. Configure per-agent restrictions for sensitive workflows

High Availability

To maximize uptime:

1. Configure Failover Mapping for all critical models
2. Ensure failover targets are from different providers
3. Use Soft Downgrade to handle quota limits gracefully
4. Monitor error rates in Observability

Next Steps

Capabilities

Manage tools, MCP servers, and guardrails

Observability

Monitor model costs and performance