> ## Documentation Index
> Fetch the complete documentation index at: https://docs.prisme.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Analytics

> Measure usage, performance, and impact of your agents

Analytics show how users interact with your published agents. Use these insights to improve agent quality and demonstrate value.

## The Analytics Page

Open any agent and go to the **Analytics** section. You'll see metrics for the selected time period.

## Time Periods

Select a period to analyze:

* **Today** - Current day
* **Last 7 days** - Past week
* **Last 30 days** - Past month
* **Last 90 days** - Past quarter
* **Custom** - Specific date range

## Key Metrics

### Users & Engagement

| Metric                        | Description                                                                   |
| ----------------------------- | ----------------------------------------------------------------------------- |
| **Active Users**              | Unique users who chatted with the agent                                       |
| **Conversations**             | Total chat sessions started                                                   |
| **Messages**                  | Total messages exchanged (user-facing only, excludes tool-calling iterations) |
| **LLM Calls**                 | Total LLM invocations (includes tool-calling and async tasks)                 |
| **Avg Messages/Conversation** | Depth of conversations (shown as subtitle)                                    |

### Quality

| Metric                  | Description                    |
| ----------------------- | ------------------------------ |
| **Average Rating**      | User feedback score (1-5)      |
| **Ratings Count**       | Number of ratings received     |
| **Rating Distribution** | Breakdown by star level        |
| **Resolution Rate**     | Percentage of resolved queries |

### Performance

| Metric                | Description             |
| --------------------- | ----------------------- |
| **P50 Response Time** | Median response latency |
| **P95 Response Time** | 95th percentile latency |
| **Error Count**       | Failed requests         |
| **Error Rate**        | Percentage of failures  |

### Cost

| Metric               | Description                |
| -------------------- | -------------------------- |
| **Input Tokens**     | Tokens from user messages  |
| **Output Tokens**    | Tokens in agent responses  |
| **Total Cost**       | Estimated API cost         |
| **Carbon (kgCO2eq)** | Estimated carbon footprint |

## Trend Charts

Visualize metrics over time:

* **Users trend** - Growth or decline in user base
* **Conversations trend** - Usage patterns by day
* **Messages trend** - Engagement over time

## Tool Usage

See which capabilities are used most:

| Tool             | Calls | Success Rate |
| ---------------- | ----- | ------------ |
| Knowledge Search | 1,234 | 98%          |
| Web Search       | 567   | 95%          |
| Calendar         | 89    | 100%         |

Use this to identify:

* Underutilized capabilities (consider removing)
* High-failure tools (investigate issues)
* Most valuable integrations

## Model Usage

If your agent uses multiple models, a breakdown is displayed showing:

* **Model name** - Each model used
* **Calls** - Number of LLM invocations per model
* **Tokens** - Total tokens consumed per model
* **Cost** - Estimated cost per model

## Interpreting Results

### Healthy Signs

* Steady or growing active users
* Ratings above 4.0
* Low error rates
* Good avg messages per conversation

### Warning Signs

* Declining active users
* Ratings below 3.5
* Increasing error rates
* Very low messages per conversation (users giving up)

### Action Items

Based on analytics, consider:

| Observation    | Possible Action                                |
| -------------- | ---------------------------------------------- |
| Low engagement | Improve welcome message, add suggested prompts |
| High errors    | Check tool configurations, review logs         |
| Poor ratings   | Review feedback, improve instructions          |
| Slow responses | Consider faster model or simplify tools        |
| Low tool usage | Update instructions to use tools more          |

## How It Works

### Pipeline Overview

```
 ES events                    agent_metrics (Mongo)              API response
 ─────────                    ────────────────────              ────────────

 agents.conversations.created ─┐                                ┌─ summary (cached)
 analytics.agent.rated ────────┼─→ Bulk Aggregator ─→ hourly ───┤
 analytics.llm.completion ─────┘      (cron)          daily     └─ series[]
                                                      summary
```

Analytics flow through three stages:

1. **Event emission** — As agents are used, Elasticsearch events are emitted by agent-factory and llm-gateway
2. **Aggregation** — A scheduled job reads these events and writes pre-aggregated rows into the `agent_metrics` collection
3. **Read** — The analytics endpoint reads from `agent_metrics`, self-heals if needed, and returns the result

### Event Sources

| Event                          | Emitted by    | Contains                                                   |
| ------------------------------ | ------------- | ---------------------------------------------------------- |
| `agents.conversations.created` | agent-factory | User ID, conversation ID, agent ID                         |
| `analytics.agent.rated`        | agent-factory | Rating value (1–5)                                         |
| `analytics.llm.completion`     | llm-gateway   | Model, token counts, cost, duration, tool names, call type |

### Caching

Summary metrics are cached for 1 hour in the `agent_metrics` collection. With warm cache, response time drops from \~1.5s to \~100ms.

### Self-Healing

The analytics endpoint performs two checks:

* **Current interval refresh**: The current in-progress hour or day is re-aggregated if stale (15-minute threshold for hourly, 1-hour for daily)
* **Gap detection**: If fewer intervals exist than expected (e.g., cron missed a run), a full re-aggregation is triggered

### Scheduled Jobs

| Job                    | Schedule                          | Description                                         |
| ---------------------- | --------------------------------- | --------------------------------------------------- |
| Aggregate metrics      | `30 1 * * *` (daily at 01:30 UTC) | Aggregates daily metrics for all active agents      |
| Cleanup old metrics    | `0 3 * * *` (daily)               | Deletes metric rows older than 30 days              |
| Refresh agent counters | `*/15 * * * *` (every 15 min)     | Updates per-agent conversation and message counters |

## Refreshing Data

Analytics are cached for performance. To see the latest data:

1. Click the **Refresh** button
2. Wait for metrics to update

<Note>
  Analytics may have a delay of up to 15 minutes for very recent activity.
</Note>

You can also force a re-aggregation via the API: `POST v1/agents/:agent_id/refresh-metrics` with an optional `hours_back` parameter (max 168 = 1 week).

## Exporting Data

Export analytics for reporting:

1. Select your time period
2. Click **Export CSV**
3. Download the file

## Privacy Considerations

Analytics are aggregated and anonymized:

* Individual conversations are not exposed
* User identities are not revealed in metrics
* Only owners and admins see analytics

For detailed conversation review, use Insights with appropriate permissions.

## Next Steps

<CardGroup cols="2">
  <Card title="Improve based on feedback" icon="pen" href="./instructions">
    Use insights to refine your agent's instructions
  </Card>

  <Card title="Insights" icon="chart-mixed" href="/products/ai-insights/overview">
    Deep dive into conversations and user feedback
  </Card>
</CardGroup>