Track agent performance with Conversation KPIs in Copilot Studio Kit

Conversation KPIs (key performance indicators) help you track and analyze custom agent performance. This feature complements existing analytics in Copilot Studio by providing aggregated data in Dataverse. With this feature, you can understand conversation outcomes without analyzing complex transcripts.

Key features

Key features of Conversation KPIs include:

• Aggregated data: Conversation KPIs surface simplified conversation outcome results, making it easier to understand the performance of your custom agents. Data includes metrics such as the number of sessions, turns, and the global outcome of conversations (for example, resolved, partially resolved, escalated, or abandoned).

• Sample Power BI reports: The Copilot Studio Kit offers sample Power BI reports based on aggregated KPIs. These reports provide a visual representation of the data, so you can quickly identify trends and areas for improvement. They provide a solid starting point for your own custom reports.

• Tracked variables: You can define specific variables to track as part of the aggregated conversation KPIs, such as custom Net Promoter Score (NPS) or other relevant metrics. Tailor the KPIs to fit your specific needs.

• Optional full transcript storage: You can include the full conversation transcript with the KPI records as a file. This storage option provides a more detailed analysis using the transcript visualizer built into the Copilot Studio Kit.

• Long-term tracking: You can store conversation KPIs indefinitely, so you can track improvements and custom copilot performance over time

Conversation KPIs are generated automatically twice per day and can also be generated on demand.

Generate KPIs on demand

To generate KPIs on demand:

1. Go to the agent configuration.
2. Select Generate KPIs.
3. Select a date range.

Each automatic run processes up to 75,000 transcripts and each on-demand run processes up to 50,000 transcripts.

Next step