Azure OpenAI model monitoring does not provide evaluation metrics - Job Status "Queued"

ferrand 55 Reputation points
2024-03-06T22:56:24.43+00:00

I am using Azure Machine Learning Studio as it provides more possibilities in terms of customization of the monitoring. Model is in production but so far I am sending questions only in the test tab of the endpoint in Azure Studio. I have sent 50 questions in this test tab as suggested by Microsoft.

I followed step by step this official Microsoft guide: https://dori-uw-1.kuma-moon.com/en-us/azure/machine-learning/prompt-flow/how-to-monitor-generative-ai-applications?view=azureml-api-2

I also watched this official tutorial: https://www.youtube.com/watch?v=hVsjcsoGAww

However, after deploying my flow to an endpoint and creating the monitoring I only get this:
Screenshot 2024-03-06 at 23.44.29

It gets stuck here:User's image

The model data collector with inputs and outputs seems to work. Sampled files are being collected.

If I click on the jobs related to the output I see that the Status is on "Queued" with the message "Information: This job is waiting for available quota in scheduling queue."
Screenshot 2024-03-06 at 23.49.34

How can I proceed? Many thanks in advance for your kind support.

Azure Machine Learning
Azure OpenAI Service
Azure OpenAI Service
An Azure service that provides access to OpenAI’s GPT-3 models with enterprise capabilities.
Azure AI services
Azure AI services
A group of Azure services, SDKs, and APIs designed to make apps more intelligent, engaging, and discoverable.
{count} votes

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.