AI foundry agents stopped searching the knowledge base (AI search)

Hi maapinho

When an AI Foundry agent suddenly stops using the knowledge source and starts hallucinating, it usually indicates a configuration or connectivity issue rather than a global outage. Common triggers include expired keys, RBAC changes, private endpoint restrictions, or overly strict retrieval thresholds. These changes can silently cause the agent to skip the Azure AI Search tool and fall back to generating answers without grounding

Immediate health checks

• Connection & Auth: Revalidate the connection in the Agent’s Knowledge tab. If you use Managed Identity, confirm it still has the Search Index Data Reader role.
• Network: If your Search resource uses a private endpoint, ensure DNS and firewall rules allow the agent to reach it.
• Index & Semantic Config: Check that the index exists and the semantic configuration hasn’t been renamed or removed.
• Tool Parameters: Review top_k and confidence thresholds. If they’re too strict, the agent may return zero hits and skip retrieval.

How to force the agent to run a search

You have two main options:

• Portal Settings: Enable “Always use knowledge” or “Block answers without citations” in the agent’s Knowledge settings. This forces the agent to query Azure AI Search before responding.
• SDK/REST: Use tool_choice: "required" or specify the Azure AI Search tool name in your run request. This ensures the model cannot skip the search step.

Example SDK snippet

run = agents.runs.create(
    thread_id=thread.id,
    agent_id=agent.id,
    tool_choice={"type": "tool", "tool_name": ai_search.name}  # or {"type": "required"}
)

This configuration guarantees that the agent invokes the Azure AI Search tool before generating an answer.

Observability

Use Foundry Observability or Application Insights to confirm whether the tool was invoked. Check traces for tool calls and whether they returned zero results or failed due to connectivity/auth issues.

Common root causes

• Key rotation or expired credentials.
• RBAC changes removing Search permissions.
• Private endpoint or DNS misconfiguration.
• Semantic config mismatch.
• Overly strict retrieval thresholds.

I Hope this helps. Do let me know if you have any further queries.

Thank you!

Thank you. We have deleted and reconnected the search resources. Re-added the knowledge and still doesn't search the knowledge like it used to.

We are using API Key which has not changed.

You mentioned "Enable “Always use knowledge”". Where do I find this option? I don't see any options like these when adding or modifying the agent's knowledge

I am having a similar issue.

The agent often says it "had trouble retrieving the information" when challenged about hallucinating answers. And sometimes mentions "security" as the reason why it cannot extract information from the knowledge sources. Not sure whether this is due to security changes in my companies' documents or the fault of the AI however some of the documents are my own and it still cannot extract information which makes me think it is a problem on the agent's side.

Hi maapinho

The “Always use knowledge” setting is part of the Azure AI Agent Service configuration, but it’s not exposed when you simply add or edit knowledge sources in the portal. It’s controlled at the agent level, specifically in the conversation orchestration settings.

Here’s how you can find and enable it:

Where to Find It

1. Go to your Agent in Azure AI Foundry
   • Navigate to Agents in your Azure AI Foundry workspace.
   • Select the agent you’ve created.
2. Open Agent Settings
   • Click on Configuration or Settings for the agent.
   • Look for the section called Conversation Behavior or Knowledge Settings.
3. Enable “Always use knowledge”
   • There will be a toggle or checkbox labeled Always use knowledge.
   • When enabled, the agent will always query the knowledge base for answers before falling back to the model.

Important Notes

• If you don’t see this option, it usually means:
  • You’re using an older version of the agent configuration.
  • Or you’re editing the knowledge source directly instead of the agent’s behavior settings.
  • In the REST API, this corresponds to the parameter:

  •     "knowledgeBehavior": {
               "alwaysUseKnowledge": true
        
    

    You can set this when updating the agent via API.

Hi.

"Click on Configuration or Settings for the agent."

There isn't such an option