Tutorial: Build an AI assistant with tool calling

Build an AI assistant that goes beyond conversation — it can call functions to perform actions. The assistant decides when a function is needed, you execute it, and feed the result back. Everything runs locally with the Foundry Local SDK.

In this tutorial, you learn how to:

Set up a project and install the Foundry Local SDK
Define tools the assistant can call
Send a message that triggers tool use
Execute the tool and return results to the model
Handle the complete tool calling loop
Clean up resources

Prerequisites

A Windows, macOS, or Linux computer with at least 8 GB of RAM.

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/cs/tutorial-tool-calling

Install packages

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

Windows
Cross-Platform

dotnet add package Microsoft.AI.Foundry.Local.WinML
dotnet add package OpenAI

dotnet add package Microsoft.AI.Foundry.Local
dotnet add package OpenAI

The C# samples in the GitHub repository are preconfigured projects. If you're building from scratch, you should read the Foundry Local SDK reference for more details on how to set up your C# project with Foundry Local.

Define tools

Tool calling lets the model request that your code runs a function and returns the result. You define the available tools as a list of JSON schemas that describe each function's name, purpose, and parameters.

Open Program.cs and add the following tool definitions:

// --- Tool definitions ---
List<ToolDefinition> tools =
[
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "get_weather",
            Description = "Get the current weather for a location",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "location", new PropertyDefinition() { Type = "string", Description = "The city or location" } },
                    { "unit", new PropertyDefinition() { Type = "string", Description = "Temperature unit (celsius or fahrenheit)" } }
                },
                Required = ["location"]
            }
        }
    },
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "calculate",
            Description = "Perform a math calculation",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "expression", new PropertyDefinition() { Type = "string", Description = "The math expression to evaluate" } }
                },
                Required = ["expression"]
            }
        }
    }
];

// --- Tool implementations ---
string ExecuteTool(string functionName, JsonElement arguments)
{
    switch (functionName)
    {
        case "get_weather":
            var location = arguments.GetProperty("location")
                .GetString() ?? "unknown";
            var unit = arguments.TryGetProperty("unit", out var u)
                ? u.GetString() ?? "celsius"
                : "celsius";
            var temp = unit == "celsius" ? 22 : 72;
            return JsonSerializer.Serialize(new
            {
                location,
                temperature = temp,
                unit,
                condition = "Sunny"
            });

        case "calculate":
            var expression = arguments.GetProperty("expression")
                .GetString() ?? "";
            try
            {
                var result = new System.Data.DataTable()
                    .Compute(expression, null);
                return JsonSerializer.Serialize(new
                {
                    expression,
                    result = result?.ToString()
                });
            }
            catch (Exception ex)
            {
                return JsonSerializer.Serialize(new
                {
                    error = ex.Message
                });
            }

        default:
            return JsonSerializer.Serialize(new
            {
                error = $"Unknown function: {functionName}"
            });
    }
}

Each tool definition includes a name, a description that helps the model decide when to use it, and a parameters schema that describes the expected input.

Add the C# methods that implement each tool:

// --- Tool definitions ---
List<ToolDefinition> tools =
[
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "get_weather",
            Description = "Get the current weather for a location",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "location", new PropertyDefinition() { Type = "string", Description = "The city or location" } },
                    { "unit", new PropertyDefinition() { Type = "string", Description = "Temperature unit (celsius or fahrenheit)" } }
                },
                Required = ["location"]
            }
        }
    },
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "calculate",
            Description = "Perform a math calculation",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "expression", new PropertyDefinition() { Type = "string", Description = "The math expression to evaluate" } }
                },
                Required = ["expression"]
            }
        }
    }
];

// --- Tool implementations ---
string ExecuteTool(string functionName, JsonElement arguments)
{
    switch (functionName)
    {
        case "get_weather":
            var location = arguments.GetProperty("location")
                .GetString() ?? "unknown";
            var unit = arguments.TryGetProperty("unit", out var u)
                ? u.GetString() ?? "celsius"
                : "celsius";
            var temp = unit == "celsius" ? 22 : 72;
            return JsonSerializer.Serialize(new
            {
                location,
                temperature = temp,
                unit,
                condition = "Sunny"
            });

        case "calculate":
            var expression = arguments.GetProperty("expression")
                .GetString() ?? "";
            try
            {
                var result = new System.Data.DataTable()
                    .Compute(expression, null);
                return JsonSerializer.Serialize(new
                {
                    expression,
                    result = result?.ToString()
                });
            }
            catch (Exception ex)
            {
                return JsonSerializer.Serialize(new
                {
                    error = ex.Message
                });
            }

        default:
            return JsonSerializer.Serialize(new
            {
                error = $"Unknown function: {functionName}"
            });
    }
}

The model doesn't run these functions directly. It returns a tool call request with the function name and arguments, and your code executes the function.

Send a message that triggers tool use

Initialize the Foundry Local SDK, load a model, and send a message that the model can answer by calling a tool.

// --- Main application ---
var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(
        Microsoft.Extensions.Logging.LogLevel.Information
    );
});
var logger = loggerFactory.CreateLogger<Program>();

await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;

var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Model not found");

await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await model.LoadAsync();
Console.WriteLine("Model loaded and ready.");

var chatClient = await model.GetChatClientAsync();
chatClient.Settings.ToolChoice = ToolChoice.Auto;

var messages = new List<ChatMessage>
{
    new ChatMessage
    {
        Role = "system",
        Content = "You are a helpful assistant with access to tools. " +
                  "Use them when needed to answer questions accurately."
    }
};

When the model determines that a tool is needed, the response contains ToolCalls instead of a regular text message. The next step shows how to detect and handle these calls.

Execute the tool and return results

After the model responds with a tool call, you extract the function name and arguments, run the function, and send the result back.

Console.WriteLine("\nTool-calling assistant ready! Type 'quit' to exit.\n");

while (true)
{
    Console.Write("You: ");
    var userInput = Console.ReadLine();
    if (string.IsNullOrWhiteSpace(userInput) ||
        userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
        userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
    {
        break;
    }

    messages.Add(new ChatMessage
    {
        Role = "user",
        Content = userInput
    });

    var response = await chatClient.CompleteChatAsync(
        messages, tools, ct
    );

    var choice = response.Choices[0].Message;

    if (choice.ToolCalls is { Count: > 0 })
    {
        messages.Add(choice);

        foreach (var toolCall in choice.ToolCalls)
        {
            var toolArgs = JsonDocument.Parse(
                toolCall.FunctionCall.Arguments
            ).RootElement;
            Console.WriteLine(
                $"  Tool call: {toolCall.FunctionCall.Name}({toolArgs})"
            );

            var result = ExecuteTool(
                toolCall.FunctionCall.Name, toolArgs
            );
            messages.Add(new ChatMessage
            {
                Role = "tool",
                ToolCallId = toolCall.Id,
                Content = result
            });
        }

        var finalResponse = await chatClient.CompleteChatAsync(
            messages, tools, ct
        );
        var answer = finalResponse.Choices[0].Message.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
    else
    {
        var answer = choice.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
}

await model.UnloadAsync();
Console.WriteLine("Model unloaded. Goodbye!");

The key steps in the tool calling loop are:

Detect tool calls — check response.Choices[0].Message.ToolCalls.
Execute the function — parse the arguments and call your local function.
Return the result — add a message with role tool and the matching ToolCallId.
Get the final answer — the model uses the tool result to generate a natural response.

Handle the complete tool calling loop

Here's the complete application that combines tool definitions, SDK initialization, and the tool calling loop into a single runnable file.

Replace the contents of Program.cs with the following complete code:

using System.Text.Json;
using Microsoft.AI.Foundry.Local;
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
using Betalgo.Ranul.OpenAI.ObjectModels.ResponseModels;
using Betalgo.Ranul.OpenAI.ObjectModels.SharedModels;
using Microsoft.Extensions.Logging;

CancellationToken ct = CancellationToken.None;

// --- Tool definitions ---
List<ToolDefinition> tools =
[
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "get_weather",
            Description = "Get the current weather for a location",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "location", new PropertyDefinition() { Type = "string", Description = "The city or location" } },
                    { "unit", new PropertyDefinition() { Type = "string", Description = "Temperature unit (celsius or fahrenheit)" } }
                },
                Required = ["location"]
            }
        }
    },
    new ToolDefinition
    {
        Type = "function",
        Function = new FunctionDefinition()
        {
            Name = "calculate",
            Description = "Perform a math calculation",
            Parameters = new PropertyDefinition()
            {
                Type = "object",
                Properties = new Dictionary<string, PropertyDefinition>()
                {
                    { "expression", new PropertyDefinition() { Type = "string", Description = "The math expression to evaluate" } }
                },
                Required = ["expression"]
            }
        }
    }
];

// --- Tool implementations ---
string ExecuteTool(string functionName, JsonElement arguments)
{
    switch (functionName)
    {
        case "get_weather":
            var location = arguments.GetProperty("location")
                .GetString() ?? "unknown";
            var unit = arguments.TryGetProperty("unit", out var u)
                ? u.GetString() ?? "celsius"
                : "celsius";
            var temp = unit == "celsius" ? 22 : 72;
            return JsonSerializer.Serialize(new
            {
                location,
                temperature = temp,
                unit,
                condition = "Sunny"
            });

        case "calculate":
            var expression = arguments.GetProperty("expression")
                .GetString() ?? "";
            try
            {
                var result = new System.Data.DataTable()
                    .Compute(expression, null);
                return JsonSerializer.Serialize(new
                {
                    expression,
                    result = result?.ToString()
                });
            }
            catch (Exception ex)
            {
                return JsonSerializer.Serialize(new
                {
                    error = ex.Message
                });
            }

        default:
            return JsonSerializer.Serialize(new
            {
                error = $"Unknown function: {functionName}"
            });
    }
}

// --- Main application ---
var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(
        Microsoft.Extensions.Logging.LogLevel.Information
    );
});
var logger = loggerFactory.CreateLogger<Program>();

await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;

var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Model not found");

await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await model.LoadAsync();
Console.WriteLine("Model loaded and ready.");

var chatClient = await model.GetChatClientAsync();
chatClient.Settings.ToolChoice = ToolChoice.Auto;

var messages = new List<ChatMessage>
{
    new ChatMessage
    {
        Role = "system",
        Content = "You are a helpful assistant with access to tools. " +
                  "Use them when needed to answer questions accurately."
    }
};

Console.WriteLine("\nTool-calling assistant ready! Type 'quit' to exit.\n");

while (true)
{
    Console.Write("You: ");
    var userInput = Console.ReadLine();
    if (string.IsNullOrWhiteSpace(userInput) ||
        userInput.Equals("quit", StringComparison.OrdinalIgnoreCase) ||
        userInput.Equals("exit", StringComparison.OrdinalIgnoreCase))
    {
        break;
    }

    messages.Add(new ChatMessage
    {
        Role = "user",
        Content = userInput
    });

    var response = await chatClient.CompleteChatAsync(
        messages, tools, ct
    );

    var choice = response.Choices[0].Message;

    if (choice.ToolCalls is { Count: > 0 })
    {
        messages.Add(choice);

        foreach (var toolCall in choice.ToolCalls)
        {
            var toolArgs = JsonDocument.Parse(
                toolCall.FunctionCall.Arguments
            ).RootElement;
            Console.WriteLine(
                $"  Tool call: {toolCall.FunctionCall.Name}({toolArgs})"
            );

            var result = ExecuteTool(
                toolCall.FunctionCall.Name, toolArgs
            );
            messages.Add(new ChatMessage
            {
                Role = "tool",
                ToolCallId = toolCall.Id,
                Content = result
            });
        }

        var finalResponse = await chatClient.CompleteChatAsync(
            messages, tools, ct
        );
        var answer = finalResponse.Choices[0].Message.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
    else
    {
        var answer = choice.Content ?? "";
        messages.Add(new ChatMessage
        {
            Role = "assistant",
            Content = answer
        });
        Console.WriteLine($"Assistant: {answer}\n");
    }
}

await model.UnloadAsync();
Console.WriteLine("Model unloaded. Goodbye!");

Run the tool-calling assistant:

dotnet run

You see output similar to:

Downloading model: 100.00%
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

The model decides when to call a tool based on the user's message. For a weather question it calls get_weather, for math it calls calculate, and for general questions it responds directly without any tool calls.

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/js/tutorial-tool-calling

npm install foundry-local-sdk-winml openai

npm install foundry-local-sdk openai

Define tools

Create a file called index.js.

Add the following tool definitions:

// --- Tool definitions ---
const tools = [
    {
        type: 'function',
        function: {
            name: 'get_weather',
            description: 'Get the current weather for a location',
            parameters: {
                type: 'object',
                properties: {
                    location: {
                        type: 'string',
                        description: 'The city or location'
                    },
                    unit: {
                        type: 'string',
                        enum: ['celsius', 'fahrenheit'],
                        description: 'Temperature unit'
                    }
                },
                required: ['location']
            }
        }
    },
    {
        type: 'function',
        function: {
            name: 'calculate',
            description: 'Perform a math calculation',
            parameters: {
                type: 'object',
                properties: {
                    expression: {
                        type: 'string',
                        description:
                            'The math expression to evaluate'
                    }
                },
                required: ['expression']
            }
        }
    }
];

// --- Tool implementations ---
function getWeather(location, unit = 'celsius') {
    return {
        location,
        temperature: unit === 'celsius' ? 22 : 72,
        unit,
        condition: 'Sunny'
    };
}

function calculate(expression) {
    // Input is validated against a strict allowlist of numeric/math characters,
    // making this safe from code injection in this tutorial context.
    const allowed = /^[0-9+\-*/(). ]+$/;
    if (!allowed.test(expression)) {
        return { error: 'Invalid expression' };
    }
    try {
        const result = Function(
            `"use strict"; return (${expression})`
        )();
        return { expression, result };
    } catch (err) {
        return { error: err.message };
    }
}

const toolFunctions = {
    get_weather: (args) => getWeather(args.location, args.unit),
    calculate: (args) => calculate(args.expression)
};

Each tool definition includes a name, a description that helps the model decide when to use it, and a parameters schema that describes the expected input.

Add the JavaScript functions that implement each tool:

// --- Tool definitions ---
const tools = [
    {
        type: 'function',
        function: {
            name: 'get_weather',
            description: 'Get the current weather for a location',
            parameters: {
                type: 'object',
                properties: {
                    location: {
                        type: 'string',
                        description: 'The city or location'
                    },
                    unit: {
                        type: 'string',
                        enum: ['celsius', 'fahrenheit'],
                        description: 'Temperature unit'
                    }
                },
                required: ['location']
            }
        }
    },
    {
        type: 'function',
        function: {
            name: 'calculate',
            description: 'Perform a math calculation',
            parameters: {
                type: 'object',
                properties: {
                    expression: {
                        type: 'string',
                        description:
                            'The math expression to evaluate'
                    }
                },
                required: ['expression']
            }
        }
    }
];

// --- Tool implementations ---
function getWeather(location, unit = 'celsius') {
    return {
        location,
        temperature: unit === 'celsius' ? 22 : 72,
        unit,
        condition: 'Sunny'
    };
}

function calculate(expression) {
    // Input is validated against a strict allowlist of numeric/math characters,
    // making this safe from code injection in this tutorial context.
    const allowed = /^[0-9+\-*/(). ]+$/;
    if (!allowed.test(expression)) {
        return { error: 'Invalid expression' };
    }
    try {
        const result = Function(
            `"use strict"; return (${expression})`
        )();
        return { expression, result };
    } catch (err) {
        return { error: err.message };
    }
}

const toolFunctions = {
    get_weather: (args) => getWeather(args.location, args.unit),
    calculate: (args) => calculate(args.expression)
};

The model doesn't run these functions directly. It returns a tool call request with the function name and arguments, and your code executes the function.

Send a message that triggers tool use

Initialize the Foundry Local SDK, load a model, and send a message that the model can answer by calling a tool.

// --- Main application ---
const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});

const model = await manager.catalog.getModel('qwen2.5-0.5b');

await model.download((progress) => {
    process.stdout.write(
        `\rDownloading model: ${progress.toFixed(2)}%`
    );
});
console.log('\nModel downloaded.');

await model.load();
console.log('Model loaded and ready.');

const chatClient = model.createChatClient();

const messages = [
    {
        role: 'system',
        content:
            'You are a helpful assistant with access to tools. ' +
            'Use them when needed to answer questions accurately.'
    }
];

const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
});

const askQuestion = (prompt) =>
    new Promise((resolve) => rl.question(prompt, resolve));

console.log(
    '\nTool-calling assistant ready! Type \'quit\' to exit.\n'
);

while (true) {
    const userInput = await askQuestion('You: ');
    if (
        userInput.trim().toLowerCase() === 'quit' ||
        userInput.trim().toLowerCase() === 'exit'
    ) {
        break;
    }

    messages.push({ role: 'user', content: userInput });

    const response = await chatClient.completeChat(
        messages, { tools }
    );
    const answer = await processToolCalls(
        messages, response, chatClient
    );

    messages.push({ role: 'assistant', content: answer });
    console.log(`Assistant: ${answer}\n`);
}

await model.unload();
console.log('Model unloaded. Goodbye!');
rl.close();

When the model determines that a tool is needed, the response contains tool_calls instead of a regular text message. The next step shows how to detect and handle these calls.

Execute the tool and return results

After the model responds with a tool call, you extract the function name and arguments, run the function, and send the result back.

async function processToolCalls(messages, response, chatClient) {
    let choice = response.choices[0]?.message;

    while (choice?.tool_calls?.length > 0) {
        messages.push(choice);

        for (const toolCall of choice.tool_calls) {
            const functionName = toolCall.function.name;
            const args = JSON.parse(toolCall.function.arguments);
            console.log(
                `  Tool call: ${functionName}` +
                `(${JSON.stringify(args)})`
            );

            const result = toolFunctions[functionName](args);
            messages.push({
                role: 'tool',
                tool_call_id: toolCall.id,
                content: JSON.stringify(result)
            });
        }

        response = await chatClient.completeChat(
            messages, { tools }
        );
        choice = response.choices[0]?.message;
    }

    return choice?.content ?? '';
}

The key steps in the tool calling loop are:

Detect tool calls — check response.choices[0]?.message?.tool_calls.
Execute the function — parse the arguments and call your local function.
Return the result — add a message with role tool and the matching tool_call_id.
Get the final answer — the model uses the tool result to generate a natural response.

Handle the complete tool calling loop

Here's the complete application that combines tool definitions, SDK initialization, and the tool calling loop into a single runnable file.

Create a file named index.js and add the following complete code:

import { FoundryLocalManager } from 'foundry-local-sdk';
import * as readline from 'readline';

// --- Tool definitions ---
const tools = [
    {
        type: 'function',
        function: {
            name: 'get_weather',
            description: 'Get the current weather for a location',
            parameters: {
                type: 'object',
                properties: {
                    location: {
                        type: 'string',
                        description: 'The city or location'
                    },
                    unit: {
                        type: 'string',
                        enum: ['celsius', 'fahrenheit'],
                        description: 'Temperature unit'
                    }
                },
                required: ['location']
            }
        }
    },
    {
        type: 'function',
        function: {
            name: 'calculate',
            description: 'Perform a math calculation',
            parameters: {
                type: 'object',
                properties: {
                    expression: {
                        type: 'string',
                        description:
                            'The math expression to evaluate'
                    }
                },
                required: ['expression']
            }
        }
    }
];

// --- Tool implementations ---
function getWeather(location, unit = 'celsius') {
    return {
        location,
        temperature: unit === 'celsius' ? 22 : 72,
        unit,
        condition: 'Sunny'
    };
}

function calculate(expression) {
    // Input is validated against a strict allowlist of numeric/math characters,
    // making this safe from code injection in this tutorial context.
    const allowed = /^[0-9+\-*/(). ]+$/;
    if (!allowed.test(expression)) {
        return { error: 'Invalid expression' };
    }
    try {
        const result = Function(
            `"use strict"; return (${expression})`
        )();
        return { expression, result };
    } catch (err) {
        return { error: err.message };
    }
}

const toolFunctions = {
    get_weather: (args) => getWeather(args.location, args.unit),
    calculate: (args) => calculate(args.expression)
};

async function processToolCalls(messages, response, chatClient) {
    let choice = response.choices[0]?.message;

    while (choice?.tool_calls?.length > 0) {
        messages.push(choice);

        for (const toolCall of choice.tool_calls) {
            const functionName = toolCall.function.name;
            const args = JSON.parse(toolCall.function.arguments);
            console.log(
                `  Tool call: ${functionName}` +
                `(${JSON.stringify(args)})`
            );

            const result = toolFunctions[functionName](args);
            messages.push({
                role: 'tool',
                tool_call_id: toolCall.id,
                content: JSON.stringify(result)
            });
        }

        response = await chatClient.completeChat(
            messages, { tools }
        );
        choice = response.choices[0]?.message;
    }

    return choice?.content ?? '';
}

// --- Main application ---
const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});

const model = await manager.catalog.getModel('qwen2.5-0.5b');

await model.download((progress) => {
    process.stdout.write(
        `\rDownloading model: ${progress.toFixed(2)}%`
    );
});
console.log('\nModel downloaded.');

await model.load();
console.log('Model loaded and ready.');

const chatClient = model.createChatClient();

const messages = [
    {
        role: 'system',
        content:
            'You are a helpful assistant with access to tools. ' +
            'Use them when needed to answer questions accurately.'
    }
];

const rl = readline.createInterface({
    input: process.stdin,
    output: process.stdout
});

const askQuestion = (prompt) =>
    new Promise((resolve) => rl.question(prompt, resolve));

console.log(
    '\nTool-calling assistant ready! Type \'quit\' to exit.\n'
);

while (true) {
    const userInput = await askQuestion('You: ');
    if (
        userInput.trim().toLowerCase() === 'quit' ||
        userInput.trim().toLowerCase() === 'exit'
    ) {
        break;
    }

    messages.push({ role: 'user', content: userInput });

    const response = await chatClient.completeChat(
        messages, { tools }
    );
    const answer = await processToolCalls(
        messages, response, chatClient
    );

    messages.push({ role: 'assistant', content: answer });
    console.log(`Assistant: ${answer}\n`);
}

await model.unload();
console.log('Model unloaded. Goodbye!');
rl.close();

Run the tool-calling assistant:

node index.js

You see output similar to:

Downloading model: 100.00%
Model downloaded.
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/python/tutorial-tool-calling

Install packages

Windows
Cross-Platform

pip install foundry-local-sdk-winml openai

pip install foundry-local-sdk openai

Define tools

Create a file called main.py and add the following tool definitions:

# --- Tool definitions ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": (
                            "The math expression to evaluate"
                        )
                    }
                },
                "required": ["expression"]
            }
        }
    }
]


# --- Tool implementations ---
def get_weather(location, unit="celsius"):
    """Simulate a weather lookup."""
    return {
        "location": location,
        "temperature": 22 if unit == "celsius" else 72,
        "unit": unit,
        "condition": "Sunny"
    }


def calculate(expression):
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/(). ")
    if not all(c in allowed for c in expression):
        return {"error": "Invalid expression"}
    try:
        result = eval(expression)
        return {"expression": expression, "result": result}
    except Exception as e:
        return {"error": str(e)}


tool_functions = {
    "get_weather": get_weather,
    "calculate": calculate
}

Each tool definition includes a name, a description that helps the model decide when to use it, and a parameters schema that describes the expected input.

Next, add the Python functions that implement each tool:

# --- Tool definitions ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": (
                            "The math expression to evaluate"
                        )
                    }
                },
                "required": ["expression"]
            }
        }
    }
]


# --- Tool implementations ---
def get_weather(location, unit="celsius"):
    """Simulate a weather lookup."""
    return {
        "location": location,
        "temperature": 22 if unit == "celsius" else 72,
        "unit": unit,
        "condition": "Sunny"
    }


def calculate(expression):
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/(). ")
    if not all(c in allowed for c in expression):
        return {"error": "Invalid expression"}
    try:
        result = eval(expression)
        return {"expression": expression, "result": result}
    except Exception as e:
        return {"error": str(e)}


tool_functions = {
    "get_weather": get_weather,
    "calculate": calculate
}

The model doesn't run these functions directly. It returns a tool call request with the function name and arguments, and your code executes the function.

Send a message that triggers tool use

Initialize the Foundry Local SDK, load a model, and send a message that the model can answer by calling a tool.

async def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Select and load a model
    model = manager.catalog.get_model("qwen2.5-0.5b")
    model.download(
        lambda progress: print(
            f"\rDownloading model: {progress:.2f}%",
            end="",
            flush=True
        )
    )
    print()
    model.load()
    print("Model loaded and ready.")

    # Get a chat client
    client = model.get_chat_client()

    # Conversation with a system prompt
    messages = [
        {
            "role": "system",
            "content": "You are a helpful assistant with access to tools. "
                       "Use them when needed to answer questions accurately."
        }
    ]

    print("\nTool-calling assistant ready! Type 'quit' to exit.\n")

    while True:
        user_input = input("You: ")
        if user_input.strip().lower() in ("quit", "exit"):
            break

        messages.append({"role": "user", "content": user_input})

        response = client.complete_chat(messages, tools=tools)
        answer = process_tool_calls(messages, response, client)

        messages.append({"role": "assistant", "content": answer})
        print(f"Assistant: {answer}\n")

    # Clean up
    model.unload()
    print("Model unloaded. Goodbye!")

When the model determines that a tool is needed, the response contains tool_calls instead of a regular text message. The next step shows how to detect and handle these calls.

Execute the tool and return results

After the model responds with a tool call, you extract the function name and arguments, run the function, and send the result back.

def process_tool_calls(messages, response, client):
    """Handle tool calls in a loop until the model produces a final answer."""
    choice = response.choices[0].message

    while choice.tool_calls:
        # Convert the assistant message to a dict for the SDK
        assistant_msg = {
            "role": "assistant",
            "content": choice.content,
            "tool_calls": [
                {
                    "id": tc.id,
                    "type": tc.type,
                    "function": {
                        "name": tc.function.name,
                        "arguments": tc.function.arguments,
                    },
                }
                for tc in choice.tool_calls
            ],
        }
        messages.append(assistant_msg)

        for tool_call in choice.tool_calls:
            function_name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            print(f"  Tool call: {function_name}({arguments})")

            # Execute the function and add the result
            func = tool_functions[function_name]
            result = func(**arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            })

        # Send the updated conversation back
        response = client.complete_chat(messages, tools=tools)
        choice = response.choices[0].message

    return choice.content

The key steps in the tool calling loop are:

Detect tool calls — check response.choices[0].message.tool_calls.
Execute the function — parse the arguments and call your local function.
Return the result — add a message with role tool and the matching tool_call_id.
Get the final answer — the model uses the tool result to generate a natural response.

Handle the complete tool calling loop

Here's the complete application that combines tool definitions, SDK initialization, and the tool calling loop into a single runnable file.

Create a file named main.py and add the following complete code:

import asyncio
import json
from foundry_local_sdk import Configuration, FoundryLocalManager


# --- Tool definitions ---
tools = [
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description": "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description": "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description": (
                            "The math expression to evaluate"
                        )
                    }
                },
                "required": ["expression"]
            }
        }
    }
]


# --- Tool implementations ---
def get_weather(location, unit="celsius"):
    """Simulate a weather lookup."""
    return {
        "location": location,
        "temperature": 22 if unit == "celsius" else 72,
        "unit": unit,
        "condition": "Sunny"
    }


def calculate(expression):
    """Evaluate a math expression safely."""
    allowed = set("0123456789+-*/(). ")
    if not all(c in allowed for c in expression):
        return {"error": "Invalid expression"}
    try:
        result = eval(expression)
        return {"expression": expression, "result": result}
    except Exception as e:
        return {"error": str(e)}


tool_functions = {
    "get_weather": get_weather,
    "calculate": calculate
}


def process_tool_calls(messages, response, client):
    """Handle tool calls in a loop until the model produces a final answer."""
    choice = response.choices[0].message

    while choice.tool_calls:
        # Convert the assistant message to a dict for the SDK
        assistant_msg = {
            "role": "assistant",
            "content": choice.content,
            "tool_calls": [
                {
                    "id": tc.id,
                    "type": tc.type,
                    "function": {
                        "name": tc.function.name,
                        "arguments": tc.function.arguments,
                    },
                }
                for tc in choice.tool_calls
            ],
        }
        messages.append(assistant_msg)

        for tool_call in choice.tool_calls:
            function_name = tool_call.function.name
            arguments = json.loads(tool_call.function.arguments)
            print(f"  Tool call: {function_name}({arguments})")

            # Execute the function and add the result
            func = tool_functions[function_name]
            result = func(**arguments)
            messages.append({
                "role": "tool",
                "tool_call_id": tool_call.id,
                "content": json.dumps(result)
            })

        # Send the updated conversation back
        response = client.complete_chat(messages, tools=tools)
        choice = response.choices[0].message

    return choice.content


async def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Select and load a model
    model = manager.catalog.get_model("qwen2.5-0.5b")
    model.download(
        lambda progress: print(
            f"\rDownloading model: {progress:.2f}%",
            end="",
            flush=True
        )
    )
    print()
    model.load()
    print("Model loaded and ready.")

    # Get a chat client
    client = model.get_chat_client()

    # Conversation with a system prompt
    messages = [
        {
            "role": "system",
            "content": "You are a helpful assistant with access to tools. "
                       "Use them when needed to answer questions accurately."
        }
    ]

    print("\nTool-calling assistant ready! Type 'quit' to exit.\n")

    while True:
        user_input = input("You: ")
        if user_input.strip().lower() in ("quit", "exit"):
            break

        messages.append({"role": "user", "content": user_input})

        response = client.complete_chat(messages, tools=tools)
        answer = process_tool_calls(messages, response, client)

        messages.append({"role": "assistant", "content": answer})
        print(f"Assistant: {answer}\n")

    # Clean up
    model.unload()
    print("Model unloaded. Goodbye!")


if __name__ == "__main__":
    asyncio.run(main())

Run the tool-calling assistant:

python main.py

You see output similar to:

Downloading model: 100.00%
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({'location': 'current location'})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({'expression': '245 * 38'})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/rust/tutorial-tool-calling

Install packages

Windows
Cross-Platform

cargo add foundry-local-sdk --features winml
cargo add tokio --features full
cargo add tokio-stream anyhow

cargo add foundry-local-sdk
cargo add tokio --features full
cargo add tokio-stream anyhow

Define tools

Add the serde_json dependency for JSON handling:
```
cargo add serde_json
```

Open src/main.rs and add the following tool definitions:

// --- Tool definitions ---
let tools: Vec<ChatCompletionTools> = serde_json::from_value(json!([
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description":
                "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description":
                            "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description":
                            "The math expression to evaluate"
                    }
                },
                "required": ["expression"]
            }
        }
    }
]))?;

Each tool definition includes a name, a description that helps the model decide when to use it, and a parameters schema that describes the expected input.

Add the Rust functions that implement each tool:

// --- Tool definitions ---
let tools: Vec<ChatCompletionTools> = serde_json::from_value(json!([
    {
        "type": "function",
        "function": {
            "name": "get_weather",
            "description":
                "Get the current weather for a location",
            "parameters": {
                "type": "object",
                "properties": {
                    "location": {
                        "type": "string",
                        "description":
                            "The city or location"
                    },
                    "unit": {
                        "type": "string",
                        "enum": ["celsius", "fahrenheit"],
                        "description": "Temperature unit"
                    }
                },
                "required": ["location"]
            }
        }
    },
    {
        "type": "function",
        "function": {
            "name": "calculate",
            "description": "Perform a math calculation",
            "parameters": {
                "type": "object",
                "properties": {
                    "expression": {
                        "type": "string",
                        "description":
                            "The math expression to evaluate"
                    }
                },
                "required": ["expression"]
            }
        }
    }
]))?;

The model doesn't run these functions directly. It returns a tool call request with the function name and arguments, and your code executes the function.

Send a message that triggers tool use

Initialize the Foundry Local SDK, load a model, and send a message that the model can answer by calling a tool.

// Initialize the Foundry Local SDK
let manager = FoundryLocalManager::create(
    FoundryLocalConfig::new("tool-calling-app"),
)?;

// Select and load a model
let model = manager
    .catalog()
    .get_model("qwen2.5-0.5b")
    .await?;

if !model.is_cached().await? {
    println!("Downloading model...");
    model
        .download(Some(|progress: f64| {
            print!("\r  {progress:.1}%");
            io::stdout().flush().ok();
        }))
        .await?;
    println!();
}

model.load().await?;
println!("Model loaded and ready.");

// Create a chat client
let client = model
    .create_chat_client()
    .temperature(0.7)
    .max_tokens(512)
    .tool_choice(ChatToolChoice::Auto);

// Conversation with a system prompt
let mut messages: Vec<ChatCompletionRequestMessage> = vec![
    ChatCompletionRequestSystemMessage::from(
        "You are a helpful assistant with access to tools. \
         Use them when needed to answer questions accurately.",
    )
    .into(),
];

When the model determines that a tool is needed, the response contains tool_calls instead of a regular text message. The next step shows how to detect and handle these calls.

Execute the tool and return results

After the model responds with a tool call, you extract the function name and arguments, run the function, and send the result back.

println!(
    "\nTool-calling assistant ready! Type 'quit' to exit.\n"
);

let stdin = io::stdin();
loop {
    print!("You: ");
    io::stdout().flush()?;

    let mut input = String::new();
    stdin.lock().read_line(&mut input)?;
    let input = input.trim();

    if input.eq_ignore_ascii_case("quit")
        || input.eq_ignore_ascii_case("exit")
    {
        break;
    }

    messages.push(
        ChatCompletionRequestUserMessage::from(input).into(),
    );

    let mut response = client
        .complete_chat(&messages, Some(&tools))
        .await?;

    // Process tool calls in a loop
    while response.choices[0].message.tool_calls.is_some() {
        let tool_calls = response.choices[0]
            .message
            .tool_calls
            .as_ref()
            .unwrap();

        // Append the assistant's tool_calls message via JSON
        let assistant_msg: ChatCompletionRequestMessage =
            serde_json::from_value(json!({
                "role": "assistant",
                "content": null,
                "tool_calls": tool_calls,
            }))?;
        messages.push(assistant_msg);

        for tc_enum in tool_calls {
            let tool_call = match tc_enum {
                ChatCompletionMessageToolCalls::Function(
                    tc,
                ) => tc,
                _ => continue,
            };
            let function_name =
                &tool_call.function.name;
            let arguments: Value =
                serde_json::from_str(
                    &tool_call.function.arguments,
                )?;
            println!(
                "  Tool call: {}({})",
                function_name, arguments
            );

            let result =
                execute_tool(function_name, &arguments);
            messages.push(
                ChatCompletionRequestToolMessage {
                    content: result.to_string().into(),
                    tool_call_id: tool_call.id.clone(),
                }
                .into(),
            );
        }

        response = client
            .complete_chat(&messages, Some(&tools))
            .await?;
    }

    let answer = response.choices[0]
        .message
        .content
        .as_deref()
        .unwrap_or("");
    let assistant_msg: ChatCompletionRequestMessage =
        serde_json::from_value(json!({
            "role": "assistant",
            "content": answer,
        }))?;
    messages.push(assistant_msg);
    println!("Assistant: {}\n", answer);
}

// Clean up
model.unload().await?;
println!("Model unloaded. Goodbye!");

The key steps in the tool calling loop are:

Detect tool calls — check response.choices[0].message.tool_calls.
Execute the function — parse the arguments and call your local function.
Return the result — add a message with role tool and the matching tool call ID.
Get the final answer — the model uses the tool result to generate a natural response.

Handle the complete tool calling loop

Here's the complete application that combines tool definitions, SDK initialization, and the tool calling loop into a single runnable file.

Replace the contents of src/main.rs with the following complete code:

use foundry_local_sdk::{
    ChatCompletionRequestMessage,
    ChatCompletionRequestSystemMessage,
    ChatCompletionRequestToolMessage,
    ChatCompletionRequestUserMessage,
    ChatCompletionMessageToolCalls,
    ChatCompletionTools, ChatToolChoice,
    FoundryLocalConfig, FoundryLocalManager,
};
use serde_json::{json, Value};
use std::io::{self, BufRead, Write};

// --- Tool implementations ---
fn execute_tool(
    name: &str,
    arguments: &Value,
) -> Value {
    match name {
        "get_weather" => {
            let location = arguments["location"]
                .as_str()
                .unwrap_or("unknown");
            let unit = arguments["unit"]
                .as_str()
                .unwrap_or("celsius");
            let temp = if unit == "celsius" { 22 } else { 72 };
            json!({
                "location": location,
                "temperature": temp,
                "unit": unit,
                "condition": "Sunny"
            })
        }
        "calculate" => {
            let expression = arguments["expression"]
                .as_str()
                .unwrap_or("");
            let is_valid = expression
                .chars()
                .all(|c| "0123456789+-*/(). ".contains(c));
            if !is_valid {
                return json!({"error": "Invalid expression"});
            }
            match eval_expression(expression) {
                Ok(result) => json!({
                    "expression": expression,
                    "result": result
                }),
                Err(e) => json!({"error": e}),
            }
        }
        _ => json!({"error": format!("Unknown function: {}", name)}),
    }
}

fn eval_expression(expr: &str) -> Result<f64, String> {
    let expr = expr.replace(' ', "");
    let chars: Vec<char> = expr.chars().collect();
    let mut pos = 0;
    let result = parse_add(&chars, &mut pos)?;
    if pos < chars.len() {
        return Err("Unexpected character".to_string());
    }
    Ok(result)
}

fn parse_add(
    chars: &[char],
    pos: &mut usize,
) -> Result<f64, String> {
    let mut result = parse_mul(chars, pos)?;
    while *pos < chars.len()
        && (chars[*pos] == '+' || chars[*pos] == '-')
    {
        let op = chars[*pos];
        *pos += 1;
        let right = parse_mul(chars, pos)?;
        result = if op == '+' {
            result + right
        } else {
            result - right
        };
    }
    Ok(result)
}

fn parse_mul(
    chars: &[char],
    pos: &mut usize,
) -> Result<f64, String> {
    let mut result = parse_atom(chars, pos)?;
    while *pos < chars.len()
        && (chars[*pos] == '*' || chars[*pos] == '/')
    {
        let op = chars[*pos];
        *pos += 1;
        let right = parse_atom(chars, pos)?;
        result = if op == '*' {
            result * right
        } else {
            result / right
        };
    }
    Ok(result)
}

fn parse_atom(
    chars: &[char],
    pos: &mut usize,
) -> Result<f64, String> {
    if *pos < chars.len() && chars[*pos] == '(' {
        *pos += 1;
        let result = parse_add(chars, pos)?;
        if *pos < chars.len() && chars[*pos] == ')' {
            *pos += 1;
        }
        return Ok(result);
    }
    let start = *pos;
    while *pos < chars.len()
        && (chars[*pos].is_ascii_digit() || chars[*pos] == '.')
    {
        *pos += 1;
    }
    if start == *pos {
        return Err("Expected number".to_string());
    }
    let num_str: String = chars[start..*pos].iter().collect();
    num_str.parse::<f64>().map_err(|e| e.to_string())
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // --- Tool definitions ---
    let tools: Vec<ChatCompletionTools> = serde_json::from_value(json!([
        {
            "type": "function",
            "function": {
                "name": "get_weather",
                "description":
                    "Get the current weather for a location",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": {
                            "type": "string",
                            "description":
                                "The city or location"
                        },
                        "unit": {
                            "type": "string",
                            "enum": ["celsius", "fahrenheit"],
                            "description": "Temperature unit"
                        }
                    },
                    "required": ["location"]
                }
            }
        },
        {
            "type": "function",
            "function": {
                "name": "calculate",
                "description": "Perform a math calculation",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "expression": {
                            "type": "string",
                            "description":
                                "The math expression to evaluate"
                        }
                    },
                    "required": ["expression"]
                }
            }
        }
    ]))?;

    // Initialize the Foundry Local SDK
    let manager = FoundryLocalManager::create(
        FoundryLocalConfig::new("tool-calling-app"),
    )?;

    // Select and load a model
    let model = manager
        .catalog()
        .get_model("qwen2.5-0.5b")
        .await?;

    if !model.is_cached().await? {
        println!("Downloading model...");
        model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                io::stdout().flush().ok();
            }))
            .await?;
        println!();
    }

    model.load().await?;
    println!("Model loaded and ready.");

    // Create a chat client
    let client = model
        .create_chat_client()
        .temperature(0.7)
        .max_tokens(512)
        .tool_choice(ChatToolChoice::Auto);

    // Conversation with a system prompt
    let mut messages: Vec<ChatCompletionRequestMessage> = vec![
        ChatCompletionRequestSystemMessage::from(
            "You are a helpful assistant with access to tools. \
             Use them when needed to answer questions accurately.",
        )
        .into(),
    ];

    println!(
        "\nTool-calling assistant ready! Type 'quit' to exit.\n"
    );

    let stdin = io::stdin();
    loop {
        print!("You: ");
        io::stdout().flush()?;

        let mut input = String::new();
        stdin.lock().read_line(&mut input)?;
        let input = input.trim();

        if input.eq_ignore_ascii_case("quit")
            || input.eq_ignore_ascii_case("exit")
        {
            break;
        }

        messages.push(
            ChatCompletionRequestUserMessage::from(input).into(),
        );

        let mut response = client
            .complete_chat(&messages, Some(&tools))
            .await?;

        // Process tool calls in a loop
        while response.choices[0].message.tool_calls.is_some() {
            let tool_calls = response.choices[0]
                .message
                .tool_calls
                .as_ref()
                .unwrap();

            // Append the assistant's tool_calls message via JSON
            let assistant_msg: ChatCompletionRequestMessage =
                serde_json::from_value(json!({
                    "role": "assistant",
                    "content": null,
                    "tool_calls": tool_calls,
                }))?;
            messages.push(assistant_msg);

            for tc_enum in tool_calls {
                let tool_call = match tc_enum {
                    ChatCompletionMessageToolCalls::Function(
                        tc,
                    ) => tc,
                    _ => continue,
                };
                let function_name =
                    &tool_call.function.name;
                let arguments: Value =
                    serde_json::from_str(
                        &tool_call.function.arguments,
                    )?;
                println!(
                    "  Tool call: {}({})",
                    function_name, arguments
                );

                let result =
                    execute_tool(function_name, &arguments);
                messages.push(
                    ChatCompletionRequestToolMessage {
                        content: result.to_string().into(),
                        tool_call_id: tool_call.id.clone(),
                    }
                    .into(),
                );
            }

            response = client
                .complete_chat(&messages, Some(&tools))
                .await?;
        }

        let answer = response.choices[0]
            .message
            .content
            .as_deref()
            .unwrap_or("");
        let assistant_msg: ChatCompletionRequestMessage =
            serde_json::from_value(json!({
                "role": "assistant",
                "content": answer,
            }))?;
        messages.push(assistant_msg);
        println!("Assistant: {}\n", answer);
    }

    // Clean up
    model.unload().await?;
    println!("Model unloaded. Goodbye!");

    Ok(())
}

Run the tool-calling assistant:

cargo run

You see output similar to:

Downloading model: 100.00%
Model loaded and ready.

Tool-calling assistant ready! Type 'quit' to exit.

You: What's the weather like today?
  Tool call: get_weather({"location":"current location"})
Assistant: The weather today is sunny with a temperature of 22°C.

You: What is 245 * 38?
  Tool call: calculate({"expression":"245 * 38"})
Assistant: 245 multiplied by 38 equals 9,310.

You: quit
Model unloaded. Goodbye!

Clean up resources

The model weights remain in your local cache after you unload a model. This means the next time you run the application, the download step is skipped and the model loads faster. No extra cleanup is needed unless you want to reclaim disk space.

Feedback

Was this page helpful?

Last updated on 2026-04-09

Share via

Tutorial: Build an AI assistant with tool calling

Prerequisites

Samples repository

Install packages

Define tools

Send a message that triggers tool use

Execute the tool and return results

Handle the complete tool calling loop

Samples repository

Install packages

Define tools

Send a message that triggers tool use

Execute the tool and return results

Handle the complete tool calling loop

Samples repository

Install packages

Define tools

Send a message that triggers tool use

Execute the tool and return results

Handle the complete tool calling loop

Samples repository

Install packages

Define tools

Send a message that triggers tool use

Execute the tool and return results

Handle the complete tool calling loop

Clean up resources

Related content

Feedback

Additional resources