Edit

Share via

Tutorial: Build a document summarizer

Build an application that reads text files and generates concise summaries — entirely on your device. This is useful when you need to quickly understand the content of documents without reading them in full, and when the documents contain sensitive information that shouldn't leave your machine.

In this tutorial, you learn how to:

  • Set up a project and install the Foundry Local SDK
  • Read a text document from the file system
  • Load a model and generate a summary
  • Control summary output with system prompts
  • Process multiple documents in a batch
  • Clean up resources

Prerequisites

  • A Windows, macOS, or Linux computer with at least 8 GB of RAM.

Install packages

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/cs/tutorial-document-summarizer

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

dotnet add package Microsoft.AI.Foundry.Local.WinML
dotnet add package OpenAI

The C# samples in the GitHub repository are preconfigured projects. If you're building from scratch, you should read the Foundry Local SDK reference for more details on how to set up your C# project with Foundry Local.

Read a text document

Before you summarize anything, you need a sample document to work with. Create a file called document.txt in your project directory and add the following content:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

Now open Program.cs and add the following code to read the document:

var target = args.Length > 0 ? args[0] : "document.txt";

The code accepts an optional file path as a command-line argument and falls back to document.txt if none is provided.

Generate a summary

Initialize the Foundry Local SDK, load a model, and send the document content along with a system prompt that instructs the model to summarize.

Replace the contents of Program.cs with the following code:

var systemPrompt =
    "Summarize the following document into concise bullet points. " +
    "Focus on the key points and main ideas.";

var target = args.Length > 0 ? args[0] : "document.txt";

if (Directory.Exists(target))
{
    await SummarizeDirectoryAsync(chatClient, target, systemPrompt, ct);
}
else
{
    Console.WriteLine($"--- {Path.GetFileName(target)} ---");
    await SummarizeFileAsync(chatClient, target, systemPrompt, ct);
}

The GetModelAsync method accepts a model alias, which is a short friendly name that maps to a specific model in the catalog. The DownloadAsync method fetches the model weights to your local cache (and skips the download if they're already cached), and LoadAsync makes the model ready for inference. The system prompt tells the model to produce bullet-point summaries focused on key ideas.

Control summary output

Different situations call for different summary styles. You can change the system prompt to control how the model structures its output. Here are three useful variations:

Bullet points (default from the previous step):

var systemPrompt =
    "Summarize the following document into concise bullet points. " +
    "Focus on the key points and main ideas.";

One-paragraph summary:

var systemPrompt =
    "Summarize the following document in a single, concise paragraph. " +
    "Capture the main argument and supporting points.";

Key takeaways:

var systemPrompt =
    "Extract the three most important takeaways from the following document. " +
    "Number each takeaway and keep each to one or two sentences.";

To try a different style, replace the Content value in the system message with one of the prompts. The model follows the instructions in the system prompt to shape the format and depth of the summary.

Process multiple documents

Extend the application to summarize every .txt file in a directory. This is useful when you have a folder of documents that all need summaries.

The following method iterates over all .txt files in a given directory and summarizes each one:

async Task SummarizeDirectoryAsync(
    dynamic chatClient,
    string directory,
    string systemPrompt,
    CancellationToken ct)
{
    var txtFiles = Directory.GetFiles(directory, "*.txt")
        .OrderBy(f => f)
        .ToArray();

    if (txtFiles.Length == 0)
    {
        Console.WriteLine($"No .txt files found in {directory}");
        return;
    }

    foreach (var txtFile in txtFiles)
    {
        var fileContent = await File.ReadAllTextAsync(txtFile, ct);
        var msgs = new List<ChatMessage>
        {
            new ChatMessage { Role = "system", Content = systemPrompt },
            new ChatMessage { Role = "user", Content = fileContent }
        };

        Console.WriteLine($"--- {Path.GetFileName(txtFile)} ---");
        var resp = await chatClient.CompleteChatAsync(msgs, ct);
        Console.WriteLine(resp.Choices[0].Message.Content);
        Console.WriteLine();
    }
}

Each file is read, paired with the same system prompt, and sent to the model independently. The model doesn't carry context between files, so each summary is self-contained.

Complete code

Replace the contents of Program.cs with the following complete code:

using Microsoft.AI.Foundry.Local;
using Betalgo.Ranul.OpenAI.ObjectModels.RequestModels;
using Microsoft.Extensions.Logging;

CancellationToken ct = CancellationToken.None;

var config = new Configuration
{
    AppName = "foundry_local_samples",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information
};

using var loggerFactory = LoggerFactory.Create(builder =>
{
    builder.SetMinimumLevel(Microsoft.Extensions.Logging.LogLevel.Information);
});
var logger = loggerFactory.CreateLogger<Program>();

// Initialize the singleton instance
await FoundryLocalManager.CreateAsync(config, logger);
var mgr = FoundryLocalManager.Instance;

// Select and load a model from the catalog
var catalog = await mgr.GetCatalogAsync();
var model = await catalog.GetModelAsync("qwen2.5-0.5b")
    ?? throw new Exception("Model not found");

await model.DownloadAsync(progress =>
{
    Console.Write($"\rDownloading model: {progress:F2}%");
    if (progress >= 100f) Console.WriteLine();
});

await model.LoadAsync();
Console.WriteLine("Model loaded and ready.\n");

// Get a chat client
var chatClient = await model.GetChatClientAsync();

var systemPrompt =
    "Summarize the following document into concise bullet points. " +
    "Focus on the key points and main ideas.";

var target = args.Length > 0 ? args[0] : "document.txt";

if (Directory.Exists(target))
{
    await SummarizeDirectoryAsync(chatClient, target, systemPrompt, ct);
}
else
{
    Console.WriteLine($"--- {Path.GetFileName(target)} ---");
    await SummarizeFileAsync(chatClient, target, systemPrompt, ct);
}

// Clean up
await model.UnloadAsync();
Console.WriteLine("\nModel unloaded. Done!");

async Task SummarizeFileAsync(
    dynamic client,
    string filePath,
    string prompt,
    CancellationToken token)
{
    var fileContent = await File.ReadAllTextAsync(filePath, token);
    var messages = new List<ChatMessage>
    {
        new ChatMessage { Role = "system", Content = prompt },
        new ChatMessage { Role = "user", Content = fileContent }
    };

    var response = await client.CompleteChatAsync(messages, token);
    Console.WriteLine(response.Choices[0].Message.Content);
}

async Task SummarizeDirectoryAsync(
    dynamic client,
    string directory,
    string prompt,
    CancellationToken token)
{
    var txtFiles = Directory.GetFiles(directory, "*.txt")
        .OrderBy(f => f)
        .ToArray();

    if (txtFiles.Length == 0)
    {
        Console.WriteLine($"No .txt files found in {directory}");
        return;
    }

    foreach (var txtFile in txtFiles)
    {
        Console.WriteLine($"--- {Path.GetFileName(txtFile)} ---");
        await SummarizeFileAsync(client, txtFile, prompt, token);
        Console.WriteLine();
    }
}

Summarize a single file:

dotnet run -- document.txt

Or summarize every .txt file in a directory:

dotnet run -- ./docs

You see output similar to:

Downloading model: 100.00%
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

Install packages

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/js/tutorial-document-summarizer

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

npm install foundry-local-sdk-winml openai

Read a text document

Before you summarize anything, you need a sample document to work with. Create a file called document.txt in your project directory and add the following content:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

Now create a file called index.js and add the following code to read the document:

const target = process.argv[2] || 'document.txt';

The script accepts an optional file path as a command-line argument and falls back to document.txt if none is provided.

Generate a summary

Initialize the Foundry Local SDK, load a model, and send the document content along with a system prompt that instructs the model to summarize.

Replace the contents of index.js with the following code:

const systemPrompt =
    'Summarize the following document into concise bullet points. ' +
    'Focus on the key points and main ideas.';

const target = process.argv[2] || 'document.txt';

try {
    const stats = statSync(target);
    if (stats.isDirectory()) {
        await summarizeDirectory(chatClient, target, systemPrompt);
    } else {
        console.log(`--- ${basename(target)} ---`);
        await summarizeFile(chatClient, target, systemPrompt);
    }
} catch {
    console.log(`--- ${basename(target)} ---`);
    await summarizeFile(chatClient, target, systemPrompt);
}

The getModel method accepts a model alias, which is a short friendly name that maps to a specific model in the catalog. The download method fetches the model weights to your local cache (and skips the download if they're already cached), and load makes the model ready for inference. The system prompt tells the model to produce bullet-point summaries focused on key ideas.

Control summary output

Different situations call for different summary styles. You can change the system prompt to control how the model structures its output. Here are three useful variations:

Bullet points (default from the previous step):

const systemPrompt =
    'Summarize the following document into concise bullet points. ' +
    'Focus on the key points and main ideas.';

One-paragraph summary:

const systemPrompt =
    'Summarize the following document in a single, concise paragraph. ' +
    'Capture the main argument and supporting points.';

Key takeaways:

const systemPrompt =
    'Extract the three most important takeaways from the following document. ' +
    'Number each takeaway and keep each to one or two sentences.';

To try a different style, replace the content value in the system message with one of the prompts. The model follows the instructions in the system prompt to shape the format and depth of the summary.

Process multiple documents

Extend the application to summarize every .txt file in a directory. This is useful when you have a folder of documents that all need summaries.

The following function iterates over all .txt files in a given directory and summarizes each one:

import { readdirSync } from 'fs';
import { join, basename } from 'path';

async function summarizeDirectory(chatClient, directory, systemPrompt) {
    const txtFiles = readdirSync(directory)
        .filter(f => f.endsWith('.txt'))
        .sort();

    if (txtFiles.length === 0) {
        console.log(`No .txt files found in ${directory}`);
        return;
    }

    for (const fileName of txtFiles) {
        const fileContent = readFileSync(join(directory, fileName), 'utf-8');
        const msgs = [
            { role: 'system', content: systemPrompt },
            { role: 'user', content: fileContent }
        ];

        console.log(`--- ${fileName} ---`);
        const resp = await chatClient.completeChat(msgs);
        console.log(resp.choices[0]?.message?.content);
        console.log();
    }
}

Each file is read, paired with the same system prompt, and sent to the model independently. The model doesn't carry context between files, so each summary is self-contained.

Complete code

Create a file named index.js and add the following complete code:

import { FoundryLocalManager } from 'foundry-local-sdk';
import { readFileSync, readdirSync, statSync } from 'fs';
import { join, basename } from 'path';

async function summarizeFile(chatClient, filePath, systemPrompt) {
    const content = readFileSync(filePath, 'utf-8');
    const messages = [
        { role: 'system', content: systemPrompt },
        { role: 'user', content: content }
    ];

    const response = await chatClient.completeChat(messages);
    console.log(response.choices[0]?.message?.content);
}

async function summarizeDirectory(chatClient, directory, systemPrompt) {
    const txtFiles = readdirSync(directory)
        .filter(f => f.endsWith('.txt'))
        .sort();

    if (txtFiles.length === 0) {
        console.log(`No .txt files found in ${directory}`);
        return;
    }

    for (const fileName of txtFiles) {
        console.log(`--- ${fileName} ---`);
        await summarizeFile(chatClient, join(directory, fileName), systemPrompt);
        console.log();
    }
}

// Initialize the Foundry Local SDK
const manager = FoundryLocalManager.create({
    appName: 'foundry_local_samples',
    logLevel: 'info'
});

// Select and load a model from the catalog
const model = await manager.catalog.getModel('qwen2.5-0.5b');

await model.download((progress) => {
    process.stdout.write(`\rDownloading model: ${progress.toFixed(2)}%`);
});
console.log('\nModel downloaded.');

await model.load();
console.log('Model loaded and ready.\n');

// Create a chat client
const chatClient = model.createChatClient();

const systemPrompt =
    'Summarize the following document into concise bullet points. ' +
    'Focus on the key points and main ideas.';

const target = process.argv[2] || 'document.txt';

try {
    const stats = statSync(target);
    if (stats.isDirectory()) {
        await summarizeDirectory(chatClient, target, systemPrompt);
    } else {
        console.log(`--- ${basename(target)} ---`);
        await summarizeFile(chatClient, target, systemPrompt);
    }
} catch {
    console.log(`--- ${basename(target)} ---`);
    await summarizeFile(chatClient, target, systemPrompt);
}

// Clean up
await model.unload();
console.log('\nModel unloaded. Done!');

Summarize a single file:

node index.js document.txt

Or summarize every .txt file in a directory:

node index.js ./docs

You see output similar to:

Downloading model: 100.00%
Model downloaded.
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

Install packages

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/python/tutorial-document-summarizer

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

pip install foundry-local-sdk-winml openai

Read a text document

Before you summarize anything, you need a sample document to work with. Create a file called document.txt in your project directory and add the following content:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

Now create a file called main.py and add the following code to read the document:

target = sys.argv[1] if len(sys.argv) > 1 else "document.txt"
target_path = Path(target)

The script accepts an optional file path as a command-line argument and falls back to document.txt if none is provided. The Path.read_text method reads the entire file into a string.

Generate a summary

Initialize the Foundry Local SDK, load a model, and send the document content along with a system prompt that instructs the model to summarize.

Replace the contents of main.py with the following code:

system_prompt = (
    "Summarize the following document into concise bullet points. "
    "Focus on the key points and main ideas."
)

target = sys.argv[1] if len(sys.argv) > 1 else "document.txt"
target_path = Path(target)

if target_path.is_dir():
    await summarize_directory(client, target_path, system_prompt)
else:
    print(f"--- {target_path.name} ---")
    await summarize_file(client, target_path, system_prompt)

The get_model method accepts a model alias, which is a short friendly name that maps to a specific model in the catalog. The download method fetches the model weights to your local cache (and skips the download if they're already cached), and load makes the model ready for inference. The system prompt tells the model to produce bullet-point summaries focused on key ideas.

Control summary output

Different situations call for different summary styles. You can change the system prompt to control how the model structures its output. Here are three useful variations:

Bullet points (default from the previous step):

system_prompt = (
    "Summarize the following document into concise bullet points. "
    "Focus on the key points and main ideas."
)

One-paragraph summary:

system_prompt = (
    "Summarize the following document in a single, concise paragraph. "
    "Capture the main argument and supporting points."
)

Key takeaways:

system_prompt = (
    "Extract the three most important takeaways from the following document. "
    "Number each takeaway and keep each to one or two sentences."
)

To try a different style, replace the "content" value in the system message with one of the prompts. The model follows the instructions in the system prompt to shape the format and depth of the summary.

Process multiple documents

Extend the application to summarize every .txt file in a directory. This is useful when you have a folder of documents that all need summaries.

The following function iterates over all .txt files in a given directory and summarizes each one:

async def summarize_directory(client, directory):
    txt_files = sorted(Path(directory).glob("*.txt"))

    if not txt_files:
        print(f"No .txt files found in {directory}")
        return

    for txt_file in txt_files:
        content = txt_file.read_text(encoding="utf-8")
        messages = [
            {
                "role": "system",
                "content": "Summarize the following document into concise bullet points. "
                           "Focus on the key points and main ideas."
            },
            {"role": "user", "content": content}
        ]

        print(f"--- {txt_file.name} ---")
        response = client.complete_chat(messages)
        print(response.choices[0].message.content)
        print()

Each file is read, paired with the same system prompt, and sent to the model independently. The model doesn't carry context between files, so each summary is self-contained.

Complete code

Create a file named main.py and add the following complete code:

import asyncio
import sys
from pathlib import Path
from foundry_local_sdk import Configuration, FoundryLocalManager


async def summarize_file(client, file_path, system_prompt):
    """Summarize a single file and print the result."""
    content = Path(file_path).read_text(encoding="utf-8")
    messages = [
        {"role": "system", "content": system_prompt},
        {"role": "user", "content": content}
    ]
    response = client.complete_chat(messages)
    print(response.choices[0].message.content)


async def summarize_directory(client, directory, system_prompt):
    """Summarize all .txt files in a directory."""
    txt_files = sorted(Path(directory).glob("*.txt"))

    if not txt_files:
        print(f"No .txt files found in {directory}")
        return

    for txt_file in txt_files:
        print(f"--- {txt_file.name} ---")
        await summarize_file(client, txt_file, system_prompt)
        print()


async def main():
    # Initialize the Foundry Local SDK
    config = Configuration(app_name="foundry_local_samples")
    FoundryLocalManager.initialize(config)
    manager = FoundryLocalManager.instance

    # Select and load a model from the catalog
    model = manager.catalog.get_model("qwen2.5-0.5b")
    model.download(lambda p: print(f"\rDownloading model: {p:.2f}%", end="", flush=True))
    print()
    model.load()
    print("Model loaded and ready.\n")

    # Get a chat client
    client = model.get_chat_client()

    system_prompt = (
        "Summarize the following document into concise bullet points. "
        "Focus on the key points and main ideas."
    )

    target = sys.argv[1] if len(sys.argv) > 1 else "document.txt"
    target_path = Path(target)

    if target_path.is_dir():
        await summarize_directory(client, target_path, system_prompt)
    else:
        print(f"--- {target_path.name} ---")
        await summarize_file(client, target_path, system_prompt)

    # Clean up
    model.unload()
    print("\nModel unloaded. Done!")


if __name__ == "__main__":
    asyncio.run(main())

Summarize a single file:

python main.py document.txt

Or summarize every .txt file in a directory:

python main.py ./docs

You see output similar to:

Downloading model: 100.00%
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

Install packages

Samples repository

The complete sample code for this article is available in the Foundry Local GitHub repository. To clone the repository and navigate to the sample use:

git clone https://github.com/microsoft/Foundry-Local.git
cd Foundry-Local/samples/rust/tutorial-document-summarizer

If you're developing or shipping on Windows, select the Windows tab. The Windows package integrates with the Windows ML runtime — it provides the same API surface area with a wider breadth of hardware acceleration.

cargo add foundry-local-sdk --features winml
cargo add tokio --features full
cargo add tokio-stream anyhow

Read a text document

Before you summarize anything, you need a sample document to work with. Create a file called document.txt in your project directory and add the following content:

Automated testing is a practice in software development where tests are written and executed
by specialized tools rather than performed manually. There are several categories of automated
tests, including unit tests, integration tests, and end-to-end tests. Unit tests verify that
individual functions or methods behave correctly in isolation. Integration tests check that
multiple components work together as expected. End-to-end tests simulate real user workflows
across the entire application.

Adopting automated testing brings measurable benefits to a development team. It catches
regressions early, before they reach production. It reduces the time spent on repetitive
manual verification after each code change. It serves as living documentation of expected
behavior, which helps new team members understand the codebase. Continuous integration
pipelines rely on automated tests to gate deployments and maintain release quality.

Effective test suites follow a few guiding principles. Tests should be deterministic, meaning
they produce the same result every time they run. Tests should be independent, so that one
failing test does not cascade into false failures elsewhere. Tests should run fast, because
slow tests discourage developers from running them frequently. Finally, tests should be
maintained alongside production code so they stay accurate as the application evolves.

Now open src/main.rs and add the following code to read the document:

let target = env::args()
    .nth(1)
    .unwrap_or_else(|| "document.txt".to_string());
let target_path = Path::new(&target);

The code accepts an optional file path as a command-line argument and falls back to document.txt if none is provided.

Generate a summary

Initialize the Foundry Local SDK, load a model, and send the document content along with a system prompt that instructs the model to summarize.

Replace the contents of src/main.rs with the following code:

let system_prompt = "Summarize the following document \
     into concise bullet points. Focus on the key \
     points and main ideas.";

let target = env::args()
    .nth(1)
    .unwrap_or_else(|| "document.txt".to_string());
let target_path = Path::new(&target);

if target_path.is_dir() {
    summarize_directory(
        &client,
        target_path,
        system_prompt,
    )
    .await?;
} else {
    let file_name = target_path
        .file_name()
        .map(|n| n.to_string_lossy().to_string())
        .unwrap_or_else(|| target.clone());
    println!("--- {} ---", file_name);
    summarize_file(
        &client,
        target_path,
        system_prompt,
    )
    .await?;
}

The get_model method accepts a model alias, which is a short friendly name that maps to a specific model in the catalog. The download method fetches the model weights to your local cache (and skips the download if they're already cached), and load makes the model ready for inference. The system prompt tells the model to produce bullet-point summaries focused on key ideas.

Control summary output

Different situations call for different summary styles. You can change the system prompt to control how the model structures its output. Here are three useful variations:

Bullet points (default from the previous step):

let system_prompt =
    "Summarize the following document into concise bullet points. \
     Focus on the key points and main ideas.";

One-paragraph summary:

let system_prompt =
    "Summarize the following document in a single, concise paragraph. \
     Capture the main argument and supporting points.";

Key takeaways:

let system_prompt =
    "Extract the three most important takeaways from the following document. \
     Number each takeaway and keep each to one or two sentences.";

To try a different style, replace the system message content with one of the prompts. The model follows the instructions in the system prompt to shape the format and depth of the summary.

Process multiple documents

Extend the application to summarize every .txt file in a directory. This is useful when you have a folder of documents that all need summaries.

The following function iterates over all .txt files in a given directory and summarizes each one:

use std::path::Path;

async fn summarize_directory(
    client: &foundry_local_sdk::ChatClient,
    directory: &Path,
    system_prompt: &str,
) -> anyhow::Result<()> {
    let mut txt_files: Vec<_> = fs::read_dir(directory)?
        .filter_map(|entry| entry.ok())
        .filter(|entry| {
            entry.path().extension()
                .map(|ext| ext == "txt")
                .unwrap_or(false)
        })
        .collect();

    txt_files.sort_by_key(|e| e.path());

    if txt_files.is_empty() {
        println!("No .txt files found in {}", directory.display());
        return Ok(());
    }

    for entry in &txt_files {
        let file_content = fs::read_to_string(entry.path())?;
        let messages: Vec<ChatCompletionRequestMessage> = vec![
            ChatCompletionRequestSystemMessage::new(system_prompt).into(),
            ChatCompletionRequestUserMessage::new(&file_content).into(),
        ];

        let file_name = entry.file_name();
        println!("--- {} ---", file_name.to_string_lossy());
        let resp = client.complete_chat(&messages, None).await?;
        let text = resp.choices[0]
            .message
            .content
            .as_deref()
            .unwrap_or("");
        println!("{}\n", text);
    }

    Ok(())
}

Each file is read, paired with the same system prompt, and sent to the model independently. The model doesn't carry context between files, so each summary is self-contained.

Complete code

Replace the contents of src/main.rs with the following complete code:

use foundry_local_sdk::{
    ChatCompletionRequestMessage,
    ChatCompletionRequestSystemMessage,
    ChatCompletionRequestUserMessage, FoundryLocalConfig,
    FoundryLocalManager,
};
use std::io::{self, Write};
use std::path::Path;
use std::{env, fs};

async fn summarize_file(
    client: &foundry_local_sdk::openai::ChatClient,
    file_path: &Path,
    system_prompt: &str,
) -> anyhow::Result<()> {
    let content = fs::read_to_string(file_path)?;
    let messages: Vec<ChatCompletionRequestMessage> = vec![
        ChatCompletionRequestSystemMessage::from(system_prompt)
            .into(),
        ChatCompletionRequestUserMessage::from(content.as_str())
            .into(),
    ];

    let response =
        client.complete_chat(&messages, None).await?;
    let summary = response.choices[0]
        .message
        .content
        .as_deref()
        .unwrap_or("");
    println!("{}", summary);
    Ok(())
}

async fn summarize_directory(
    client: &foundry_local_sdk::openai::ChatClient,
    directory: &Path,
    system_prompt: &str,
) -> anyhow::Result<()> {
    let mut txt_files: Vec<_> = fs::read_dir(directory)?
        .filter_map(|entry| entry.ok())
        .filter(|entry| {
            entry
                .path()
                .extension()
                .map(|ext| ext == "txt")
                .unwrap_or(false)
        })
        .collect();

    txt_files.sort_by_key(|e| e.path());

    if txt_files.is_empty() {
        println!(
            "No .txt files found in {}",
            directory.display()
        );
        return Ok(());
    }

    for entry in &txt_files {
        let file_name = entry.file_name();
        println!(
            "--- {} ---",
            file_name.to_string_lossy()
        );
        summarize_file(
            client,
            &entry.path(),
            system_prompt,
        )
        .await?;
        println!();
    }

    Ok(())
}

#[tokio::main]
async fn main() -> anyhow::Result<()> {
    // Initialize the Foundry Local SDK
    let manager = FoundryLocalManager::create(
        FoundryLocalConfig::new("doc-summarizer"),
    )?;

    // Select and load a model from the catalog
    let model = manager
        .catalog()
        .get_model("qwen2.5-0.5b")
        .await?;

    if !model.is_cached().await? {
        println!("Downloading model...");
        model
            .download(Some(|progress: f64| {
                print!("\r  {progress:.1}%");
                io::stdout().flush().ok();
            }))
            .await?;
        println!();
    }

    model.load().await?;
    println!("Model loaded and ready.\n");

    // Create a chat client
    let client = model
        .create_chat_client()
        .temperature(0.7)
        .max_tokens(512);

    let system_prompt = "Summarize the following document \
         into concise bullet points. Focus on the key \
         points and main ideas.";

    let target = env::args()
        .nth(1)
        .unwrap_or_else(|| "document.txt".to_string());
    let target_path = Path::new(&target);

    if target_path.is_dir() {
        summarize_directory(
            &client,
            target_path,
            system_prompt,
        )
        .await?;
    } else {
        let file_name = target_path
            .file_name()
            .map(|n| n.to_string_lossy().to_string())
            .unwrap_or_else(|| target.clone());
        println!("--- {} ---", file_name);
        summarize_file(
            &client,
            target_path,
            system_prompt,
        )
        .await?;
    }

    // Clean up
    model.unload().await?;
    println!("\nModel unloaded. Done!");

    Ok(())
}

Summarize a single file:

cargo run -- document.txt

Or summarize every .txt file in a directory:

cargo run -- ./docs

You see output similar to:

Downloading model: 100.00%
Model loaded and ready.

--- document.txt ---
- Automated testing uses specialized tools to execute tests instead of manual verification.
- Tests fall into three main categories: unit tests (individual functions), integration tests
  (component interactions), and end-to-end tests (full user workflows).
- Key benefits include catching regressions early, reducing manual effort, serving as living
  documentation, and gating deployments through continuous integration pipelines.
- Effective test suites should be deterministic, independent, fast, and maintained alongside
  production code.

Model unloaded. Done!

Clean up resources

The model weights remain in your local cache after you unload a model. This means the next time you run the application, the download step is skipped and the model loads faster. No extra cleanup is needed unless you want to reclaim disk space.

Related content

  • Last updated on