Foundry Local SDK migration guide

C# SDK Migration Guide

To improve your ability to ship applications using on-device AI, there are substantial changes to the architecture of the C# SDK in version 0.8.0 and later. In this section, we outline the key changes to help you migrate your applications to the latest version of the SDK.

The following diagram shows how the previous architecture - for versions earlier than 0.8.0 - relied heavily on using a REST webserver to manage models and inference like chat completions:

The SDK would use a Remote Procedural Call (RPC) to find Foundry Local CLI executable on the machine, start the webserver, and then communicate with it over HTTP. This architecture had several limitations, including:

• Complexity in managing the webserver lifecycle.
• Challenging deployment: End users needed to have the Foundry Local CLI installed on their machines and your application.
• Version management of the CLI and SDK could lead to compatibility issues.

To address these issues, the redesigned architecture in version 0.8.0 and later uses a more streamlined approach. The new architecture is as follows:

In this new architecture:

• Your application is self-contained. It doesn't require the Foundry Local CLI to be installed separately on the end user's machine making it easier for you to deploy applications.
• The REST web server is optional. You can still use the web server if you want to integrate with other tools that communicate over HTTP. Read Use chat completions via REST server with Foundry Local for details on how to use this feature.
• The SDK has native support for chat completions and audio transcriptions, allowing you to build conversational AI applications with fewer dependencies. Read Use Foundry Local native chat completions API for details on how to use this feature.
• On Windows devices, you can use a Windows ML build that handles hardware acceleration for models on the device by pulling in the right runtime and drivers.

API changes

Version 0.8.0 and later provides a more object-orientated and composable API. The main entry point continues to be the FoundryLocalManager class, but instead of being a flat set of methods that operate via static calls to a stateless HTTP API, the SDK now exposes methods on the FoundryLocalManager instance that maintain state about the service and models.

The API allows Foundry Local to be more configurable over the web server, logging, cache location, and model variant selection. For example, the Configuration class allows you to set up the application name, logging level, web server URLs, and directories for application data, model cache, and logs:

var config = new Configuration
{
    AppName = "app-name",
    LogLevel = Microsoft.AI.Foundry.Local.LogLevel.Information,
    Web = new Configuration.WebService
    {
        Urls = "http://127.0.0.1:55588"
    },
    AppDataDir = "./foundry_local_data",
    ModelCacheDir = "{AppDataDir}/model_cache",
    LogsDir = "{AppDataDir}/logs"
};

API reference

• For more details on the Foundry Local C# SDK read Foundry Local C# SDK API Reference.

JavaScript SDK Migration Guide

To improve your ability to ship applications using on-device AI, there are substantial changes to the architecture of the JavaScript SDK. In this section, we outline the key changes to help you migrate your applications to the latest version of the SDK.

The following diagram shows how the previous architecture relied heavily on using a REST webserver to manage models and inference like chat completions:

The SDK would use a Remote Procedural Call (RPC) to find Foundry Local CLI executable on the machine, start the webserver, and then communicate with it over HTTP. This architecture had several limitations, including:

• Complexity in managing the webserver lifecycle.
• Challenging deployment: End users needed to have the Foundry Local CLI installed on their machines and your application.
• Version management of the CLI and SDK could lead to compatibility issues.

To address these issues, the redesigned architecture uses a more streamlined approach. The new architecture is as follows:

In this new architecture:

• Your application is self-contained. It doesn't require the Foundry Local CLI to be installed separately on the end user's machine making it easier for you to deploy applications.
• The REST web server is optional. You can still use the web server if you want to integrate with other tools that communicate over HTTP.
• The SDK has native support for chat completions and audio transcriptions, allowing you to build conversational AI applications with fewer dependencies.

API changes

The latest version provides a more object-oriented and composable API. The main entry point continues to be the FoundryLocalManager class, but instead of being a flat set of methods that operate via static calls to a stateless HTTP API, the SDK now exposes methods on the FoundryLocalManager instance that maintain state about the service and models.

The API allows Foundry Local to be more configurable over the web server, logging, cache location, and model variant selection. For example, the config allows you to set up the application name, logging level, web server URLs, and directories for application data, model cache, and logs:

const config = {
  appName: "app-name",
  logLevel: "info",
  webServiceUrls: "http://127.0.0.1:55588",
  appDataDir: "./foundry_local_data",
  modelCacheDir: "{appDataDir}/model_cache",
  logsDir: "{appDataDir}/logs",
};

In the previous version of the Foundry Local JavaScript SDK, you couldn't configure these settings directly through the SDK, which limited your ability to customize the behavior of the service.

References

• Integrate with inference SDKs
• Use Foundry Local native chat completions API

Python SDK Migration Guide

To improve your ability to ship applications using on-device AI, there are substantial changes to the architecture of the Python SDK. In this section, we outline the key changes to help you migrate your applications to the latest version of the SDK.

The following diagram shows how the previous architecture relied heavily on using a REST webserver to manage models and inference like chat completions:

The SDK would use a Remote Procedural Call (RPC) to find Foundry Local CLI executable on the machine, start the webserver, and then communicate with it over HTTP. This architecture had several limitations, including:

• Complexity in managing the webserver lifecycle.
• Challenging deployment: End users needed to have the Foundry Local CLI installed on their machines and your application.
• Version management of the CLI and SDK could lead to compatibility issues.

To address these issues, the redesigned architecture uses a more streamlined approach. The new architecture is as follows:

In this new architecture:

• Your application is self-contained. It doesn't require the Foundry Local CLI to be installed separately on the end user's machine making it easier for you to deploy applications.
• The REST web server is optional. You can still use the web server if you want to integrate with other tools that communicate over HTTP.
• The SDK has native support for chat completions and audio transcriptions, allowing you to build conversational AI applications with fewer dependencies.

API changes

The latest version provides a more object-oriented and composable API. The main entry point continues to be the FoundryLocalManager class, but the initialization pattern, model management, and inference have all changed significantly.

The API allows Foundry Local to be more configurable over the web server, logging, cache location, and model variant selection. For example, the Configuration class allows you to set up the application name, logging level, web server URLs, and directories for application data, model cache, and logs:

from foundry_local_sdk import Configuration

config = Configuration(
    app_name="app-name",
    log_level="info",
    web={"urls": "http://127.0.0.1:55588"},
    model_cache_dir="./foundry_local_data/model_cache",
)

References

• Use chat completions via REST server
• Use Foundry Local native chat completions API

Rust SDK Migration Guide

To improve your ability to ship applications using on-device AI, there are substantial changes to the architecture of the Rust SDK. In this section, we outline the key changes to help you migrate your applications to the latest version of the SDK.

The following diagram shows how the previous architecture relied heavily on using a REST webserver to manage models and inference like chat completions:

The SDK would use a Remote Procedural Call (RPC) to find the Foundry Local CLI executable on the machine, start the webserver, and then communicate with it over HTTP. This architecture had several limitations, including:

• Complexity in managing the webserver lifecycle.
• Challenging deployment: End users needed to have the Foundry Local CLI installed on their machines and your application.
• Version management of the CLI and SDK could lead to compatibility issues.

To address these issues, the redesigned architecture uses a more streamlined approach. The new architecture is as follows:

In this new architecture:

• Your application is self-contained. It doesn't require the Foundry Local CLI to be installed separately on the end user's machine making it easier for you to deploy applications.
• The REST web server is optional. You can still use the web server if you want to integrate with other tools that communicate over HTTP.
• The SDK has native support for chat completions and audio transcriptions, allowing you to build conversational AI applications with fewer dependencies.

API changes

The latest version provides a more object-oriented and composable API. The builder pattern has been replaced with a configuration-based approach, and model management now uses dedicated Model objects.

The new SDK provides native chat and audio clients, eliminating the need for manual HTTP requests in most cases:

use foundry_local_sdk::{FoundryLocalConfig, FoundryLocalManager};

let config = FoundryLocalConfig::new("app-name");
let manager = FoundryLocalManager::create(config)?;

let model = manager.catalog().get_model("qwen2.5-0.5b").await?;
model.download(None).await?;
model.load().await?;

// Native chat client - no HTTP server needed
let client = model.create_chat_client().temperature(0.7).max_tokens(256);

References

• Use chat completions via REST server
• Use Foundry Local native chat completions API