Dragon Copilot web app for mobile - overview

Introduction

Dragon Copilot web app for mobile is an SDK that you can integrate with your iOS and Android apps. When initialized, the SDK launches the Dragon Copilot web UI and gives your app access to the full suite of Dragon Copilot features and capabilities.

This document outlines the steps to embed Dragon Copilot web app for mobile seamlessly into your mobile apps.

This enables your users to:

• Perform ambient recording and stream audio to Dragon Copilot for generative AI processing in a reliable and secure manner.
• Review AI-generated notes and transcripts in the web UI.
• Use hands-free dictation of speech to text.
• Ask patient-related questions and receive answers from the AI engine.

See also: Dragon Copilot web app for mobile use cases.

API reference

iOS: Dragon Copilot web app for mobile - iOS API reference.

Android: Dragon Copilot web app for mobile - Android API reference.

See also: Platform comparison - iOS and Android.

Sample code

iOS: For Swift and React Native sample code, installation instructions, and additional resources, see the sample apps on GitHub.

Android: See Android sample code.

Supported platforms

• iOS 17 and 18 on iPhone and iPad devices.
• Android 13 and higher on phones and tablets.

Supported development environments

• Xcode 16.2.
• Swift, Objective-C, and React Native.

Swift and Objective-C

Xcode setup: Open your project in Xcode and add the SDK as a package dependency. Make sure the necessary frameworks and libraries are linked to your project.

React Native

Middleware: Since Dragon Copilot web app for mobile is built with native iOS technology, you need to write a middleware as a bridge between React Native and the SDK.

Dependencies: Add the SDK as a package dependency in your React Native project. Make sure the necessary frameworks and libraries are linked to your project.

Application logs

Dragon Copilot web app for mobile generates application logs, which are stored locally on the device. Additionally, these logs are sent to Azure Application Insights for centralized monitoring and analysis.

These logs don't contain any PHI (protected health information) or PII (personally identifiable information). The logs include Microsoft identifiers to provide context, such as the device ID, partner ID, customer ID, product ID, and correlation ID.

Error handling and codes

Dragon Copilot web app for mobile performs various HTTP requests to retrieve information on the app and the user. It includes a robust error handling mechanism to ensure reliable and consistent behavior during these requests. When making HTTP requests, our system automatically retries the request if it encounters any retryable HTTP status codes. This helps to handle transient errors and improve the reliability of the system.

Retry logic:

• Retryable status codes: If the HTTP request fails with a retryable status code (for example, 500, 502, 503, 504), the system retries the request.
• Retry attempts: The system retries the request up to three times.
• Delay between retries: There's a consistent delay of two seconds between each retry attempt.

Common issues

Authentication issues: If the partner-issued authentication token doesn't meet Dragon Copilot's requirements, the SDK fails to authenticate. Make sure the token is valid and meets all necessary criteria. For more information, see App authentication.

Missing required fields: If any required fields are missing, the app might not start, or the UI will display a generic error page with an error code.

A valid server environment ID is needed to start, and a correlation ID is required to show session details.

Resiliency

Streaming is the primary method of delivering audio to Dragon Copilot. However, Dragon Copilot web app for mobile also stores the audio locally as a backup in case of network interruptions. This backup enables users to record continuously even when network interruptions occur. The upload finishes when the connection is restored. Your mobile app doesn't need to implement audio backup functionality because the SDK fully manages this functionality.

Dragon Copilot web app for mobile saves backup audio files locally to iOS Core Data and encrypts them using AES 256. The SDK uses encryption keys that are stored in the iOS keychain. These keys are unique per recording and never leave the device, so the audio can only be decrypted on the respective device.

The following diagram illustrates the backup and encryption strategy employed by Dragon Copilot web app for mobile.