Livsy Code

Foundation Models Utilities

Greetings, traveler!

Apple’s Foundation Models framework provides a Swift API for working with language models through sessions, tools, structured output, and dynamic instructions. Apple has now published a separate open-source package called Foundation Models framework utilities that adds several experimental components on top of that framework.

The package focuses on three areas:

  • connecting LanguageModelSession to hosted models
  • controlling the growth of a session transcript
  • loading task-specific instructions only when the model needs them

Apple describes these APIs as emerging and experimental patterns, so they should not be treated as stable additions to the main Foundation Models framework yet.

Adding the package

You can add the package through Xcode using this repository URL:

https://github.com/apple/foundation-models-utilities

For a Swift package, add the dependency and the FoundationModelsUtilities product:

let package = Package(
    name: "YourApp",
    dependencies: [
        .package(
            url: "https://github.com/apple/foundation-models-utilities",
            from: "1.0.0"
        )
    ],
    targets: [
        .target(
            name: "YourApp",
            dependencies: [
                .product(
                    name: "FoundationModelsUtilities",
                    package: "foundation-models-utilities"
                )
            ]
        )
    ]
)

The package uses Swift tools version 6.2 and Swift 6 language mode. Its package manifest declares support for iOS 27, macOS 27, visionOS 27, and watchOS 27. The README also lists selected Linux distributions, including Ubuntu, among the supported environments.

Connecting to a hosted model

ChatCompletionsLanguageModel connects the Foundation Models API to a server that implements the Chat Completions REST protocol.

import FoundationModels
import FoundationModelsUtilities

let model = ChatCompletionsLanguageModel(
    name: "my-model",
    url: serverURL
)

let session = LanguageModelSession(model: model)

let response = try await session.respond(
    to: "Explain actor isolation in Swift."
)

print(response.content)

The important part is that the custom model is passed to a regular LanguageModelSession. Code that works with the session can continue using familiar Foundation Models concepts instead of introducing a separate API for each model provider.

This makes the utility useful when an app needs to work with a local model server, a hosted model, or other infrastructure built around the Chat Completions protocol.

Some servers do not support guided generation. You can describe that limitation when creating the model:

let model = ChatCompletionsLanguageModel(
    name: "my-model",
    url: serverURL,
    supportsGuidedGeneration: false
)

Guided generation matters when the session expects the model to produce output that follows a requested structure. The flag prevents the client from assuming that the connected server supports this feature.

ChatCompletionsLanguageModel does not turn every Chat Completions server into a fully equivalent replacement for Apple’s on-device model. The server still needs to support the features used by the session, such as tools or structured generation.

Managing session history

A LanguageModelSession keeps a transcript containing instructions, prompts, responses, tool calls, and tool output. That transcript consumes the model’s context window.

Apple documents a 4,096-token context window for its on-device model. Instructions and tool definitions are part of that budget, not only the visible conversation. A long-running session therefore needs a strategy for removing or compressing older content.

Foundation Models Utilities provides profile modifiers for this purpose.

Dropping completed tool calls

The droppingCompletedToolCalls() modifier removes completed tool interactions from the history.

Profile {
    Instructions("You are a helpful assistant.")
    SearchTool()
}
.droppingCompletedToolCalls()

Tool calls can add substantial data to a transcript. A result may contain records, identifiers, or structured JSON that the model no longer needs after it has produced the final response.

Removing these entries saves context, but it also removes information that may become relevant later. This strategy works best when completed tool results do not need to be referenced in a future turn.

Keeping a rolling window

The rollingWindow(entries:) modifier keeps only a limited number of recent transcript entries.

Profile {
    Instructions("You are a helpful assistant.")
}
.rollingWindow(entries: 10)

This approach is predictable and inexpensive. The oldest entries disappear once the history grows beyond the configured size.

The limit is based on transcript entries rather than their token count. Ten short messages and ten large tool outputs can consume very different amounts of context. A rolling window is therefore useful as a simple boundary, but it does not provide an exact token budget.

Summarizing older interactions

The summarizeHistory modifier replaces older conversation content with a generated summary.

Profile {
    Instructions("You are a helpful assistant.")
}
.summarizeHistory(
    entryThreshold: 10,
    model: summarizerModel
)

The summarizer can use a different language model from the main session. An application may choose a smaller or cheaper model for compression while reserving another model for user-facing responses.

Summarization preserves more context than simply deleting old entries, but it is not lossless. A summary can omit details or change their meaning. Important application state should remain in typed models or persistent storage instead of existing only in the transcript.

Composing history strategies

The modifiers can be combined:

struct AssistantProfile: LanguageModelSession.DynamicProfile {
    let summarizerModel: any LanguageModel

    var body: some DynamicProfile {
        Profile {
            Instructions(
                "A conversation between a user and a helpful assistant."
            )
            ToggleDarkModeTool()
        }
        .summarizeHistory(
            entryThreshold: 10,
            model: summarizerModel
        )
        .rollingWindow(entries: 10)
        .droppingCompletedToolCalls()
    }
}

The order matters. Apple states that these modifiers apply from the outside in. In this example, the profile first removes completed tool calls, then limits the remaining history, and finally summarizes it when the configured threshold is reached.

There is no universal strategy for transcript management. A short-lived generation session may need none of these modifiers. A tool-based assistant that remains open for hours may need all three.

Loading instructions with skills

The package also introduces runtime skills.

A skill contains instructions for a specific kind of task. Instead of adding every possible instruction to the session at startup, the app provides the model with available skills. The model activates a skill through a tool call when it decides that the skill is relevant.

@Observable
final class Assistant {
    var activations = SkillActivations()
}
struct AssistantProfile: LanguageModelSession.DynamicProfile {
    let assistant: Assistant

    var body: some DynamicProfile {
        Profile {
            Instructions(
                "A conversation between a user and a helpful assistant."
            )

            Skills(activations: assistant.activations) {
                Skill(
                    name: "style-guide",
                    description: "Applies the project's writing rules",
                    prompt: styleGuide
                )

                Skill(
                    name: "calendar",
                    description: "Handles calendar-related tasks",
                    instructions: calendarInstructions
                )
            }
        }
    }
}

Skills conforms to DynamicInstructions and uses a result builder to define its contents. SkillActivations tracks the active skills and conforms to both Observable and RandomAccessCollection, so an app can use the same state to update its SwiftUI interface.

Prompt-based skills

A prompt-based skill adds its content to the transcript as the output of the activation tool call.

Skill(
    name: "style-guide",
    description: "Applies the project's writing rules",
    prompt: styleGuide
)

This approach does not modify the original instructions entry. According to the repository documentation, that avoids invalidating the model’s key-value cache.

Prompt-based skills are suitable for reference material such as style guides, product documentation, or domain notes. Their content becomes part of the conversation rather than part of the highest-priority instructions.

Instructions-based skills

An instructions-based skill appends its content to the first instructions entry:

Skill(
    name: "calendar",
    description: "Handles calendar-related tasks",
    instructions: calendarInstructions
)

Models generally give instructions higher priority than regular prompt content. This makes the variant appropriate for rules that should directly control behavior.

The trade-off is that changing the instructions can invalidate the key-value cache. The next response may require the model to process the instruction prefix again, which can affect time to first token.

Instructions-based skills can also allow deactivation:

Skill(
    name: "calendar",
    description: "Handles calendar-related tasks",
    instructions: calendarInstructions,
    allowsDeactivation: true
)

After finishing the task, the model may issue another tool call that removes the skill from the active instructions. This keeps unrelated instructions out of later requests. Deactivation can be combined with droppingCompletedToolCalls() to remove the completed activation sequence from the transcript as well.

Why skills matter

Tools and instructions consume context even when the current request does not need them. An assistant that supports mail, calendars, travel, writing, and document search can quickly accumulate a large initial prompt.

Skills allow the session to expose a short description first and load the full content only when required. This reduces context usage and avoids placing unrelated rules into every request.

The activation is model-driven, so it is not fully deterministic. Applications should not depend on a skill being activated for security checks, authorization, or other rules that must always run. Those constraints still belong in application code.

Development-time and runtime skills

The repository also includes a skills directory for coding agents. These files teach a development agent how to use the package.

They are separate from the runtime Skill type:

  • repository skills guide a coding agent while it works on the project
  • runtime skills change the behavior of a LanguageModelSession inside the application

Both use the same general idea of loading focused instructions when they become relevant, but they run in different environments and have different lifecycles.

When to use the package

Foundation Models Utilities is most useful for applications that need one or more of the following:

  • a hosted model behind a Chat Completions endpoint
  • long-running sessions with a growing transcript
  • multiple groups of task-specific instructions
  • skills that can be activated and removed at runtime
  • an observable representation of active model capabilities

A small feature that sends one prompt and receives one response probably does not need these utilities.

Final thoughts

Foundation Models Utilities adds a small set of focused building blocks rather than another complete agent framework.

ChatCompletionsLanguageModel lets a standard LanguageModelSession communicate with external models. History modifiers provide several ways to control transcript growth. Skills keep optional instructions out of the context until they are needed.

The package is still experimental, and its minimum Apple platform versions are currently set to version 27. It is best treated as a reference for new Foundation Models patterns and as a package whose public API may continue to change.

It might be interesting: