phi-core

Simple, effective agent loop in Rust.

phi-core is a library for building LLM-powered agents that can use tools. It provides the core loop — prompt the model, execute tool calls, feed results back — and gets out of your way.

Philosophy

The loop is the product. An agent is just a loop: send messages to an LLM, get back text and tool calls, execute the tools, repeat until the model stops. phi-core implements this loop with streaming, cancellation, context management, and multi-provider support — so you don't have to.

Features

• Streaming events — Real-time AgentEvent stream for UI updates (text deltas, thinking, tool execution)
• Multi-provider — Anthropic, OpenAI, Google Gemini, Amazon Bedrock, Azure OpenAI, and any OpenAI-compatible API
• Tool system — AgentTool trait with built-in coding tools (bash, file read/write/edit, search)
• Context management — Automatic token estimation, tiered compaction (truncate tool outputs → summarize → drop old messages)
• Execution limits — Max turns, tokens, and wall-clock time
• Steering & follow-ups — Interrupt the agent mid-run or queue work for after it finishes
• Cancellation — CancellationToken-based abort at any point
• Builder pattern — Ergonomic BasicAgent struct with chainable configuration; Agent trait for polymorphism
• Config-driven construction — TOML/JSON/YAML config → agent_from_config() → Arc<dyn Agent>
• Session persistence — SessionRecorder materializes structured session/loop/turn records from events
• Sub-agents — Delegate tasks to child agent loops via SubAgentTool
• MCP integration — Connect to external tool servers via Model Context Protocol (stdio + HTTP)
• Evaluational parallelism — Run N configs concurrently, select the best result via EvaluationStrategy

Ecosystem

phi-core is part of the LazyBouy ecosystem. It powers the agent backend for Phi applications.

• Repository: github.com/LazyBouy/phi-core
• License: MIT

Installation

Requirements

• Rust 2021 edition (1.75+)
• Tokio async runtime

Add to Cargo.toml

[dependencies]
phi-core = "0.8"

Dependencies

phi-core brings in these key dependencies automatically:

Feature Flags

All providers and built-in tools are included by default. Optional features:

Enable in Cargo.toml:

[dependencies]
phi-core = { version = "0.7", features = ["openapi"] }

Quick Start

Basic Example with Anthropic

use phi_core::{BasicAgent, AgentEvent, StreamDelta};
use phi_core::provider::ModelConfig;
use phi_core::tools::default_tools;

#[tokio::main]
async fn main() {
    let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
    let mut agent = BasicAgent::new(ModelConfig::anthropic(
        "claude-sonnet-4-20250514",
        "Claude Sonnet 4",
        &api_key,
    ))
    .with_system_prompt("You are a helpful coding assistant.")
    .with_tools(default_tools());

    let mut rx = agent.prompt("List the files in the current directory").await;

    while let Some(event) = rx.recv().await {
        match event {
            AgentEvent::MessageUpdate { delta, .. } => match delta {
                StreamDelta::Text { delta } => print!("{}", delta),
                StreamDelta::Thinking { delta } => print!("[thinking] {}", delta),
                _ => {}
            },
            AgentEvent::ToolExecutionStart { tool_name, .. } => {
                println!("\n→ Running tool: {}", tool_name);
            }
            AgentEvent::ToolExecutionEnd { tool_name, is_error, .. } => {
                if is_error {
                    println!("  ✗ {} failed", tool_name);
                } else {
                    println!("  ✓ {} done", tool_name);
                }
            }
            AgentEvent::AgentEnd { .. } => {
                println!("\n\nDone.");
            }
            _ => {}
        }
    }
}

Example with OpenAI-Compatible Provider

For OpenAI, xAI, Groq, or any compatible API, use ModelConfig::openai() or ModelConfig::local():

use phi_core::{BasicAgent, AgentEvent, StreamDelta};
use phi_core::provider::ModelConfig;
use phi_core::tools::default_tools;

#[tokio::main]
async fn main() {
    let api_key = std::env::var("OPENAI_API_KEY").unwrap();
    let mut agent = BasicAgent::new(ModelConfig::openai("gpt-4o", "GPT-4o", &api_key))
        .with_system_prompt("You are a helpful assistant.")
        .with_tools(default_tools());

    let mut rx = agent.prompt("What is 2 + 2?").await;

    while let Some(event) = rx.recv().await {
        match event {
            AgentEvent::MessageUpdate { delta, .. } => {
                if let StreamDelta::Text { delta } = delta {
                    print!("{}", delta);
                }
            }
            AgentEvent::AgentEnd { .. } => println!(),
            _ => {}
        }
    }
}

Real-Time Streaming

By default, agent.prompt() blocks until the loop finishes and returns a receiver with all events buffered. To consume events in real-time, use prompt_with_sender() with a caller-provided channel:

use phi_core::{BasicAgent, AgentEvent, StreamDelta};
use phi_core::provider::ModelConfig;
use phi_core::tools::default_tools;

#[tokio::main]
async fn main() {
    let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
    let mut agent = BasicAgent::new(ModelConfig::anthropic(
        "claude-sonnet-4-20250514",
        "Claude Sonnet 4",
        &api_key,
    ))
    .with_system_prompt("You are a helpful assistant.")
    .with_tools(default_tools());

    let (tx, mut rx) = tokio::sync::mpsc::unbounded_channel();

    // Consume events in real-time on a separate task
    tokio::spawn(async move {
        while let Some(event) = rx.recv().await {
            match event {
                AgentEvent::MessageUpdate { delta, .. } => {
                    if let StreamDelta::Text { delta } = delta {
                        print!("{}", delta);
                    }
                }
                AgentEvent::AgentEnd { .. } => println!(),
                _ => {}
            }
        }
    });

    // This blocks until the loop finishes; state is restored automatically
    agent.prompt_with_sender("What is 2 + 2?", tx).await;

    // Agent is ready for another prompt immediately
    let _rx = agent.prompt("Follow up question").await;
}

Using the Low-Level API

For more control, use agent_loop() directly:

use phi_core::agent_loop::{agent_loop, AgentLoopConfig};
use phi_core::provider::ModelConfig;
use phi_core::types::*;
use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken;

#[tokio::main]
async fn main() {
    let (tx, mut rx) = mpsc::unbounded_channel();
    let cancel = CancellationToken::new();

    let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();

    let mut context = AgentContext {
        system_prompt: "You are helpful.".into(),
        messages: Vec::new(),
        tools: phi_core::tools::default_tools(),
        ..Default::default()
    };

    let config = AgentLoopConfig {
        model_config: ModelConfig::anthropic(
            "claude-sonnet-4-20250514",
            "Claude Sonnet 4",
            &api_key,
        ),
        thinking_level: ThinkingLevel::Off,
        max_tokens: None,
        temperature: None,
        convert_to_llm: None,
        transform_context: None,
        get_steering_messages: None,
        get_follow_up_messages: None,
        context_config: None,
        execution_limits: None,
        cache_config: CacheConfig::default(),
        tool_execution: ToolExecutionStrategy::default(),
        retry_config: phi_core::RetryConfig::default(),
        before_turn: None,
        after_turn: None,
        on_error: None,
        input_filters: vec![],
        ..Default::default()
    };

    let prompts = vec![AgentMessage::Llm(Message::user("Hello!"))];
    let new_messages = agent_loop(prompts, &mut context, &config, tx, cancel).await;

    // Drain events
    while let Ok(_event) = rx.try_recv() {
        // handle events...
    }

    println!("Got {} new messages", new_messages.len());
}

The Agent Loop

The agent loop is the core of phi-core. It implements the fundamental cycle:

User prompt → LLM call → Tool execution → LLM call → ... → Final response

The agent_loop module contains the core loop logic in mod.rs and the evaluation sub-module for evaluational parallelism strategies.

How It Works

┌──────────────────────────────────────────────┐
│                  agent_loop()                │
│                                              │
│  1. Add prompts to context                   │
│  2. Emit AgentStart + TurnStart              │
│                                              │
│  ┌─────────── Inner Loop ──────────────┐     │
│  │  • Check steering messages          │     │
│  │  • Check execution limits           │     │
│  │  • Compact context (if configured)  │     │
│  │  • Stream LLM response              │     │
│  │  • Extract tool calls               │     │
│  │  • Execute tools (with steering)    │     │
│  │  • Emit TurnEnd                     │     │
│  │  • Continue if tool_calls or steer  │     │
│  └─────────────────────────────────────┘     │
│                                              │
│  3. Check follow-up messages                 │
│  4. If follow-ups exist, loop again          │
│  5. Emit AgentEnd                            │
└──────────────────────────────────────────────┘

Entry Points

agent_loop()

Starts a new agent run with prompt messages:

#![allow(unused)]
fn main() {
pub async fn agent_loop(
    prompts: Vec<AgentMessage>,
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

The prompts are added to context, then the loop runs. Returns all new messages generated during the run.

agent_loop_continue()

Resumes from existing context (e.g., after an error, retry, or branch):

#![allow(unused)]
fn main() {
pub async fn agent_loop_continue(
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

Preconditions: context.agent_id and context.session_id must be Some — the function panics with a descriptive message otherwise. In practice, any context that passed through agent_loop() at least once already has these set. When constructing a context manually (e.g., from a persisted snapshot), set them explicitly before calling this function.

The last message in context must also not be an assistant message.

AgentLoopConfig

#![allow(unused)]
fn main() {
pub struct AgentLoopConfig {
    /// REQUIRED — complete provider identity: model id, api_key, base_url, protocol, cost rates.
    pub model_config: ModelConfig,
    /// Optional override — bypasses ProviderRegistry, used for MockProvider in tests.
    pub provider_override: Option<Arc<dyn StreamProvider>>,
    pub config_id: Option<String>,
    pub thinking_level: ThinkingLevel,
    pub max_tokens: Option<u32>,
    pub temperature: Option<f32>,
    pub convert_to_llm: Option<ConvertToLlmFn>,
    pub transform_context: Option<TransformContextFn>,
    pub get_steering_messages: Option<GetMessagesFn>,
    pub get_follow_up_messages: Option<GetMessagesFn>,
    pub context_config: Option<ContextConfig>,
    pub execution_limits: Option<ExecutionLimits>,
    pub cache_config: CacheConfig,
    pub tool_execution: ToolExecutionStrategy,
    pub retry_config: RetryConfig,
    pub before_loop: Option<BeforeLoopFn>,
    pub after_loop: Option<AfterLoopFn>,
    pub before_turn: Option<BeforeTurnFn>,
    pub after_turn: Option<AfterTurnFn>,
    pub on_error: Option<OnErrorFn>,
    pub before_tool_execution: Option<BeforeToolExecutionFn>,
    pub after_tool_execution: Option<AfterToolExecutionFn>,
    pub before_tool_execution_update: Option<BeforeToolExecutionUpdateFn>,
    pub after_tool_execution_update: Option<AfterToolExecutionUpdateFn>,
    pub before_compaction_start: Option<BeforeCompactionStartFn>,
    pub after_compaction_end: Option<AfterCompactionEndFn>,
    pub input_filters: Vec<Arc<dyn InputFilter>>,
    pub first_turn_trigger: TurnTrigger,
    pub context_translation: Option<Arc<dyn ContextTranslationStrategy>>,
    pub prun_pending: Option<Arc<Mutex<Vec<PrunRequest>>>>,
}
}

0.9.0 — async lifecycle hooks. BeforeLoopFn, AfterLoopFn, BeforeTurnFn, AfterTurnFn, OnErrorFn, BeforeToolExecutionFn, AfterToolExecutionFn, BeforeCompactionStartFn, and AfterCompactionEndFn are now async — their function bodies return Pin<Box<dyn Future<Output = T> + Send>> (alias: HookFuture<'_, T>). Sync closure bodies migrate by wrapping in Box::pin(async move { ... }). Closures can now .await LLM calls and other async work directly without a tokio::task::block_in_place bridge.

Pre-existing-behaviour preservation note (phi-core 0.9.0): tool-update hooks stay sync. BeforeToolExecutionUpdateFn and AfterToolExecutionUpdateFn remain Arc<dyn Fn(&str, &str, &str) -> bool + Send + Sync> / Arc<dyn Fn(&str, &str, &str) + Send + Sync> respectively. Async-ifying them would cascade into the ToolUpdateFn callback type and every AgentTool::execute body that invokes ctx.on_update(...) — a materially wider migration than the 0.9.0 scope. The veto decision in BeforeToolExecutionUpdateFn is synchronous so the surrounding emit gate works without an .await at every streamed tool-update; consumers that need async work at update-time should dispatch via tokio::spawn(...) inside the sync closure body. Tracked under the CHANGELOG [Unreleased] "Forward markers" section for a future release. InputFilter::filter() is also now async fn via #[async_trait] — see Tools and the per-turn debug-capture surface at debugging.md.

Steering & Follow-Ups

Steering

Steering messages interrupt the agent between tool executions. When the agent is executing multiple tool calls from a single LLM response, steering is checked after each tool completes. If a steering message is found:

1. The current tool finishes normally
2. All remaining tool calls are skipped with is_error: true and "Skipped due to queued user message"
3. The steering message is injected into context
4. The loop continues with a new LLM call that sees the interruption

#![allow(unused)]
fn main() {
// While agent is running tools, redirect it:
agent.steer(AgentMessage::Llm(Message::user("Stop that. Instead, explain what you found.")));
}

Follow-Ups

Follow-up messages are checked after the agent would normally stop (no more tool calls, no steering). If follow-ups exist, the loop continues with them as new input — the agent doesn't need to be re-prompted.

#![allow(unused)]
fn main() {
// Queue work for after the agent finishes its current task:
agent.follow_up(AgentMessage::Llm(Message::user("Now run the tests.")));
agent.follow_up(AgentMessage::Llm(Message::user("Then commit the changes.")));
}

Queue Modes

Both queues support two delivery modes:

#![allow(unused)]
fn main() {
agent.set_steering_mode(QueueMode::All);
agent.set_follow_up_mode(QueueMode::OneAtATime);
}

Queue Management

#![allow(unused)]
fn main() {
agent.clear_steering_queue();   // Drop all pending steers
agent.clear_follow_up_queue();  // Drop all pending follow-ups
agent.clear_all_queues();       // Drop everything
}

Low-Level API

When using agent_loop() directly, steering and follow-ups are provided via callback functions:

#![allow(unused)]
fn main() {
let config = AgentLoopConfig {
    get_steering_messages: Some(Box::new(|| {
        // Return Vec<AgentMessage> — checked between tool calls
        vec![]
    })),
    get_follow_up_messages: Some(Box::new(|| {
        // Return Vec<AgentMessage> — checked when agent would stop
        vec![]
    })),
    // ...
};
}

Custom Compaction

By default, when context exceeds the token budget in ContextConfig, phi-core runs a 3-level compaction strategy: truncate tool outputs → summarize old turns → drop middle messages (legacy in-memory path via compact_messages()). When a Session is available, the modern system uses non-destructive CompactionBlock overlays — see compaction. You can replace this with your own CompactionStrategy.

CompactionStrategy vs BlockCompactionStrategy

• CompactionStrategy — Legacy in-memory approach. Destructive: it mutates the message list directly. Used when AgentContext.session is None (no session persistence).
• BlockCompactionStrategy — New overlay approach. Non-destructive: it creates a CompactionBlock on the LoopRecord rather than altering the original messages. Used when AgentContext.session is Some (session-backed execution). Original messages remain authoritative for replay and branching.

Example of a custom CompactionStrategy:

#![allow(unused)]
fn main() {
use phi_core::context::{CompactionStrategy, ContextConfig, CompactionConfig, compact_messages};
use phi_core::types::*;
use std::sync::Arc;

struct MyCompaction;

impl CompactionStrategy for MyCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        // Your logic here — then optionally delegate to the default:
        compact_messages(messages, config)
    }
}

// Modern pattern: set strategies via ContextConfig.compaction
let context_config = ContextConfig {
    compaction: CompactionConfig {
        // in_memory_strategy: used when AgentContext.session is None (sub-agents, tests)
        in_memory_strategy: Some(Arc::new(MyCompaction)),
        // block_strategy: used when AgentContext.session is Some (session-backed execution)
        // block_strategy: Some(Arc::new(MyBlockCompaction)),
        ..CompactionConfig::default()
    },
    ..ContextConfig::default()
};

let agent = BasicAgent::new(model_config)
    .with_context_config(context_config);
}

The in-memory strategy is called once per turn, right before the LLM call, whenever context_config is Some and AgentContext.session is None. When in_memory_strategy is None, DefaultCompaction (which wraps compact_messages()) is used automatically. When a session is present, block_strategy is used instead (defaulting to DefaultBlockCompaction).

Use Cases

Memory-aware compaction — Index messages into a vector store before they're dropped, so the agent can recall them later via a search tool:

#![allow(unused)]
fn main() {
struct MemoryAwareCompaction {
    memory: Arc<dyn MemoryStore>,
}

impl CompactionStrategy for MemoryAwareCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        let compacted = compact_messages(messages.clone(), config);

        // Index what was dropped
        let dropped: Vec<_> = messages.iter()
            .filter(|m| !compacted.contains(m))
            .collect();
        if !dropped.is_empty() {
            self.memory.index(dropped);
        }

        compacted
    }
}
}

Semantic pointer compaction — Replace dropped messages with a marker so the agent knows context was lost:

#![allow(unused)]
fn main() {
struct SemanticPointerCompaction;

impl CompactionStrategy for SemanticPointerCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        let compacted = compact_messages(messages.clone(), config);
        let dropped_count = messages.len() - compacted.len();

        if dropped_count == 0 {
            return compacted;
        }

        // Insert a marker after the first kept messages
        let mut result = compacted;
        let insert_at = config.compaction.keep_first_turns.min(result.len());
        result.insert(insert_at, AgentMessage::Extension(
            ExtensionMessage::new("compaction_marker", serde_json::json!({
                "dropped": dropped_count,
                "note": format!("{} earlier messages were compacted", dropped_count),
            }))
        ));
        result
    }
}
}

Priority-preserving compaction — Never drop messages containing important keywords:

#![allow(unused)]
fn main() {
struct PriorityPreservingCompaction {
    preserve_keywords: Vec<String>,
}

impl CompactionStrategy for PriorityPreservingCompaction {
    fn compact(
        &self,
        messages: Vec<AgentMessage>,
        config: &ContextConfig,
    ) -> Vec<AgentMessage> {
        let (priority, normal): (Vec<_>, Vec<_>) = messages.into_iter()
            .partition(|m| self.is_priority(m));

        let mut compacted = compact_messages(normal, config);

        // Re-insert priority messages — they're never dropped
        for msg in priority {
            compacted.push(msg);
        }
        compacted
    }
}
}

Evaluational Parallelism

agent_loop_parallel runs the same prompt through multiple AgentLoopConfigs concurrently, evaluates the results with a pluggable EvaluationStrategy, and returns the winning branch. This is useful for multi-model comparison, A/B prompt testing, and selecting the best response among different reasoning approaches.

#![allow(unused)]
fn main() {
use phi_core::{agent_loop_parallel, PickFirstEvaluation, AgentContext, AgentLoopConfig};
use std::sync::Arc;

let result = agent_loop_parallel(
    prompts,
    base_context,           // cloned per branch; Arc tools shared
    vec![config_a, config_b],
    Arc::new(PickFirstEvaluation),
    tx,
    cancel,
).await;

// result.selected_context feeds directly into agent_loop_continue()
// result.selected_messages is the winning branch's output
}

See Evaluational Parallelism for the full guide including built-in strategies, the LLM judge, and session continuity.

Evaluational Parallelism

Evaluational parallelism runs the same prompt through multiple AgentLoopConfigs concurrently, evaluates the results with a pluggable strategy, and delivers the single best outcome. This lets you compare models, prompt variants, or reasoning settings in one call — then continue the session normally with the winner.

Overview

             ┌─ Config A ─► Branch A ─► response A ─┐
prompt ──────┤                                        ├─► Evaluate ─► selected response
             └─ Config B ─► Branch B ─► response B ─┘

Every branch receives an identical copy of the base context (message history, tools) and the same prompt. Branches run concurrently. After all branches finish, the EvaluationStrategy picks the winner and returns its context and messages.

When to use evaluational parallelism vs. parallel sub-agents

Entry point

#![allow(unused)]
fn main() {
pub async fn agent_loop_parallel(
    prompts: Vec<AgentMessage>,
    base_context: AgentContext,           // cloned once per config
    configs: Vec<AgentLoopConfig>,        // one per branch
    strategy: Arc<dyn EvaluationStrategy>,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> ParallelLoopResult
}

base_context is cloned once per config entry — tools are Arc-shared (zero copy); the message history is deep-cloned so branches start from identical state but diverge independently.

Minimal example

#![allow(unused)]
fn main() {
use phi_core::{agent_loop_parallel, PickFirstEvaluation, AgentContext, AgentLoopConfig};
use phi_core::provider::ModelConfig;
use std::sync::Arc;
use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken;

let config_a = AgentLoopConfig {
    model_config: ModelConfig::anthropic("claude-opus-4-6", "my-key", "claude-opus-4-6"),
    ..AgentLoopConfig::default()
};
let config_b = AgentLoopConfig {
    model_config: ModelConfig::anthropic("claude-haiku-4-5", "my-key", "claude-haiku-4-5"),
    ..AgentLoopConfig::default()
};

let (tx, mut rx) = mpsc::unbounded_channel();
let result = agent_loop_parallel(
    vec![AgentMessage::Llm(Message::user("Explain quantum entanglement."))],
    AgentContext { system_prompt: "Be concise.".into(), ..Default::default() },
    vec![config_a, config_b],
    Arc::new(PickFirstEvaluation),  // or any EvaluationStrategy
    tx,
    CancellationToken::new(),
)
.await;

println!("Selected branch: {}", result.selected_index);
// Continue the session with the winning context
// agent_loop_continue(&mut result.selected_context, &next_config, tx, cancel).await;
}

ParallelLoopResult

#![allow(unused)]
fn main() {
pub struct ParallelLoopResult {
    pub selected_context: AgentContext,        // winning branch's full context
    pub selected_messages: Vec<AgentMessage>,  // messages produced by the winner
    pub selected_index: usize,                 // 0-based index into original configs
    pub all_outcomes: Vec<ParallelLoopOutcome>,// remaining (non-selected) outcomes
    pub total_usage: Usage,                    // all branch usages + evaluation usage
}
}

Feed selected_context directly into agent_loop_continue() to resume the session normally — parallel execution is a single-loop operation, not a special session mode.

Built-in strategies

TransparentEvaluation

Single-branch pass-through. Panics if more than one config is provided.

Use this when you want the parallel plumbing (events, ParallelLoopResult) for a single config — zero evaluation overhead.

#![allow(unused)]
fn main() {
Arc::new(TransparentEvaluation)
}

PickFirstEvaluation

Always selects index 0 regardless of content.

Deterministic, zero-cost. Useful for testing and debugging multi-branch setups where you only care about the first config's output.

#![allow(unused)]
fn main() {
Arc::new(PickFirstEvaluation)
}

TokenEfficientEvaluation

Selects the branch with the lowest total token usage.

Prefer when cost or latency matters more than response depth. The model that solved the task most concisely wins.

#![allow(unused)]
fn main() {
Arc::new(TokenEfficientEvaluation)
}

ElaborateEvaluation

Selects the branch with the highest total token usage.

Prefer when depth and thoroughness are the priority. The most verbose response wins — useful when you want the most comprehensive analysis.

#![allow(unused)]
fn main() {
Arc::new(ElaborateEvaluation)
}

LlmJudgeEvaluation

Uses a separate LLM call to evaluate which branch produced the best response.

#![allow(unused)]
fn main() {
use phi_core::LlmJudgeEvaluation;

Arc::new(LlmJudgeEvaluation {
    judge_config: AgentLoopConfig {
        model_config: ModelConfig::anthropic("claude-opus-4-6", "my-key", "claude-opus-4-6"),
        context_config: Some(ContextConfig {
            max_context_tokens: 100_000,
            ..Default::default()
        }),
        ..AgentLoopConfig::default()
    },
    system_prompt: None, // use built-in judge prompt
})
}

agent_loop_continue mode

When prompts is empty, agent_loop_parallel routes each branch to agent_loop_continue instead of agent_loop. This lets you run parallel evaluation from an existing conversation context — the user query is already the last message in base_context.

#![allow(unused)]
fn main() {
// The user query is the last message in context (no new prompts to add).
let result = agent_loop_parallel(
    vec![],          // empty → agent_loop_continue mode
    base_context,    // must be non-empty and not end on an assistant message
    configs,
    strategy,
    tx,
    cancel,
)
.await;
}

Same preconditions as agent_loop_continue apply: base_context.messages must be non-empty and must not end on an assistant message.

original_context_len on ParallelLoopOutcome

Each outcome carries original_context_len: usize — the number of messages in the cloned context at the moment the branch was dispatched:

#![allow(unused)]
fn main() {
pub struct ParallelLoopOutcome {
    // ...
    pub original_context_len: usize,
}
}

context.messages[..original_context_len] is the shared base context all branches started from. Messages at [original_context_len..] are new messages produced by that branch.

Evaluation strategies use this field to extract the original user query and prior conversation history without separate bookkeeping, regardless of whether agent_loop or agent_loop_continue mode was used.

LLM Judge — prompt construction and comprehension criteria

What the judge sees

The judge receives only clean, relevant content:

• Prior conversation context (new): the conversation history before the user query, formatted as a human-readable transcript. Tool call arguments and images are stripped — only Content::Text survives. Omitted from the prompt when empty.
• Original query: text extracted from user messages in prompts (agent_loop mode), or from the last Message::User in context.messages[..original_context_len] (agent_loop_continue mode). Tool calls, images, and thinking are stripped.
• Per-branch response: the text of the last Message::Assistant in each branch's new_messages. Tool calls, tool results, and intermediate multi-turn exchanges are stripped entirely — the judge evaluates outcomes, not reasoning traces.

Example judge prompt (with prior context):

Prior conversation context:
User: What is quantum mechanics?
Assistant: Quantum mechanics is the branch of physics that...

Original query:
Can you explain quantum entanglement in simple terms?

Response 1:
Quantum entanglement is when two particles share a quantum state...

Response 2:
Think of two magic dice...

Which response is best? Reply with ONLY the response number (e.g., "1" or "2").

Query extraction in agent_loop_continue mode

When prompts is empty, the judge cannot read the query directly from the prompts slice. It instead locates the last Message::User in outcome.context.messages[..original_context_len] and extracts its text content. Everything before that message becomes the prior conversation context.

Judge's comprehension criteria

The judge can only make a fair comparison when it sees all N branch final responses simultaneously alongside the prior context and query. For this to work, the combined content must fit within the judge model's context window.

This condition — all content fitting in the judge's context at once — is called the judge's comprehension criteria.

The budget is derived automatically from judge_config.context_config.max_context_tokens (if set). About 20% of the budget is reserved for the system prompt, query framing, and overhead; the remaining 80% is allocated for prior context + branch responses combined.

When no context_config is set on judge_config, no compaction is applied (all content is passed through as-is).

2-iteration compaction strategy

When the combined content exceeds the budget, compaction is applied in two iterations:

Iteration 1 — compact prior context only, outputs intact

The prior conversation context is compacted through 3 progressive tiers while branch outputs are preserved verbatim:

1. Tier 1 — tail truncation: keep only the last 80 lines of the context transcript.
2. Tier 2 — paragraph summary: keep only the first paragraph and last paragraph (separated by ...).
3. Tier 3 — hard char limit: truncate to a per-response char limit derived from the remaining budget, minimum 200 chars. The formula is max(200, (token_budget * 4) / n) where n is the number of texts being compacted and the * 4 factor converts from tokens to chars (1 token ~ 4 chars estimate).

After each tier, the combined token estimate is re-checked. If the budget is satisfied, the judge proceeds with the compacted context and intact outputs.

Iteration 2 — compact both context and outputs independently

If iteration 1 cannot satisfy the budget even at tier 3, the context stays at its most- compacted (tier-3) form and branch outputs are now compacted independently through the same tiered compaction pipeline (legacy compact_messages(); see compaction for the modern CompactionBlock system).

prior context (tier-3)  +  outputs (tier-1 → 2 → 3)  →  check budget after each tier

If the criteria still cannot be satisfied after iteration 2, a ProgressMessage warning is emitted to tx and the judge proceeds best-effort.

Why context is compacted first

Iteration 1 biases the judge towards seeing the complete, uncompacted branch outputs — the actual decision material. Prior conversation history is ancillary; trimming it first preserves the most important information for fair comparison.

Original responses are always preserved

Compaction only affects what the judge reads. The selected_messages field in ParallelLoopResult always contains the original, uncompacted winning branch response.

Setting the judge's context limit

Set judge_config.context_config.max_context_tokens to the judge model's context window size (in tokens). This enables the comprehension-criteria check:

#![allow(unused)]
fn main() {
context_config: Some(ContextConfig {
    max_context_tokens: 200_000, // Claude Opus 4.6 context window
    ..Default::default()
}),
}

Different judge models have different context windows — the limit is co-located with the model config that actually has the constraint.

Design decisions

original_context_len on outcome (not a separate parameter) The EvaluationStrategy trait receives only outcomes and prompts. Embedding original_context_len in each outcome avoids changing the trait signature and keeps all outcome data co-located. Since all branches share the same base context, the value is identical across outcomes — using outcomes[0] is idiomatic.

Same tier functions for context and output compaction compact_tier1/2/3 were designed for document text but work equally well on a formatted conversation transcript. Reusing the same primitives minimises code surface and keeps compaction behaviour consistent.

Budget allocation — context gets priority (iteration 1) Iteration 1 compacts only the prior context, keeping outputs intact. This preserves the complete branch responses — the actual decision material — while trimming ancillary history first. Outputs are only compacted in iteration 2 when the context alone cannot satisfy the budget.

Session identity and loop IDs

All branches share the same session_id for traceability. Each branch gets a distinct loop_id following the format:

{session_id}.{config_segment}.{N}

where config_segment is derived from config.config_id (if set) or auto-derived as {provider}.{model-slug}[.thinking].

Example with two configs:

ses_abc123.anthropic.claude-opus-4-6.1
ses_abc123.anthropic.claude-haiku-4-5.2

The judge loop (if used) also runs in the same session:

ses_abc123.anthropic.claude-opus-4-6.3   ← judge's loop

Observability

Two events bracket the entire parallel execution:

#![allow(unused)]
fn main() {
AgentEvent::ParallelLoopStart {
    session_id: String,
    loop_ids: Vec<String>,   // one per branch, in config order
    timestamp: DateTime<Utc>,
}

AgentEvent::ParallelLoopEnd {
    session_id: String,
    selected_loop_id: String,
    selected_config_index: usize,
    evaluation_usage: Usage,  // judge LLM usage (zero if no judge)
    timestamp: DateTime<Utc>,
}
}

Events from all branches are interleaved in tx. Demultiplex by loop_id from each branch's AgentStart event.

Session continuity

agent_loop_parallel is a single-loop operation. After it returns, call agent_loop_continue on result.selected_context to continue the session:

#![allow(unused)]
fn main() {
let result = agent_loop_parallel(prompts, base_ctx, configs, strategy, tx, cancel).await;

// The session continues normally with the winning branch's context
let follow_up = agent_loop_continue(
    &mut result.selected_context,
    &next_config,
    tx2,
    cancel2,
)
.await;
}

Complete example — multi-model comparison with LLM judge

use phi_core::{
    agent_loop_parallel, agent_loop_continue,
    AgentContext, AgentLoopConfig, AgentMessage, AgentEvent, Message,
};
use phi_core::context::ContextConfig;
use phi_core::LlmJudgeEvaluation;
use phi_core::provider::ModelConfig;
use std::sync::Arc;
use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken;

#[tokio::main]
async fn main() {
    // Branch A: fast, cost-efficient model
    let config_a = AgentLoopConfig {
        model_config: ModelConfig::anthropic("claude-haiku-4-5", API_KEY, "claude-haiku-4-5"),
        ..AgentLoopConfig::default()
    };

    // Branch B: powerful model
    let config_b = AgentLoopConfig {
        model_config: ModelConfig::anthropic("claude-opus-4-6", API_KEY, "claude-opus-4-6"),
        ..AgentLoopConfig::default()
    };

    // Judge: evaluates which response is better
    let judge_config = AgentLoopConfig {
        model_config: ModelConfig::anthropic("claude-opus-4-6", API_KEY, "claude-opus-4-6"),
        context_config: Some(ContextConfig {
            max_context_tokens: 200_000,
            ..Default::default()
        }),
        ..AgentLoopConfig::default()
    };

    let (tx, mut rx) = mpsc::unbounded_channel::<AgentEvent>();
    let cancel = CancellationToken::new();

    let result = agent_loop_parallel(
        vec![AgentMessage::Llm(Message::user("What is the most important physics discovery of the 20th century?"))],
        AgentContext {
            system_prompt: "You are a knowledgeable assistant.".into(),
            ..Default::default()
        },
        vec![config_a, config_b],
        Arc::new(LlmJudgeEvaluation { judge_config, system_prompt: None }),
        tx,
        cancel,
    )
    .await;

    println!("Selected branch: {}", result.selected_index);
    println!("Total tokens used: {}", result.total_usage.total_tokens);

    // Collect and display the winning response
    for msg in &result.selected_messages {
        if let phi_core::AgentMessage::Llm(phi_core::Message::Assistant { content, .. }) = msg {
            for block in content {
                if let phi_core::Content::Text { text } = block {
                    println!("Response: {}", text);
                }
            }
        }
    }

    // Continue the session with the winner
    // let (tx2, _rx2) = mpsc::unbounded_channel();
    // agent_loop_continue(&mut result.selected_context, &next_config, tx2, cancel2).await;
}

Custom evaluation strategies

Implement EvaluationStrategy for custom evaluation logic:

#![allow(unused)]
fn main() {
use phi_core::{AgentEvent, AgentMessage, ParallelLoopOutcome, Usage};
use phi_core::{EvaluationDecision, EvaluationStrategy};
use async_trait::async_trait;
use tokio::sync::mpsc;
use tokio_util::sync::CancellationToken;

struct LongestResponseEvaluation;

#[async_trait::async_trait]
impl EvaluationStrategy for LongestResponseEvaluation {
    async fn evaluate(
        &self,
        _prompts: &[AgentMessage],
        outcomes: &[ParallelLoopOutcome],
        _tx: &mpsc::UnboundedSender<AgentEvent>,
        _cancel: CancellationToken,
    ) -> (EvaluationDecision, Usage) {
        let idx = outcomes
            .iter()
            .enumerate()
            .max_by_key(|(_, o)| {
                // Sum all text content lengths across new messages
                o.new_messages.iter().filter_map(|m| m.as_llm()).flat_map(|msg| {
                    if let phi_core::Message::Assistant { content, .. } = msg {
                        content.iter().filter_map(|c| {
                            if let phi_core::Content::Text { text } = c { Some(text.len()) } else { None }
                        }).collect::<Vec<_>>()
                    } else {
                        vec![]
                    }
                }).sum::<usize>()
            })
            .map(|(i, _)| i)
            .unwrap_or(0);
        (EvaluationDecision::Select(idx), Usage::default())
    }
}
}

Messages & Events

Message Types

Message

The core LLM message type, tagged by role:

#![allow(unused)]
fn main() {
pub enum Message {
    User {
        content: Vec<Content>,
        timestamp: u64,
    },
    Assistant {
        content: Vec<Content>,
        stop_reason: StopReason,
        model: String,
        provider: String,
        usage: Usage,
        timestamp: u64,
        error_message: Option<String>,
    },
    ToolResult {
        tool_call_id: String,
        tool_name: String,
        content: Vec<Content>,
        is_error: bool,
        timestamp: u64,
        child_loop_id: Option<String>,  // set by sub-agent tools
    },
}
}

Create user messages easily:

#![allow(unused)]
fn main() {
let msg = Message::user("Hello, world!");
}

AgentMessage

Wraps Message with support for extension messages (UI-only, notifications, etc.):

#![allow(unused)]
fn main() {
pub enum AgentMessage {
    Llm(LlmMessage),
    Extension(ExtensionMessage),
}

pub struct LlmMessage {
    pub message: Message,
    /// Which turn produced this message. `None` for messages that predate
    /// turn tracking or are created outside the agent loop.
    pub turn_id: Option<TurnId>,
}

pub struct ExtensionMessage {
    pub role: String,
    pub kind: String,
    pub data: serde_json::Value,
}
}

Create extension messages with the convenience constructor:

#![allow(unused)]
fn main() {
let ext = ExtensionMessage::new("status_update", serde_json::json!({"status": "running"}));
let msg = AgentMessage::Extension(ext);
}

The kind field categorizes the extension (e.g., "status_update", "ui_event", "notification"). Use as_llm() to extract the Message if it's an LLM message. LlmMessage wraps a Message with an optional TurnId { loop_id, turn_index } for compaction tracking — this allows the compaction system to identify which turn produced each message. The default convert_to_llm function filters out Extension messages before sending to the provider.

All core message types implement Serialize, Deserialize, Clone, and PartialEq, enabling state persistence and test assertions.

Content

Each message contains Vec<Content>:

#![allow(unused)]
fn main() {
pub enum Content {
    Text { text: String },
    Image { data: String, mime_type: String },
    Thinking { thinking: String, signature: Option<String> },
    ToolCall { id: String, name: String, arguments: serde_json::Value },
}
}

An assistant message can contain multiple content blocks — e.g., thinking + text + tool calls.

The signature field on Content::Thinking is a cryptographic integrity token issued by the LLM provider (Anthropic calls it signature, OpenAI calls it encrypted_content, Gemini calls it thought_signature). It must be echoed back unmodified in multi-turn conversations — tampering or omitting it causes the provider to reject the request. It is None on providers that don't support extended thinking or on the first-turn generation.

StopReason

#![allow(unused)]
fn main() {
pub enum StopReason {
    Stop,              // Natural completion
    Length,            // Hit max tokens
    ToolUse,           // Wants to call tools
    Error,             // Provider error
    Aborted,           // Cancelled by user
    MaxTurns,          // Reached maximum allowed turns
    UserStop,          // Explicit user stop command
    Handoff,           // Handing off to a human operator
    GuardRail,         // Stopped by content moderation / safety filter
    ContextCompacted,  // Context was compacted to fit within limits
    Paused,            // Paused waiting for external input
}
}

Usage

Token usage from the provider:

#![allow(unused)]
fn main() {
pub struct Usage {
    pub input: u64,
    pub output: u64,
    pub cache_read: u64,
    pub cache_write: u64,
    pub total_tokens: u64,
}
}

AgentEvent

Events emitted during the agent loop for real-time UI updates:

StreamDelta

Deltas within MessageUpdate:

#![allow(unused)]
fn main() {
pub enum StreamDelta {
    Text { delta: String },
    Thinking { delta: String },
    ToolCallDelta { delta: String },
}
}

Agent State

The Agent struct provides access to its current state:

#![allow(unused)]
fn main() {
// Check if the agent is currently streaming a response
if agent.is_streaming() {
    // Use steer() or follow_up() instead of prompt()
    agent.steer(AgentMessage::Llm(Message::user("New instruction")));
}

// Access the full message history
let messages: &[AgentMessage] = agent.messages();

// Check the last message
if let Some(last) = messages.last() {
    println!("Last message role: {}", last.role());
}
}

The is_streaming() flag is true between prompt()/continue_loop() call and completion. While streaming, calling prompt() will panic — use steer() or follow_up() instead.

Tools

The AgentTool Trait

Every tool implements AgentTool:

#![allow(unused)]
fn main() {
#[async_trait]
pub trait AgentTool: Send + Sync {
    fn name(&self) -> &str;
    fn label(&self) -> &str;
    fn description(&self) -> &str;
    fn parameters_schema(&self) -> serde_json::Value;
    async fn execute(
        &self,
        params: serde_json::Value,
        ctx: ToolContext,
    ) -> Result<ToolResult, ToolError>;
}
}

ToolContext

All execution context is bundled into a single struct, making the trait easier to extend in the future:

#![allow(unused)]
fn main() {
pub struct ToolContext {
    pub tool_call_id: String,
    pub tool_name: String,
    pub cancel: CancellationToken,
    pub on_update: Option<ToolUpdateFn>,
    pub on_progress: Option<ProgressFn>,
}
}

ToolContext implements Clone and Debug.

ToolResult

#![allow(unused)]
fn main() {
pub struct ToolResult {
    pub content: Vec<Content>,
    pub details: serde_json::Value,
}
}

The content is sent back to the LLM. The details field holds metadata (not sent to the LLM) for UI/logging.

ToolError

#![allow(unused)]
fn main() {
pub enum ToolError {
    Failed(String),
    NotFound(String),
    InvalidArgs(String),
    Cancelled,
}
}

Errors are converted to ToolResult with is_error: true and sent back to the LLM so it can recover.

Implementing a Custom Tool

#![allow(unused)]
fn main() {
use phi_core::types::*;
use async_trait::async_trait;

pub struct WeatherTool;

#[async_trait]
impl AgentTool for WeatherTool {
    fn name(&self) -> &str { "get_weather" }
    fn label(&self) -> &str { "Weather" }
    fn description(&self) -> &str {
        "Get current weather for a city."
    }

    fn parameters_schema(&self) -> serde_json::Value {
        serde_json::json!({
            "type": "object",
            "properties": {
                "city": {
                    "type": "string",
                    "description": "City name"
                }
            },
            "required": ["city"]
        })
    }

    async fn execute(
        &self,
        params: serde_json::Value,
        _ctx: ToolContext,
    ) -> Result<ToolResult, ToolError> {
        let city = params["city"].as_str()
            .ok_or(ToolError::InvalidArgs("missing city".into()))?;

        // Call weather API...
        Ok(ToolResult {
            content: vec![Content::Text {
                text: format!("Weather in {}: 72°F, sunny", city),
            }],
            details: serde_json::Value::Null,
        })
    }
}
}

Register custom tools alongside defaults:

#![allow(unused)]
fn main() {
use phi_core::tools::default_tools;

let mut tools = default_tools();
tools.push(Box::new(WeatherTool));
let agent = BasicAgent::new(model_config).with_tools(tools);
}

Error Handling

Return Err(ToolError) on failure, not Ok with error text. When a tool returns Err, the agent loop converts it to a Message::ToolResult with is_error: true and sends it to the LLM. The LLM sees the error and can self-correct — retry with different arguments, try a different approach, or explain the failure to the user.

#![allow(unused)]
fn main() {
async fn execute(&self, params: serde_json::Value, _ctx: ToolContext) -> Result<ToolResult, ToolError> {
    let path = params["path"].as_str()
        .ok_or(ToolError::InvalidArgs("missing 'path'".into()))?;

    let content = std::fs::read_to_string(path)
        .map_err(|e| ToolError::Failed(format!("Cannot read {}: {}", path, e)))?;

    Ok(ToolResult {
        content: vec![Content::Text { text: content }],
        details: serde_json::Value::Null,
    })
}
}

Exception: BashTool. The built-in BashTool returns Ok even on non-zero exit codes, with both stdout and stderr in the result. This is intentional — the LLM needs to see the actual error output (compilation errors, test failures, etc.) to diagnose and fix issues. Only truly exceptional failures (e.g., command not found, cancellation) return Err.

Tool Execution Flow

1. LLM returns Content::ToolCall blocks in its response
2. Agent loop emits ToolExecutionStart for each
3. Tool's execute() is called with parsed arguments
4. Result (or error) is wrapped in Message::ToolResult
5. ToolExecutionEnd is emitted
6. All tool results are added to context
7. Loop continues with another LLM call

Streaming Tool Output

Long-running tools can stream progress updates to the UI via the on_update callback. Each call emits a ToolExecutionUpdate event. Partial results are for UI/logging only — they are not sent to the LLM. Only the final ToolResult returned from execute() becomes part of the conversation.

The ToolUpdateFn type

#![allow(unused)]
fn main() {
pub type ToolUpdateFn = Arc<dyn Fn(ToolResult) + Send + Sync>;
}

Basic usage

Call on_update whenever you have progress to report:

#![allow(unused)]
fn main() {
use phi_core::types::*;

struct DataProcessorTool;

#[async_trait]
impl AgentTool for DataProcessorTool {
    // ... name, label, description, parameters_schema ...

    async fn execute(
        &self,
        params: serde_json::Value,
        ctx: ToolContext,
    ) -> Result<ToolResult, ToolError> {
        let rows = fetch_rows(&params)?;
        let total = rows.len();

        for (i, row) in rows.iter().enumerate() {
            // Check for cancellation
            if ctx.cancel.is_cancelled() {
                return Err(ToolError::Cancelled);
            }

            process_row(row);

            // Stream progress every 100 rows
            if i % 100 == 0 {
                if let Some(ref cb) = &ctx.on_update {
                    cb(ToolResult {
                        content: vec![Content::Text {
                            text: format!("Processed {}/{} rows", i, total),
                        }],
                        details: serde_json::json!({"progress": i as f64 / total as f64}),
                    });
                }
            }
        }

        Ok(ToolResult {
            content: vec![Content::Text {
                text: format!("Processed all {} rows", total),
            }],
            details: serde_json::Value::Null,
        })
    }
}
}

Consuming updates in your UI

Updates arrive as AgentEvent::ToolExecutionUpdate events on the same event stream as all other agent events:

#![allow(unused)]
fn main() {
while let Some(event) = rx.recv().await {
    match event {
        AgentEvent::ToolExecutionStart { tool_name, .. } => {
            println!("⏳ {} started", tool_name);
        }
        AgentEvent::ToolExecutionUpdate { tool_name, partial_result, .. } => {
            // Show progress in your UI
            if let Some(Content::Text { text }) = partial_result.content.first() {
                println!("  📊 {}: {}", tool_name, text);
            }
        }
        AgentEvent::ToolExecutionEnd { tool_name, is_error, .. } => {
            println!("{} {}", if is_error { "❌" } else { "✅" }, tool_name);
        }
        AgentEvent::ProgressMessage { tool_name, text, .. } => {
            println!("  💬 {}: {}", tool_name, text);
        }
        _ => {}
    }
}
}

Progress Messages

In addition to on_update (which streams partial ToolResult values), tools can emit lightweight text-only progress messages via ctx.on_progress. These appear as AgentEvent::ProgressMessage events:

#![allow(unused)]
fn main() {
async fn execute(&self, params: serde_json::Value, ctx: ToolContext) -> Result<ToolResult, ToolError> {
    if let Some(ref progress) = &ctx.on_progress {
        progress("Starting analysis...".into());
    }

    // ... do work ...

    if let Some(ref progress) = &ctx.on_progress {
        progress("Almost done...".into());
    }

    Ok(ToolResult { /* ... */ })
}
}

Use on_progress for simple status text. Use on_update when you need structured data (progress percentages, partial results).

Guidelines

• Call on_update as often as useful — there's no rate limit. The callback is synchronous and cheap.
• Always check ctx.on_update.is_some() before building the ToolResult. If None, the loop isn't interested in updates (e.g., testing).
• Use details for structured data — content is for human-readable text, details can carry progress percentages, byte counts, etc.
• Don't rely on updates reaching the LLM — they won't. Only the final return value is added to context.
• Simple tools don't need it — if your tool completes in <1 second, just ignore ctx (prefix with _ctx to suppress the warning).

End-to-end example

Here's a complete example: a CLI agent with a deploy tool that streams progress. The human sees real-time output while the LLM only gets the final result.

use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;
use phi_core::types::*;

/// A tool that deploys an app and streams each step.
struct DeployTool;

#[async_trait]
impl AgentTool for DeployTool {
    fn name(&self) -> &str { "deploy" }
    fn label(&self) -> &str { "Deploy App" }
    fn description(&self) -> &str { "Deploy the application to production." }
    fn parameters_schema(&self) -> serde_json::Value {
        serde_json::json!({
            "type": "object",
            "properties": {
                "env": { "type": "string", "description": "Target environment" }
            },
            "required": ["env"]
        })
    }

    async fn execute(
        &self,
        params: serde_json::Value,
        ctx: ToolContext,
    ) -> Result<ToolResult, ToolError> {
        let env = params["env"].as_str().unwrap_or("staging");

        let steps = ["Building image", "Running tests", "Pushing to registry", "Rolling out"];
        for (i, step) in steps.iter().enumerate() {
            if ctx.cancel.is_cancelled() {
                return Err(ToolError::Cancelled);
            }

            // Stream each step to the UI
            if let Some(ref cb) = &ctx.on_update {
                cb(ToolResult {
                    content: vec![Content::Text {
                        text: format!("[{}/{}] {}...", i + 1, steps.len(), step),
                    }],
                    details: serde_json::json!({
                        "step": i + 1,
                        "total": steps.len(),
                        "phase": step,
                    }),
                });
            }

            // Simulate work
            tokio::time::sleep(std::time::Duration::from_secs(2)).await;
        }

        // Only this final result is sent to the LLM
        Ok(ToolResult {
            content: vec![Content::Text {
                text: format!("Successfully deployed to {}", env),
            }],
            details: serde_json::json!({"env": env, "status": "success"}),
        })
    }
}

#[tokio::main]
async fn main() {
    let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
    let mut agent = BasicAgent::new(ModelConfig::anthropic(
        "claude-sonnet-4-20250514",
        "Claude Sonnet 4",
        &api_key,
    ))
    .with_system_prompt("You are a deployment assistant.")
    .with_tools(vec![Box::new(DeployTool)]);

    let mut rx = agent.prompt("Deploy to production").await;

    while let Some(event) = rx.recv().await {
        match event {
            // LLM text streaming
            AgentEvent::MessageUpdate {
                delta: StreamDelta::Text { delta }, ..
            } => print!("{}", delta),

            // Tool progress streaming
            AgentEvent::ToolExecutionStart { tool_name, .. } => {
                println!("\n🚀 Starting {}...", tool_name);
            }
            AgentEvent::ToolExecutionUpdate { partial_result, .. } => {
                if let Some(Content::Text { text }) = partial_result.content.first() {
                    println!("  {}", text);
                }
            }
            AgentEvent::ToolExecutionEnd { tool_name, is_error, .. } => {
                if is_error {
                    println!("  ❌ {} failed", tool_name);
                } else {
                    println!("  ✅ {} complete", tool_name);
                }
            }
            AgentEvent::ProgressMessage { text, .. } => {
                println!("  💬 {}", text);
            }

            AgentEvent::AgentEnd { .. } => break,
            _ => {}
        }
    }
}

Running this produces:

🚀 Starting deploy...
  [1/4] Building image...
  [2/4] Running tests...
  [3/4] Pushing to registry...
  [4/4] Rolling out...
  ✅ deploy complete
Successfully deployed to production. The deployment completed all 4 stages.

The human sees each step as it happens. The LLM only sees "Successfully deployed to production" and can continue the conversation from there.

How agents benefit

When an AI agent (like a coding assistant) uses phi-core, streaming tool output helps in two ways:

1. Human oversight — The human watching the agent work sees real-time progress instead of waiting for a tool to finish. A bash command running cargo build can stream compiler output as it happens, so the human can interrupt early if something is wrong.

2. Agent UIs — Tools like web dashboards, IDE extensions, or chat interfaces can render live progress bars, log tails, or status indicators. The details field in ToolResult carries structured data (progress percentage, byte counts, etc.) that UIs can render however they want.

The LLM itself doesn't see updates — it works with final results only. This is intentional: partial output would waste context tokens and confuse the model. The streaming is purely a human-facing feature.

Execution Strategies

When the LLM returns multiple tool calls in a single response (e.g., "read file A, read file B, run bash C"), ToolExecutionStrategy controls how they run:

The enum is defined with #[derive(Default)] and Parallel carries the #[default] attribute:

#![allow(unused)]
fn main() {
pub enum ToolExecutionStrategy {
    Sequential,
    #[default]
    Parallel,
    Batched { size: usize },
}
}

Configuration

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;
use phi_core::types::ToolExecutionStrategy;

// Default — parallel (fastest)
let agent = BasicAgent::new(model_config.clone());

// Sequential (debug / shared state)
let agent = BasicAgent::new(model_config.clone())
    .with_tool_execution(ToolExecutionStrategy::Sequential);

// Batched — 3 at a time
let agent = BasicAgent::new(model_config.clone())
    .with_tool_execution(ToolExecutionStrategy::Batched { size: 3 });
}

When to use each

• Parallel (default): Most tool calls are independent — file reads, searches, API calls. Running them concurrently can cut latency dramatically (3 tools × 50ms = ~50ms instead of ~150ms).
• Sequential: When tools have side effects that depend on order, or when you need fine-grained steering control between each tool.
• Batched: When you want parallelism but also want steering checkpoints. For example, Batched { size: 3 } runs 3 tools concurrently, checks for user interrupts, then runs the next 3.

Steering messages are always checked between execution units (between each tool in Sequential, after all tools in Parallel, between batches in Batched). If a user interrupts, remaining tools are skipped.

Context Management

Long-running agents accumulate messages that exceed the model's context window. phi-core provides token tracking, overflow detection, tiered compaction, and execution limits.

The context module is split into sub-modules: token, config, tracker, compaction, strategy, compact_messages, execution, orchestration.

Token Estimation

Fast estimation without external tokenizer dependencies:

#![allow(unused)]
fn main() {
use phi_core::context::{estimate_tokens, message_tokens, total_tokens};

estimate_tokens("Hello world");          // ~3 tokens (chars / 4)
message_tokens(&agent_message);          // estimate for a single message
total_tokens(&messages);                 // estimate for all messages
}

Context Tracking

ContextTracker combines real token counts from provider responses with estimation for new messages — more accurate than pure estimation:

#![allow(unused)]
fn main() {
use phi_core::context::ContextTracker;

let mut tracker = ContextTracker::new();

// After each assistant response, record the real usage:
tracker.record_usage(&assistant_usage, message_index);

// Get current context size (real usage + estimated trailing):
let tokens = tracker.estimate_context_tokens(agent.messages());

// After compaction, reset the tracker:
tracker.reset();
}

When no usage data is available, it falls back to chars/4 estimation.

Context Overflow Detection

When the context exceeds a model's window, providers return overflow errors. phi-core detects these automatically across all major providers.

HTTP-level detection

Providers that check before streaming (Google, Bedrock, Vertex) return ProviderError::ContextOverflow:

#![allow(unused)]
fn main() {
use phi_core::provider::ProviderError;

match agent.prompt("...").await {
    // The loop already handles this — but you can also match it:
    Err(ProviderError::ContextOverflow { message }) => {
        // Compact and retry
    }
    _ => {}
}
}

ProviderError::classify() auto-detects overflow from error messages covering Anthropic, OpenAI, Google, AWS Bedrock, xAI, Groq, OpenRouter, llama.cpp, LM Studio, MiniMax, Kimi, GitHub Copilot, and generic patterns.

Message-level detection

SSE-based providers (Anthropic, OpenAI) return overflow as a StopReason::Error message. Check with:

#![allow(unused)]
fn main() {
if message.is_context_overflow() {
    // Compact and retry
}
}

Handling overflow in your application

phi-core provides the detection and building blocks. Your application wires the compaction strategy:

#![allow(unused)]
fn main() {
// Proactive: check before each prompt
let tokens = tracker.estimate_context_tokens(agent.messages());
if tokens > context_window - reserve {
    let compacted = compact_messages(agent.messages().to_vec(), &config);
    agent.replace_messages(compacted);
}

// Reactive: catch overflow errors
// ... on ContextOverflow or message.is_context_overflow():
//   compact, then retry with agent.continue_loop()
}

For LLM-based summarization (asking the model to summarize old messages), implement that in your application layer — phi-core provides replace_messages() and compact_messages() as building blocks.

ContextConfig

#![allow(unused)]
fn main() {
pub struct ContextConfig {
    pub max_context_tokens: usize,      // Default: 100,000
    pub system_prompt_tokens: usize,    // Default: 4,000
    pub compaction: CompactionConfig,   // Primary compaction settings

    // Custom token counter (serde-skipped). None → HeuristicTokenCounter (chars/4).
    pub token_counter: Option<Arc<dyn TokenCounter>>,

    // Legacy backward-compat fields (prefer CompactionConfig equivalents):
    pub keep_recent: usize,             // Default: 10
    pub keep_first: usize,             // Default: 2
    pub tool_output_max_lines: usize,  // Default: 50
}

pub struct CompactionConfig {
    // ── WHEN to compact ──
    pub compact_at_pct: f64,                     // Default: 0.90 (90%)
    pub compact_budget_threshold_pct: f64,       // Default: 0.05 (5%)
    pub compaction_scope: CompactionScope,       // Default: FixedCount(3)

    // ── HOW to compact ──
    pub keep_first_turns: usize,                 // Default: 2
    pub keep_recent_turns: usize,                // Default: 10
    pub max_summary_tokens: usize,               // Default: 2_000 (budget, not per-turn)
    pub tool_output_max_lines: usize,            // Default: 50
}
}

CompactionScope

Controls how many earlier loops are included in compaction and context loading:

See compaction.md for full details on the non-destructive overlay model.

Tiered Compaction

compact_messages() tries each level in order, stopping as soon as messages fit the budget:

Level 1: Truncate Tool Outputs

Replaces long tool outputs with head + tail (keeping first N/2 and last N/2 lines). This is the cheapest — preserves conversation structure, typically saves 50-70% in coding sessions.

Level 2: Summarize Old Turns

Keeps the last keep_recent messages in full detail. Older assistant messages are replaced with one-line summaries like "[Summary] [Assistant used 3 tool(s)]", and their tool results are dropped.

Level 3: Drop Middle Messages

Keeps keep_first messages from the start and keep_recent from the end, dropping everything in between. A marker message notes how many were removed.

ExecutionLimits

Prevents runaway agents:

#![allow(unused)]
fn main() {
pub struct ExecutionLimits {
    pub max_turns: usize,              // Default: 50
    pub max_total_tokens: usize,       // Default: 1,000,000
    pub max_duration: Duration,        // Default: 600s (10 min)
    pub max_cost: Option<f64>,         // Default: None (no cost cap)
}
}

max_cost caps cumulative dollar cost for the run. Requires AgentLoopConfig.cost_config to be set — without pricing rates the accumulated cost is always 0.0 and this limit has no effect.

When a limit is reached, the agent stops with a message like "[Agent stopped: Max turns reached (50/50)]".

Disabling Context Management

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(model_config)
    .without_context_management();
}

This sets both context_config and execution_limits to None.

Prompt Caching

phi-core automatically optimizes API costs through prompt caching. For providers that support it, stable content (system prompts, tool definitions, conversation history) is cached between turns, giving you up to 90% savings on input tokens.

How It Works

In a multi-turn agent loop, each request sends the full context: system prompt + tools + conversation history. Without caching, you pay full price for all of it every turn. With caching, the provider reuses previously processed prefixes.

Provider Support

What Gets Cached (Anthropic)

phi-core places up to 3 cache breakpoints automatically:

1. System prompt — stable across all turns
2. Tool definitions — rarely change between turns
3. Conversation history — second-to-last message, so the growing prefix is cached

This means on a typical multi-turn conversation, only the latest user message and the new assistant response cost full price.

Configuration

Caching is enabled by default with automatic breakpoint placement. No configuration needed for optimal behavior.

Disable Caching

Use CacheStrategy::Disabled to turn off all cache breakpoint placement while keeping the config structure intact. Alternatively, set enabled: false on the CacheConfig master switch.

#![allow(unused)]
fn main() {
use phi_core::{BasicAgent, CacheConfig, CacheStrategy};
use phi_core::provider::ModelConfig;

let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();

// Option 1: CacheStrategy::Disabled (preferred — explicit intent)
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_cache_config(CacheConfig {
    strategy: CacheStrategy::Disabled,
    ..Default::default()
});

// Option 2: Master switch (equivalent effect)
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_cache_config(CacheConfig {
    enabled: false,
    ..Default::default()
});
}

Fine-Grained Control

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_cache_config(CacheConfig {
    enabled: true,
    strategy: CacheStrategy::Manual {
        cache_system: true,
        cache_tools: true,
        cache_messages: false, // Don't cache conversation history
    },
});
}

Monitoring Cache Usage

Every Usage struct includes cache statistics:

#![allow(unused)]
fn main() {
// After a response:
let usage = message.usage(); // from assistant message
println!("Cache read: {} tokens", usage.cache_read);
println!("Cache write: {} tokens", usage.cache_write);
println!("Cache hit rate: {:.1}%", usage.cache_hit_rate() * 100.0);
}

• cache_read — tokens served from cache (cheap)
• cache_write — tokens written to cache (slightly more than base price)
• cache_hit_rate() — fraction of input tokens from cache (0.0–1.0)

Cost Impact

For a typical 10-turn agent conversation with Anthropic Claude:

That's an 84% cost reduction with zero configuration.

Best Practices

1. Keep system prompts stable — changing the system prompt between turns invalidates the cache
2. Don't shuffle tools — tool order matters for cache prefix matching
3. Let it work automatically — the default CacheStrategy::Auto is optimal for most use cases. The three strategies are Auto (recommended), Disabled (no breakpoints), and Manual (fine-grained control)
4. Monitor cache_hit_rate() — if it's consistently low, check if your system prompt or tools are changing unexpectedly

Retry with Backoff

When an LLM provider returns a transient error — rate limit (HTTP 429) or network failure — phi-core automatically retries with exponential backoff and jitter. No configuration required; it works out of the box.

How it works

Request → Error? → Retryable? → Wait (backoff + jitter) → Retry → ...
                       ↓ No
                  Fail immediately

1. The agent loop calls the provider
2. If the provider returns a retryable error:
   • If a retry-after delay was provided (rate limits), use that
   • Otherwise, calculate delay: initial_delay × multiplier^(attempt-1) with ±20% jitter
   • Wait, then retry
3. After max_retries attempts, the error propagates normally

What gets retried

Default configuration

#![allow(unused)]
fn main() {
RetryConfig {
    max_retries: 3,          // Up to 3 retry attempts
    initial_delay_ms: 1000,  // 1 second before first retry
    backoff_multiplier: 2.0, // Double the delay each attempt
    max_delay_ms: 30_000,    // Cap at 30 seconds
}
}

With defaults, the retry delays are approximately:

• Attempt 1: ~1s
• Attempt 2: ~2s
• Attempt 3: ~4s

(±20% jitter to avoid thundering herd when multiple agents hit the same provider)

Configuration

Using the Agent builder

#![allow(unused)]
fn main() {
use phi_core::{BasicAgent, RetryConfig};
use phi_core::provider::ModelConfig;

let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();

// Default — 3 retries, exponential backoff (recommended)
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
));

// Custom — more retries, longer initial delay
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_retry_config(RetryConfig {
    max_retries: 5,
    initial_delay_ms: 2000,
    backoff_multiplier: 2.0,
    max_delay_ms: 60_000,
});

// Disable retries entirely
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_retry_config(RetryConfig::none());
}

Using AgentLoopConfig directly

#![allow(unused)]
fn main() {
use phi_core::agent_loop::AgentLoopConfig;
use phi_core::RetryConfig;

let config = AgentLoopConfig {
    // ...other fields...
    retry_config: RetryConfig {
        max_retries: 3,
        initial_delay_ms: 1000,
        backoff_multiplier: 2.0,
        max_delay_ms: 30_000,
    },
    ..Default::default()
};
}

Rate limit headers

When a provider returns ProviderError::RateLimited { retry_after_ms: Some(5000) }, phi-core uses that exact delay instead of the calculated backoff. This respects the provider's guidance — if Anthropic says "retry after 5 seconds", we wait 5 seconds, not our own estimate.

If no retry_after_ms is provided, the exponential backoff kicks in.

Observability

Retry attempts are logged via tracing at the WARN level:

WARN Provider error (attempt 1/3), retrying in 1.1s: Rate limited, retry after 1000ms
WARN Provider error (attempt 2/3), retrying in 2.3s: Rate limited, retry after 2000ms

Subscribe to tracing events in your application to surface these in your UI:

#![allow(unused)]
fn main() {
use tracing_subscriber;

// Simple stderr logging
tracing_subscriber::fmt::init();

// Or filter to just retries
tracing_subscriber::fmt()
    .with_env_filter("phi_core::provider::retry=warn")
    .init();
}

Design notes

• Retry lives in the agent loop, not inside individual providers. One config controls all retry behavior.
• Jitter prevents thundering herd: when many agents hit a rate limit simultaneously, jitter spreads their retries so they don't all retry at the same instant.
• Cancellation is respected: if the user cancels while waiting for a retry, the loop exits immediately.
• No retry on API errors: a malformed request will fail the same way every time. Retrying wastes time and tokens.

Skills

Skills extend an agent with domain expertise using the AgentSkills open standard. A skill is a directory containing a SKILL.md file with instructions the agent can load on demand.

How it works

Skills use progressive disclosure to manage context efficiently:

1. Metadata (~100 tokens/skill) — name + description, always in the system prompt
2. Instructions (<5k tokens) — SKILL.md body, loaded when the agent decides the skill is relevant
3. Resources (unlimited) — scripts, references, assets — loaded only when needed

The agent decides when to activate a skill based on the description alone. No trigger engine needed.

Skill format

my-skill/
├── SKILL.md          # Required: YAML frontmatter + instructions
├── scripts/          # Optional: executable code
├── references/       # Optional: documentation loaded on demand
└── assets/           # Optional: templates, static resources

SKILL.md uses YAML frontmatter:

---
name: git
description: Git operations — commit, branch, merge, rebase. Use when the user mentions version control.
---

# Git Skill

## Workflow
1. Run `git status` first
2. Stage changes, write conventional commit messages
3. For merges, check for conflicts first

## Scripts
For complex diffs: `bash {baseDir}/scripts/diff_summary.sh`

Loading skills

#![allow(unused)]
fn main() {
use phi_core::SkillSet;
use std::path::PathBuf;

// Load from multiple directories (later dirs override earlier on name conflict)
let skills = SkillSet::load(&[PathBuf::from("./skills"), PathBuf::from("~/.phi-core/skills")]);

// Or load from a single directory with a label
let workspace_skills = SkillSet::load_dir("./skills", "workspace");
}

Using with Agent

#![allow(unused)]
fn main() {
use phi_core::{BasicAgent, SkillSet};
use phi_core::provider::ModelConfig;
use std::path::PathBuf;

let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
let skills = SkillSet::load(&[PathBuf::from("./skills")]);

let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_system_prompt("You are a coding assistant.")
.with_skills(skills)  // Appends skill index to system prompt
.with_tools(tools);
}

The agent's system prompt will include:

<available_skills>
  <skill>
    <name>git</name>
    <description>Git operations — commit, branch, merge, rebase.</description>
    <location>/path/to/skills/git/SKILL.md</location>
  </skill>
</available_skills>

When the agent encounters a task matching a skill, it reads the SKILL.md using the read_file tool and follows the instructions. No special infrastructure needed.

Precedence

When loading from multiple directories, later directories take precedence. A skill in ./skills/ overrides the same-named skill in ~/.phi-core/skills/.

You can also merge skill sets explicitly:

#![allow(unused)]
fn main() {
let mut base = SkillSet::load_dir("/usr/share/phi-core/skills", "bundled")?;
let user = SkillSet::load_dir("~/.phi-core/skills", "user")?;
let workspace = SkillSet::load_dir("./skills", "workspace")?;

base.merge(user);
base.merge(workspace); // workspace wins on conflict
}

Compatibility

By following the AgentSkills standard, skills written for phi-core work with Claude Code, Codex CLI, Gemini CLI, Cursor, OpenCode, Goose, and any other compatible agent. Write once, use everywhere.

Design philosophy

Skills are deliberately simple:

• No trigger engine — the LLM decides from descriptions
• No compile-time registration — skills use existing tools (read_file, bash)
• No plugin API — skills are just files
• No runtime loading — loaded at startup, that's it

If a skill needs a custom tool, it can provide an MCP server.

Sub-Agents

Sub-agents let a parent agent delegate tasks to child agent loops, each with their own system prompt, tools, and ModelConfig. The parent LLM invokes them like any other tool.

Overview

Parent Agent
├── prompt("Research X and implement Y")
│   ├── calls SubAgentTool("researcher", task="Research X")
│   │   └── child agent_loop() with read/search tools → returns findings
│   ├── calls SubAgentTool("coder", task="Implement Y based on findings")
│   │   └── child agent_loop() with edit/write tools → returns result
│   └── summarizes both results

Each sub-agent invocation starts a fresh conversation — no state leaks between calls.

Creating Sub-Agents

#![allow(unused)]
fn main() {
use phi_core::agents::SubAgentTool;
use phi_core::provider::ModelConfig;
use phi_core::tools;
use std::sync::Arc;

let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();

let researcher = SubAgentTool::new(
    "researcher",
    ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key),
)
.with_description("Searches and reads files to gather information.")
.with_system_prompt("You are a research assistant. Be thorough and concise.")
.with_tools(vec![
    Arc::new(tools::ReadFileTool::new()),
    Arc::new(tools::SearchTool::new()),
])
.with_max_turns(10);
}

Registering on a Parent Agent

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;

let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
let mut agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_system_prompt("You coordinate between sub-agents.")
.with_sub_agent(researcher)
.with_sub_agent(coder);
}

The parent sees sub-agents as regular tools. It decides when to delegate based on its system prompt.

Parallel Execution

When the parent LLM calls multiple sub-agents in a single response, they run concurrently (default Parallel strategy). Two sub-agents each taking 50ms complete in ~50ms total, not 100ms.

Configuration

Event Forwarding

When the parent provides an on_update callback (standard for all tools), sub-agent events are forwarded as ToolExecutionUpdate events. The parent's UI sees real-time progress from the child:

• Text deltas from the sub-agent's LLM responses
• Tool call notifications from the sub-agent's tool usage

When the child loop completes, the parent emits ToolExecutionEnd with child_loop_id: Some(loop_id) set to the child's loop_id. This lets you correlate ToolExecutionEnd on the parent side with AgentStart/AgentEnd on the child side when both event streams are consumed.

Design Decisions

• Context isolation: Each invocation starts fresh. Sub-agents don't accumulate history across calls.
• No nesting: Sub-agents are not given other SubAgentTools. This prevents infinite delegation chains.
• Cancellation propagation: The parent's cancellation token is forwarded. Aborting the parent aborts all sub-agents.
• Turn limiting: The default 10-turn limit prevents runaway execution. The parent's execution limits also apply to total wall-clock time.

Example

See examples/sub_agent.rs for a complete coordinator with researcher and coder sub-agents.

State Persistence

phi-core supports saving and restoring agent conversation state, enabling pause/resume workflows, state transfer between processes, and conversation checkpointing.

Save and Restore

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;

// After running some conversation turns...
let json = agent.save_messages();
std::fs::write("conversation.json", &json)?;

// Later, in a new process:
let json = std::fs::read_to_string("conversation.json")?;
let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
let mut agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
))
.with_system_prompt("You are helpful.");

agent.restore_messages(&json)?;

// Continue the conversation — the agent sees the full history
let rx = agent.prompt("Follow up question").await;
}

Builder Initialization

For constructing an agent with pre-existing history:

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;

let saved: Vec<AgentMessage> = serde_json::from_str(&json)?;
let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .with_messages(saved)
    .with_system_prompt("...");
}

JSON Format

Messages serialize as a JSON array. Each message is tagged by role:

[
  {
    "role": "user",
    "content": [{"type": "text", "text": "Hello"}],
    "timestamp": 1700000000000
  },
  {
    "role": "assistant",
    "content": [{"type": "text", "text": "Hi there!"}],
    "stopReason": "stop",
    "model": "claude-sonnet-4-20250514",
    "provider": "anthropic",
    "usage": {"input": 100, "output": 50, "cache_read": 0, "cache_write": 0, "total_tokens": 150},
    "timestamp": 1700000001000
  }
]

Extension messages use a nested structure:

{
  "role": "extension",
  "kind": "status_update",
  "data": {"status": "running"}
}

Context Tracking

ContextTracker and ExecutionTracker are runtime-only and not persisted. This is by design — both are created fresh each agent_loop() invocation and operate on whatever messages are in context at that point. Restoring messages and calling prompt() works correctly without any special recalculation.

What's Serializable

Lifecycle Callbacks

phi-core provides four tiers of lifecycle callbacks that let you observe and control the agent loop without modifying its internals. Loop-level, turn-level, and tool-level callbacks are set on AgentLoopConfig (or via Agent builder methods). Session-level callbacks (before_task / after_task) are set on SessionRecorderConfig.

0.9.0 — async hook bodies. All loop-level, turn-level, and the non-update tool-level hooks below (plus the two compaction hooks) are now async. The on_* builders on BasicAgent accept closures whose bodies return Pin<Box<dyn Future<Output = T> + Send>> — wrap sync bodies in Box::pin(async move { ... }), or .await LLM and other async work directly. The two tool-update hooks (before_tool_execution_update / after_tool_execution_update) stay sync — see the note next to their sections for the rationale. CHANGELOG [0.9.0] § Migration carries the full mechanical recipe.

Tiers Overview

Loop-Level Hooks

before_loop

Called once before AgentStart is emitted. Receives the current message history and an initial usage counter of 0. Return false to abort the entire run — AgentEnd is emitted with an empty message list and the loop exits immediately.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_before_loop(|messages, _usage| {
        println!("Starting run with {} existing messages", messages.len());
        true // return false to abort
    });
}

after_loop

Called once after AgentEnd is emitted. Receives the new messages produced during the run and the accumulated Usage across all turns.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_after_loop(|new_messages, total_usage| {
        println!(
            "Run complete: {} new messages, {} total tokens",
            new_messages.len(),
            total_usage.total_tokens
        );
    });
}

Turn-Level Hooks

before_turn

Called before each LLM call. Receives the current message history and the turn number (0-indexed). Return false to abort the loop.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_before_turn(|messages, turn| {
        println!("Turn {} starting with {} messages", turn, messages.len());
        turn < 10 // Stop after 10 turns
    });
}

after_turn

Called after each LLM response and tool execution. Receives the updated message history and the turn's token usage.

#![allow(unused)]
fn main() {
use std::sync::{Arc, Mutex};

let total_cost = Arc::new(Mutex::new(0u64));
let cost_tracker = total_cost.clone();

let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_after_turn(move |_messages, usage| {
        let mut cost = cost_tracker.lock().unwrap();
        *cost += usage.input + usage.output;
        println!("Cumulative tokens: {}", *cost);
    });
}

on_error

Called when the LLM returns a StopReason::Error. Receives the error message string.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_error(|err| {
        eprintln!("LLM error: {}", err);
        // Log to monitoring, send alert, etc.
    });
}

Tool-Level Hooks

before_tool_execution

Called before each tool starts, after the ToolExecutionStart event would normally emit. Receives the tool name, call ID, and arguments. Return false to skip the tool — a ToolExecutionEnd with an error result is emitted and the tool's execute() is never called.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_before_tool_execution(|name, call_id, _args| {
        println!("About to run tool: {}", name);
        // Return false to block specific tools:
        name != "bash" // block bash, allow everything else
    });
}

after_tool_execution

Called after each tool finishes (after ToolExecutionEnd is emitted). Receives the tool name, call ID, and whether the result was an error.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_after_tool_execution(|name, call_id, is_error| {
        if is_error {
            eprintln!("Tool {} ({}) failed", name, call_id);
        }
    });
}

before_tool_execution_update (sync — see note below)

Called before each ToolExecutionUpdate event (streaming progress from a running tool). Return false to suppress the event — the tool keeps running and the final ToolResult is unaffected; only the intermediate streaming update is dropped.

Pre-existing-behaviour preservation note (phi-core 0.9.0). The two tool-update hooks (before_tool_execution_update / after_tool_execution_update) remain sync after the 0.9.0 async-trait migration. Async-ifying them would cascade into the ToolUpdateFn callback type and every AgentTool::execute body that invokes ctx.on_update(...) — materially wider than the 0.9.0 scope. The veto decision in before_tool_execution_update must be synchronous so the surrounding emit gate works without an .await suspension at every streamed tool-update. Async work at update-time should be dispatched via tokio::spawn(...) inside the sync closure body. Tracked in the CHANGELOG [Unreleased] "Forward markers" for a future release.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_before_tool_execution_update(|name, call_id, text| {
        // Only forward updates for bash tool
        name == "bash"
    });
}

after_tool_execution_update

Called after each ToolExecutionUpdate event, only if it was not suppressed by before_tool_execution_update.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_after_tool_execution_update(|name, call_id, text| {
        // e.g., log streaming updates to a file
    });
}

Script Callbacks

In addition to Rust closures, callbacks can be implemented as external shell or Python scripts. This allows non-Rust consumers to hook into the agent lifecycle without compiling Rust code.

Script callbacks are specified as command strings (e.g., "./scripts/on_task_start.sh" or "python3 scripts/after_turn.py"). The agent loop spawns the script as a subprocess, passing relevant context (such as session ID, turn number, or tool name) as environment variables or arguments. The script's exit code determines whether the action proceeds (0 = continue, non-zero = abort, for Before* hooks).

Script callbacks can be configured in the [callbacks] section of the config file or set programmatically via the Agent trait.

All callback tiers are wired in the script callback bridge. Loop-level (before_loop, after_loop), tool-level (before_tool_execution, after_tool_execution), compaction-level (before_compaction_start, after_compaction_end), and turn-level (before_turn, after_turn) hooks are all resolved from the [callbacks] config section and bridged to external scripts. The bridge passes hook context as JSON (message count, turn index, tool name, etc.) via stdin to the subprocess.

Hook Ordering

The hooks fire in strict order relative to their paired events. This ordering is an invariant — it is enforced at runtime:

before_loop
  → AgentStart
    before_turn
      → TurnStart
        [MessageStart / MessageUpdate* / MessageEnd]
        [per tool call:]
          before_tool_execution
            → ToolExecutionStart
              (before_tool_execution_update → ToolExecutionUpdate → after_tool_execution_update)*
            ToolExecutionEnd →
          after_tool_execution
        [if context budget exceeded:]
          before_compaction_start
            → CompactionStarted
            CompactionEnded →
          after_compaction_end
      TurnEnd →
    after_turn
  AgentEnd →
after_loop

Short-Circuit Rules

Steering Checkpoints

Steering messages (injected via the agent's steering queue) are checked at six specific points in the turn cycle. These checkpoints give the caller opportunities to redirect the agent mid-run without waiting for the current loop iteration to complete.

The Six Checkpoints

1. Before turn -- After before_turn fires, before the LLM call. The steering message is prepended to the message history as a User message before the model sees it.
2. After turn -- After the LLM response is received and after_turn fires. Steering is appended before the next turn begins.
3. Between tool executions (Sequential) -- When tool_strategy = "sequential", the steering queue is checked between each individual tool call. This is the finest-grained checkpoint.
4. Between batches (Batched) -- When tool_strategy = "batched", the steering queue is checked after each batch completes, before the next batch starts.
5. After all tools (Parallel) -- When tool_strategy = "parallel", steering is checked once after all tool calls complete. No mid-batch interruption.
6. On loop re-entry -- At the top of each loop iteration, before before_turn fires.

Per-Strategy Behavior

In all strategies, checkpoints 1, 2, and 6 always apply. The strategy only affects when steering is checked during tool execution (checkpoints 3-5).

Why Mid-Stream and Mid-Tool Steering Is Not Supported

Steering is intentionally not checked:

• During an LLM streaming response -- The SSE stream is atomic from the agent loop's perspective. Interrupting a partial response would produce an inconsistent message (partial assistant text with no stop reason). The model's response must complete or fail before steering can take effect.
• During a single tool's execution -- A tool call is an atomic unit. Interrupting a bash command mid-execution or a file write mid-stream would leave the environment in an undefined state. The tool must return its ToolResult before steering is considered.

These boundaries are not limitations but invariants that keep the message history and environment consistent.

Hard Abort with CancellationToken

For cases where waiting for the next steering checkpoint is unacceptable (e.g., runaway tool, user-initiated cancel), CancellationToken provides a hard abort:

#![allow(unused)]
fn main() {
use tokio_util::sync::CancellationToken;

let cancel = CancellationToken::new();
let cancel_clone = cancel.clone();

// In another task:
cancel_clone.cancel(); // triggers immediate abort
}

When the token is cancelled:

• The current LLM stream is dropped (partial response discarded)
• Running tools are cancelled via their async cancellation
• The loop emits AgentEnd with StopReason::Aborted
• No further turns or tool calls are attempted

CancellationToken is a last resort. Prefer steering for graceful redirection; use cancellation only when the agent must stop immediately.

Combining Callbacks

All callbacks are optional and independent:

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key))
    .on_before_loop(|_msgs, _| true)
    .on_after_loop(|msgs, usage| {
        println!("Done: {} messages, {} tokens", msgs.len(), usage.total_tokens);
    })
    .on_before_turn(|_msgs, turn| turn < 20)
    .on_after_turn(|msgs, usage| {
        println!("Messages: {}, Tokens: {}/{}", msgs.len(), usage.input, usage.output);
    })
    .on_error(|err| eprintln!("Error: {}", err))
    .on_before_tool_execution(|name, _id, _args| {
        println!("Running: {}", name);
        true
    })
    .on_after_tool_execution(|name, _id, is_error| {
        println!("Tool {} finished (error={})", name, is_error);
    });
}

Using with AgentLoopConfig

For direct loop usage without the Agent wrapper:

#![allow(unused)]
fn main() {
use std::sync::Arc;
use phi_core::agent_loop::AgentLoopConfig;
use phi_core::provider::ModelConfig;

let config = AgentLoopConfig {
    model_config: ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key),
    // Loop-level
    before_loop: Some(Arc::new(|_msgs, _| true)),
    after_loop: Some(Arc::new(|msgs, usage| { /* log */ })),
    // Turn-level
    before_turn: Some(Arc::new(|_msgs, turn| turn < 5)),
    after_turn: Some(Arc::new(|_msgs, _usage| { /* log */ })),
    on_error: Some(Arc::new(|err| eprintln!("{}", err))),
    // Tool-level
    before_tool_execution: Some(Arc::new(|name, id, args| true)),
    after_tool_execution: Some(Arc::new(|name, id, is_error| {})),
    before_tool_execution_update: Some(Arc::new(|name, id, text| true)),
    after_tool_execution_update: Some(Arc::new(|name, id, text| {})),
    ..Default::default()
};
}

Sessions

A Session is a named container (keyed by session_id) that groups all LoopRecords belonging to one agent session. Sessions provide persistent, structured memory of every agent interaction — suitable for logging, replay, branching, and tracing agent-spawning chains.

The session module is split into sub-modules: model, recorder, storage, helpers.

Session (session_id)
├── LoopRecord (loop_id: A)       ← origin loop
│   ├── LoopRecord (loop_id: B)   ← continuation of A
│   └── LoopRecord (loop_id: C)   ← another continuation of A
│       ├── LoopRecord (loop_id: D)  ← parallel branch
│       └── LoopRecord (loop_id: E)  ← parallel branch (selected)
└── child_loop_refs → Session (sub-agent session)

Overview

Relationship to loops

One session contains many loops. Loops within a session form a tree via parent_loop_id / children_loop_ids links. Parallel-evaluation branches form a sibling group linked by ParallelGroupRecord. Sub-agent loops are cross-session (different session_id) and connected via ChildLoopRef / SpawnRef instead.

Session Formation

A new Session is opened when SessionRecorder first encounters a session_id it has not seen before. Three scenarios produce a new session:

PerSessionId (default)

One Session per session_id. Maps naturally onto BasicAgent lifetime — one BasicAgent instance = one session for its entire lifetime.

#![allow(unused)]
fn main() {
let mut recorder = SessionRecorder::new(SessionRecorderConfig::default());
// Every event from a single BasicAgent feeds into one Session.
}

When to use: The default for most applications. No infrastructure needed.

InactivityTimeout

Opens a new session when the agent has been idle for longer than a configured threshold. Requires the caller to rotate session_id beforehand — the recorder detects the new session_id on the next AgentStart.

#![allow(unused)]
fn main() {
// In your agent orchestrator, before prompting:
if agent.check_and_rotate(Duration::from_secs(1800)).is_some() {
    println!("Started new session after 30 minutes idle");
}
}

When to use: Long-running assistants where each "conversation" should be a distinct session even if the BasicAgent object persists.

Explicit rotation

Call BasicAgent::new_session() directly to rotate immediately.

#![allow(unused)]
fn main() {
let new_id = agent.new_session();
// All subsequent loops belong to the new session.
}

When to use: At conversation boundaries you control explicitly (e.g. "clear chat" button, new document context).

LoopRecord Anatomy

Field table

LoopStatus lifecycle

                              AgentEnd (no rejection)
┌─────────┐  AgentStart  ┌─────────┐ ───────────────────────► ┌───────────┐
│ Pending ├─────────────►│ Running │                           │ Completed │
└─────────┘              └────┬────┘ AgentEnd (rejection Some) └───────────┘
                               │ ────────────────────────────► ┌──────────┐
                               │                               │ Rejected │
                               │ flush() before AgentEnd       └──────────┘
                               └─────────────────────────────► ┌─────────┐
                                                               │ Aborted  │
                                                               └─────────┘

Pending is only used for parallel-evaluation branches: they are pre-registered when ParallelLoopStart arrives, before their individual AgentStart fires.

continuation_kind classification

LoopConfigSnapshot

LoopConfigSnapshot captures model identity and key configuration from the AgentLoopConfig that ran the loop:

#![allow(unused)]
fn main() {
pub struct LoopConfigSnapshot {
    pub model: String,                    // e.g. "claude-opus-4-6"
    pub provider: String,                 // e.g. "anthropic"
    pub config_id: Option<String>,        // from AgentLoopConfig.config_id
    pub name: Option<String>,             // model display name
    pub api: Option<ApiProtocol>,         // which API protocol was used
    pub base_url: Option<String>,         // provider base URL
    pub reasoning: Option<bool>,          // whether model supports reasoning/thinking
    pub context_window: Option<u32>,      // model context window size
    pub max_tokens: Option<u32>,          // max output tokens
    pub thinking_level: Option<ThinkingLevel>, // reasoning depth for this loop
    pub temperature: Option<f32>,         // sampling temperature
}
}

The first three fields (model, provider, config_id) are always populated. The remaining fields use Option with #[serde(skip_serializing_if = "Option::is_none")] so they only appear when set, keeping serialized output compact.

Why not store the full AgentLoopConfig? The full config contains API keys (in ModelConfig.api_key) and non-serialisable hook closures. Storing it would require stripping secrets and skipping closures for little extra value. LoopConfigSnapshot is sufficient for cost attribution, replay (the caller reconstructs the config), and identifying parallel branches (e.g. "haiku vs. opus").

Note: thinking_level and temperature were previously stored on the Session struct. They are now tracked per-loop in LoopConfigSnapshot, which more accurately reflects that these settings can vary between loops (e.g. across parallel evaluation branches with different configs).

events field

LoopRecord.events contains every AgentEvent emitted during the loop, in order, tagged with a monotonic sequence counter.

MessageUpdate (streaming delta) events are excluded by default — they are 100–1 000× more numerous than final messages and are not needed for replay. Enable them with SessionRecorderConfig { include_streaming_events: true, .. }.

AgentEnd.messages is the authoritative message source for a loop. LoopRecord.messages is populated directly from it. Reconstructing messages from MessageStart/MessageEnd events would be fragile.

compaction_block field

LoopRecord.compaction_block holds a non-destructive compaction overlay. When present, the context loader uses this block instead of the raw messages field to reconstruct the agent's working context. The original messages remain authoritative for replay and branching — they are never mutated or discarded. This overlay model means compaction is always reversible: removing or replacing the CompactionBlock restores the original conversation without data loss.

Bidirectional parent↔child links

Both directions of the loop tree are maintained:

• LoopRecord.parent_loop_id — child → parent (set at loop creation)
• LoopRecord.children_loop_ids — parent → children (appended at AgentEnd)

This allows O(1) traversal in either direction without scanning the full loops vec.

Loop Tree Navigation

Session provides four navigation methods:

#![allow(unused)]
fn main() {
// Root loops — no parent in this session.
session.root_loops();

// Direct same-session children of a loop.
session.children_of("loop-id-A");

// All parallel siblings (including the loop itself).
session.parallel_siblings("loop-id-branch-1");

// Lookup by id.
session.get_loop("loop-id-X");

// Cumulative token usage for the whole session.
session.total_usage();
}

Reconstructing a conversation thread

Follow the parent→child chain from a root:

#![allow(unused)]
fn main() {
fn print_thread(session: &Session, loop_id: &str, indent: usize) {
    if let Some(lr) = session.get_loop(loop_id) {
        println!("{:indent$}{loop_id}: {:?}", "", lr.status, indent = indent);
        for child_id in &lr.children_loop_ids {
            print_thread(session, child_id, indent + 2);
        }
    }
}

for root in session.root_loops() {
    print_thread(&session, &root.loop_id, 0);
}
}

Identifying branches

Branches share the same parent_loop_id and each has parallel_group set:

#![allow(unused)]
fn main() {
let branches: Vec<_> = session.parallel_siblings("branch-loop-id").collect();
let winner = branches.iter().find(|l| {
    l.parallel_group.as_ref().map(|pg| pg.is_selected).unwrap_or(false)
});
}

Cross-Session Sub-Agent Tracking

Sub-agents run with their own session_id. phi-core maintains bidirectional links between the parent session and the child session:

Parent Session                         Child Session
──────────────────────                 ──────────────────────────
LoopRecord (loop-P)                    Session
  child_loop_refs:                       parent_spawn_ref:
    ChildLoopRef {                         SpawnRef {
      tool_call_id: "call-1"               parent_session_id: "sess-P"
      tool_name: "sub_agent"               parent_loop_id: "loop-P"
      child_loop_id: "loop-C"              tool_call_id: "call-1"
      child_session_id: "sess-C"           tool_name: "sub_agent"
    }                                    }

Tracing a full spawn chain

#![allow(unused)]
fn main() {
// Load parent and child sessions from disk.
let parent = load_session("sess-P", dir)?;
let child = load_session("sess-C", dir)?;

// From parent: find all sub-agent spawns.
for lr in &parent.loops {
    for child_ref in &lr.child_loop_refs {
        println!("Tool {} spawned sub-agent loop {}",
            child_ref.tool_name, child_ref.child_loop_id);
    }
}

// From child: find the parent that triggered it.
if let Some(ref sr) = child.parent_spawn_ref {
    println!("This session was spawned by {} in session {}",
        sr.tool_name, sr.parent_session_id);
}
}

Why sub-agents get separate sessions

Sub-agents have clean identity boundaries — they can be loaded and analyzed independently of their parent. Embedding child data inside the parent session would bloat the parent record and couple two independent execution traces. The bidirectional ChildLoopRef / SpawnRef pair provides a complete spawn graph without that coupling.

Parallel Evaluation Groups

When agent_loop_parallel runs N branches, each branch gets its own LoopRecord. All N records are linked by ParallelGroupRecord:

#![allow(unused)]
fn main() {
pub struct ParallelGroupRecord {
    pub all_loop_ids: Vec<String>,       // all branch loop_ids in config order
    pub selected_loop_id: String,        // winner chosen by EvaluationStrategy
    pub selected_config_index: usize,    // 0-based index into original configs
    pub evaluation_usage: Usage,         // judge LLM tokens (zero if no judge)
    pub is_selected: bool,               // true only on the winner's record
}
}

LoopStatus::Pending is used before AgentStart arrives for each branch. ParallelLoopStart announces all loop_ids in advance, so the group can be registered immediately without retroactive wiring.

SessionRecorder Usage

Wire the recorder to your agent's event channel:

#![allow(unused)]
fn main() {
use phi_core::session::{SessionRecorder, SessionRecorderConfig, save_session};
use phi_core::AgentEvent;
use std::path::Path;
use tokio::sync::mpsc;

let (tx, mut rx) = mpsc::unbounded_channel::<AgentEvent>();
let mut recorder = SessionRecorder::new(SessionRecorderConfig::default());

// Spawn a task to consume events from the channel.
tokio::spawn(async move {
    while let Some(event) = rx.recv().await {
        recorder.on_event(event);
    }
    // Channel closed — flush and persist.
    recorder.flush();
    for session in recorder.drain_completed() {
        save_session(&session, Path::new("./sessions")).unwrap();
    }
});

// Pass tx to agent_loop / agent_loop_continue / BasicAgent.
}

include_streaming_events

Enable only when you need to replay or audit the raw token stream:

#![allow(unused)]
fn main() {
SessionRecorderConfig {
    include_streaming_events: true,
    ..Default::default()
}
}

Storage implications: a single turn with extended thinking may produce thousands of MessageUpdate events. Each is a full clone of the accumulated message plus the delta.

Session Lifecycle Callbacks

SessionRecorderConfig supports two session-level callbacks for billing, audit, and metrics:

These are session-level (not loop-level) hooks. Unlike before_loop/after_loop on AgentLoopConfig which fire around each individual agent loop, before_task and after_task fire once per session lifecycle:

#![allow(unused)]
fn main() {
use phi_core::session::{SessionRecorder, SessionRecorderConfig};

let config = SessionRecorderConfig {
    before_task: Some(Arc::new(|session: &phi_core::session::Session| -> bool {
        println!("Session started: {}", session.session_id);
        // Initialize billing, start audit trail, etc.
        true // return false to reject the session
    })),
    after_task: Some(Arc::new(|session| {
        println!("Session finalized: {} ({} loops)", session.session_id, session.loops.len());
        // Finalize billing, emit metrics, etc.
    })),
    ..Default::default()
};

let mut recorder = SessionRecorder::new(config);
}

Persistence API

File format: pretty-printed JSON (serde_json::to_writer_pretty). Directory layout: flat — {dir}/{session_id}.json, no sub-directories, no index. Writes are atomic: the implementation writes to a temp file then renames over the target, so readers never observe a partially-written file.

Pluggable store trait (0.7.0+)

For callers that want to swap the persistence backend (e.g. S3, SQLite, an in-memory fake for tests) or that need concurrent-writer safety, phi-core exposes a SessionStore async trait alongside the free functions:

#![allow(unused)]
fn main() {
use phi_core::session::{SessionStore, FileSystemSessionStore};

let store = FileSystemSessionStore::new("./sessions");
store.save(&session).await?;          // acquires fs2 exclusive lock
let loaded = store.load("sess-1").await?;
let ids    = store.list_ids().await?;
store.delete("sess-1").await?;
}

FileSystemSessionStore::save() takes an advisory fs2 exclusive lock on the target file before the atomic rename. Concurrent writers to the same session_id get back SessionError::Locked { session_id } instead of silently producing a corrupt file. Readers take a shared lock and so coexist with themselves.

The free save_session() / load_session() / etc. functions remain available and unchanged — use the trait when you need pluggability or contention safety.

When to call flush()

Call flush() before saving to finalize any loops that have not received AgentEnd yet (e.g. on process shutdown). Flushed loops get status Aborted.

#![allow(unused)]
fn main() {
recorder.flush();
let sessions = recorder.drain_completed();
for s in &sessions {
    save_session(s, Path::new("./sessions"))?;
}
}

Design Decisions

1. loop_id on every AgentEvent variant

Decision: Add loop_id: String to all 11 AgentEvent variants that lacked it.

Why: agent_loop_parallel interleaves branch events on one tx channel. Without loop_id on every event, TurnStart, ToolExecutionEnd, etc. cannot be reliably attributed to the correct branch LoopRecord. The only alternative — heuristically assigning events to the last-opened loop — produces incorrect records when two branches overlap.

Rejected alternative: Last-opened-loop heuristic. Rejected because parallel branches genuinely interleave; the heuristic would silently misattribute events.

2. LoopStatus::Pending for parallel branches

Decision: Pre-register LoopRecord { status: Pending } entries when ParallelLoopStart arrives, before their AgentStart events fire.

Why: ParallelLoopStart announces all loop_ids in advance. Pre-creating records lets the ParallelGroupRecord be registered immediately, so no retroactive wiring is needed when each branch's AgentStart arrives later.

Rejected alternative: Create LoopRecords only on AgentStart and retroactively set ParallelGroupRecord on ParallelLoopEnd. Rejected because it requires a second pass over all records and makes the group state inconsistent during the parallel execution window.

3. Messages from AgentEnd, not reconstructed from events

Decision: LoopRecord.messages is populated directly from AgentEnd.messages.

Why: AgentEnd.messages is the authoritative, ordered list of all messages produced by a loop. The LLM loop already assembles this — there is no value in re-assembling it from MessageStart/MessageEnd events in the recorder.

Rejected alternative: Reconstruct messages from streaming events. Rejected because it duplicates work, is fragile (missed events, ordering edge cases), and requires special handling for partial messages.

4. Bidirectional parent↔child within a session

Decision: Maintain both parent_loop_id (child→parent) and children_loop_ids (parent→children) on every LoopRecord.

Why: O(1) traversal in both directions without scanning the full loops vec. The recorder appends to parent.children_loop_ids when a loop's AgentEnd arrives and its parent_loop_id is in the same session.

Rejected alternative: Single-direction links + O(N) scan. Rejected because deep continuation trees (10+ loops) would incur O(N²) cost for common tree operations.

5. continuation_kind classifies loop origin

Decision: Reuse the existing ContinuationKind enum (Initial, Default, Rerun, Branch, Compaction) to classify loop relationships, supplemented by the parent_loop_id/session_id cross-session check.

Why: ContinuationKind is already threaded through AgentStart — no new enum is needed. The full classification table (origin / continuation / retry / branch / sub-agent) is derivable from (parent_loop_id, session_id, continuation_kind).

Rejected alternative: A dedicated LoopOrigin enum on LoopRecord. Rejected because it would duplicate information already present in the existing fields and require an additional mapping step in the recorder.

6. Sub-agents are separate sessions with bidirectional cross-session links

Decision: Sub-agents always get their own session_id. The parent records ChildLoopRef (outbound); the child Session records SpawnRef (inbound).

Why: Clean agent identity boundaries — sub-agent sessions can be loaded and analyzed independently. The bidirectional link pair provides a complete spawn graph without coupling the parent and child session records.

Rejected alternative: Embed sub-agent loops inside the parent session. Rejected because a sub-agent may have many of its own continuations, parallel branches, and even nested sub-agents — treating it as a flat loop inside the parent session would obscure this structure.

7. SpawnRef on Session (not on LoopRecord)

Decision: The inbound cross-session spawn reference lives on Session.parent_spawn_ref, not on an individual LoopRecord.

Why: Sub-agent spawning is a session-level concern. The entire child session was triggered by one parent loop — the reference belongs at the session level, not on individual loop records within it. Placing it on a LoopRecord would require choosing which loop gets the ref (the first? the origin?) arbitrarily.

Rejected alternative: LoopRecord.parent_spawn_ref. Rejected because a sub-agent session may have multiple origin loops (e.g. after new_session()) and the spawn ref would be duplicated or placed inconsistently.

8. include_streaming_events: bool (default false)

Decision: MessageUpdate (streaming delta) events are excluded from LoopRecord.events by default.

Why: Streaming deltas are 100–1 000× more numerous than final messages and are not needed for replay or branching. The final message content in AgentEnd.messages is authoritative. Opt-in ensures that session files stay compact by default.

Rejected alternative: Always store all events. Rejected because a single session with a few extended-thinking turns could easily produce megabytes of delta events.

9. Flat file layout: {dir}/{session_id}.json

Decision: One JSON file per session. No index file, no sub-directories.

Why: Simplest observable format — files can be inspected directly with any JSON tool. list_session_ids is a directory listing. No index to maintain or synchronize.

Rejected alternative: Indexed layout (e.g. sessions/index.json + sessions/{id}.json). Rejected because the index requires atomic updates (write to two files) and can fall out of sync. An indexed layout can be added in a future iteration when query patterns (filtering, pagination) are clearer.

Context Compaction

Compaction manages context window pressure by creating non-destructive overlays on session history. Nothing is deleted or replaced — original messages remain authoritative in LoopRecord.messages.

How it works

When the context approaches the token budget, a CompactionBlock is created on the current LoopRecord. This block controls what gets loaded into context for subsequent LLM calls, replacing the raw messages with a compacted view.

CompactionBlock anatomy

A block has three sections:

┌─────────────────────────────────────────────┐
│  keep_first    │ Original turns, verbatim    │  Most recent loop only
│  (turns 0..1)  │ No modification              │
├────────────────┼────────────────────────────-│
│  keep_compacted│ Summarised one-liners       │  All loops
│  (turns 2..N-6)│ ≤ max_summary_tokens        │
├────────────────┼────────────────────────────-│
│  keep_recent   │ Tool outputs truncated      │  Most recent loop only
│  (turns N-5..N)│ Rest unchanged              │
└─────────────────────────────────────────────┘

• keep_first — verbatim turns from the start. Only for the most recent loop. Original messages in this range are used as-is.
• keep_compacted — fully summarised middle section. For the most recent loop this is the gap between keep_first and keep_recent. For older loops this covers the entire loop.
• keep_recent — recent turns with only tool outputs truncated. Only for the most recent loop.

When compaction fires

Compaction uses a percentage-based threshold:

headroom = compact_at_pct − (system_tokens / max_tokens) − (current_tokens / max_tokens)

Compaction fires when headroom < compact_budget_threshold_pct.

With defaults (100k max, 4k system, 90% ceiling, 5% threshold): fires when current tokens exceed ~81k.

Configuration

ContextConfig

#![allow(unused)]
fn main() {
ContextConfig {
    max_context_tokens: 100_000,   // Model's context window
    system_prompt_tokens: 4_000,   // Reserved for system prompt
    compaction: CompactionConfig { // Always present when limits are set
        // WHEN
        compact_at_pct: 0.90,
        compact_budget_threshold_pct: 0.05,
        compaction_scope: CompactionScope::FixedCount(3),
        // HOW
        keep_first_turns: 2,
        keep_recent_turns: 10,
        max_summary_tokens: 2_000,
        tool_output_max_lines: 50,
    },
}
}

Compaction is disabled entirely by setting context_config: None on AgentLoopConfig.

CompactionScope

Controls how many earlier loops are included in compaction and context loading:

• FixedCount(n) — Compact a fixed number of earlier loops. Simple and predictable.
• TokenBudget — Walk the chain backward, accumulating per-loop token estimates, and stop when max_context_tokens would be exceeded.

TokenBudget and exceeding the window

The TokenBudget scope can include loops whose raw messages exceed max_context_tokens. This is intentional: the compacted summaries of those loops will fit in the window, even though the originals did not. This enables richer context for expensive summarisation strategies (e.g. LLM summarisers) that compress large loops into compact representations that then fit within the budget.

For example, if a loop has 50k tokens of raw messages and the window is 100k, TokenBudget includes it in scope. The strategy's keep_compacted method produces a ~500 token summary of that loop, which fits easily.

Cross-loop compaction

When compaction fires, blocks are created for the current loop and earlier loops within the compaction_scope on the active chain.

The "active chain" is the linear path from root to current loop via parent_loop_id links:

• Parallel branches — only the selected branch is on the chain. Unselected siblings get their own compaction if/when they become active.
• Reruns — the rerun's parent points to the pre-rerun loop. Superseded runs are siblings, not ancestors.

Loading rule

When building context from session history:

• Most recent loop: keep_first + keep_compacted + keep_recent
• Earlier loops (within compaction_scope): only keep_compacted
• Loops older than that: skipped entirely

Custom strategies

Compaction strategies are fields on CompactionConfig, not on AgentLoopConfig. The dispatch logic in run.rs reads them from ctx_config.compaction:

• in_memory_strategy — custom in-memory compaction strategy (used when session is None)
• block_strategy — block-based compaction strategy (used when session is Some; falls back to DefaultBlockCompaction)

Implement BlockCompactionStrategy to customise any section.

As of phi-core 0.9.0, BlockCompactionStrategy is #[async_trait]-marked and all four methods are async fn — implementations can issue LLM calls inside keep_compacted / keep_recent without block_in_place workarounds:

#![allow(unused)]
fn main() {
use async_trait::async_trait;
use phi_core::{BlockCompactionStrategy, CompactionConfig, CompactedSection, TurnRange, TurnMap, DefaultBlockCompaction};
use phi_core::session::LoopRecord;

struct MyStrategy;

#[async_trait]
impl BlockCompactionStrategy for MyStrategy {
    async fn keep_first(&self, record: &LoopRecord, turn_map: &TurnMap, config: &CompactionConfig) -> Option<TurnRange> {
        DefaultBlockCompaction.keep_first(record, turn_map, config).await // delegate
    }

    async fn keep_recent(&self, record: &LoopRecord, turn_map: &TurnMap, config: &CompactionConfig) -> Option<CompactedSection> {
        DefaultBlockCompaction.keep_recent(record, turn_map, config).await // delegate
    }

    async fn keep_compacted(&self, record: &LoopRecord, turn_map: &TurnMap, config: &CompactionConfig, is_most_recent: bool) -> Option<CompactedSection> {
        // Custom LLM-based summarisation — issue LLM calls directly without bridging.
        my_llm_summarize(record, turn_map, config, is_most_recent).await
    }
}
}

Sync impls that don't .await anything migrate by adding #[async_trait::async_trait] + the async keyword on each method signature; the bodies remain unchanged. See the per-turn debug-capture surface in debugging.md for the canonical pattern to inspect what each compacted turn looked like to the model.

Set the custom strategy on CompactionConfig:

#![allow(unused)]
fn main() {
let compaction_config = CompactionConfig {
    block_strategy: Some(Arc::new(MyStrategy)),
    ..Default::default()
};
}

Public APIs

Orchestration functions

• compact_session_loops(session, loop_id, strategy, config, max_tokens) — Creates CompactionBlocks for the current loop and earlier loops within the configured scope. Mutates the session in place; caller persists to disk.
• build_context_from_session(session, loop_id, config, max_tokens) — Builds a compacted context by walking the loop chain, loading from blocks where available and raw messages otherwise.

BasicAgent methods

• compact_context_with_sender(&mut self, tx) — Standalone compaction with full event lifecycle: AgentStart(Compaction) → CompactionStarted → compact → CompactionEnded → AgentEnd. No-op if session or config is missing.
• compact_context(&mut self) -> usize — Fire-and-forget compaction. Returns the number of loops that received new CompactionBlocks. Returns 0 if session or config is missing.

Events

Two events bracket compaction:

• CompactionStarted { loop_id, estimated_tokens, message_count, timestamp }
• CompactionEnded { loop_id, messages_before, messages_after, estimated_tokens_before, estimated_tokens_after, loops_compacted, timestamp }

For standalone compaction (compact_context_with_sender), these appear inside a dedicated LoopRecord with continuation_kind: Compaction.

TurnId tracking

Every message pushed during the agent loop carries a TurnId { loop_id, turn_index } identifying which turn produced it. This enables TurnMap::from_messages() to group messages by turn without replaying the event stream.

TurnId is stored on LlmMessage.turn_id and serialized as an optional turnId field alongside the existing message JSON. Old data without turnId deserializes with turn_id: None.

Data model

Struct definitions

#![allow(unused)]
fn main() {
pub struct CompactionBlock {
    pub keep_first: Option<TurnRange>,         // verbatim turns from start (most recent loop only)
    pub keep_recent: Option<CompactedSection>,  // truncated tool outputs (most recent loop only)
    pub keep_compacted: Option<CompactedSection>,// summarised section (all loops)
    pub created_at: DateTime<Utc>,
}

pub struct TurnRange {
    pub start_turn: u32,  // inclusive, matches TurnId.turn_index
    pub end_turn: u32,    // inclusive
}

pub struct CompactedSection {
    pub range: TurnRange,
    pub messages: Vec<AgentMessage>,  // replacement messages for this range
}

pub struct TurnId {
    pub loop_id: String,
    pub turn_index: u32,
}
}

Serialization format

CompactionBlock on LoopRecord:

{
  "loop_id": "session123.model.1",
  "messages": [ ... ],
  "compaction_block": {
    "keep_first": { "startTurn": 0, "endTurn": 1 },
    "keep_compacted": {
      "range": { "startTurn": 2, "endTurn": 7 },
      "messages": [
        { "role": "user", "content": [{"type": "text", "text": "[Summary] User asked about X"}], "timestamp": 123 }
      ]
    },
    "keep_recent": {
      "range": { "startTurn": 8, "endTurn": 12 },
      "messages": [ ... ]
    },
    "createdAt": "2026-03-28T10:00:00Z"
  }
}

TurnId on LlmMessage:

{
  "role": "assistant",
  "content": [...],
  "stopReason": "stop",
  "model": "claude-sonnet-4-6",
  "provider": "anthropic",
  "usage": { ... },
  "timestamp": 123,
  "turnId": { "loopId": "session123.model.1", "turnIndex": 3 }
}

Old data without turnId deserializes as turn_id: None.

Invariants

1. If keep_first is Some, keep_compacted must also be Some (there must be a middle to summarise).
2. If keep_recent is Some, keep_compacted must also be Some.
3. For older loops (not most recent), keep_first and keep_recent are always None.
4. CompactedSection.range bounds must be within the loop's turn count.
5. If a loop has a compaction_block, all older loops on the same chain must also have one.
6. If a ToolCall content block is within a section's turn range, its corresponding ToolResult message must also be within the same section. Turn-based grouping (via TurnId) enforces this.

Summary budget semantics

max_summary_tokens is a token budget for the summarised output, not a per-turn limit. Strategies should aim to summarise ALL turns within this budget (e.g. shorter summaries or LLM-generated digests), not merely process turns until the budget runs out. DefaultBlockCompaction is a basic implementation that drops remaining turns when exhausted.

Backward compatibility

• LoopRecord.compaction_block uses #[serde(default, skip_serializing_if = "Option::is_none")] — old records without the field deserialize as None.
• LlmMessage.turn_id uses #[serde(default, skip_serializing_if = "Option::is_none")] — old messages without turnId deserialize as None.
• The CompactionConfig field on ContextConfig uses #[serde(default)] — old configs get CompactionConfig::default().

Focused Compaction

Focused compaction extends the context compaction system with two features: focus messages that steer what the compaction summary emphasizes, and compaction instances that let you define named compaction configurations reusable across agent profiles.

Focus Message

The focus_message field on CompactionConfig is an optional string prepended to the compacted section before the LLM summarizes it. It tells the summarizer what to prioritize when condensing conversation history.

Without a focus message, compaction produces a generic summary. With one, the summary retains details relevant to your domain:

#![allow(unused)]
fn main() {
use phi_core::context::{ContextConfig, CompactionConfig};

let config = ContextConfig {
    max_context_tokens: 200_000,
    compaction: CompactionConfig {
        focus_message: Some(
            "Focus on specification details, API contracts, and architectural decisions.".to_string()
        ),
        ..Default::default()
    },
    ..Default::default()
};
}

The focus message does not change the compaction trigger logic (thresholds, turn counts). It only affects the content of the summarized middle section.

When to use a focus message

• Domain-specific agents: An agent reviewing legal contracts should retain clause references, not general pleasantries.
• Long coding sessions: Focus on file paths, function signatures, and design rationale so the agent can continue working after compaction.
• Research agents: Preserve citations, data points, and methodology notes.

Compaction Instances

Compaction instances are named variations of the compaction defaults, declared with [[context.compaction.instances]] in the config file. Each instance uses the {{...}} ID reference protocol to declare its name, and overrides specific fields from the parent [compaction] section. Fields not set on the instance fall through to the parent defaults.

Config example

# ── Context config (max_context_tokens lives on ContextConfig, not CompactionConfig) ──
[context]
max_context_tokens = 200000

# ── Compaction defaults ─────────────────────────────────────────
[context.compaction]
compact_at_pct = 0.85
compact_budget_threshold_pct = 0.05
keep_first_turns = 2
keep_recent_turns = 4
max_summary_tokens = 2000
tool_output_max_lines = 50
focus_message = "Retain key decisions and code changes."

# ── Named compaction instances ──────────────────────────────────
[[context.compaction.instances]]
id = "{{%coding%}}"
description = "Compaction tuned for coding tasks"
focus_message = "Focus on file paths, function signatures, and design rationale."
keep_recent_turns = 6
max_summary_tokens = 3000

[[context.compaction.instances]]
id = "{{%research%}}"
description = "Compaction tuned for research tasks"
focus_message = "Preserve citations, data sources, and methodology."
keep_first_turns = 3
max_summary_tokens = 4000

Referencing from an agent profile

Agent profiles reference a compaction instance via the compaction field, using the {{...}} ID protocol:

[agent.profile]
name = "coding-agent"
system_prompt = "You are an expert software engineer."
compaction = "{{compaction.coding}}"

[[agent.profile.instances]]
id = "{{%researcher%}}"
description = "A research-focused profile variant"
compaction = "{{compaction.research}}"

When the agent is constructed from config, the referenced compaction instance is resolved and its fields are merged with the compaction defaults to produce the final CompactionConfig.

Programmatic Usage

When building agents in Rust without a config file, focused compaction is set directly on CompactionConfig:

#![allow(unused)]
fn main() {
use phi_core::context::CompactionConfig;
use phi_core::agent_loop::AgentLoopConfig;
use phi_core::provider::ModelConfig;

let context = phi_core::context::ContextConfig {
    max_context_tokens: 200_000,
    compaction: CompactionConfig {
        compact_at_pct: 0.85,
        compact_budget_threshold_pct: 0.05,
        keep_first_turns: 2,
        keep_recent_turns: 6,
        max_summary_tokens: 3_000,
        tool_output_max_lines: 50,
        focus_message: Some(
            "Focus on file paths, function signatures, and design rationale.".to_string()
        ),
        ..Default::default()
    },
    ..Default::default()
};

let config = AgentLoopConfig {
    model_config: ModelConfig::anthropic("claude-sonnet-4-20250514", "Sonnet", &api_key),
    context_config: Some(context),
    ..Default::default()
};
}

Or via BasicAgent builder methods:

#![allow(unused)]
fn main() {
use phi_core::{BasicAgent, context::CompactionConfig};
use phi_core::provider::ModelConfig;

let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Sonnet", &api_key))
    .with_context_config(phi_core::context::ContextConfig {
        max_context_tokens: 200_000,
        compaction: CompactionConfig {
            focus_message: Some("Retain specification details and API contracts.".to_string()),
            ..Default::default()
        },
        ..Default::default()
    });
}

Summary

Context Translation

Context translation solves a fundamental problem in multi-provider agent systems: when an agent switches providers mid-session, content types from the original provider may be silently dropped or cause errors on the new provider. The ContextTranslationStrategy trait provides a read-only translation layer that produces temporary copies of messages, never modifying the canonical history.

Why it is needed

Different LLM providers support different content types. For example:

• Anthropic emits Content::Thinking blocks (chain-of-thought reasoning)
• OpenAI has no native thinking block format
• Google/Bedrock do not support thinking blocks at all

Without translation, switching from Anthropic to OpenAI mid-session would cause thinking blocks to be silently dropped or rejected. The agent loses reasoning context it previously produced.

Design principles

The canonical Message format IS the master layout

phi-core's Message enum (User, Assistant, ToolResult) and Content enum (Text, Image, Thinking, ToolCall) define the canonical format. All providers parse into this format and all session history is stored in it. Translation happens only at the boundary, right before messages are sent to a provider.

Read-only translation

Translation produces temporary copies of the message slice. The original messages in LoopRecord.messages are never modified. This means:

• Session persistence always stores the full-fidelity canonical format
• Multiple providers can read the same history with different translations
• No information is permanently lost

Lossless round-trip guarantee

Consider this scenario:

Turn 1-3: Anthropic (produces Content::Thinking blocks)
Turn 4:   Switch to OpenAI
Turn 5-6: Switch back to Anthropic

Here is what happens:

1. Turns 1-3 are stored with full Content::Thinking blocks in canonical format.
2. Turn 4: Before calling OpenAI, the translator converts Content::Thinking to Content::Text prefixed with [Reasoning]. OpenAI sees text, not thinking blocks. The canonical history is untouched.
3. Turns 5-6: Back on Anthropic. The translator passes Content::Thinking through unchanged. Anthropic sees the original thinking blocks from turns 1-3 exactly as they were produced.

The original thinking blocks from turns 1-3 are never lost. They remain in the canonical history and are available whenever the session returns to a provider that supports them.

Content type translation rules

The DefaultContextTranslation implementation applies these rules per target provider:

Content::Thinking

All other content types

Content::Text, Content::Image, and Content::ToolCall pass through unchanged for all providers.

Message-level behavior

Only Message::Assistant messages are translated (since they are the only ones that carry provider-specific content types). Message::User and Message::ToolResult pass through unchanged.

The ContextTranslationStrategy trait

#![allow(unused)]
fn main() {
pub trait ContextTranslationStrategy: Send + Sync {
    /// Translate a slice of messages for the given target provider protocol.
    fn translate_for_provider(&self, messages: &[Message], target: ApiProtocol) -> Vec<Message>;
}
}

The trait receives the full message slice and the target ApiProtocol enum variant. It returns a new Vec<Message> with translations applied.

DefaultContextTranslation

The built-in implementation applies the content type rules described above. It is the default when no custom strategy is provided.

Custom strategies

Implement the trait to define custom translation logic:

#![allow(unused)]
fn main() {
use phi_core::provider::context_translation::{ContextTranslationStrategy, DefaultContextTranslation};
use phi_core::provider::model::ApiProtocol;
use phi_core::types::content::Message;

struct MyTranslation;

impl ContextTranslationStrategy for MyTranslation {
    fn translate_for_provider(&self, messages: &[Message], target: ApiProtocol) -> Vec<Message> {
        // Custom logic here — e.g., strip all images for text-only providers
        // Fall back to default for everything else
        DefaultContextTranslation.translate_for_provider(messages, target)
    }
}
}

Usage

On AgentLoopConfig

Set the context_translation field to inject a strategy into the agent loop:

#![allow(unused)]
fn main() {
use std::sync::Arc;
use phi_core::agent_loop::AgentLoopConfig;
use phi_core::provider::context_translation::DefaultContextTranslation;
use phi_core::provider::ModelConfig;

let config = AgentLoopConfig {
    model_config: ModelConfig::openai("gpt-4o", "GPT-4o", &api_key),
    context_translation: Some(Arc::new(DefaultContextTranslation)),
    ..Default::default()
};
}

When context_translation is Some, the loop calls translate_for_provider() on the message slice before each LLM call. When None, messages are passed to the provider as-is.

When to enable translation

Enable context translation when:

• Your agent may switch providers mid-session (e.g., using different models for different tasks)
• You are loading session history that was produced by a different provider
• You are running parallel sub-agents on different providers that share context

If your agent always uses a single provider, translation is unnecessary.

Context Pruning

Context pruning is a model-directed mechanism for surgically removing irrelevant content from the working context during a run. Unlike compaction (which is threshold-triggered and bulk), pruning gives the model fine-grained control over what stays in the context window.

Philosophy

Context pruning saves context length (tokens in the context window), not monetary cost. The token cost of a pruned message has already been paid -- pruning cannot reclaim it. What pruning reclaims is space: room in the context window for the model to continue working without hitting the context limit.

Think of it as a researcher working through a stack of papers. The researcher freely explores tangential references, reads through lengthy tool outputs, and investigates dead ends. When a line of inquiry turns out to be irrelevant, the researcher sets those papers aside rather than keeping them on the desk. The desk has limited space; the filing cabinet does not. Pruning moves content from desk to cabinet.

This freedom to explore without anxiety about context length is the core value proposition. The model can request verbose tool outputs, try multiple approaches, and investigate broadly -- knowing it can prune the dead ends and keep only what matters for the current task.

Static vs In-Run Context

Every message in the context belongs to one of two streams:

• user_context -- All User messages: the initial prompt, follow-ups, steering messages, and any user-injected content. These represent user intent.
• inrun_context -- All model-generated content: Assistant messages, ToolCall content, and ToolResult messages. These are the model's working memory.

The system_prompt is separate from both streams. It is a dedicated field on AgentContext, always occupies the first position, and is never subject to pruning or compaction.

Pruning Rules

• user_context is NEVER pruned. User intent is sacred. The model cannot discard the user's words, steering messages, or follow-up instructions.
• inrun_context CAN be surgically pruned by the model using the PrunTool. The model decides what is no longer relevant and removes it.
• system_prompt is never pruned. It is not part of either stream.

PrunTool Variants

phi-core provides two pruning operations, both invoked by the model as tool calls:

prun(tokens)

Silent removal. The model specifies a token budget to reclaim, and the oldest inrun_context entries (by timestamp) are removed from the working context until the budget is met.

• Removed content is preserved in the session log -- nothing is lost permanently
• Removed entries become invisible to the LLM on subsequent turns
• The model sees a ToolResult confirming how many tokens were reclaimed

prun_with_memo(tokens, memo)

Removal with summary. Same as prun, but the model provides a concise memo string that replaces the pruned content in the working context.

• Each pruned message with a memo creates a separate PrunedMemo entry at its original timestamp, preserving chronological order
• Useful when the pruned content contained decisions or conclusions the model wants to remember
• The memo should be concise -- a few sentences, not a reproduction of the pruned content

Model Autonomy

The model decides which variant to use and when. Typical patterns:

• Silent prune after exploring a dead end (e.g., reading a file that turned out to be irrelevant)
• Memo prune after a productive investigation (e.g., "Investigated auth module: uses JWT with RS256, tokens expire after 1h, refresh handled in middleware")
• No prune when all context remains relevant to the current task

Working Context Rebuild

Each turn, the working context sent to the LLM is rebuilt from scratch by build_working_context(), merging the two streams:

1. Collect all user_context entries with their timestamps
2. Collect all live inrun_context entries with their timestamps
3. For each PrunedMemo entry, create a separate User message with the memo text at the entry's original timestamp
4. Sort all collected entries by timestamp to preserve chronological order
5. Prepend the system_prompt

The result is a coherent conversation history where:

• User messages are always present
• Pruned-silent entries are invisible (the conversation flows as if they never existed)
• Each pruned-with-memo entry appears as a separate brief summary message at its original timestamp position, preserving the chronological position of the message it replaced

Session Log Integrity

The session log (context.messages) records everything that happened during the run. Pruning never modifies the session log -- it only affects what the LLM sees in the working context.

PrunRecord

Each prune operation emits a PrunApplied event (recorded in LoopRecord.events by SessionRecorder) containing:

• pruned_timestamps -- Vec<u64> of timestamps identifying the pruned messages
• tokens_removed -- Total tokens reclaimed
• messages_removed -- Number of messages pruned
• memo -- Optional summary string (present only for prun_with_memo)

On session reload, the two context streams are reconstructed by walking LoopRecord.events to find PrunApplied events. The pruned_timestamps field identifies which messages were pruned. These messages are placed in the pruned state (PrunedSilent or PrunedMemo depending on whether memo is Some), and their memo (if any) is restored as a separate message at the correct chronological position.

Compaction Interaction

Pruning and compaction are complementary mechanisms that operate at different levels:

After Compaction

When compaction fires, it summarizes a range of messages into a CompactionBlock. After compaction:

• All surviving messages (the summary, kept-first, and kept-recent) become part of user_context -- they are treated as established context and are unprunable
• New model-generated content after compaction starts a fresh inrun_context stream
• The model can prune this new inrun_context as usual

This means compaction resets the pruning boundary. Content that was once prunable inrun_context, if it survives compaction, becomes permanent user_context.

Configuration

TOML

[tools]
enabled = ["bash", "read_file", "write_file", "edit_file", "search", "prun"]

Adding "prun" to the enabled tools list makes both prun and prun_with_memo available to the model. They are two operations exposed through a single tool registration.

Rust (Programmatic)

#![allow(unused)]
fn main() {
use phi_core::agents::BasicAgent;
use phi_core::provider::ModelConfig;

let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Sonnet", &api_key))
    .with_default_tools()
    .with_prun_tool();  // enables context pruning
}

The with_prun_tool() builder method registers the PrunTool, making both pruning variants available. It can be combined with any other tool configuration.

Recommended Setup

Context pruning works best when compaction is also configured, providing both surgical (model-directed) and bulk (automatic) context management:

[tools]
enabled = ["bash", "read_file", "write_file", "edit_file", "search", "prun"]

[compaction]
max_context_tokens = 200000
compact_at_pct = 0.85
keep_first_turns = 2
keep_recent_turns = 4

With this setup, the model can prune irrelevant exploration results as it works, and compaction provides a safety net if the context still grows too large.

Configuration Guide

Define your entire agent in a config file — model, tools, compaction, limits — and construct it with two lines of Rust:

use phi_core::{parse_config_file, agent_from_config, Agent};
use std::path::Path;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = parse_config_file(Path::new("agent.toml"))?;
    let agent = agent_from_config(&config)?;

    // agent is Arc<dyn Agent> — ready to prompt
    println!("Agent model: {:?}", agent.model_config().unwrap().id);
    Ok(())
}

Overview

The configuration system replaces scattered Rust builder calls with a declarative config file. Instead of this:

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(ModelConfig::anthropic("claude-sonnet-4-20250514", "Sonnet", &key))
    .with_system_prompt("You are a coding assistant.")
    .with_thinking(ThinkingLevel::High)
    .with_temperature(0.2)
    .with_execution_limits(ExecutionLimits { max_turns: 50, .. })
    .with_context_config(ContextConfig { .. });
}

You write a TOML file:

[agent]
system_prompt = "You are a coding assistant."

[agent.profile]
thinking_level = "high"
temperature = 0.2

[provider]
model = "claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"

[execution]
max_turns = 50

Three formats supported: TOML (primary, Rust-idiomatic), JSON (programmatic generation), YAML (human-friendly alternative).

Pipeline: Config file → parse_config_file() → AgentConfig struct → agent_from_config() → Arc<dyn Agent>

Quick Start

1. Create agent.toml:

[provider]
model = "claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"

2. Load and use it:

use phi_core::{parse_config_file, agent_from_config, Agent};
use std::path::Path;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let config = parse_config_file(Path::new("agent.toml"))?;
    let agent = agent_from_config(&config)?;

    // The agent is an Arc<dyn Agent> wrapping a BasicAgent internally.
    // Access configuration through trait methods:
    let model = agent.model_config().unwrap();
    println!("Using model: {} via {}", model.id, model.provider);

    Ok(())
}

Only the [provider] section is required. Everything else has sensible defaults.

Config Formats

TOML (Recommended)

The primary format. Clean, readable, Rust-idiomatic.

#![allow(unused)]
fn main() {
use phi_core::config::{parse_config, ConfigFormat};

let toml_str = r#"
[provider]
model = "claude-sonnet-4-20250514"
api_key = "sk-..."
"#;
let config = parse_config(toml_str, ConfigFormat::Toml)?;
}

JSON

Useful when generating config programmatically.

#![allow(unused)]
fn main() {
let json_str = r#"{ "provider": { "model": "gpt-4o", "api_key": "sk-...", "api": "openai" } }"#;
let config = parse_config(json_str, ConfigFormat::Json)?;
}

YAML

Human-friendly alternative.

#![allow(unused)]
fn main() {
let yaml_str = "provider:\n  model: claude-sonnet-4-20250514\n  api_key: sk-...";
let config = parse_config(yaml_str, ConfigFormat::Yaml)?;
}

Auto-Detection

parse_config_file detects format from the file extension:

parse_config_auto tries all formats in order (TOML → JSON → YAML) and returns the first successful parse.

Environment Variable Substitution

Any string field in the config can reference environment variables with ${VAR}:

[provider]
api_key = "${ANTHROPIC_API_KEY}"
base_url = "${CUSTOM_API_URL}"

[agent]
system_prompt = "Running in ${ENVIRONMENT} mode."

How it works:

• Substitution happens before parsing (pre-parse text replacement)
• Works in all three formats (TOML, JSON, YAML)
• Missing variables produce ConfigError::MissingEnvVar
• Malformed patterns like ${UNCLOSED are passed through literally
• Empty ${} is passed through literally

Agent Profile

An AgentProfile is a reusable blueprint that defines default configuration. Multiple agent instances can share the same profile while overriding specific fields.

[agent.profile]
name = "coding-agent"
description = "An agent specialized for code generation and review"
system_prompt = "You are an expert software engineer."
thinking_level = "high"
temperature = 0.2
max_tokens = 16384
config_id = "coder"
skills = ["code-review", "debugging"]

System Prompt Resolution

The system prompt is resolved through a priority chain. The first non-empty value wins:

1. [agent].system_prompt — explicit agent-level override (highest priority)
2. Profile instance system_prompt — when an agent instance references a profile via {{agent_profile.name}}
3. [agent.profile].system_prompt — base inline profile fallback
4. Empty string (no system prompt)

Inline Text

The simplest form — write the prompt directly:

[agent.profile]
system_prompt = "You are an expert software engineer."

Agent-level overrides the profile:

[agent.profile]
system_prompt = "You are a general assistant."   # default from blueprint

[agent]
system_prompt = "You are a Python specialist."   # overrides the profile

File Reference (file: prefix)

Load the prompt from a file. Relative paths resolve from the agent's workspace directory:

[agent]
workspace = "workspace"

[agent.profile]
system_prompt = "file:system_prompt.md"        # resolves to workspace/system_prompt.md

Absolute paths are used as-is:

[agent.profile]
system_prompt = "file:/etc/phi/prompts/coder.md"

The file: prefix works at all levels: [agent].system_prompt, [agent.profile].system_prompt, and [[agent.profile.instances]].system_prompt.

Strategy Reference ({{...}} protocol)

For advanced multi-block prompt composition, reference a system prompt instance. This uses a 3-entity chain: strategy (block template) → prompt instance (block content) → agent reference.

# 1. Define the strategy template — block structure with ordering and size limits
[[system_prompt_strategy.instances]]
id = "{{coding_strategy}}"

[[system_prompt_strategy.instances.blocks]]
name = "identity"
order = 0
max_length = 2000

[[system_prompt_strategy.instances.blocks]]
name = "instructions"
order = 1
max_length = 3000

[[system_prompt_strategy.instances.blocks]]
name = "constraints"
order = 2
max_length = 1000

# 2. Define the prompt instance — fills content into the strategy's blocks
#    Block values can be inline text or file: references (relative to workspace)
[[system_prompt.instances]]
id = "{{coding_prompt}}"
description = "System prompt for coding agents"
type = "{{system_prompt_strategy.coding_strategy}}"
identity = "You are an expert software engineer at a fintech company."
instructions = "file:prompts/coding_instructions.md"
constraints = "Never modify production databases. Always write tests."

# 3. Reference the prompt instance from the agent
[agent]
system_prompt = "{{system_prompt.coding_prompt}}"
workspace = "workspace"

The builder resolves the chain: finds the prompt instance → finds its strategy → sorts blocks by order → resolves file: paths → truncates each block to max_length → joins with double newlines.

See the Field Reference for [system_prompt_strategy] and [system_prompt] sections.

Profile Instance Override

When using named profile instances, the instance's system_prompt participates in resolution. The system_prompt field on a profile instance supports all three modes — inline text, file: path, or {{...}} reference to a system_prompt instance:

# ── System prompt strategy + instance (reusable prompt definition) ───
[[system_prompt_strategy.instances]]
id = "{{simple}}"

[[system_prompt_strategy.instances.blocks]]
name = "identity"
order = 0
max_length = 5000

[[system_prompt.instances]]
id = "{{coder_prompt}}"
type = "{{system_prompt_strategy.simple}}"
identity = "file:prompts/coder.md"

[[system_prompt.instances]]
id = "{{reviewer_prompt}}"
type = "{{system_prompt_strategy.simple}}"
identity = "file:prompts/reviewer.md"

# ── Profile instances reference system_prompt instances ──────────────
[agent.profile]
name = "base"
system_prompt = "You are a general assistant."   # base fallback

[[agent.profile.instances]]
id = "{{coder}}"
system_prompt = "{{system_prompt.coder_prompt}}"   # profile → system_prompt instance
temperature = 0.2
max_tokens = 16384

[[agent.profile.instances]]
id = "{{reviewer}}"
system_prompt = "{{system_prompt.reviewer_prompt}}" # profile → system_prompt instance
temperature = 0.1
max_tokens = 8192

# ── Agent instances reference profile instances ──────────────────────
[[agent.instances]]
name = "code-writer"
agent_profile = "{{agent_profile.coder}}"          # agent → profile → system_prompt

[[agent.instances]]
name = "code-reviewer"
agent_profile = "{{agent_profile.reviewer}}"       # agent → profile → system_prompt

[[agent.instances]]
name = "generalist"
# no agent_profile → uses base [agent.profile].system_prompt

Full reference chain: agent.instances → agent.profile.instances (via agent_profile) → system_prompt.instances (via system_prompt) → system_prompt_strategy.instances (via type). Each layer can override or inherit from the one above.

When an agent instance omits agent_profile, it is built using the base [agent.profile] directly (no instance override). The base profile's system_prompt, temperature, and other fields apply as defaults.

Workspace-relative Resolution

The file: prefix resolves relative to the agent's workspace directory. Each agent instance can set its own workspace, so the same file: reference resolves to different files per agent:

[agent.profile]
name = "base"

[[agent.profile.instances]]
id = "{{copywriter}}"
system_prompt = "file:system_prompt.md"   # same file ref, different workspace resolution
temperature = 0.7

[[agent.instances]]
name = "alpha-writer"
agent_profile = "{{agent_profile.copywriter}}"
workspace = "projects/alpha"              # reads projects/alpha/system_prompt.md

[[agent.instances]]
name = "beta-writer"
agent_profile = "{{agent_profile.copywriter}}"
workspace = "projects/beta"               # reads projects/beta/system_prompt.md

Workspace resolution order:

1. [[agent.instances]].workspace — per-agent instance (highest priority)
2. [agent].workspace — shared agent-level
3. default_workspace — global default
4. "." — current directory

Thinking Level

Controls depth of model reasoning. Specified as a string in config:

Parsing is case-insensitive: "High", "HIGH", "high" all work.

Skills vs Tools

skills in the profile are skill names loaded via SkillSet from SKILL.md files (per the AgentSkills standard). They are NOT tools. See Skills for details.

Profile Instances

Profile instances are named variations of the profile blueprint. Each instance inherits the profile defaults and overrides specific fields. This lets you define a single profile and then create specialized variants without duplicating the entire configuration.

Use [[agent.profile.instances]] to define instances. Each instance requires an id field using the {{...}} ID reference protocol (see below). Instance fields override the corresponding profile defaults; any field not specified falls through to the profile value.

Agent instances reference a profile instance via the agent_profile field, using either a qualified reference ({{agent_profile.name}}) or an unqualified reference ({{name}}) if the name is unique across all namespaces.

Example

# ── Profile defaults ──────────────────────────────────────────
[agent.profile]
name = "coding-agent"
description = "An agent specialized for code tasks"
system_prompt = "You are an expert software engineer."
thinking_level = "high"
temperature = 0.2
max_tokens = 16384

# ── Profile instances (override specific fields) ─────────────
[[agent.profile.instances]]
id = "{{%coder%}}"
description = "A code generation specialist"
thinking_level = "high"
temperature = 0.2
max_tokens = 16384
config_id = "coder"

[[agent.profile.instances]]
id = "{{%reviewer%}}"
description = "A code review specialist"
thinking_level = "high"
temperature = 0.1
max_tokens = 8192
config_id = "reviewer"

# ── Agent instances referencing profile instances ─────────────
[[agent.instances]]
name = "code-writer"
agent_profile = "{{agent_profile.coder}}"
system_prompt = "You write clean, well-tested code. Follow existing patterns."

[[agent.instances]]
name = "code-reviewer"
agent_profile = "{{agent_profile.reviewer}}"
system_prompt = "You review code for bugs, security issues, and style violations."

The code-writer agent inherits all profile defaults and applies the coder instance overrides. The code-reviewer agent uses the reviewer instance, which sets a lower temperature and smaller token budget for more focused review output.

ID Reference Protocol

The {{...}} syntax is a lightweight reference protocol for linking configuration entities (providers, profile instances, sub-agents) by name. It appears in id fields (to declare an entity) and in reference fields like provider and agent_profile (to point to an entity).

Syntax

Namespaces

References are resolved within namespaces. The three namespaces are:

• agent_profile -- Profile instances declared in [[agent.profile.instances]]
• provider -- Provider instances declared in [[provider.instances]]
• sub_agent -- Sub-agent instances declared in [[sub_agents.instances]]

Resolution

Qualified references ({{type.name}}) include the namespace prefix and always resolve unambiguously. Use these when multiple namespaces could contain the same name.

Unqualified references ({{name}}) omit the namespace. The system searches all namespaces and resolves the reference only if the name is unique. If multiple entities share the same name across namespaces, an unqualified reference is ambiguous and will produce an error.

Recreation Semantics

The % sigil controls whether an entity is recreated when referenced:

• Without % ({{name}} or {{type.name}}): The entity is recreated each time it is resolved. Use this when you want fresh instances.
• With % ({{%name%}} or {{%type.name%}}): The entity is reused if it already exists (matched by latest creation date). Use this for shared singletons like provider connections.

The {{#system_id#}} form references a literal system-generated ID and never triggers recreation.

Usage in ID Fields

When declaring an entity, the id field establishes the entity's name within its namespace:

[[provider.instances]]
id = "{{%openai%}}"          # declares "openai" in the provider namespace
model = "gpt-4o"

Usage in Reference Fields

When referencing an entity from another section, use the reference syntax:

[[agent.instances]]
name = "my-agent"
provider = "{{provider.openai}}"       # qualified reference
agent_profile = "{{reviewer}}"         # unqualified (must be unique)

Provider Configuration

The [provider] section defines the LLM model, API credentials, and protocol.

[provider]
model = "claude-sonnet-4-20250514"    # Model ID sent to the API
api_key = "${ANTHROPIC_API_KEY}"      # API credential
api = "anthropic_messages"            # API protocol
provider = "anthropic"                # Provider name
name = "Claude Sonnet 4"             # Human-friendly display name
reasoning = true                      # Model supports thinking
context_window = 200000               # Context window in tokens
max_tokens = 8192                     # Default max output tokens

API Protocols

Default base URLs are set automatically per protocol when base_url is omitted:

• Anthropic: https://api.anthropic.com
• OpenAI: https://api.openai.com
• Google: https://generativelanguage.googleapis.com
• Others: empty (uses provider defaults)

Important: The API protocol is NOT auto-detected from the model name. If you set model = "gpt-4o", you must also set api = "openai" explicitly.

Cost Rates

Enable cost tracking by setting per-token rates:

[provider.cost]
input_per_million = 3.0       # $ per million input tokens
output_per_million = 15.0     # $ per million output tokens
cache_read_per_million = 0.3  # $ per million cache-read tokens
cache_write_per_million = 3.75

Cost is tracked automatically after each turn. Combine with [execution].max_cost to enforce a budget.

Custom Headers

[provider]
model = "my-model"

[provider.headers]
"X-Custom-Header" = "value"
"Authorization" = "Bearer ${CUSTOM_TOKEN}"

Multiple Providers

Use [[provider.instances]] to define named providers alongside the default. Each instance uses the {{...}} ID reference protocol to declare its name in the provider namespace. The url field is an alias for base_url.

# Default provider — Anthropic (used unless overridden)
[provider]
model = "claude-sonnet-4-20250514"
name = "Claude Sonnet 4"
api_key = "${ANTHROPIC_API_KEY}"
api = "anthropic_messages"
provider = "anthropic"

[provider.cost]
input_per_million = 3.0
output_per_million = 15.0
cache_read_per_million = 0.3
cache_write_per_million = 3.75

# OpenAI
[[provider.instances]]
id = "{{%openai%}}"
description = "OpenAI GPT-4o provider"
name = "GPT-4o"
model = "gpt-4o"
api_key = "${OPENAI_API_KEY}"
api = "openai_completions"
url = "https://api.openai.com/v1"

# OpenRouter
[[provider.instances]]
id = "{{%openrouter%}}"
description = "OpenRouter multi-model gateway"
name = "OpenRouter"
model = "anthropic/claude-sonnet-4"
api_key = "${OPENROUTER_API_KEY}"
api = "openai_completions"
url = "https://openrouter.ai/api/v1"
provider = "openrouter"

# Google Gemini
[[provider.instances]]
id = "{{%gemini%}}"
description = "Google Gemini 2.5 Flash provider"
name = "Gemini 2.5 Flash"
model = "gemini-2.5-flash"
api_key = "${GOOGLE_API_KEY}"
api = "google_generative_ai"

# Ollama (local)
[[provider.instances]]
id = "{{%ollama%}}"
description = "Local Ollama instance for development"
name = "Ollama Llama 3.2"
model = "llama3.2"
api = "openai_completions"
url = "http://localhost:11434/v1"
api_key = "not-needed"
provider = "ollama"

Agent instances and sub-agents reference these via the ID protocol (e.g., provider = "{{provider.openai}}" or provider = "{{ollama}}" if unique).

Session Configuration

The [session] section controls session scope.

[session]
scope = "persistent"       # "ephemeral" (default) or "persistent"

Session Scope

Note: Setting scope = "persistent" declares intent but does not automatically configure a storage backend. The caller must set up session persistence using the session recorder.

Thinking level and temperature are configured per-loop via LoopConfigSnapshot (captured on each AgentStart event) rather than at the session level. Set them on the agent profile or AgentLoopConfig.

Tools

The [tools] section declares which tools the agent can use and how they execute.

[tools]
enabled = ["bash", "file_read", "file_write", "search"]
tool_strategy = "parallel"   # "sequential", "parallel", or "batched"
batch_size = 3               # Only used when strategy is "batched"

Tool Execution Strategies

Context Pruning

Enable model-directed context pruning with with_prun_tool(). This lets the model surgically remove irrelevant inrun content (its own messages, tool calls, tool results) from the working context to reclaim space in the context window. User messages are never pruned. See Context Pruning for details.

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(model_config)
    .with_default_tools()
    .with_prun_tool();
}

Or via config:

[tools]
enabled = ["bash", "read_file", "write_file", "prun"]

Registering Tools at Runtime

Tools are NOT instantiated from the config file. The config specifies tool names only. You must register tool instances after constructing the agent:

#![allow(unused)]
fn main() {
use phi_core::{parse_config_file, agent_from_config, Agent};
use phi_core::tools::{BashTool, ReadFileTool, WriteFileTool, SearchTool};
use std::sync::Arc;

let config = parse_config_file(Path::new("agent.toml"))?;
let agent = agent_from_config(&config)?;

// Cast to mutable and register tools
let agent_mut = Arc::get_mut(&mut agent).unwrap();
agent_mut.set_tools(vec![
    Arc::new(BashTool::default()),
    Arc::new(ReadFileTool::new()),
    Arc::new(WriteFileTool::new()),
    Arc::new(SearchTool::new()),
]);
}

Tool Registry

Instead of manually registering tools after construction, use agent_from_config_with_registry() to resolve tool names from the config automatically:

#![allow(unused)]
fn main() {
use phi_core::{parse_config_file, agent_from_config_with_registry, Agent};
use phi_core::tools::ToolRegistry;
use std::path::Path;

let config = parse_config_file(Path::new("agent.toml"))?;

// Create a registry with the 6 built-in tools
let registry = ToolRegistry::new().with_defaults();

// Tools listed in config.tools.enabled are resolved through the registry
let agent = agent_from_config_with_registry(&config, &registry)?;
}

The default registry includes all 6 built-in tools: bash, read_file, write_file, edit_file, list_files, search. You can also register custom tools:

#![allow(unused)]
fn main() {
let mut registry = ToolRegistry::new().with_defaults();
registry.register("my_tool", || Arc::new(MyCustomTool::new()));

let agent = agent_from_config_with_registry(&config, &registry)?;
}

Unknown tool names in tools.enabled are silently skipped. Use registry.contains(name) to check availability before construction if needed.

Context & Compaction

The [compaction] section controls automatic context management. When the conversation grows too long, compaction summarizes older messages to stay within the model's context window.

[compaction]
max_context_tokens = 200000     # Model's context window
system_prompt_tokens = 4000     # Tokens reserved for system prompt
compact_at_pct = 0.85           # Start measuring at 85% capacity
compact_budget_threshold_pct = 0.05  # Compact when < 5% headroom remains
keep_first_turns = 2            # Keep first 2 turns verbatim
keep_recent_turns = 4           # Keep last 4 turns verbatim
max_summary_tokens = 2000       # Token budget for the summarized middle
tool_output_max_lines = 50      # Truncate tool outputs to 50 lines

Compaction must be explicitly enabled by setting max_context_tokens. If omitted, compaction is disabled entirely.

How Compaction Works

1. Before each LLM turn, the loop estimates current token usage
2. If usage exceeds the trigger threshold, compaction fires
3. First N turns are kept verbatim (preserves initial context)
4. Middle turns are summarized (aggressive token reduction)
5. Last M turns are kept verbatim (preserves recent history)
6. Tool outputs in kept turns are truncated to max_lines

See Context Compaction for the full algorithm.

Focused Compaction

The focus_message field steers what the compaction summary emphasizes. Compaction instances let you define named variations that agent profiles can reference.

[compaction]
max_context_tokens = 200000
focus_message = "Retain key decisions and code changes."

# Named compaction instances
[[compaction.instances]]
id = "{{%coding%}}"
focus_message = "Focus on file paths, function signatures, and design rationale."
keep_recent_turns = 6
max_summary_tokens = 3000

[[compaction.instances]]
id = "{{%research%}}"
focus_message = "Preserve citations, data sources, and methodology."
keep_first_turns = 3
max_summary_tokens = 4000

Profiles reference a compaction instance via compaction = "{{compaction.coding}}":

[agent.profile]
name = "coding-agent"
compaction = "{{compaction.coding}}"

See Focused Compaction for full details.

Execution Limits

The [execution] section sets safety guards that prevent runaway loops and budget overruns.

[execution]
max_turns = 50              # Maximum LLM turns (default: 50)
max_total_tokens = 1000000  # Total token budget (default: 1,000,000)
max_duration_secs = 600     # Wall-clock timeout in seconds (default: 600)
max_cost = 5.0              # Dollar cost cap (requires [provider.cost] rates)

Cost Tracking

Cost enforcement requires both cost rates and a budget:

[provider.cost]
input_per_million = 3.0
output_per_million = 15.0

[execution]
max_cost = 5.0   # Stop when accumulated cost reaches $5

Without cost rates (all zeros), max_cost has no effect. Token usage is always tracked regardless.

Retry Configuration

Automatic retry for transient provider errors (rate limits, network issues):

[execution.retry]
max_retries = 3           # Retry attempts (default: 3, 0 = disabled)
initial_delay_ms = 1000   # First retry delay in ms
backoff_multiplier = 2.0  # Exponential backoff multiplier
max_delay_ms = 30000      # Maximum delay cap

Only RateLimited and Network errors are retried. Invalid requests and context overflows fail immediately.

Cache Configuration

Control prompt caching behavior:

[execution.cache]
enabled = true        # Master switch (default: true)
strategy = "auto"     # "auto" or "disabled"

Sub-Agents

Define sub-agents that run their own agent loops when invoked as tools:

[[sub_agents.instances]]
name = "researcher"
description = "Searches the web for information"
system_prompt = "You are a research assistant. Search thoroughly."
model = "claude-haiku-4-5-20251001"
max_turns = 10
tools = ["web_search"]

[[sub_agents.instances]]
name = "code_writer"
description = "Writes and edits code files"
system_prompt = "You are a code generation expert."
provider = "openai"    # References a [[provider.instances]] by name
max_turns = 20
tools = ["bash", "file_write"]

Sub-agents do NOT inherit the parent agent's configuration. Each sub-agent is fully independent — set all needed fields explicitly.

Multi-Agent Configurations

For complex setups, combine named providers with named agent instances:

# Providers
[provider]
model = "claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"

[[provider.instances]]
name = "fast"
model = "claude-haiku-4-5-20251001"
api_key = "${ANTHROPIC_API_KEY}"

# Agent instances
[[agent.instances]]
name = "planner"
system_prompt = "You are an architect. Plan the approach."
provider = "fast"

[[agent.instances]]
name = "executor"
system_prompt = "You are an implementer. Write the code."

Agent Workspace

The workspace field sets the working directory for an agent. Tools that interact with the filesystem (bash, file read/write, etc.) use this as their base path.

There are two levels of workspace configuration:

• default_workspace (top-level config field): Sets the default workspace for all agents. If omitted, the current working directory is used.
• workspace (per-agent field on [agent.profile] or [[agent.instances]]): Overrides default_workspace for a specific agent.

default_workspace = "/home/user/projects"

[agent.profile]
workspace = "/home/user/projects/my-app"   # overrides default_workspace for this agent

Callbacks & Hooks

The config schema accepts [callbacks] and [hooks] sections for lifecycle hooks:

[callbacks]
before_loop = "my_plugin::before_loop"
after_turn = "my_plugin::after_turn"
before_task = "./scripts/on_task_start.sh"
after_task = "python3 scripts/after_task.py"

[hooks]
transform_context = "my_plugin::transform"

Script-based callbacks (shell scripts, Python scripts) are supported. The agent spawns the script as a subprocess, passing context via environment variables. Exit code 0 means continue; non-zero aborts the action (for Before* hooks). WASM plugin loading for Rust-native callbacks is planned for Phase 2.

Session-Level Callbacks

before_task and after_task are session-level callbacks configured on SessionRecorderConfig:

• before_task: Fires on the first AgentStart event with a new session_id. Use for task-level setup, metrics initialization, or audit logging.
• after_task: Fires on flush(). Use for task-level teardown, billing, or summary generation.

Programmatic Hooks

To set hooks programmatically, use the Agent trait setter methods after construction:

#![allow(unused)]
fn main() {
let agent = agent_from_config(&config)?;
let agent_mut = Arc::get_mut(&mut agent).unwrap();
agent_mut.set_before_loop(Some(Arc::new(|msgs, n| {
    println!("Loop starting with {} messages", msgs.len());
    true // return false to abort
})));
}

Complete Example

A full coding agent configuration using every section:

# ── Agent identity ────────────────────────────────────────────
[agent]
system_prompt = "You are an expert software engineer."

[agent.profile]
name = "coding-agent"
description = "Full-featured coding assistant"
thinking_level = "high"
temperature = 0.2
max_tokens = 16384
config_id = "coder-v1"
skills = ["code-review"]

# ── Provider ──────────────────────────────────────────────────
[provider]
model = "claude-sonnet-4-20250514"
api_key = "${ANTHROPIC_API_KEY}"
reasoning = true
context_window = 200000

[provider.cost]
input_per_million = 3.0
output_per_million = 15.0
cache_read_per_million = 0.3
cache_write_per_million = 3.75

# ── Session ───────────────────────────────────────────────────
[session]
scope = "persistent"

# ── Tools ─────────────────────────────────────────────────────
[tools]
enabled = ["bash", "file_read", "file_write", "search", "edit_file"]
tool_strategy = "parallel"

# ── Context management ────────────────────────────────────────
[compaction]
max_context_tokens = 200000
system_prompt_tokens = 4000
compact_at_pct = 0.85
keep_first_turns = 2
keep_recent_turns = 4
max_summary_tokens = 2000
tool_output_max_lines = 50

# ── Execution limits ──────────────────────────────────────────
[execution]
max_turns = 100
max_total_tokens = 2000000
max_duration_secs = 1800
max_cost = 10.0

[execution.retry]
max_retries = 3
initial_delay_ms = 1000
backoff_multiplier = 2.0

[execution.cache]
enabled = true
strategy = "auto"

# ── Sub-agents ────────────────────────────────────────────────
[[sub_agents.instances]]
name = "researcher"
description = "Searches for information and documentation"
system_prompt = "Find relevant information. Be thorough."
model = "claude-haiku-4-5-20251001"
max_turns = 10
tools = ["web_search"]

Field Reference

[agent]

[agent.profile]

ProfileInstanceSection

Each entry in [[agent.profile.instances]]:

AgentInstanceSection

Each entry in [[agent.instances]]:

[provider]

ProviderInstanceSection

Each entry in [[provider.instances]] accepts all fields from [provider] above, plus:

[provider.cost]

[session]

[tools]

[compaction]

[system_prompt_strategy]

Strategy templates define block structure for multi-block system prompts.

[[system_prompt_strategy.instances]]

[[system_prompt_strategy.instances.blocks]]

[system_prompt]

Prompt instances fill content into a strategy's blocks.

[[system_prompt.instances]]

Note: Block content fields use #[serde(flatten)] — they appear as top-level keys on the instance, not nested under a blocks table.

[execution]

[execution.retry]

[execution.cache]

Error Handling

agent_from_config() and the parse functions return ConfigError:

Common Mistakes

Forgetting to set the API protocol for non-Anthropic models:

# Wrong — defaults to anthropic_messages, fails at runtime
[provider]
model = "gpt-4o"
api_key = "${OPENAI_API_KEY}"

# Correct
[provider]
model = "gpt-4o"
api_key = "${OPENAI_API_KEY}"
api = "openai"

Setting max_cost without cost rates:

# max_cost is ignored — no rates to compute cost from
[execution]
max_cost = 5.0

# Correct — set rates AND budget
[provider.cost]
input_per_million = 3.0
output_per_million = 15.0

[execution]
max_cost = 5.0

Expecting tools to be instantiated from config:

[tools]
enabled = ["bash", "file_read"]
# These are names only — you must call agent.set_tools() in Rust

Programmatic Usage

Using AgentConfig Directly

You can construct AgentConfig in Rust without a file:

#![allow(unused)]
fn main() {
use phi_core::config::schema::{AgentConfig, ProviderSection, ProfileSection, AgentSection};

let config = AgentConfig {
    provider: ProviderSection {
        model: Some("claude-sonnet-4-20250514".into()),
        api_key: Some(std::env::var("ANTHROPIC_API_KEY")?),
        ..Default::default()
    },
    agent: AgentSection {
        system_prompt: Some("You are helpful.".into()),
        profile: ProfileSection {
            thinking_level: Some("high".into()),
            ..Default::default()
        },
        ..Default::default()
    },
    ..Default::default()
};

let agent = agent_from_config(&config)?;
}

Mixing Config with Programmatic Overrides

After agent_from_config(), use Agent trait methods to add hooks, tools, or modify settings:

#![allow(unused)]
fn main() {
use phi_core::{parse_config_file, agent_from_config, Agent};
use std::sync::Arc;

let config = parse_config_file(Path::new("agent.toml"))?;
let mut agent = agent_from_config(&config)?;

// Get mutable access to add tools and hooks
let a = Arc::get_mut(&mut agent).unwrap();
a.set_tools(vec![Arc::new(phi_core::tools::BashTool::default())]);
a.set_before_loop(Some(Arc::new(|msgs, _| {
    println!("Starting with {} messages", msgs.len());
    true
})));
}

Reading Config Through the Agent Trait

All configuration is accessible through Agent trait methods:

#![allow(unused)]
fn main() {
let agent = agent_from_config(&config)?;

// Config accessors (all have defaults)
agent.model_config();       // Option<&ModelConfig>
agent.profile();            // Option<&AgentProfile>
agent.system_prompt();      // &str
agent.thinking_level();     // ThinkingLevel
agent.temperature();        // Option<f32>
agent.max_tokens();         // Option<u32>
agent.context_config();     // Option<&ContextConfig>
agent.execution_limits();   // Option<&ExecutionLimits>
agent.cache_config();       // CacheConfig
agent.tool_execution();     // ToolExecutionStrategy
agent.retry_config();       // RetryConfig
agent.session();            // Option<&Session>
agent.build_config();       // Result<AgentLoopConfig, AgentBuildError>
                            // Default impl returns Err(MissingModelConfig)
                            // if model_config() returns None. BasicAgent's
                            // override always returns Ok(...).
}

MCP Integration

What is MCP?

The Model Context Protocol (MCP) is a JSON-RPC 2.0 protocol that lets AI agents discover and call tools from external servers. It defines a standard way for agents to connect to tool providers over two transports:

• Stdio — spawn a child process, communicate via stdin/stdout (newline-delimited JSON)
• HTTP — POST JSON-RPC requests to an HTTP endpoint

Connecting to MCP Servers

Stdio Transport

Use with_mcp_server_stdio() to spawn an MCP server process and register its tools:

use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("ANTHROPIC_API_KEY")?;
    let mut agent = BasicAgent::new(ModelConfig::anthropic(
        "claude-sonnet-4-20250514",
        "Claude Sonnet 4",
        &api_key,
    ))
    .with_system_prompt("You are a helpful assistant with file access.")
    .with_mcp_server_stdio(
        "npx",
        &["-y", "@modelcontextprotocol/server-filesystem", "/tmp"],
        None,
    )
    .await?;

    let rx = agent.prompt("List files in /tmp").await;
    // handle events...
    Ok(())
}

You can pass environment variables to the server process:

#![allow(unused)]
fn main() {
use std::collections::HashMap;

let mut env = HashMap::new();
env.insert("API_TOKEN".into(), "secret".into());

let agent = BasicAgent::new(model_config)
    .with_mcp_server_stdio("my-mcp-server", &["--port", "0"], Some(env))
    .await?;
}

HTTP Transport

For remote MCP servers exposed over HTTP:

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(model_config)
    .with_mcp_server_http("http://localhost:8080/mcp")
    .await?;
}

How MCP Tools Work

When you call with_mcp_server_stdio() or with_mcp_server_http(), phi-core:

1. Connects to the MCP server and performs the initialize handshake
2. Calls tools/list to discover available tools
3. Wraps each MCP tool as an AgentTool via McpToolAdapter
4. Adds them to the agent's tool list

MCP tools appear alongside built-in tools. The LLM sees them with their original names, descriptions, and JSON Schema parameters — it can call them just like any other tool.

Mixing Built-in and MCP Tools

#![allow(unused)]
fn main() {
use phi_core::tools::default_tools;

let agent = BasicAgent::new(model_config)
    .with_tools(default_tools())  // bash, read, write, edit, list, search
    .with_mcp_server_stdio("my-db-server", &[], None)
    .await?;
// Agent now has both built-in coding tools AND MCP database tools
}

Using the MCP Client Directly

For lower-level control, use McpClient directly:

#![allow(unused)]
fn main() {
use phi_core::mcp::{McpClient, McpToolAdapter};
use std::sync::Arc;
use tokio::sync::Mutex;

let client = McpClient::connect_stdio("my-server", &[], None).await?;
let tools = client.list_tools().await?;

for tool in &tools {
    println!("{}: {}", tool.name, tool.description.as_deref().unwrap_or(""));
}

// Call a tool directly
let result = client.call_tool("read_file", serde_json::json!({"path": "/tmp/test.txt"})).await?;

// Or wrap as AgentTool adapters
let client = Arc::new(Mutex::new(client));
let adapters = McpToolAdapter::from_client(client).await?;
}

Error Handling

MCP operations return McpError:

• McpError::Transport — connection or I/O failure
• McpError::Protocol — unexpected response format
• McpError::JsonRpc — server returned a JSON-RPC error (code + message)
• McpError::Serialization — JSON serialization/deserialization failure
• McpError::Io — standard I/O error
• McpError::ConnectionClosed — server process exited

When an MCP tool returns isError: true, the adapter converts it to a ToolError::Failed, which the agent loop sends back to the LLM with is_error: true so it can self-correct.

OpenAPI Tool Adapter

Auto-generate AgentTool implementations from OpenAPI 3.0 specs. Point an agent at any API spec and it instantly gets callable tools for every operation.

Feature-gated — add features = ["openapi"] to your Cargo.toml.

Quick Start

use phi_core::BasicAgent;
use phi_core::openapi::{OpenApiToolAdapter, OpenApiConfig, OperationFilter};
use phi_core::provider::ModelConfig;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let api_key = std::env::var("ANTHROPIC_API_KEY")?;
    let config = OpenApiConfig::new()
        .with_bearer_token("sk-...");

    let agent = BasicAgent::new(ModelConfig::anthropic(
        "claude-sonnet-4-20250514",
        "Claude Sonnet 4",
        &api_key,
    ))
    .with_system_prompt("You are an API assistant.")
    .with_openapi_file("petstore.yaml", config, &OperationFilter::All)
    .await?;

    Ok(())
}

Loading Specs

Three ways to load an OpenAPI spec:

#![allow(unused)]
fn main() {
// From a file
let agent = agent.with_openapi_file("spec.yaml", config, &filter).await?;

// From a URL
let agent = agent.with_openapi_url("https://api.example.com/openapi.json", config, &filter).await?;

// From a string (sync)
let agent = agent.with_openapi_spec(&spec_string, config, &filter)?;
}

Or create adapters directly for more control:

#![allow(unused)]
fn main() {
let adapters = OpenApiToolAdapter::from_str(&spec, config, &OperationFilter::All)?;
let tools: Vec<Box<dyn AgentTool>> = adapters.into_iter().map(|a| Box::new(a) as _).collect();
}

Configuration

OpenApiConfig controls auth, headers, timeouts, and response limits:

#![allow(unused)]
fn main() {
let config = OpenApiConfig::new()
    .with_base_url("https://api.staging.example.com") // Override spec's servers
    .with_bearer_token("sk-...")                       // Bearer auth
    .with_header("X-Custom", "value")                  // Extra headers
    .with_timeout_secs(60)                             // Request timeout
    .with_max_response_bytes(128 * 1024)               // Truncate large responses
    .with_name_prefix("github");                       // Tool names: github__listRepos
}

Authentication

#![allow(unused)]
fn main() {
// Bearer token
let config = OpenApiConfig::new().with_bearer_token("token");

// API key in a custom header
let config = OpenApiConfig::new().with_api_key("X-API-Key", "key-value");

// No auth
let config = OpenApiConfig::new(); // default
}

Filtering Operations

Most API specs have dozens or hundreds of operations. Use OperationFilter to select which ones become tools:

#![allow(unused)]
fn main() {
// All operations (default)
let filter = OperationFilter::All;

// Specific operations by ID
let filter = OperationFilter::ByOperationId(vec![
    "listRepos".into(),
    "getRepo".into(),
    "createIssue".into(),
]);

// All operations with a specific tag
let filter = OperationFilter::ByTag(vec!["repos".into()]);

// All operations under a path prefix
let filter = OperationFilter::ByPathPrefix("/repos".into());
}

How It Works

Each OpenAPI operation becomes one AgentTool:

When the LLM calls a tool, the adapter:

1. Substitutes path parameters in the URL (/pets/{petId} → /pets/123)
2. Adds query parameters as ?key=value
3. Adds header parameters
4. Applies auth from config
5. Sends the request body as JSON (if the operation has one)
6. Returns the response text (with status code) to the LLM

Non-2xx responses are not treated as errors — they're returned as text so the LLM can reason about them and retry or adjust.

Mixing with Other Tools

OpenAPI tools work alongside built-in tools and MCP tools:

#![allow(unused)]
fn main() {
use phi_core::tools::default_tools;

let agent = BasicAgent::new(model_config)
    .with_tools(default_tools())
    .with_openapi_file("github.yaml", github_config, &github_filter).await?
    .with_mcp_server_stdio("db-server", &[], None).await?;
}

Limitations (v1)

• OpenAPI 3.0.x only (not 3.1.x)
• JSON request/response bodies only (no multipart/form-data)
• No OAuth2 or token refresh (pass tokens via OpenApiConfig)
• Operations without operationId are skipped
• Path-level $ref items are skipped

Providers Overview

phi-core supports multiple LLM providers through the StreamProvider trait and ApiProtocol dispatch. Callers never name a provider struct directly — ModelConfig is the single descriptor for every provider connection.

Supported Protocols

ApiProtocol Enum

#![allow(unused)]
fn main() {
pub enum ApiProtocol {
    AnthropicMessages,
    OpenAiCompletions,
    OpenAiResponses,
    AzureOpenAiResponses,
    GoogleGenerativeAi,
    GoogleVertex,
    BedrockConverseStream,
}
}

ModelConfig

ModelConfig is the single, complete description of a provider connection. Pass it to BasicAgent::new(), SubAgentTool::new(), or AgentLoopConfig.model_config:

#![allow(unused)]
fn main() {
pub struct ModelConfig {
    pub id: String,              // e.g. "gpt-4o" — model name sent to the API
    pub name: String,            // e.g. "GPT-4o" — display label for logging/UI
    pub api: ApiProtocol,        // Which wire protocol to use (dispatch key)
    pub provider: String,        // e.g. "openai" — logging label
    pub base_url: String,        // API endpoint (no trailing slash)
    pub api_key: String,         // Auth credential (sk-..., or "access_key:secret" for Bedrock)
    pub reasoning: bool,         // Supports thinking/reasoning
    pub context_window: u32,     // Context size in tokens
    pub max_tokens: u32,         // Default max output
    pub cost: CostConfig,        // Pricing per million tokens (0.0 = no tracking)
    pub headers: HashMap<String, String>,  // Extra HTTP headers
    pub compat: Option<OpenAiCompat>,      // Quirk flags (OpenAiCompletions only)
}
}

Factory methods (all accept api_key as the auth parameter):

#![allow(unused)]
fn main() {
let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
let anthropic = ModelConfig::anthropic("claude-sonnet-4-20250514", "Claude Sonnet 4", &api_key);

let openai_key = std::env::var("OPENAI_API_KEY").unwrap();
let openai = ModelConfig::openai("gpt-4o", "GPT-4o", &openai_key);

let gemini_key = std::env::var("GEMINI_API_KEY").unwrap();
let google = ModelConfig::google("gemini-2.0-flash", "Gemini 2.0 Flash", &gemini_key);

// Local server — pass empty string for api_key if unauthenticated
let local = ModelConfig::local("http://localhost:1234/v1", "my-model", "");

// OpenRouter — dedicated factory with correct compat flags
let or_key = std::env::var("OPENROUTER_API_KEY").unwrap();
let openrouter = ModelConfig::openrouter("anthropic/claude-sonnet-4", &or_key);
}

ProviderRegistry

Maps ApiProtocol → StreamProvider. The default registry includes all built-in providers:

#![allow(unused)]
fn main() {
let registry = ProviderRegistry::default();

// Use it to stream with any model
let result = registry.stream(&model_config, stream_config, tx, cancel).await?;
}

Custom registries (advanced — for adding a fully custom StreamProvider implementation):

#![allow(unused)]
fn main() {
use phi_core::provider::{ProviderRegistry, ApiProtocol};

let mut registry = ProviderRegistry::new();
registry.register(ApiProtocol::AnthropicMessages, my_custom_provider);
// Then pass to AgentLoopConfig... (most users should use provider_override instead)
}

StreamProvider Trait

#![allow(unused)]
fn main() {
#[async_trait]
pub trait StreamProvider: Send + Sync {
    async fn stream(
        &self,
        config: StreamConfig,
        tx: mpsc::UnboundedSender<StreamEvent>,
        cancel: CancellationToken,
    ) -> Result<Message, ProviderError>;
}
}

All providers receive a StreamConfig, emit StreamEvents through the channel, and return the final Message.

OpenAPI Tool Adapter

In addition to LLM providers, phi-core can auto-generate tools from any OpenAPI 3.0 spec. This is a tool integration (not a provider), but it complements the provider system by letting agents call external APIs.

Enable with features = ["openapi"]. See the OpenAPI Tools guide for details.

Anthropic Provider

Handles the Anthropic Messages API with SSE streaming. Selected automatically when ModelConfig.api == ApiProtocol::AnthropicMessages.

Usage

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;

let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
));
}

Features

Streaming SSE

Uses reqwest-eventsource to parse Anthropic's SSE stream. Events handled:

• message_start — Input token usage, cache stats
• content_block_start — Text, thinking, or tool_use block
• content_block_delta — Text, thinking, input JSON, or signature deltas
• content_block_stop — Block complete
• message_delta — Stop reason, output usage
• message_stop — Stream complete

Extended Thinking

Set thinking_level to enable thinking with a token budget:

Thinking content is streamed as Content::Thinking with a cryptographic signature for verification.

Cache Control

Automatic prompt caching via cache_control markers:

• System prompt: Always cached with {"type": "ephemeral"}
• Second-to-last message: Gets cache_control on its last content block, creating a cache breakpoint

This means on repeated calls, only the latest message is processed at full price.

Configuration

Environment Variables

OpenAI Compatible Provider

One implementation (OpenAiCompatProvider) covers OpenAI, xAI, Groq, Cerebras, OpenRouter, Mistral, DeepSeek, and any other OpenAI Chat Completions-compatible API. The provider is selected automatically when ModelConfig.api == ApiProtocol::OpenAiCompletions.

Per-service behavior is controlled by OpenAiCompat flags stored in ModelConfig.compat.

Usage

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;

// OpenAI
let api_key = std::env::var("OPENAI_API_KEY").unwrap();
let agent = BasicAgent::new(ModelConfig::openai("gpt-4o", "GPT-4o", &api_key));

// OpenRouter
let or_key = std::env::var("OPENROUTER_API_KEY").unwrap();
let agent = BasicAgent::new(ModelConfig::openrouter("anthropic/claude-sonnet-4", &or_key));

// Local server (LM Studio, Ollama, llama.cpp, vLLM)
let agent = BasicAgent::new(ModelConfig::local(
    "http://localhost:1234/v1",
    "my-model",
    "",  // empty string — most local servers don't require auth
));
}

OpenAiCompat Quirk Flags

Different providers have behavioral differences even though they share the same API:

#![allow(unused)]
fn main() {
pub struct OpenAiCompat {
    pub supports_store: bool,
    pub supports_developer_role: bool,
    pub supports_reasoning_effort: bool,
    pub supports_usage_in_streaming: bool,
    pub max_tokens_field: MaxTokensField,       // MaxTokens or MaxCompletionTokens
    pub requires_tool_result_name: bool,
    pub requires_assistant_after_tool_result: bool,
    pub thinking_format: ThinkingFormat,        // OpenAi, Xai, Qwen, or OpenRouter
}
}

Provider Presets

Adding a New Compatible Provider

1. Add a constructor to OpenAiCompat:

#![allow(unused)]
fn main() {
impl OpenAiCompat {
    pub fn my_provider() -> Self {
        Self {
            supports_usage_in_streaming: true,
            // set flags as needed...
            ..Default::default()
        }
    }
}
}

2. Create a ModelConfig that uses it:

#![allow(unused)]
fn main() {
use phi_core::provider::{ModelConfig, ApiProtocol, OpenAiCompat};

let config = ModelConfig {
    id: "my-model".into(),
    name: "My Model".into(),
    api: ApiProtocol::OpenAiCompletions,
    provider: "my-provider".into(),
    base_url: "https://api.myprovider.com/v1".into(),
    api_key: std::env::var("MY_API_KEY").unwrap_or_default(),
    compat: Some(OpenAiCompat::my_provider()),
    ..Default::default()
};
BasicAgent::new(config)
}

Thinking/Reasoning

The ThinkingFormat enum controls how reasoning content is parsed from streams:

• ThinkingFormat::OpenAi — Uses reasoning_content field (most providers, default)
• ThinkingFormat::Xai — Uses reasoning field (Grok)
• ThinkingFormat::Qwen — Uses reasoning_content field (Qwen variant)
• ThinkingFormat::OpenRouter — Uses reasoning_details array (OpenRouter extended thinking)

Auth

Uses Authorization: Bearer {api_key} header. Extra headers can be added via ModelConfig.headers.

Google Gemini Provider

Two providers for Google's Gemini models:

• GoogleProvider — Google AI Studio (Generative AI API) via ApiProtocol::GoogleGenerativeAi
• GoogleVertexProvider — Google Cloud Vertex AI via ApiProtocol::GoogleVertex

Google AI Studio

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::ModelConfig;

let api_key = std::env::var("GOOGLE_API_KEY").unwrap();
let agent = BasicAgent::new(ModelConfig::google(
    "gemini-2.0-flash",
    "Gemini 2.0 Flash",
    &api_key,
));
}

API Details

• Endpoint: {base_url}/v1beta/models/{model}:streamGenerateContent?alt=sse&key={api_key}
• Auth: API key as query parameter
• Default base URL: https://generativelanguage.googleapis.com
• Default context window: 1,000,000 tokens

Message Format

Google uses a different message format than OpenAI/Anthropic:

Streaming

Uses SSE format (alt=sse). Each chunk contains candidates with content.parts and optional usageMetadata.

Google Vertex AI

GoogleVertexProvider uses the same message format but with Vertex AI authentication and endpoints.

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::{ModelConfig, ApiProtocol};

// Vertex AI uses OAuth2 Bearer tokens as the api_key
let access_token = get_access_token(); // your OAuth2 helper
let agent = BasicAgent::new(ModelConfig {
    id: "gemini-2.0-flash".into(),
    name: "Gemini 2.0 Flash (Vertex)".into(),
    api: ApiProtocol::GoogleVertex,
    provider: "google_vertex".into(),
    base_url: "https://us-central1-aiplatform.googleapis.com".into(),
    api_key: access_token,
    ..Default::default()
});
}

• Protocol: ApiProtocol::GoogleVertex
• Auth: OAuth2 / service account credentials (Bearer token in api_key)
• Endpoint pattern: https://{region}-aiplatform.googleapis.com/v1/projects/{project}/locations/{region}/publishers/google/models/{model}:streamGenerateContent

Amazon Bedrock Provider

Handles the AWS Bedrock ConverseStream API. Selected automatically when ModelConfig.api == ApiProtocol::BedrockConverseStream.

Usage

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::{ModelConfig, ApiProtocol};

// With static credentials in api_key: "ACCESS_KEY:SECRET_KEY" or "ACCESS_KEY:SECRET_KEY:SESSION_TOKEN"
let creds = std::env::var("AWS_BEDROCK_CREDENTIALS").unwrap_or_default();
let agent = BasicAgent::new(ModelConfig {
    id: "anthropic.claude-3-sonnet-20240229-v1:0".into(),
    name: "Claude Sonnet (Bedrock)".into(),
    api: ApiProtocol::BedrockConverseStream,
    provider: "bedrock".into(),
    base_url: "https://bedrock-runtime.us-east-1.amazonaws.com".into(),
    api_key: creds, // "access_key:secret_key[:session_token]", or "" for IAM roles
    ..Default::default()
});
}

Authentication

The api_key field uses a colon-separated format:

{access_key_id}:{secret_access_key}
{access_key_id}:{secret_access_key}:{session_token}

For IAM roles (e.g., EC2 instance profiles, ECS task roles), pass an empty api_key and provide pre-computed Authorization headers via ModelConfig.headers.

API Details

• Endpoint: {base_url}/model/{model}/converse-stream
• Default base URL: https://bedrock-runtime.us-east-1.amazonaws.com
• Protocol: ApiProtocol::BedrockConverseStream

Message Format

Bedrock uses its own content block format:

Stream Events

Bedrock's ConverseStream returns these event types:

• contentBlockStart — New content block (text or tool use)
• contentBlockDelta — Text or tool use input delta
• contentBlockStop — Block complete
• messageStop — Stop reason (end_turn, max_tokens, tool_use)
• metadata — Token usage

Azure OpenAI Provider

Handles the OpenAI Responses API format with Azure-specific authentication and URL patterns. Selected automatically when ModelConfig.api == ApiProtocol::AzureOpenAiResponses.

Usage

#![allow(unused)]
fn main() {
use phi_core::BasicAgent;
use phi_core::provider::{ModelConfig, ApiProtocol};

let api_key = std::env::var("AZURE_OPENAI_API_KEY").unwrap();
let agent = BasicAgent::new(ModelConfig {
    id: "gpt-4o".into(),
    name: "GPT-4o (Azure)".into(),
    api: ApiProtocol::AzureOpenAiResponses,
    provider: "azure_openai".into(),
    base_url: "https://my-resource.openai.azure.com/openai/deployments/my-deployment".into(),
    api_key,
    ..Default::default()
});
}

Authentication

Uses the api-key header (not Authorization: Bearer):

api-key: {your_api_key}

Additional headers can be set via ModelConfig.headers (e.g., for Azure AD Bearer tokens).

URL Format

https://{resource}.openai.azure.com/openai/deployments/{deployment}

Set this as ModelConfig.base_url. The provider appends /responses?api-version=2025-01-01-preview.

API Details

• Protocol: ApiProtocol::AzureOpenAiResponses
• Format: OpenAI Responses API (not Chat Completions)
• Streaming: SSE with event types:
  • response.output_text.delta — Text content
  • response.function_call_arguments.start — Tool call start
  • response.function_call_arguments.delta — Tool call arguments
  • response.completed — Final usage data

Message Format

Uses the Responses API input format:

Built-in Tools

phi-core ships with six coding-oriented tools. Get them all with default_tools():

#![allow(unused)]
fn main() {
use phi_core::tools::default_tools;
let tools = default_tools();
}

BashTool

Execute shell commands with timeout and output capture.

• Name: bash
• Parameters: command (string, required)

Configuration

#![allow(unused)]
fn main() {
pub struct BashTool {
    pub cwd: Option<String>,           // Working directory
    pub timeout: Duration,             // Default: 120s
    pub max_output_bytes: usize,       // Default: 256KB
    pub deny_patterns: Vec<String>,    // Blocked commands
    pub confirm_fn: Option<ConfirmFn>, // Confirmation callback
}
}

Default deny patterns: rm -rf /, rm -rf /*, mkfs, dd if=, fork bomb.

Example

#![allow(unused)]
fn main() {
let bash = BashTool::default();
// Or customize:
let bash = BashTool {
    cwd: Some("/workspace".into()),
    timeout: Duration::from_secs(60),
    ..Default::default()
};
}

ReadFileTool

Read file contents with optional line range.

• Name: read_file
• Parameters: path (required), offset (optional, 1-indexed line), limit (optional, number of lines)

Configuration

#![allow(unused)]
fn main() {
pub struct ReadFileTool {
    pub max_bytes: usize,              // Default: 1MB
    pub allowed_paths: Vec<String>,    // Path restrictions (empty = no restriction)
}
}

WriteFileTool

Write content to a file. Creates parent directories automatically.

• Name: write_file
• Parameters: path (required), content (required)

EditFileTool

Surgical search/replace edits. The most important tool for coding agents — instead of rewriting entire files, the agent specifies exact text to find and replace.

• Name: edit_file
• Parameters: path (required), old_text (required), new_text (required)

The old_text must match exactly, including whitespace and indentation.

ListFilesTool

List files and directories with optional glob filtering.

• Name: list_files
• Parameters: path (optional, default: .), pattern (optional glob)

Configuration

#![allow(unused)]
fn main() {
pub struct ListFilesTool {
    pub max_results: usize,    // Default: 200
    pub timeout: Duration,     // Default: 10s
}
}

Uses find or fd for efficient traversal.

SearchTool

Search files using grep (or ripgrep if available).

• Name: search
• Parameters: pattern (required, regex), path (optional root directory)

Configuration

#![allow(unused)]
fn main() {
pub struct SearchTool {
    pub root: Option<String>,      // Root directory
    pub max_results: usize,        // Default: 50
    pub timeout: Duration,         // Default: 30s
}
}

Returns matching lines with file paths and line numbers.

PrunTool

Model-directed context pruning. Removes the oldest inrun_context entries (model-generated messages) from the working context to reclaim space in the context window. Pruned content is preserved in the session log.

• Name: prun
• Parameters: tokens (integer, required) -- approximate number of tokens to reclaim

The tool removes inrun_context entries oldest-first until the requested token budget is met. User messages are never affected. Returns a confirmation with the actual token count reclaimed.

Configuration

#![allow(unused)]
fn main() {
let agent = BasicAgent::new(model_config)
    .with_prun_tool();  // enables both prun and prun_with_memo
}

PrunWithMemoTool

Context pruning with a summary replacement. Same removal behavior as prun, but inserts a concise memo at the position of the earliest pruned message so the model retains key takeaways.

• Name: prun_with_memo
• Parameters: tokens (integer, required) -- approximate number of tokens to reclaim; memo (string, required) -- concise summary to retain in working context

The memo appears at the original timestamp of the earliest pruned message, preserving conversation chronology. Useful when pruned content contained decisions or conclusions worth remembering.

See Context Pruning for the full design.

Configuration

AgentLoopConfig

The main configuration for the agent loop:

#![allow(unused)]
fn main() {
pub struct AgentLoopConfig {
    /// REQUIRED — Complete provider identity: model id, api_key, base_url, protocol, compat flags, cost rates.
    pub model_config: ModelConfig,
    /// Custom provider override. When Some, bypasses ProviderRegistry. Use for MockProvider in tests.
    pub provider_override: Option<Arc<dyn StreamProvider>>,
    /// Stable config identity for loop_id generation.
    pub config_id: Option<String>,
    pub thinking_level: ThinkingLevel,
    pub max_tokens: Option<u32>,
    pub temperature: Option<f32>,
    pub convert_to_llm: Option<ConvertToLlmFn>,
    pub transform_context: Option<TransformContextFn>,
    pub get_steering_messages: Option<GetMessagesFn>,
    pub get_follow_up_messages: Option<GetMessagesFn>,
    /// Context config (includes CompactionConfig with strategies and token counter).
    pub context_config: Option<ContextConfig>,
    pub execution_limits: Option<ExecutionLimits>,
    pub cache_config: CacheConfig,
    pub tool_execution: ToolExecutionStrategy,
    pub retry_config: RetryConfig,
    // ── Lifecycle callbacks ──
    pub before_turn: Option<BeforeTurnFn>,
    pub after_turn: Option<AfterTurnFn>,
    pub before_loop: Option<BeforeLoopFn>,
    pub after_loop: Option<AfterLoopFn>,
    pub before_tool_execution: Option<BeforeToolExecutionFn>,
    pub after_tool_execution: Option<AfterToolExecutionFn>,
    pub before_tool_execution_update: Option<BeforeToolExecutionUpdateFn>,
    pub after_tool_execution_update: Option<AfterToolExecutionUpdateFn>,
    /// Compaction lifecycle callbacks (G1).
    pub before_compaction_start: Option<BeforeCompactionStartFn>,
    pub after_compaction_end: Option<AfterCompactionEndFn>,
    pub on_error: Option<OnErrorFn>,
    pub input_filters: Vec<Arc<dyn InputFilter>>,
    pub first_turn_trigger: TurnTrigger,
    /// Context translation strategy for cross-provider compatibility (G8).
    pub context_translation: Option<Arc<dyn ContextTranslationStrategy>>,
    /// Shared state for PrunTool to communicate pruning requests to the loop.
    pub prun_pending: Option<Arc<Mutex<Vec<PrunRequest>>>>,
}
}

Note: Compaction strategies (in_memory_strategy, block_strategy) are fields on CompactionConfig (inside ContextConfig), not on AgentLoopConfig. The token_counter for pluggable token counting is also on ContextConfig.

StreamConfig

Internal config passed to StreamProvider::stream(). All provider identity comes from model_config:

#![allow(unused)]
fn main() {
pub struct StreamConfig {
    /// REQUIRED — full provider identity: id, api_key, base_url, compat, cost.
    pub model_config: ModelConfig,
    pub system_prompt: String,
    pub messages: Vec<Message>,
    pub tools: Vec<ToolDefinition>,
    pub thinking_level: ThinkingLevel,
    pub max_tokens: Option<u32>,  // overrides model_config.max_tokens when Some
    pub temperature: Option<f32>,
    pub cache_config: CacheConfig,
}
}

ContextConfig

Controls context window management and compaction:

#![allow(unused)]
fn main() {
pub struct ContextConfig {
    pub max_context_tokens: usize,                          // Default: 100,000
    pub system_prompt_tokens: usize,                        // Default: 4,000
    pub compaction: CompactionConfig,                       // Full compaction policy (nested)
    pub token_counter: Option<Arc<dyn TokenCounter>>,       // Pluggable token counting (REQ-162)
    // Legacy fields (backward compat — use compaction.* instead):
    pub keep_recent: usize,                                 // Default: 10
    pub keep_first: usize,                                  // Default: 2
    pub tool_output_max_lines: usize,                       // Default: 50
}
}

CompactionConfig

#![allow(unused)]
fn main() {
pub struct CompactionConfig {
    // WHEN to compact:
    pub compact_at_pct: f64,                                // Default: 0.90
    pub compact_budget_threshold_pct: f64,                  // Default: 0.05
    pub compaction_scope: CompactionScope,                  // Default: FixedCount(3)
    // HOW to compact:
    pub keep_first_turns: usize,                            // Default: 2
    pub keep_recent_turns: usize,                           // Default: 10
    pub max_summary_tokens: usize,                          // Default: 2,000
    pub tool_output_max_lines: usize,                       // Default: 50
    pub focus_message: Option<String>,                      // Guides summarization focus
    // Strategy objects (G5 — moved from AgentLoopConfig):
    pub in_memory_strategy: Option<Arc<dyn CompactionStrategy>>,
    pub block_strategy: Option<Arc<dyn BlockCompactionStrategy>>,
}
}

ExecutionLimits

Prevents runaway agents:

#![allow(unused)]
fn main() {
pub struct ExecutionLimits {
    pub max_turns: usize,              // Default: 50
    pub max_total_tokens: usize,       // Default: 1,000,000
    pub max_duration: Duration,        // Default: 600s
    pub max_cost: Option<f64>,         // Default: None (no cost cap)
}
}

ThinkingLevel

#![allow(unused)]
fn main() {
pub enum ThinkingLevel {
    Off,        // No thinking (default)
    Minimal,    // 128 tokens (Anthropic budget)
    Low,        // 512 tokens
    Medium,     // 2,048 tokens
    High,       // 8,192 tokens
}
}

CostConfig

Token pricing per million:

#![allow(unused)]
fn main() {
pub struct CostConfig {
    pub input_per_million: f64,
    pub output_per_million: f64,
    pub cache_read_per_million: f64,
    pub cache_write_per_million: f64,
}
}

API Reference

Top-Level Functions

agent_loop()

#![allow(unused)]
fn main() {
pub async fn agent_loop(
    prompts: Vec<AgentMessage>,
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

Start an agent loop with new prompt messages. Returns all messages generated during the run.

agent_loop_continue()

#![allow(unused)]
fn main() {
pub async fn agent_loop_continue(
    context: &mut AgentContext,
    config: &AgentLoopConfig,
    tx: mpsc::UnboundedSender<AgentEvent>,
    cancel: CancellationToken,
) -> Vec<AgentMessage>
}

Resume from existing context. The last message must not be an assistant message.

default_tools()

#![allow(unused)]
fn main() {
pub fn default_tools() -> Vec<Arc<dyn AgentTool>>
}

Returns: BashTool, ReadFileTool, WriteFileTool, EditFileTool, ListFilesTool, SearchTool.

Agent Trait

The runtime interface for all agent implementations. Programs against this trait to remain independent of the specific implementation.

#![allow(unused)]
fn main() {
use phi_core::Agent;  // trait must be in scope to call trait methods
}

Trait methods cover: prompting (prompt, prompt_messages, prompt_with_sender, prompt_messages_with_sender, continue_loop, continue_loop_with_sender), state access (messages, is_streaming, agent_id, session_id, last_loop_id), message mutation (clear_messages, append_message, replace_messages, save_messages, restore_messages, set_tools), control (abort, reset), and steering/follow-up queues (steer, follow_up, clear_steering_queue, clear_follow_up_queue, clear_all_queues, set_steering_mode, set_follow_up_mode).

The trait is object-safe: Box<dyn Agent> and &mut dyn Agent work for runtime polymorphism.

phi_core::* re-exports Agent, so use phi_core::* brings it into scope automatically.

BasicAgent Struct

The default in-memory Agent implementation. Owns a single linear message history, tool registry, and model configuration.

Construction

#![allow(unused)]
fn main() {
let api_key = std::env::var("ANTHROPIC_API_KEY").unwrap();
let agent = BasicAgent::new(ModelConfig::anthropic(
    "claude-sonnet-4-20250514",
    "Claude Sonnet 4",
    &api_key,
));
}

Builder Methods

All return Self for chaining (unless noted as Result).

Core

Tools & Integrations

Workspace & System Prompt

Context & Limits

Behavior

Callbacks

Prompting

State Access

State Mutation

Steering & Follow-Up Queues

Control

Session Callback Types

These are set on SessionRecorderConfig and fire at the session level (not per-loop). See Sessions for usage.

Re-exports

The crate re-exports key types from lib.rs:

#![allow(unused)]
fn main() {
// Agent system
pub use agents::{Agent, AgentProfile, BasicAgent, QueueMode};
pub use agents::SubAgentTool;

// Agent loop
pub use agent_loop::{agent_loop, agent_loop_continue, agent_loop_parallel};
pub use agent_loop::evaluation::{
    ElaborateEvaluation, LlmJudgeEvaluation, PickFirstEvaluation,
    TokenEfficientEvaluation, TransparentEvaluation,
};

// Config-driven construction
pub use config::{
    agent_from_config, agent_from_config_with_registry, agents_from_config,
    parse_config, parse_config_file, AgentConfig, ConfigError, ConfigFormat,
};

// Context management
pub use context::{
    CompactionStrategy, CompactionConfig, CompactionScope, ContextConfig,
    DefaultCompaction, DefaultBlockCompaction, BlockCompactionStrategy,
    ContextTracker, CompactionBlock, CompactedSection, TurnMap, TurnRange,
    build_context_from_session, compact_session_loops,
};
pub use context::skills::SkillSet;

// Session persistence
pub use session::{
    Session, SessionRecorder, SessionRecorderConfig, SessionScope, SessionError,
    LoopRecord, LoopEvent, LoopStatus, Turn, LoopConfigSnapshot,
    ParallelGroupRecord, ChildLoopRef, SpawnRef, SessionFormation,
    save_session, load_session, list_session_ids, delete_session, load_sessions_for_agent,
};

// Provider
pub use provider::retry::RetryConfig;

// Types (glob re-export)
pub use types::*;  // Message, Content, AgentMessage, AgentEvent, Usage, LlmMessage,
                    // StopReason, StreamDelta, TurnTrigger, ThinkingLevel, CacheConfig, etc.
}

0.7.0 additions (reachable via module paths)

These symbols were added in 0.7.0 but are not (yet) part of the top-level glob. Import them via their module path:

#![allow(unused)]
fn main() {
// Session: trait-based pluggable store + atomic-write filesystem impl with
// advisory locks (fs2 exclusive lock; returns SessionError::Locked on contention).
use phi_core::session::{SessionStore, FileSystemSessionStore};

// Provider: credential refresh hook for long-running agents whose token expires
// mid-run. On ProviderError::Auth, the loop invalidates the cached credential
// and retries once before propagating.
use phi_core::provider::{CredentialProvider, StaticCredentialProvider};

// Provider: structured-output contract. JsonObject = free-form JSON;
// JsonSchema = strict schema (native where supported, tool-call emulation on
// Anthropic and Anthropic-on-Bedrock, SchemaMismatch on others).
use phi_core::provider::ResponseFormat;

// Agent: fallible build_config(). Default impl returns Err(MissingModelConfig)
// instead of panicking when model_config() is None.
use phi_core::agents::AgentBuildError;

// MCP: configurable per-request timeout (default 30s) on both stdio + HTTP
// transports. Use McpClientConfig with connect_stdio_with_config /
// connect_http_with_config.
use phi_core::mcp::{McpClientConfig, DEFAULT_REQUEST_TIMEOUT};
}

phi-core — Project Overview

1. Purpose Statement

phi-core is a Rust async library for building stateful, multi-turn LLM agents that can autonomously execute tools to accomplish tasks. The library solves the core engineering problems of agent construction: routing between many LLM provider APIs through a unified interface, running a prompt-then-tool-call loop until the model signals completion, streaming real-time events to UI consumers, and automatically managing context windows so conversations do not exceed model token limits. It is designed to be embedded as a dependency in application code — it provides no standalone binary, no HTTP server, and no user interface of its own.

2. Key Capabilities

3. Inputs & Outputs

Inputs

Outputs

4. Actors & Use Cases

Application Developer

The primary consumer. Embeds phi-core as a library dependency.

End User (via application)

Interacts through the application wrapping this library. Uses cases match what the application exposes (e.g., CLI prompts in examples/cli.rs: /quit, /clear, /model).

LLM Provider

External service receiving structured HTTP requests. The library sends conversation history and tool schemas; the provider returns streaming token deltas and final messages. Providers never call back into the library.

MCP Server

External process exposing tools over the Model Context Protocol. The library connects as a client via stdio pipe or HTTP. The server exposes tool definitions that are adapted into AgentTool instances.

Sub-Agent

A child instance of the agent loop spawned internally when a SubAgentTool is called. Operates with its own fresh context and toolset. Results are returned to the parent as a ToolResult.

5. Constraints & Non-Goals

• No built-in HTTP server. The library is embeddable only; serving the agent over HTTP requires external frameworks.
• No user interface. UI rendering (text display, color, input handling) is the application's responsibility (see examples/cli.rs for a reference implementation).
• No authentication management. API keys must be supplied by the caller. The library does not fetch, rotate, or cache credentials.
• Single event consumer per run. agent_loop() returns a single UnboundedReceiver<AgentEvent>. Fan-out to multiple consumers requires application-level bridging.
• No agent-to-agent networking. Sub-agents run in-process only. No remote agent delegation.
• No persistent storage. Conversation state is held in memory. Serialization to disk is the caller's responsibility (the library provides serialize/deserialize helpers).
• No built-in precision token counting. The default HeuristicTokenCounter uses 4 characters per token. A pluggable TokenCounter trait (src/context/token.rs) allows callers to supply a custom counter (e.g., tiktoken-based), but no precision implementation ships with the library.
• No multi-modal generation. Images can be sent to the model (as Content::Image), but image generation is not supported.
• No structured output / JSON mode. The library passes raw messages; enforcing structured output is the caller's responsibility via system prompt.
• Skipped tools on steering. When steering messages arrive mid-batch, remaining tool calls in that batch are skipped with an error result — their outputs are never computed. This is a documented behavior, not a bug.

6. Key Terminology Glossary

Architecture Overview

For detailed component specifications, trait signatures, sequence diagrams, and data models, see the full Architecture Spec. For formal algorithm descriptions, see Algorithms.

Layered Design

phi-core is organized as three conceptual layers within a single crate. Dependencies flow strictly downward — upper layers use lower layers, never the reverse.

┌─────────────────────────────────────────────┐
│  Layer 3: Orchestration          (planned)   │
│  Multi-agent, delegation, work modes         │
├─────────────────────────────────────────────┤
│  Layer 2: Agent + Providers                  │
│  Concrete providers, tools, retry, caching,  │
│  context management, MCP                     │
├─────────────────────────────────────────────┤
│  Layer 1: Core Loop                          │
│  agent_loop, types, traits                   │
│  Provider-agnostic. Tool-agnostic.           │
└─────────────────────────────────────────────┘

Layer 1: Core Loop

The pure agent loop. No opinions about LLMs, no built-in tools. Just the control flow.

Modules: types/, agent_loop/, provider/traits.rs

Owns:

• agent_loop() / agent_loop_continue() — the loop itself
• AgentTool trait — interface tools must implement
• StreamProvider trait — interface providers must implement
• AgentMessage, AgentEvent, StreamDelta — message & event types
• AgentContext — system prompt + messages + tools
• Tool execution strategies (parallel/sequential/batched)
• Streaming tool output (ToolUpdateFn)
• Steering & follow-up message injection

Does not own: Any concrete provider or tool implementation.

Layer 2: Agent + Providers

Batteries-included single-agent layer. Most users interact with this.

Modules: agents/, context/, provider/*.rs, tools/*.rs, mcp/*.rs

Adds on top of Layer 1:

• Concrete providers — Anthropic, OpenAI-compat, Google, Azure, Bedrock, Vertex
• Provider registry — dispatch by API protocol
• Context translation — cross-provider content type compatibility (G8)
• Prompt caching — automatic cache breakpoint placement
• Retry with backoff — exponential, jitter, respects retry-after
• Context management — token estimation, compaction, execution limits, cost tracking
• AgentProfile + SystemPromptStrategy — reusable agent blueprints with multi-block prompt composition
• Config-driven construction — TOML/JSON/YAML → agent_from_config() → Arc<dyn Agent>
• Built-in tools — bash, read_file, write_file, edit_file, list_files, search, prun (context pruning)
• Tool registry — name-based tool resolution from config
• Session persistence — SessionRecorder materializes Turn structs from events
• MCP client — stdio + HTTP transports, tool adapter
• Agent trait — the runtime interface (prompting, state, control, ~40 methods)
• BasicAgent struct — default in-memory implementation of Agent; stateful builder
• SubAgentTool — delegates tasks to a child agent_loop() as a tool

Layer 3: Orchestration (planned)

Multi-agent coordination. Not yet implemented — the architecture is designed to support it when needed.

Planned capabilities:

• Orchestrator struct — spawn, delegate, and coordinate multiple agents
• Work modes:
  • Interactive — multi-turn, human in the loop (current default)
  • Autonomous — runs to completion without input (background tasks, CI)
  • Pipeline — input → output, chainable (scan → fix → verify)
  • Supervisor — delegates to other agents, synthesizes results
• Fan-out — same task to multiple agents (different providers for diversity)
• Pipeline chaining — output of agent A feeds input of agent B
• Agent communication through the orchestrator event bus

Why not yet: Multi-agent orchestration adds complexity. The single-agent loop handles 95% of use cases. Layer 3 will be built when a concrete use case drives it, not speculatively.

Module Layout

phi-core/
├── src/
│   ├── lib.rs                     # Public re-exports
│   │
│   │── Layer 1: Core Loop ─────────────────────
│   ├── types/
│   │   ├── mod.rs                 # Re-exports, Message, AgentMessage
│   │   ├── content.rs             # Content enum (Text, Image, Thinking, ToolCall), StopReason
│   │   ├── extension.rs           # ExtensionMessage
│   │   ├── agent_message.rs       # AgentMessage enum, LlmMessage (Message + TurnId)
│   │   ├── usage.rs               # Usage, CacheConfig, CacheStrategy, ThinkingLevel
│   │   ├── tool.rs                # AgentTool trait, ToolDefinition, ToolContext
│   │   ├── event.rs               # AgentEvent enum, TurnTrigger, StreamDelta
│   │   ├── context.rs             # AgentContext, InRunEntry (2-stream pruning)
│   │   └── parallel.rs            # ToolExecutionStrategy
│   ├── agent_loop/
│   │   ├── core.rs                # agent_loop(), agent_loop_continue()
│   │   ├── run.rs                 # run_loop() — inner turn engine
│   │   ├── streaming.rs           # stream_assistant_response() — LLM call + retry
│   │   ├── tools.rs               # execute_tool_calls()
│   │   ├── config.rs              # AgentLoopConfig, callback type aliases
│   │   ├── helpers.rs             # Input filtering, message conversion
│   │   ├── parallel.rs            # agent_loop_parallel()
│   │   ├── evaluation.rs          # EvaluationStrategy trait + 5 built-in strategies
│   │   └── script_callback.rs     # ScriptCallback for shell/Python hooks
│   │
│   │── Layer 2: Agent + Providers ─────────────
│   ├── agents/
│   │   ├── agent.rs               # Agent trait (runtime interface, ~40 methods)
│   │   ├── basic_agent.rs         # BasicAgent struct (default in-memory impl)
│   │   ├── profile.rs             # AgentProfile struct
│   │   ├── system_prompt.rs       # SystemPromptStrategy, SystemPrompt, PromptBlockDef
│   │   └── sub_agent.rs           # SubAgentTool (child agent_loop as a tool)
│   ├── config/
│   │   ├── schema.rs              # AgentConfig + all TOML/JSON/YAML config sections
│   │   ├── builder.rs             # agent_from_config(), agents_from_config()
│   │   ├── parser.rs              # Multi-format parsing + env var substitution
│   │   └── reference.rs           # {{...}} ID reference protocol parser
│   ├── context/
│   │   ├── config.rs              # ContextConfig, CompactionConfig, CompactionScope
│   │   ├── compaction.rs          # CompactionBlock, CompactedSection
│   │   ├── compact_messages.rs    # compact_messages() — legacy tiered compaction
│   │   ├── strategy.rs            # CompactionStrategy, BlockCompactionStrategy traits
│   │   ├── orchestration.rs       # compact_session_loops(), build_context_from_session()
│   │   ├── execution.rs           # ExecutionLimits, ExecutionTracker
│   │   ├── tracker.rs             # ContextTracker (hybrid token counting)
│   │   ├── token.rs               # TokenCounter trait, HeuristicTokenCounter
│   │   └── skills.rs              # SkillSet (SKILL.md loader)
│   ├── session/
│   │   ├── model.rs               # Session, LoopRecord, Turn, LoopStatus
│   │   ├── recorder.rs            # SessionRecorder (event → session state machine)
│   │   ├── storage.rs             # save_session(), load_session(), list/delete
│   │   └── helpers.rs             # Internal utilities
│   ├── provider/
│   │   ├── traits.rs              # StreamProvider trait, StreamEvent, ProviderError
│   │   ├── model.rs               # ModelConfig, ApiProtocol, OpenAiCompat
│   │   ├── registry.rs            # ProviderRegistry (protocol → provider)
│   │   ├── retry.rs               # Retry with exponential backoff
│   │   ├── context_translation.rs # ContextTranslationStrategy (G8)
│   │   ├── anthropic.rs           # Anthropic Messages API
│   │   ├── openai_compat.rs       # OpenAI Chat Completions (15+ providers)
│   │   ├── openai_responses.rs    # OpenAI Responses API
│   │   ├── google.rs              # Google Generative AI
│   │   ├── google_vertex.rs       # Google Vertex AI
│   │   ├── bedrock.rs             # AWS Bedrock ConverseStream
│   │   ├── azure_openai.rs        # Azure OpenAI
│   │   ├── mock.rs                # Mock provider for testing
│   │   └── sse.rs                 # SSE utilities
│   ├── tools/
│   │   ├── bash.rs                # BashTool
│   │   ├── file.rs                # ReadFileTool, WriteFileTool
│   │   ├── edit.rs                # EditFileTool
│   │   ├── list.rs                # ListFilesTool
│   │   ├── search.rs              # SearchTool
│   │   ├── prun.rs                # PrunTool, PrunWithMemoTool (context pruning)
│   │   └── registry.rs            # ToolRegistry (name → factory)
│   ├── mcp/
│   │   ├── client.rs              # MCP client (stdio + HTTP)
│   │   ├── tool_adapter.rs        # McpToolAdapter (MCP tool → AgentTool)
│   │   ├── transport.rs           # Transport implementations
│   │   └── types.rs               # MCP protocol types
│   └── openapi/                   # (feature-gated: "openapi")
│       ├── adapter.rs             # OpenApiToolAdapter
│       └── types.rs               # OpenApiConfig, OperationFilter

Data Flow

                    ┌─────────────┐
                    │   Caller    │
                    └──────┬──────┘
                           │ prompt / prompt_messages
                    ┌──────▼──────┐
                    │ BasicAgent  │  Layer 2: stateful wrapper
                    │ (agents/)   │  Manages queues, tools, state
                    └──────┬──────┘
                           │
                    ┌──────▼──────┐
                    │ agent_loop  │  Layer 1: core loop
                    │             │  Prompt → LLM → Tools → Repeat
                    └──┬───────┬──┘
                       │       │
              ┌────────▼──┐ ┌──▼────────┐
              │ Provider  │ │   Tools   │  Layer 2: implementations
              │ .stream() │ │ .execute()│
              └────────┬──┘ └──┬────────┘
                       │       │
              ┌────────▼──┐ ┌──▼────────┐
              │ LLM API   │ │ OS / FS   │
              │ (HTTP)    │ │ (shell)   │
              └───────────┘ └───────────┘

Events flow back via mpsc::UnboundedSender<AgentEvent>

How Providers Plug In

1. Implement StreamProvider trait (Layer 1 interface)
2. Register with ProviderRegistry under an ApiProtocol (Layer 2)
3. Set ModelConfig.api to match that protocol
4. The registry dispatches stream() calls to the right provider

Each provider translates between phi-core's Message/Content types and the provider's native API format. All providers emit StreamEvents through the channel for real-time updates.

How Tools Plug In

1. Implement AgentTool trait (Layer 1 interface)
2. Add to the tools vec (via default_tools() or custom)
3. The agent loop converts tools to ToolDefinition (name, description, schema) for the LLM
4. When the LLM returns Content::ToolCall, the loop finds the matching tool and calls execute()
5. Results are wrapped in Message::ToolResult and added to context