Agent Orchestration

contextunity.router uses LangGraph for agent orchestration. Each agent is a state machine that processes requests through a series of nodes with conditional routing.

Model Registry & Fallbacks

contextunity.router never hardcodes LLM provider clients (like openai.ChatCompletion). Instead, all AI generation is routed through a unified Model Registry.

This registry ensures high availability. If the primary model (e.g., openai/gpt-5-mini) experiences a rate limit or a 500 error from the provider, the registry automatically retries against the configured fallback_keys (e.g., Anthropic or Vertex) without failing the user request.

Policy: models and Langfuse

Model keys belong next to router.policy.ai_model_policy (default_ai_model, default_model_secret_ref, fallbacks). Langfuse belongs only under router.policy.langfuse: optional tracing_enabled, and public_key_ref / secret_key_ref (optional host_ref) whose values are env var names (same convention as model_secret_ref on nodes). Do not use a top-level langfuse_tracing_enabled field on policy — it is not part of the schema and will fail validation.

Bootstrap resolves those env var names from the process environment and can sync values to Shield; runtime tracing still follows Router and token rules.

Project Fallbacks: Projects specify their own fallback_keys array when defining an LLM node in the contextunity.project.yaml manifest. The Router then automatically handles rate limits or provider outages by cycling through the node’s explicit fallback constraints.

Global Architecture Fallback: If a project omits fallback_keys, the graph will normally fail if the primary model errors out. However, Router administrators can configure CU_ROUTER_ALLOW_GLOBAL_FALLBACK=true along with CU_ROUTER_FALLBACK_LLMS. This creates a global safety net that intercepts failures using the Router’s own baseline models.

from contextunity.router.modules.models import model_registry

model = model_registry.get_llm_with_fallback(
    key="openai/gpt-5-mini",
    fallback_keys=["anthropic/claude-sonnet-4", "vertex/gemini-2.5-flash"],
    strategy="fallback",
    config=config,
)

response = await model.generate(request)

Supported Providers

Provider	Models	Backend
OpenAI	GPT-5, GPT-5-mini, o1, o3	openai.py
Anthropic	Claude Sonnet 4, Haiku	anthropic.py
Vertex AI	Gemini 2.5 Flash/Pro	vertex.py
Groq	Llama (ultra-fast inference)	groq.py
Perplexity	Sonar (web-grounded)	perplexity.py
RLM	Recursive Language Model	rlm.py
HuggingFace	HF Inference / Hub	huggingface.py
OpenRouter	Multi-provider gateway	openrouter.py
Local	Ollama, vLLM (OpenAI-compat)	local_openai.py
RunPod	Serverless inference	runpod.py

Agent Graphs (State Machines)

ContextRouter executes agent workflows as LangGraph state machines. Projects register graph topology declaratively via contextunity.project.yaml; Router compiles and caches each graph per tenant.

This provides:

1. Deterministic state transitions per node.
2. Conditional routing via condition_key / condition_map.
3. Typed execution telemetry (tool_result) for observability.

Manifest-Driven Graph Registration

Project manifests use a graph map under router.graph:

router:
  # Optional fallback graph when caller omits graph id
  default_graph: "analytics"
  graph:
    analytics:
      template: "yaml:retrieval_augmented"
      overrides:
        generate:
          config:
            model: "openai/gpt-5-mini"

Project-Authored Local Graphs

For project-specific topology, define an inline graph with explicit nodes and edges:

router:
  graph:
    newsroom:
      nodes:
        - name: "planner"
          type: "llm"
          model: "openai/gpt-5-mini"
        - name: "agent"
          type: "agent"
          model: "openai/gpt-5-mini"
          tools:
            - "federated:store_outputs"
          config:
            tool_choice: "auto"
            max_tool_calls: 5
        - name: "store"
          tool_binding: "federated:store_outputs"
      edges:
        - from: "__start__"
          to: "planner"
        - from: "planner"
          to: "agent"
        - from: "agent"
          to: "store"

Tool bindings are namespaced:

• platform:<tool_name>
• federated:<tool_name>

Node types are explicit:

• llm — one direct model call, no tool-calling loop
• agent — model call with an explicit tools or toolkits allowlist
• tool — deterministic execution of one tool_binding
• platform — internal ContextUnity platform tool

See Graph Compiler & Templates for the full compiler contract.

Plugin System

To extend contextunity.router with custom capabilities beyond the core bundle — custom tools, LLM providers, data connectors, or processing transformers — the architecture uses a declarative Plugin System with capability-mediated PluginContext access.

Plugins are self-contained directories with a plugin.yaml manifest and a Python entry point. See the Plugin System page for the full API reference, security model, and code examples.