Agent Orchestration

contextunity.router uses LangGraph for agent orchestration. Each agent is a state machine that processes requests through a series of nodes with conditional routing.

Model Registry & Fallbacks

contextunity.router never hardcodes LLM provider clients (like openai.ChatCompletion). Instead, all AI generation is routed through a unified Model Registry.

This registry ensures high availability. If the primary model (e.g., openai/gpt-5-mini) experiences a rate limit or a 500 error from the provider, the registry automatically retries against the configured fallback_keys (e.g., Anthropic or Vertex) without failing the user request.

Project Fallbacks: Projects specify their own fallback_keys array when defining an LLM node in the contextunity.project.yaml manifest. The Router then automatically handles rate limits or provider outages by cycling through the node’s explicit fallback constraints.

Global Architecture Fallback: If a project omits fallback_keys, the graph will normally fail if the primary model errors out. However, Router administrators can configure CU_ROUTER_ALLOW_GLOBAL_FALLBACK=true along with CU_ROUTER_FALLBACK_LLMS. This creates a global safety net that intercepts failures using the Router’s own baseline models.

from contextunity.router.modules.models import model_registry

model = model_registry.get_llm_with_fallback(
    key="openai/gpt-5-mini",
    fallback_keys=["anthropic/claude-sonnet-4", "vertex/gemini-2.5-flash"],
    strategy="fallback",
    config=config,
)

response = await model.generate(request)

Supported Providers

Provider	Models	Backend
OpenAI	GPT-5, GPT-5-mini, o1, o3	`openai.py`
Anthropic	Claude Sonnet 4, Haiku	`anthropic.py`
Vertex AI	Gemini 2.5 Flash/Pro	`vertex.py`
Groq	Llama (ultra-fast inference)	`groq.py`
Perplexity	Sonar (web-grounded)	`perplexity.py`
RLM	Recursive Language Model	`rlm.py`
HuggingFace	HF Inference / Hub	`huggingface.py`
OpenRouter	Multi-provider gateway	`openrouter.py`
Local	Ollama, vLLM (OpenAI-compat)	`local_openai.py`
RunPod	Serverless inference	`runpod.py`

Agent Graphs (State Machines)

To handle complex, multi-step workflows (like verifying news, parsing catalogs, or resolving medical anomalies), contextunity.router represents processes as Directed Acyclic Graphs (DAGs).

These Agent graphs are defined in cortex/graphs/ using LangGraph’s StateGraph. This allows agents to:

Have a defined, immutable state at each step.
Route conditionally (e.g., if a tool fails, route back to the LLM to fix the arguments).
Produce deterministic execution traces for the contextunity.view dashboard.

router-agents

Built-in Graphs

Graph	Purpose	Location
Dispatcher	Central request routing	`graphs/dispatcher.py`
RAG Retrieval	Knowledge retrieval pipeline	`graphs/rag_retrieval/`
Gardener	Product taxonomy classification	`graphs/commerce/gardener/`
Matcher	Product linking (RLM-based)	`graphs/commerce/matcher/`
News Engine	Multi-stage news pipeline	`graphs/news_engine/`
Analytics	Data analytics pipeline	`graphs/analytics/`
Self-healing	Auto-recovery logic	`graphs/self_healing/`

Plugin Architecture & Manifest Assembly

Because contextunity.router acts as the central brain for multiple distinct projects, it does not contain hardcoded project-specific pipelines. Instead, Graph Architecture is manifest-driven and assembled dynamically when a project connects.

Projects register their graphs in two ways using contextunity.project.yaml:

1. Template Instantiation

Used for standardized topologies (like RAG or Medical SQL Analyst). The project references a predefined template and provides nodes just to pass configuration (like prompts and model keys). The type and description fields of nodes are ignored for templates.

router:
  graph:
    id: "contextmed_analyst"
    template: "sql_analytics"
    nodes:
      - name: "planner"
        model: "openai/gpt-5-mini"

2. Custom Dynamic Assembly

Used when a project needs a unique topology (like Pink Pony). If template isn’t provided, or is explicitly set to custom (without quotes), contextunity.router dynamically compiles a StateGraph from the declarative nodes and edges.

router:
  graph:
    id: "my-project"
    template: custom # Or simply omit template
    nodes:
      - name: "tech_reporter"
        type: "llm"  # Required for custom
        model: "openai/gpt-5-mini"
      - name: "store_outputs"
        type: "tool"
    edges:
      - from: "__start__"
        to: "tech_reporter"
      # Conditional routing (Agentic Loops)
      - from: "tech_reporter"
        condition_key: "agent_action"
        condition_map:
          "call_tool": "store_outputs"
          "finish": "__end__"

contextunity.router’s CustomGraphBuilder natively translates nodes into execution blocks (make_llm_node, make_tool_node) and constructs a parallel LangGraph StateGraph directly from the edges, including cyclic conditional routing.

Declarative YAML Plugins

To extend contextunity.router with custom capabilities beyond the core bundle (such as specialized graphs, custom tools, or third-party connectors), the architecture uses a declarative YAML-based plugin system (plugins.yaml).

# Example plugins.yaml
plugin:
  name: "seo_optimizer"
  version: "1.0.0"
  graphs:
    - name: "seo_graph"
      entrypoint: "seo_analyzer"

Plugins can register:

Graphs — entire LangGraph definitions
Tools — custom LLM function-calling tools
Connectors — new data source adapters
Transformers — new processing stages (NER, classification, etc.)
Providers — new storage backends