Architecture¶

This is the canonical architecture reference for the SecondBrain repo. It explains how a request enters, what runtime loop handles it, where context comes from, how governance gates execution, and where state lands. Every layer points at the package that owns it so a maintainer can follow the trail in code.

For per-component depth (kernel, harness, MCP, antahkarana, data agent, patterns, marketing, travel) see docs/components/. For the request flow as prose, the sections below are ordered the same way a request actually moves.

Mental Model¶

SecondBrain is local-first. A request hits one of several entry surfaces, gets handed to one of a small number of runtime loops, picks up context from local stores, runs tools under governance, persists state, and emits observability that feeds quality loops.

There is no single global planner — each loop is purpose-built for its job.

flowchart LR
  USER["User / Operator / Channel / SDK"]

  subgraph SURFACES["Entry surfaces"]
    CLI["sb CLI<br/>brain/cli/"]
    SERVE["sb serve (FastAPI)<br/>brain/serve/"]
    UI["React operator UI + Mac shell<br/>serve-ui/, mac/"]
    SDK["Programmatic SDK<br/>brain/sdk/"]
    GW["Gateway / channels<br/>brain/gateway/, brain/channels/"]
  end

  subgraph LOOPS["Runtime loops"]
    HARNESS["AgentHarness<br/>bounded chat turn"]
    AGENTIC["AgenticRuntime<br/>Planner→Executor→Critic"]
    BG["Background sessions<br/>native + bridge engines"]
    GWRUN["Gateway AgentRuntime<br/>channel sessions"]
    WF["Tasks + workflows"]
  end

  subgraph CONTEXT["Context, memory, knowledge"]
    INGEST["Ingest + sources"]
    RET["Hybrid retrieval"]
    PACK["Context packs"]
    MEM["Memory subsystem"]
    KNOW["Knowledge + graph"]
  end

  subgraph GOV["Governance"]
    POLICY["Policies + approvals"]
    KERNEL["Kernel primitives<br/>RunContext, budgets"]
    TOOLS["Tool registries"]
  end

  subgraph EXT["Capability planes"]
    MCP["MCP"]
    INT["Connectors + integrations"]
    BRIDGE["Claude / Codex bridges"]
    PROVIDERS["Provider registry"]
  end

  subgraph STATE["Local state"]
    DB[("Split SQLite<br/>runtime / memory / knowledge /<br/>work / travel / antahkarana")]
    VEC["Vector store<br/>state/retrieval (LanceDB)"]
    FS["Vault, artifacts, logs"]
  end

  subgraph QUALITY["Observability"]
    OBS["Traces + observation envelopes"]
    QC["Quality / eval / autotune"]
  end

  USER --> CLI & UI & SDK & GW
  UI --> SERVE
  CLI --> HARNESS & AGENTIC & BG & WF
  SERVE --> HARNESS & BG
  GW --> GWRUN
  SDK --> HARNESS & AGENTIC

  HARNESS & AGENTIC & BG & GWRUN & WF --> POLICY
  HARNESS --> PACK
  PACK --> RET --> KNOW
  HARNESS --> MEM
  INGEST --> KNOW & VEC

  POLICY --> TOOLS
  TOOLS --> MCP & INT & BRIDGE
  HARNESS & AGENTIC & GWRUN --> PROVIDERS

  HARNESS & AGENTIC & GWRUN & WF & MEM & KNOW --> DB
  RET --> VEC
  HARNESS & AGENTIC & GWRUN --> OBS --> QC --> DB

  classDef surface fill:#D8ECFF,stroke:#0B5CAD,color:#0D2740,stroke-width:2px;
  classDef runtime fill:#EDE7F6,stroke:#5E35B1,color:#1A0050,stroke-width:2px;
  classDef gov fill:#FFF6CC,stroke:#B88700,color:#4C3900,stroke-width:2px;
  classDef state fill:#E9F8EB,stroke:#1B7D3A,color:#104A24,stroke-width:2px;
  classDef adapter fill:#FFE8CF,stroke:#C45E00,color:#4A2A00,stroke-width:2px;
  class CLI,SERVE,UI,SDK,GW surface;
  class HARNESS,AGENTIC,BG,GWRUN,WF,INGEST,RET,PACK,MEM,KNOW,OBS,QC runtime;
  class POLICY,KERNEL,TOOLS gov;
  class DB,VEC,FS state;
  class MCP,INT,BRIDGE,PROVIDERS adapter;

1. Entry Surfaces¶

Surface	Implementation	Notes
`sb` CLI	`brain/cli/__init__.py`, `brain/cli/*`	Broadest operator surface; ~175 top-level commands.
Interactive chat	`brain/chat/repl.py`, `brain/chat/commands/`, `brain/chat/transport.py`	Slash commands, transcripts, terminal callbacks.
Local daemon API	`brain/serve/app.py`, `brain/serve/services.py`, `brain/serve/routers/`	FastAPI; mounts chat, sessions, approvals, automations, memory, quality, MCP, A2A, wiki, WS.
React operator UI	`serve-ui/src/App.tsx`, `serve-ui/src/pages/`, `serve-ui/src/lib/api.ts`	Browser shell served from `brain/serve/static/`.
Mac shell	`mac/SecondBrainMac/`	Wraps the same local daemon, not a separate backend.
SDK facade	`brain/sdk/`	Programmatic access to stable runtime functions.
Gateway / channels	`brain/gateway/`, `brain/channels/`	Channel-side dedupe, lanes, outbound queues, approvals.
Background sessions	`brain/background_sessions/`, `brain/cli/sessions.py`, `brain/serve/routers/sessions.py`	Durable native or bridge-engine sessions.
Optional daemon worker	`brain/daemon/`, `brain/cli/daemon_cmd.py`	Sweeper / task queue / harness refresh.

For user-facing CLI changes, keep brain/ui_schema/cli_schema.json in sync via sb ui-schema --write-default.

2. Runtime Loops¶

There is no single global planner. Each loop owns its bounded job:

Loop	Implementation	Used for
Chat turn harness	`brain/agent/harness.py` (+ collaborators in `brain/agent/`)	Bounded interactive or web chat turn — tools, streaming, approvals, citations, memory.
Agentic runtime	`brain/agent/agentic.py`, `brain/tasks/llm_planner.py`	Goal-directed Planner → Executor → Critic loop.
Gateway runtime	`brain/agent_runtime/runner.py`	Channel sessions: dedupe, compaction, tools, outbound delivery, approval lifecycle.
Background session supervisor	`brain/background_sessions/runtime.py`, `brain/background_sessions/bridges.py`	Durable native chat / agentic, or external Claude / Codex bridge sessions.
Workflow / task executor	`brain/workflows/`, `brain/tasks/`, `brain/work_graph/`, `brain/orchestrator/`	Structured task graphs and operational automations.
AI workforce	`brain/workforce/`	Declarative role teams compiled into background sessions.
Daemon poll loop	`brain/daemon/supervisor.py`, `brain/daemon/workers.py`	Sweeps, queue draining, harness refresh.

Domain runtimes (brain/data_agent/, brain/grounded/, brain/meeting/, brain/marketing/, brain/travel/, brain/antahkarana/) reuse these primitives — they don't own separate policy or storage stacks.

AgentHarness collaborators¶

The chat harness is intentionally a thin façade. State lives in collaborators:

brain/agent/turn_preparer.py — turn preparation
brain/agent/turn_state.py — mutable per-turn state
brain/agent/turn_budget.py — budgets and deadlines
brain/agent/tool_scheduler.py — tool wave scheduling
brain/agent/tool_executor_v2.py — bounded execution
brain/agent/tool_outcomes.py — typed tool outcome union
brain/agent/replay_control.py — replay / idempotency
brain/agent/reflection_engine.py — reflection
brain/agent/turn_finalizer.py — finalization
brain/agent/artifact_store.py — artifacts
brain/agent/callbacks.py — event callbacks

See docs/harness/ for the deep dive.

3. Context, Memory, And Knowledge¶

The context plane is not "prompt stuffing." Depending on the surface and options, the runtime composes:

Concept	Implementation
Ingest + source tracking	`brain/ingest/`, `brain/cli/ingest.py`
Codebase indexing	`brain/codebase/`, `brain/cli/codebase_cmd.py`
Hybrid retrieval	`brain/retrieve/`, especially `brain/retrieve/hybrid.py`
Context packs	`brain/context_packs/` (service, builder, schema_v2, contracts)
Context compilation / compaction	`brain/context/`, `brain/context/compiler.py`, `brain/context/compaction.py`
Persistent memory	`brain/memory/store.py`, `brain/memory/retriever.py`, `brain/memory/review_queue.py`
Kernel memory bundle	`brain/kernel/memory.py`
Promoted knowledge	`brain/knowledge/`
Graph / entity / ontology	`brain/graph/`, `brain/identity/`, `brain/ontology/`
Institutional memory	`brain/institutional/`
Antahkarana cognitive state	`brain/antahkarana/chitta/`, `brain/antahkarana/manas/`, etc.
Vector store + file hashes	`brain/state/vector_store.py`, `brain/state/file_hashes.py`

Large contexts may be compacted before generation. Chat and serve paths share the model-aware compaction helpers.

brain/runtime/service_factory.py and brain/serve/services.py are the main places that construct shared services (settings, migrations, event logs, retrievers, memory stores, vector stores, approval policies).

4. Governance, Policy, And Approvals¶

SecondBrain treats read-only, local write, network, and externally delegated actions differently. Every loop converges on the same policy surfaces:

Concept	Implementation
Policy bundles + execution modes	`brain/policies/`
Kernel permission model	`brain/kernel/tooling/permissions.py`
Kernel tool executor	`brain/kernel/tooling/executor.py`
Chat tool scheduling / execution	`brain/agent/tool_scheduler.py`, `brain/agent/tool_executor_v2.py`
Command safety	`brain/policies/command_safety.py`
Action gateway + approvals	`brain/orchestrator/action_gateway.py`, `brain/orchestrator/approvals.py`
Serve approvals router	`brain/serve/routers/approvals.py`
Gateway approval lifecycle	`brain/gateway/`
Background-session approval state	`brain/background_sessions/runtime.py`, `brain/background_sessions/store.py`

MCP, external integrations, and Claude / Codex bridge sessions all pass through these surfaces when they can perform side effects. Serve chat publishes policy decisions over SSE; CLI chat renders permission prompts directly; background sessions can pause in awaiting_approval with an approval-request ID stored on the session row.

5. Tools, Adapters, And Capability Planes¶

Concept	Implementation
Chat / runtime tool registry	`brain/agent/tools.py`
Toolsets	`brain/agent/toolsets/`
Kernel tool registry	`brain/kernel/tooling/registry.py`
Built-in kernel tools	`brain/kernel/tools/`
MCP registry + execution	`brain/mcp/`, `brain/adapters/mcp/`
Skill discovery / routing	`brain/skills/`
Agent registry / builder	`brain/agent_builder/`, `brain/agent_sdk/`
Provider registry / router	`brain/providers/`
Integrations + connectors	`brain/integrations/`, `brain/connectors/`
Claude / Codex bridges	`brain/superpowers/`, `brain/background_sessions/bridges.py`
Agent patterns	`brain/patterns/`

These are the extension points — prefer adding a CLI command, a toolset, a kernel tool, a skill, an MCP server, an integration, a workflow, a provider, or a pattern over inventing a new runtime layer.

6. Persistence — Split SQLite Topology¶

State is split by domain in brain/db/topology.py (TOPOLOGY_VERSION = 2).

Database	Tables owned
`runtime.db`	chat sessions / messages, tool calls, approvals, integrations, gateway queues, A2A, observations, background sessions, subagent runs
`memory.db`	working memory, episodes, long-term memories, memory links, review queue, institutional memory, lifecycle
`knowledge.db`	raw / promoted artifacts, facts, evidence, graph + entity, context items, captures, ingest sources, people identity, decision catalog
`work.db`	tasks, workflows, task workspaces, projects, commitments, actions, quality / eval / autotune / improvement, agent + skill registry governance
`travel.db`	travel goals, events, preferences, plans, loyalty programs
`antahkarana.db`	Antahkarana cognitive state — chitta, manas, ahamkara, karma, sankalpa, sabha

Beyond the SQLite ledgers:

Vector store lives under state/retrieval/ (LanceDB by default; pluggable via SB_VECTOR_BACKEND — see brain/memory/indexes/). Legacy installs with data under state/chroma/ are auto-migrated on first run.
Vault, artifacts, reports, logs, backups, built UI assets live on the filesystem alongside the DBs.

Migrations: legacy chain 001_initial_schema.sql … 048_chat_turn_journal.sql (top-level), plus split-domain migrations under brain/db/migrations/{runtime, memory, knowledge, work, travel}/.

7. Observability And Quality¶

Concept	Implementation
Runtime observation envelopes	`brain/obs/`, `brain/db/migrations/runtime/008_observation_envelopes.sql`
Decision traces	`brain/obs/trace_store.py`, `brain/cli/traces.py`
Chat / session events	`brain/state/event_log.py`, `brain/chat/session_store.py`
Serve chat SSE event types	`brain/serve/chat_runtime.py::STREAM_EVENT_TYPES`
UI reducer handled set	`serve-ui/src/lib/chat.ts::HANDLED_EVENT_TYPES`
Startup drift check	`GET /stream-events` (`brain/serve/routers/core.py`) + UI diff in `serve-ui/src/App.tsx`
Quality control plane	`brain/quality/`
Evals	`brain/eval/`, `brain/evals/`
Simulations	`brain/simulations/`
Conversation improvement	`brain/improvement/`
Autotune	`brain/autotune/`
Reports	`brain/reports/`, `brain/cli/report_cmd.py`

When adding a chat stream event, update the serve event list, the UI reducer, and the chat page rendering in the same change. The browser warns when it receives a server event the reducer does not handle. See the frontend contract in docs/specs/agent-harness-frontend-contract.md.

End-to-End Flows¶

Chat turn¶

flowchart TD
  U["User message"] --> SURF{"Surface"}
  SURF --> REPL["CLI ChatRepl<br/>brain/chat/repl.py"]
  SURF --> ROUTER["Serve chat router<br/>brain/serve/routers/chat.py"]
  ROUTER --> SSE["QueueCallbacks + SSE<br/>brain/serve/chat_runtime.py"]
  REPL --> STATE["ChatSessionState"]
  SSE --> STATE

  STATE --> PROMPT["PromptBuilder<br/>profile, skills, instructions"]
  STATE --> CTX["Context assembly<br/>retrieval, packs, memory, Antahkarana"]
  PROMPT --> HARNESS["AgentHarness.run_turn"]
  CTX --> HARNESS

  HARNESS --> PREP["TurnPreparer + TurnBudget"]
  PREP --> LLM["Provider tool-call loop"]
  LLM --> DECIDE{"Tool call?"}
  DECIDE -- "no" --> FINAL["TurnFinalizer<br/>citations, memory, decisions"]
  DECIDE -- "yes" --> SCHED["ToolScheduler"]
  SCHED --> POLICY{"Policy / approval gate"}
  POLICY -- "denied" --> OUT["Typed ToolDenied outcome"]
  POLICY -- "allowed" --> EXEC["BoundedToolExecutor"]
  EXEC --> OUT2["ToolOutcome<br/>ok / error / timeout / denied / artifact"]
  OUT2 --> LLM
  FINAL --> STORE[("runtime.db<br/>chat_messages, tool_calls,<br/>session_events, observations")]

Durable background session¶

flowchart TD
  START["Start request<br/>CLI / serve UI / automation / workflow"] --> REQ["BackgroundSessionStartRequest<br/>engine, runner_kind, goal,<br/>cwd, model, approval mode"]
  REQ --> ROW["background_agent_sessions<br/>queued / running /<br/>awaiting_approval / terminal"]
  ROW --> SUP["BackgroundSessionSupervisor"]
  ROW --> DAEMON["Daemon sweeper<br/>orphan + stale cleanup"]
  SUP --> WORKER["session worker<br/>heartbeat + attempts"]
  WORKER --> ENGINE{"Engine"}

  ENGINE -- "native/chat" --> CHAT["AgentHarness chat run"]
  ENGINE -- "native/agentic" --> AGENTIC["AgenticRuntime"]
  ENGINE -- "claude/codex bridge" --> BRIDGE["External CLI bridge<br/>brain/superpowers/"]

  CHAT & AGENTIC & BRIDGE --> APPROVAL{"Write-capable?"}
  APPROVAL -- "needs approval" --> APR["approval_requests<br/>awaiting_approval"]
  APR --> WORKER
  APPROVAL -- "allowed" --> RUN["Execute + collect output"]
  RUN --> CKPT["background_agent_checkpoints<br/>summary + resume_cursor"]
  RUN --> ART["background_agent_artifacts"]
  RUN --> EVENTS["session_events<br/>session_state / checkpoint"]
  EVENTS --> WS["WebSocket / SSE<br/>operator UI updates"]

For deeper flow detail (tool waves, replay, reflection) see docs/harness/.

Reading order for maintainers¶

When deciding where a change belongs, read in this order:

This page — to identify the layer.
The component overview under docs/components/<name>/ — for the public API and contracts.
The package under brain/<name>/ — for the actual implementation.
tests/<module>/test_<name>*.py — for the contract under load.
The frontend contract (docs/specs/agent-harness-frontend-contract.md) — only if the change crosses the chat-stream boundary.

Cross-links to the deep-dive component docs:

Kernel
Agent Harness — overview, data flow, harness API, collaborators, testing
Antahkarana
Data Agent
MCP
Patterns
Travel
Marketing