Context, memory, and research configuration¶

These three configuration groups affect what supporting information Colossus prepares for model work, but they own different state and lifecycles:

Context snapshots do not delete transcript messages. Memory indexes do not own memory lifecycle state. Research evidence is not automatically promoted into general memory.

For user workflows, see Sessions and context, Memories, and Deep research.

Choose a starting point¶

Omitting all three blocks selects these defaults:

context:
  autoCompaction: true
  compactAtPercent: 70
  targetPercent: 45
  preserveRecentMessages: 8
  modelAssisted: true
memory:
  indexEnabled: true
  indexPath: null
  retrievalLimit: 6
  semantic:
    kind: disabled
research:
  maxSources: 20
  maxWorkers: 4

When a block is present, you may specify only the fields you want to override. Omitted ordinary fields keep the defaults above, while unknown fields are rejected. Tagged choices such as semantic still require an explicit kind when their block is present.

Context configuration¶

Context configuration controls when older messages are replaced by an immutable summary in the next provider request. The complete encrypted message history remains available through session commands and audit state.

Context fields¶

All percentages apply to the selected model profile's effective input budget, not its advertised context window:

safety margin = max(ceil(context window / 10), 512)
input budget  = context window - max output - safety margin
threshold     = input budget × compactAtPercent / 100
target        = input budget × targetPercent / 100

Instructions, tool schemas, binding decisions, relevant memories, snapshots, and recent messages all contribute to the prepared-request estimate. See Runtime limits configuration for a worked token example.

Automatic compaction¶

When autoCompaction is enabled and the original context estimate exceeds the threshold, Colossus creates a snapshot only when there is no useful active snapshot or the active prepared view also exceeds the threshold. It summarizes an older message range and preserves the newest configured messages.

Compaction tries to reach targetPercent, but preservation and binding context take precedence. If the newest logical turn, instructions, tool definitions, decisions, or preserved messages cannot fit in the effective input budget, Colossus fails explicitly instead of discarding them. This means compaction may have triggered even when the turn ends with a preservation error: the protected tail alone was too large to summarize. Built-in repository file summaries are byte-bounded as well as line-bounded so one generated or minified preview cannot create that condition by itself.

Setting autoCompaction: false disables threshold-triggered snapshots. Manual compaction remains available, and omitted context fields keep their defaults. The materialized values are still validated together:

context:
  autoCompaction: false
  compactAtPercent: 70
  targetPercent: 45
  preserveRecentMessages: 8
  modelAssisted: false

Use this only when an operator or application will monitor context and compact deliberately.

Model-assisted versus deterministic snapshots¶

With modelAssisted: true, Colossus resolves the context_summarizer model role. An unconfigured specialized role follows normal model routing and falls back to primary. When the resolved provider is usable, Colossus may send bounded historical messages to that model with no tools and use a valid result as a hybrid_model snapshot.

If the route is offline echo, unavailable, too small, returns no usable final text, or fails, Colossus creates a deterministic snapshot instead. Compaction failure never causes canonical session messages to be deleted.

Set modelAssisted: false when historical messages must not be sent through a separate summarization call or when fully deterministic snapshots are preferred:

context:
  autoCompaction: true
  compactAtPercent: 65
  targetPercent: 40
  preserveRecentMessages: 12
  modelAssisted: false

If model assistance is enabled, review the provider and model selected by the context_summarizer role. A dedicated route can use a smaller model or a different trust boundary than primary.

Snapshot lifecycle¶

Snapshots are immutable journal records with a source message range and either the deterministic or hybrid_model strategy. One snapshot is active for future turns; restoring an older snapshot changes that active pointer without deleting later messages or snapshots.

Inspect and manage the lifecycle with:

colossus --config .colossus/config.yaml context status SESSION_ID --role primary
colossus --config .colossus/config.yaml context compact SESSION_ID --role primary
colossus --config .colossus/config.yaml context list SESSION_ID
colossus --config .colossus/config.yaml context restore SESSION_ID SNAPSHOT_ID

context status reports the resolved model profile, raw and prepared estimates, output and safety reservations, input budget, threshold, target, and active snapshot. Manual compact and restore are independently authorized state transitions.

Memory configuration¶

Memory records are durable, scoped, non-secret background context. Active decisions are binding context and take precedence over memories; memories are explicitly presented to the model as background rather than instructions.

Memory fields¶

retrievalLimit does not change the CLI memories search --limit argument. It bounds automatic memory composition during model-context preparation.

Local Tantivy index¶

When indexEnabled is true, Colossus opens a local Tantivy lexical index. The index stores candidate IDs and disposable search fields; after searching, Colossus reloads each result from the canonical journal and rechecks status, expiry, session scope, and repository scope before release.

If indexPath is null, the path is derived from the local storage.path. This remains local derived state when PostgreSQL owns the canonical journal. Set an explicit path when local state placement or volume management requires it:

memory:
  indexEnabled: true
  indexPath: .colossus/indexes/memory-tantivy
  retrievalLimit: 8
  semantic:
    kind: disabled

Index updates are queued from canonical journal events and applied in order. Index-open, sync, or search failure does not destroy memories; Colossus can fall back to a bounded canonical term match and exposes index lag and errors through status.

Disable all indexes while preserving canonical memory operations with:

memory:
  indexEnabled: false
  indexPath: null
  retrievalLimit: 6
  semantic:
    kind: disabled

Chroma cannot be configured when indexEnabled is false.

Index operations¶

colossus --config .colossus/config.yaml memories index status
colossus --config .colossus/config.yaml memories index sync
colossus --config .colossus/config.yaml memories index rebuild

status reports each consumer's readiness, journal position, lag, retry state, and adapter status. sync retries queued journal-to-index work. rebuild resets disposable index data and recreates it from canonical active records; it does not recreate or edit memory records.

An external mutation with an unknown outcome blocks automatic Chroma retries. Inspect status and use an operator-authorized rebuild to re-establish known projection state.

Chroma semantic memory¶

Chroma adds a second candidate index alongside Tantivy. It does not become the memory source of truth. The Chroma collection and its local position file are disposable projection state that can be rebuilt from the journal.

This example uses deterministic local embeddings and sends the resulting vectors, memory text, and bounded metadata to Chroma:

memory:
  indexEnabled: true
  indexPath: .colossus/indexes/memory-tantivy
  retrievalLimit: 8
  semantic:
    kind: chroma
    baseUrl: https://chroma.internal.example
    tenant: colossus
    database: production
    collection: memories
    credentialReference: env:CHROMA_TOKEN
    timeoutMs: 30000
    positionPath: .colossus/indexes/chroma-position.json
    embedding:
      kind: local
      dimensions: 384
sandbox:
  networkDestinations:
    - https://chroma.internal.example

Under an isolating boundary, merge the sandbox destination into the deployment's sandbox block. Acknowledged full access needs no duplicate destination and adding one does not narrow ambient authority. Chroma and embedding credentials are resolved by Colossus in-process after authorization; they do not need sandbox.environment grants.

Chroma fields¶

The Chroma baseUrl must not contain a path other than /, user information, query, or fragment. Colossus constructs the Chroma v2 API paths and gets or creates the configured collection.

Under acknowledged full access, canonical non-loopback plaintext HTTP is accepted. It has no TLS confidentiality or server authentication and can expose memory text, metadata, vectors, and credentials in transit.

Under an isolating boundary, the Chroma origin must appear in sandbox.networkDestinations. Its client uses DNS pinning, no ambient proxy, no redirects, bounded requests and responses, the permit timeout, and the shared network CA bundle.

Enabling Chroma is an external disclosure decision: memory text, metadata, IDs, and vectors are sent to that service. The configured collection should be dedicated to the deployment and protected accordingly.

Local embeddings¶

embedding:
  kind: local
  dimensions: 384

Local embeddings use deterministic token and bigram feature hashing without a model or network request. dimensions must be in 64..=4096. This provides lightweight lexical similarity in vector form; it should not be described as model-derived semantic understanding.

OpenAI-compatible embeddings¶

Use a remote embedding endpoint when model-derived vectors are required:

embedding:
  kind: open_ai_compatible
  profile: memory-embeddings
  model: text-embedding-model
  baseUrl: https://embeddings.example.com/v1
  credentialReference: env:EMBEDDING_API_KEY
  timeoutMs: 30000
  dimensions: 1536

Colossus appends /embeddings to baseUrl. Under an isolating boundary, the embedding origin and Chroma origin must both be present in sandbox.networkDestinations; list both when they differ. This is the corresponding isolating-boundary grant:

sandbox:
  networkDestinations:
    - https://chroma.internal.example
    - https://embeddings.example.com

Acknowledged full access authorizes the exact configured HTTP(S) endpoints without that duplicate grant; adding the list does not narrow ambient authority.

The embedding service receives memory text during indexing and query text during search. Changing the embedding model or vector dimensions changes projection compatibility; rebuild the disposable memory indexes deliberately after the new profile is in place.

Research configuration¶

Research configuration sets run-wide evidence bounds. The caller selects depth and evidence lanes for each run; the model cannot add an unrequested lane or choose a new backend.

research:
  maxSources: 20
  maxWorkers: 4

Research fields¶

Despite its name, maxWorkers is a total work-item budget in the current runtime, not a promise of parallel execution. Each planned query combined with each selected lane is one potential collection job.

Depth determines the planned-query ceiling:

For example, standard research over repo,web,mcp can plan up to nine query/lane jobs. With the default maxWorkers: 4, Colossus attempts the first four and records the rest as skipped limitations. Source exhaustion can stop collection earlier.

To allow every potential lane for a standard three-lane run while retaining a 30-source ceiling:

research:
  maxSources: 30
  maxWorkers: 9

Deep three-lane research can plan 18 jobs, but the hard maxWorkers maximum is 16. Split an unusually broad question or select fewer lanes instead of assuming every combination will run.

Evidence lanes¶

Every collection is an ordinary authorized effect. A denied, unavailable, failed, or budget-skipped lane becomes a durable limitation while other released evidence can still produce a report.

Configure web search through Search configuration. Configure MCP evidence through MCP research templates.

Research model roles and fallback¶

Research uses the fixed research_planner, research_worker, and research_synthesizer model roles for query planning, claim extraction, and final report synthesis. Unconfigured specialized roles fall back to primary.

Model output is accepted only after strict phase-specific validation. If planning, extraction, or synthesis fails or returns invalid output, Colossus records the fallback and continues with deterministic queries, source sentences, or citation-safe report generation. Model assistance never weakens the configured source, lane, or worker bounds.

Web search route¶

Use named top-level search profiles and an explicit research route:

research:
  maxSources: 20
  maxWorkers: 4
search:
  profiles:
    internal:
      kind: searxng
      endpoint: https://search.example.com/search
      credentialReference: env:SEARCH_TOKEN
      authHeader: X-Searxng-Key
      userAgent: colossus/0.10
      timeoutMs: 30000
  roles:
    research: internal
sandbox:
  networkDestinations:
    - https://search.example.com

Top-level search supports credentials, independent profiles, and explicit agent and research routes. Its credential is resolved in-process and does not need a sandbox environment grant. See Search configuration for transport and authority requirements.

Data disclosure and trust boundaries¶

Credentials remain references in YAML and are resolved only inside permit-bearing adapters. Colossus-owned semantic and search clients use the shared CA bundle for HTTPS, declared or ambient request-bound network authorization, DNS pinning, redirect rejection, response bounds, and quarantine.

Neither an index nor an external service may directly make a memory visible. Colossus always rechecks canonical lifecycle and scope before composing memory context.

Common configuration mistakes¶

Validate the result¶

Parse all relationships and bounds without resolving credential values:

colossus --config .colossus/config.yaml config show

Then exercise the configured boundaries:

colossus --config .colossus/config.yaml models route context_summarizer
colossus --config .colossus/config.yaml context status SESSION_ID --role primary
colossus --config .colossus/config.yaml memories index status
colossus --config .colossus/config.yaml search profiles
colossus --config .colossus/config.yaml search query \
  "deployment evidence" --role research --limit 3
colossus --config .colossus/config.yaml mcp tools --server SERVER

Run only the commands relevant to enabled features. config show proves the strict shape and numeric relationships; status and live queries prove that local indexes, model routes, network policy, credentials, and remote services are actually usable.

Return to the configuration overview.