Skip to content

Configuration Reference

SPOT reads its configuration from two places: environment variables, for the infrastructure and the secrets, and spot.yaml, for the platform settings, the plugins, the workflows and the routing. This page lists both. Its environment variables are those of the development stack, the repository's own compose files and .env. An install from deploy/ sets its variables in deploy/.env: Deployment configuration lists them.

Quick Start

Copy .env.example to .env and customize:

cp .env.example .env

See .env.example for comprehensive documentation of all available variables.

Environment Control

# Set environment mode
APP_ENV=prod|dev|test

# Default: prod (for safety)

See Environment Configuration for details on environment management.

Required Variables

These variables MUST be set in production:

Variable Description Example
APP_ENV Environment mode prod, dev, or test
SECRET_KEY JWT token signing key Generate with: python -c "import secrets; print(secrets.token_urlsafe(32))"
SPOT_INTERNAL_API_KEY Key core services authenticate each other with Generate like SECRET_KEY
ENCRYPTION_KEY Fernet key the stored emails, results, plugin tokens, plugin secrets and plugin-source tokens are encrypted with python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
POSTGRES_DB PostgreSQL database name spot
POSTGRES_USER PostgreSQL username spot
POSTGRES_PASSWORD PostgreSQL password secure_password
RABBITMQ_DEFAULT_USER RabbitMQ username guest
RABBITMQ_DEFAULT_PASS RabbitMQ password secure_password

Core Configuration

Docker Compose

Variable Default Description
COMPOSE_PROJECT_NAME spot Project name for consistent container/network naming

Application Settings

Variable Default Description
LOG_LEVEL INFO Logging level (platform.log_level in spot.yaml): DEBUG, INFO, WARNING, ERROR or CRITICAL, in any case; WARN and FATAL are read as WARNING and CRITICAL. Any other value stops the service at start, and is refused with a 422 when saved through the API
LOG_FORMAT json json, or text for development; any other value stops the service at start
DEBUG false Enable debug mode
TZ UTC Timezone

Security

Variable Default Description
SECRET_KEY (required) Secret key for JWT token signing - MUST be set
SPOT_INTERNAL_API_KEY (required) Key core services authenticate each other with: the api-gateway sends it to the plugin-manager, to the analyzer-orchestrator (every route but /health and /version), to the knowledge service (Knowledge admin pages) and to the mail-orchestrator (mail-retriever views, retrievers' ingest). Whoever sends it is treated as core, so it never goes to a plugin, which uses its own token
ENCRYPTION_KEY (required) Fernet key that encrypts what SPOT stores of an email: its raw message and view, its analyses' results and their signals' evidence and location; the plugin tokens and secrets; and the auth tokens of the plugin sources. Migration 010 needs it as soon as analyses exist. Keep it with your backups: data encrypted with another key cannot be read
ALLOWED_HOSTS localhost,127.0.0.1,api-gateway,testserver The host names the api-gateway answers, comma-separated or a JSON list; any other Host is refused unless DEBUG is set

Database (PostgreSQL)

Variable Default Description
POSTGRES_DB spot Database name
POSTGRES_USER spot Database username
POSTGRES_PASSWORD spot123 Database password
DATABASE_URL (auto-constructed) Full connection URL

Redis Cache

Variable Default Description
REDIS_PASSWORD (empty) Redis password (empty = no auth)
REDIS_URL (auto-constructed) Full connection URL
REDIS_MAXMEMORY 256mb Maximum memory
REDIS_MAXMEMORY_POLICY allkeys-lru Eviction policy

RabbitMQ Message Queue

Variable Default Description
RABBITMQ_DEFAULT_USER guest RabbitMQ username
RABBITMQ_DEFAULT_PASS guest RabbitMQ password
RABBITMQ_URL (auto-constructed) Full connection URL

The analyzer-orchestrator consumes five queues:

Queue Bound to Holds Dead-letter queue
orchestrator.analysis spot.analysis / email.analyze Analysis jobs orchestrator.analysis.dead-letter
orchestrator.analysis.sync spot.analysis / email.analyze.sync The ingests of retrievers that wait for their verdict orchestrator.analysis: an ingest expired, refused, or failing past the limit is handled with the bulk
orchestrator.dispatch spot.results / analysis.complete Verdicts to dispatch to responders orchestrator.dispatch.dead-letter
orchestrator.shadow spot.results / analysis.complete Results to run the shadow workflows on, at most 1 000 orchestrator.shadow.dead-letter, at most 1 000 too
orchestrator.benchmark spot.analysis / email.benchmark The emails of a platform benchmark run, one message each, consumed one at a time orchestrator.benchmark.dead-letter

The api-gateway, the mail-orchestrator and the analyzer-orchestrator declare them once connected, so nothing is published to a queue not declared yet. The first of them to start after an upgrade empties the classic queues of earlier releases (orchestrator.analysis.requests, orchestrator.results.dispatch) into them and deletes those. A queue the analyzer-orchestrator of the earlier release still consumes is emptied but kept, since that one may hold messages it has not finished: they go back to the queue when it stops, and the new analyzer-orchestrator, which starts after it, empties and deletes the queue. The unused exchange spot.direct is deleted too, unless a queue of an older release is still bound to it.

The queues are quorum queues with a delivery limit of 10 (x-delivery-limit). A message whose handler fails is handed back after a wait that starts at 1 s and doubles up to 5 minutes; once it has failed 10 times, its next failure parks it in its dead-letter queue, 11 deliveries and about 13.5 minutes after it was first received. A message that is not JSON, or that its consumer can never process (an analysis job message that names no job, a message that is no result), is parked at once. A parked message stays in its dead-letter queue until it is purged or replayed. The sync queue's failures, expired or refused messages included, move to orchestrator.analysis and are handled there. Parking is at least once: a message being parked stays in its queue until its dead-letter queue has taken it.

orchestrator.shadow is bounded: it holds at most 1 000 results (x-max-length), and past that its oldest are dropped into its dead-letter queue (x-overflow: drop-head), which holds at most 1 000 and drops its own oldest. So when the shadow workflows cannot keep up, the oldest shadow work goes, never production's, and the drops show as the dead-letter queue's depth on the Queues page. RabbitMQ parks a bounded queue's messages at most once (at least once needs reject-publish overflow): shadow work is best effort.

The bound is part of the queue's declaration, and RabbitMQ refuses to declare the queue again with another one (PRECONDITION_FAILED). A release that changes the bound says so, and its upgrade deletes orchestrator.shadow and orchestrator.shadow.dead-letter (rabbitmqctl delete_queue <name>), losing the shadow work they hold. Until then, the services log that orchestrator.shadow cannot be declared and the shadow does not run; production's queues are declared and consumed all the same.

To replay parked messages once their cause is fixed, an admin calls POST /api/v1/observability/queues/{queue}/replay (?limit=N), runs spot-cli queues replay <queue> [--limit N], or uses the Queues page's "Replay one" and "Replay all" buttons, for orchestrator.analysis, orchestrator.dispatch, orchestrator.shadow or orchestrator.benchmark (what a replay of the last does: Messages in a dead-letter queue):

  1. Fix the cause first.
  2. Replay one message (--limit 1, ?limit=1, or "Replay one"), and check that it is handled: it does not come back to the dead-letter queue, and its job completes (for a benchmark message, its run's count moves).
  3. Then replay the rest.

Why one first: a message that fails again holds one of its queue's slots while it waits between its retries, about 13.5 minutes in all, before it is parked again. N such messages hold the analysis queue's 10 slots for about N/10 x 13.5 minutes, and new analyses wait behind them, the synchronous ingests that expire into that queue included.

A replay moves only the messages parked when it starts, at most limit, into the queue alone (through the default exchange, so no other queue bound to the same exchange gets a copy), each with its whole retry budget again. It moves each message at least once: a failure between a message's publish and its ack can leave it in both queues, and two replays at once can move a message twice. That is safe, as for any message delivered twice: an analysis delivered while its job runs waits for that run, one delivered later runs the job again and overwrites its record and result, and each responder acts once per job_id. A broker failure answers 503 and leaves what was not moved parked: replay again.

The Queues page of the dashboard shows how many messages each queue holds, dead-letter queues included. From the host of the stack:

# How many messages each queue holds (the counts refresh every few seconds)
docker compose exec rabbitmq rabbitmqctl list_queues name messages

# Look at up to 10 parked messages, leaving them parked; rabbitmqadmin signs in
# with the user RabbitMQ was created with, read from the container's environment
docker compose exec rabbitmq sh -c 'rabbitmqadmin -u "$RABBITMQ_DEFAULT_USER" \
  -p "$RABBITMQ_DEFAULT_PASS" get queue=orchestrator.analysis.dead-letter \
  count=10 ackmode=reject_requeue_true'

# Drop the parked messages of a queue
docker compose exec rabbitmq rabbitmqctl purge_queue orchestrator.analysis.dead-letter

# Replay them into their queue, once their cause is fixed, with a one-off shovel:
# it moves what the dead-letter queue holds when it starts, then removes itself
docker compose exec rabbitmq rabbitmq-plugins enable rabbitmq_shovel
docker compose exec rabbitmq rabbitmqctl set_parameter shovel replay-analysis \
  '{"src-protocol": "amqp091", "src-uri": "amqp://", "src-queue": "orchestrator.analysis.dead-letter", "src-delete-after": "queue-length", "dest-protocol": "amqp091", "dest-uri": "amqp://", "dest-queue": "orchestrator.analysis"}'

rabbitmqctl, rabbitmq-plugins and the shovel's amqp:// (a direct connection to the node) need no user.

A parked message tells nothing of its content in the logs: the handler logs its job id, the error's type and where it was raised. The shovel, like the replay above, moves what was parked when it starts, and each message gets its whole retry budget again: RabbitMQ counts its deliveries from 0. A shovelled message's first delivery still carries the x-delivery-count of the dead-letter queue (each look counts there), which the consumer ignores on a first delivery; the replay drops it.

How the broker counts deliveries is RabbitMQ 3.13's: every message handed back counts as a failed delivery, whether its consumer nacked it or closed its channel (a crash included). RabbitMQ 4.0 changes the rules and the defaults (quorum queues get a delivery limit of 20). When RabbitMQ is upgraded, run the broker integration tests again (tests/integration/test_broker_flow.py, TestDeadLettering and TestSyncLane): they check that a message that always fails is parked after 11 deliveries, that the queues of earlier releases are emptied into their successors, and that a synchronous ingest that expires (losing its expiration) or keeps failing moves to the bulk queue.

Service Configuration

API Gateway

Variable Default Description
API_GATEWAY_PORT 8001 External API port

Analyzer Orchestrator

Variable Default Description
MAX_CONCURRENT_JOBS 10 Analyses run at once from the bulk queue (orchestrator.analysis): user submissions, re-analyses and the ingests of retrievers that do not wait
MAX_CONCURRENT_SYNC_JOBS 4 Analyses run at once from the queue of synchronous ingests (orchestrator.analysis.sync), the retrievers that wait for their verdict, such as the milter

A synchronous ingest has slots of its own, so a backlog of bulk analyses never holds it up behind them. One not started before its retriever stops waiting expires and moves to the bulk queue, where it is still analysed, later: one not started in time never holds up a fresh synchronous ingest. One whose handling keeps failing (Redis down, say) holds its slot through its retries, as in the bulk queue, before it moves to the bulk queue too. The analyzers stay shared: a synchronous ingest skips the queue, not an analyzer that is saturated, so an analyzer slowed by bulk analyses slows it too.

Plugin Configuration

"Plugin" is the umbrella vocabulary for anything pluggable into SPOT. Four kinds exist, each with its section under plugins: in config/spot.yaml:

  • analyzers (plugins.analyzers) answer POST /internal/analyze with a verdict, a score and the signals behind it (AnalyzerVerdict);
  • context providers (plugins.context_providers) fill the Knowledge Store when core calls their POST /internal/sync, on their sync_schedule; analyzers query the store (Context Providers guide);
  • mail retrievers (plugins.mail_retrievers) push raw messages to the api-gateway's POST /internal/ingest; each is a mail connection, with a mode (see Observe and enforce);
  • responders (plugins.responders) act on verdicts on POST /internal/respond (see Responders).
plugins:
  analyzers:
    analyzer-nlp:
      enabled: true
      url: "http://spot-analyzer-nlp:8000"
      settings: {}
  context_providers:
    employee-dir:
      enabled: true
      url: "http://provider-employee-dir:8000"
      sync_schedule: "0 */6 * * *"
      settings:
        LDAP_URL: ldap://ldap.example.com
  mail_retrievers: {}
  responders: {}

Each entry needs at minimum url and enabled. Installed plugins also carry image, version, container_id, container_name, installed_at, source_id, source_name, requested_resources, egress, after an update previous_version (the version it replaced, which a roll back installs) and, when the plugin needs one, consent (set by the installer). resources overrides the memory and CPU the image asks for (see Plugin resources). A plugin has one kind, its section's: an entry has no kinds. The api-gateway drops, once at start, the kinds: that 2.0.0-beta.6 and earlier wrote on every entry, and refuses to start on one whose kinds names another kind or several.

Core calls every plugin with the plugin's own token, and a plugin calls SPOT with it (see Plugin tokens). An analyzer that has no token, no url, or is disabled is not called: in the workflows that name it, its failure strategy applies at once, with no retry. It counts as unknown, or fails the job when its strategy is fail (see Analyzer Configuration).

Plugin sources

The plugin sources the catalog is read from are kept in spot.yaml, under plugins.sources, each by its id (1 to 50 lowercase letters, digits and hyphens, neither first nor last a hyphen):

plugins:
  sources:
    spot-plugin-index:
      name: "SPOT plugin index"
      type: index
      url: "https://codeberg.org/SPOT_Project/spot-plugin-index/raw/branch/published/index.json"
      enabled: true
      priority: 50
    corp:
      name: "Corporate plugins"
      type: gitlab
      url: "https://gitlab.example.com"
      organization: "plugins"
      registry: "gitlab.example.com:5050"
      enabled: true
      priority: 100
      secrets:
        AUTH_TOKEN: "gAAAAAB..."
  index_offered: true
Field Required / default Meaning
name Required The name shown in the dashboard.
type Required index, gitlab, gitea, dockerhub, ghcr, gitlab_registry or gitea_registry.
url Required The forge's, or the registry listing's, base URL; for an index, the URL of its index.json.
organization Required, none for an index The organisation, group or namespace that holds the plugins; a registry source may give image_namespace instead.
registry Required, none for an index The container registry host.
image_namespace None The registry namespace, when it differs from organization.
enabled true A disabled source is not scanned.
priority 100 From 1 to 1000; a lower number comes first.
secrets.AUTH_TOKEN None, and none for an index The source's token, sealed with ENCRYPTION_KEY.

An index source is read only once a key SPOT ships is found to have signed it (see The signed plugin index). Set a token through the API (/api/v1/plugin-sources) or the dashboard's Plugins → Sources page, which seal it; the API never returns it. A start of the api-gateway that finds no sources key gives spot.yaml the sources 2.0 kept in Redis. Until 2.2, removing the key so brings back the sources 2.0 kept in Redis, those removed since included.

A start adds the signed plugin index, spot-plugin-index, when it finds in plugins.sources, those it moves from Redis included, no source of type index and none of its id, no record that it was offered (plugins.index_offered, which that start writes), and SPOT_BOOTSTRAP_DEFAULT_PLUGIN_SOURCE not set to false. Removed, it stays removed; lifting the opt-out later, or removing an index source of another id, offers it once at the next start.

Plugin settings

A plugin's image declares its settings in its spot.plugin.settings label: each a type (string, integer, number, boolean, url or secret), with optionally required, default, description, secret and enum. The dashboard renders its form from it, and GET /api/v1/config/plugin/{kind}/{id}/schema serves it from the catalog, that of the version installed, so it answers while the plugin is down: 404 for a manual entry, 503 when the catalog cannot be read, does not list the plugin at that version or cannot read its entry. An installed plugin's settings and secrets are written against that schema, which tells the secret ones: without it a write answers 503 and saves nothing, until the catalog is refreshed, the plugin's source is re-added or the plugin is updated to a version it lists. A manual entry has no catalog schema: its settings are stored as given.

The values live on the plugin's entry: settings for the plain ones, secrets for the secret ones, encrypted with ENCRYPTION_KEY. The installer starts the container with each setting as an environment variable of its name in upper case, each secret as the file /run/secrets/<NAME>, and SPOT's URLs (SPOT_GATEWAY_URL, SPOT_KNOWLEDGE_URL) and the plugin's id (SPOT_PLUGIN_ID); the SDK's PluginSettings reads them. Set secrets through the API (see Plugin secrets), not by hand.

Saving an installed plugin's settings or secrets restarts it: its container is re-created from spot.yaml, and the plugin is unavailable for a few seconds. This applies to PUT /api/v1/plugins/{id}/settings, to a settings, secrets or url change through PUT /api/v1/config/plugin/{kind}/{id}, and to a settings change through PUT /api/v1/config/options/analyzers.<id>.settings...; their answers carry restarting: true. The save and the restart are one step for the plugin: a stop or an uninstall sent meanwhile waits for both. A disabled plugin is not restarted (it takes the settings when started), nor is a manual entry. Each setting given is read as the plugin reads its text, against the plugin's catalog schema: for its type ("8000" and 8000 are an integer, "true" and true a boolean, any value a string), then its bounds (a number is finite and within its minimum and maximum, a string matches its pattern whole), then its enum when it has one. The value is stored as read. One the plugin cannot read answers 422, naming the setting and the rule. So does a setting named spot_plugin_token, the file of the plugin's token, and a secret whose name is no safe file name (letters, digits, _, . and -, not . or ..). A restart that fails answers 503, and the settings stay saved.

Each plugin image declares in spot.plugin.egress what it sends outside SPOT: none, indicators (URLs, domains, hashes and the like) or content (email content). An image without the label counts as content. The catalog shows the class, and the installer records it on the plugin's entry as egress.

Every plugin, of any class, runs on spot-plugins-isolated, a Docker network with no route out. It reaches the gateway, the knowledge service and the analyzer-orchestrator only (the bundled Ollama is out of its reach unless the plugin is of class content), so a model it would fetch by URL or by Hugging Face id cannot load: ship the model in the image or on a volume. A plugin reaches the hosts its settings of type url name, and nothing else, through the egress gateway: see Reaching configured hosts.

Installing a content plugin needs an admin's consent: "consent_to_content": true in POST /api/v1/plugins/install, else 409. The installer records the consent on the entry:

plugins:
  analyzers:
    analyzer-llm:
      url: "http://spot-analyzer-llm:8000"
      image: "registry.example.com/spot/analyzer-llm:2.0.0"
      version: "2.0.0"
      egress: content
      consent:
        egress: content
        user: admin
        at: "2026-10-08T09:30:00Z"

Updating to a version whose egress is content needs the consent again (POST /api/v1/plugins/{id}/update with "consent_to_content": true), unless a consent to content is recorded already. A manual entry (POST /api/v1/config/plugin/{kind}, no image) records no egress and no consent: the admin wires a service of their own, outside the catalog.

The catalog lists an image whose spot.plugin.contract is not 2 (an image without the label is contract 1; a label that is no number is unreadable) with a refusal that says why; installing or updating to it answers 409 with that reason. An entry listed by an earlier SPOT, which did not read the label, is refused until the catalog is refreshed.

A plugin has one kind: the catalog leaves out an image whose spot.plugin.kind names no kind, one core does not know or several, and the scan logs why ("Refused plugin : spot.plugin.kind=... names no kind, an unknown one or several").

Responders

Responders act on verdicts: quarantine, notify, open a ticket. They are the entries of plugins.responders: core calls a plugin as a responder only when it is an entry there. A mailbox connector that fetches mail and moves a malicious message to quarantine is two plugins, a retriever and a responder, each with its own id:

plugins:
  mail_retrievers:
    mailbox-connector:
      url: "http://spot-mailbox-connector:8000"
  responders:
    mailbox-quarantine:
      url: "http://spot-mailbox-quarantine:8000"
    responder-ticket:
      url: "http://spot-responder-ticket:8000"

A workflow names the responders of each verdict in on_verdict:

workflows:
  - id: default-workflow
    on_verdict:
      malicious: [mailbox-quarantine, responder-ticket]
      unknown: [responder-ticket]

Whether they act is the job's mode, not the workflow's (see Observe and enforce). Once a job has its verdict, the analyzer-orchestrator calls POST /internal/respond on each responder named, at once, each with its own token. The call carries the verdict, the score, decided_by, the signals and the job's mode: a responder acts only in enforce, and in observe only records what it would do. Each responder called is recorded as an action, respond, applied when the job is in enforce and the responder answered. A failed job is unknown and goes to on_verdict.unknown. A call that fails, or gets no answer within 10 s, is retried three times (after 1, 2 and 4 s), then given up and logged. A name that is not a configured, enabled responder is skipped with a warning, so a workflow may name a responder installed later.

When a responder gives up, the result is delivered again once every call has ended, and every responder of its verdict is called again, until the delivery limit parks the result in orchestrator.dispatch.dead-letter (see RabbitMQ Message Queue). Calling the responders that succeeded again is safe: each acts once per job_id.

Delivery is at least once: a responder may be called more than once for the same job_id. A call that timed out may still complete while it is retried, a result delivered again calls every responder of its verdict again, and a restart between the calls and the end of the dispatch calls them again. A responder acts once per job_id.

Observe and enforce

Each job is in observe or enforce mode. In observe, SPOT records what would be done with the message; in enforce, it is done. A job is in enforce only when its mail connection and its route both are:

  • a mail connection is a mail retriever's entry: its mode is observe or enforce. A new retriever is written mode: observe, and an entry without one is in observe. Switch it with PUT /api/v1/config/plugin/mail_retriever/{id}/mode (admin, body {"mode": "enforce"}) or spot-cli plugin mode <id> enforce; the history records "Set to enforce", and nothing is restarted: the retriever learns its mode from SPOT's answers to its ingests (IngestionResult.mode), so a switch reaches it with its next answer. While SPOT does not answer, the milter keeps the mode of the last answer, observe until the first. A plugin of another kind has no mode;
  • a route's mode is enforce by default, which follows its connection, or observe, which keeps its jobs in observe whatever the connection says. An email no route takes is in its connection's mode.
plugins:
  mail_retrievers:
    retriever-milter:
      url: "http://spot-retriever-milter:8000"
      mode: enforce
recipient_groups:
  finance: ["cfo@example.com", "@finance.example.com"]
routing:
  - name: finance
    match: {recipient_groups: [finance]}
    workflow: strict-workflow
    mode: observe        # this route's jobs stay in observe

A user's submission (POST /api/v1/analyze) comes through no connection: it is in observe. A re-analysis of a stored email takes the mode of the connection the email came through, and its route's when it names no workflow. One that names a workflow is not routed, so it takes the connection's mode alone: a route's observe holds only the jobs the route chooses, and an analyst's re-analysis naming a workflow, of an email whose connection is in enforce, is in enforce.

A job's mode is decided when it is queued: a switch reaches the jobs queued after it, and those already queued keep theirs. To stop the responders at once, disable them (spot-cli plugin disable <id>): the dispatcher skips a disabled responder from its next read of spot.yaml.

The mode travels with the job: the responders get it as mode, and the retriever in its answer. What was done, or would have been, is recorded as the job's actions, one per job, actor and kind: the retriever reports its own (POST /internal/jobs/{job_id}/action, scope ingest, body {"action": "reject", "applied": true}; applied is refused for a job in observe), and each responder called is one. A job's status (GET /api/v1/analyze/{job_id}) lists them, and GET /api/v1/analyze/actions?days=7 counts those of the last days by actor, kind, action, mode and whether applied, and lists every connection with its current mode (connections).

A retriever's MODE setting (the milter's, before it took its mode from SPOT) is refused by the API in a mail retriever's settings (422, naming the switch above: the plugin and config routes, an install, an update), as WORKFLOW_ID is. One written by hand leaves its settings at the api-gateway's start. It becomes the mode of an entry written before modes, which has none, when it is observe or enforce; any other value, and the setting of an entry that has a mode, is dropped with a warning: only the switch above changes a connection's mode. No plugin may take the id api, the source of user submissions. A workflow's response_mode is gone: a stored one is dropped when read, the change logged, and a workflow written with one is refused.

Mail Orchestrator

Mail retrievers are configured in config/spot.yaml under plugins.mail_retrievers.

Variable Default Description
REQUIRE_BROKER true RabbitMQ is required: every ingest is published to it, and startup fails if it is unreachable
VERDICT_POLL_INTERVAL_MS 100 How often a synchronous ingest polls Redis for the analysis job record

Workflow and Analyzer Configuration Files

Variable Default Description
SPOT_CONFIG_FILE config/spot.yaml The operator's spot.yaml (/app/config/spot.yaml in the containers, on the spot-config volume); it overrides the shipped defaults

Each image ships the defaults at /app/defaults/spot.yaml (config/defaults/spot.yaml in this repository), outside /app/config: the spot-config volume keeps the files of its first install, and the defaults change with each release. A defaults/ directory an older release left in the volume is not read. The operator's spot.yaml overrides the defaults: platform key by key; plugins, mail, intake, workflows, recipient_groups, routing, shadow, sender_lists and default_workflow_id whole.

default_workflow_id names the workflow an analysis that names none runs when no route takes it (a retriever's, the milter's, a submission's): default-workflow in the defaults. A spot.yaml with workflows of its own sets it to one of them. SPOT writes it as the last key of the file, after the routing sections.

See Workflow YAML Schema below, and Plugin settings above.

Workflow YAML Schema

Workflows define which analyzers look at an email, in which order, and how their answers become the email's verdict. They live in the workflows section of spot.yaml, which the API and the dashboard edit. The models are the SDK's (spot_sdk.workflow). Workflows explains the ideas.

Workflow Structure

workflows:
  - id: "workflow-id"           # Required: unique identifier
    name: "Human Readable Name" # Required: display name
    version: 1                  # Integer, at least 1
    description: ""             # Optional
    stages: [...]               # Required: at least one stage
    timeout_ms: 300000          # The whole run, 1000 to 600000 (default 5 min)
    verdict_policy: {...}       # How the analyzers' answers become the verdict
    on_verdict: {}              # Responders to call, by verdict
    created_by: "system"        # Creator identifier

The analyzer-orchestrator bounds the whole run by timeout_ms: an analyzer still running then, or never reached, is recorded as timeout, and the verdict is decided on what finished.

Stage Configuration

stages:
  - name: "stage-name"          # Required: unique within the workflow
    type: "parallel"            # parallel | sequential | conditional (default: sequential)
    depends_on: []              # Stages that run before this one
    condition: null             # Run the stage only when this holds
    retrieval_limits: null      # Caps on Knowledge Store queries (see below)
    analyzers: [...]            # List of analyzer configs

Stages run in the order their depends_on give; a cycle or an unknown stage fails the job.

Stage Types: - parallel - Run all analyzers at once - sequential - Run the analyzers one after another - conditional - Run the analyzers one after another, each only when its own condition holds; the others are recorded as skipped

A workflow with a stage of any other type is refused, naming the stage.

A stage whose condition does not hold is skipped: its analyzers are recorded as skipped, and a condition on that stage is false.

retrieval_limits caps what the stage's analyzers ask of the Knowledge Store: max_top_k (1 to 1000) and min_score_floor (0 to 1). They only shrink a query, never widen it; the orchestrator sends them in each AnalyzeRequest and the SDK's KnowledgeClient.for_analysis applies them.

Conditions

A stage's condition, or an analyzer's in a conditional stage, compares the outcome of an earlier stage: the workflow's policy applied to that stage's analyzers alone.

  • <stage>.score <operator> <number>: the stage's score, 0 to 100, with >, >=, <, <=, == or !=. It is false when the stage has no score, its verdict being unknown.
  • <stage>.verdict == <verdict> or !=: one of malicious, suspicious, spam, benign, unknown.
condition: "triage.score >= 50"
condition: "triage.verdict == suspicious"

A condition must parse and name a stage of the workflow, or the workflow is refused. A condition on a stage that was skipped is false; one on a stage that has not run yet fails the job.

Analyzer Configuration (in Workflow)

Each analyzer within a stage:

analyzers:
  - id: "analyzer-nlp"          # Required: its id under plugins.analyzers
    weight: 1.0                 # 0 to 10; 0 only observes
    timeout_ms: 30000           # 1000 to 300000; the workflow's timeout may cut it short
    required: false             # If true and unknown, the email is unknown
    failure_strategy: "retry"   # retry | skip | fail (default: retry)
    retry_config:               # Used by retry
      max_attempts: 3
      backoff_ms: 1000
      max_backoff_ms: 10000
      exponential_backoff: true
    condition: null             # In a conditional stage: when to run this analyzer

The orchestrator sends each analyzer an AnalyzeRequest with its own token: the job id, the email's view, the outcomes of the stages that ran before (previous, by stage name), the stage's retrieval_limits and the time left (deadline_ms). The answer must be an AnalyzerVerdict.

An analyzer that does not answer one (a timeout, an HTTP error, an answer that is invalid, larger than 1 MiB or encoded, no token, no URL) is handled by its failure strategy:

  • retry - Try again after the backoff, up to max_attempts, then count the analyzer as unknown
  • skip - Count it as unknown at once
  • fail - Fail the job: its verdict is unknown, decided by failed

An analyzer that answers unknown itself is unknown too, whatever its strategy. An analyzer of weight 0 only observes: it runs, and its decisive signals count, nothing else does.

Verdict Policy

verdict_policy:
  method: weighted_mean         # most_confident | weighted_mean | mean | max | majority
  malicious_threshold: 80       # 1 to 100
  suspicious_threshold: 50      # 1 to 100, at most malicious_threshold
  decisive_signals:
    - signal: "misp.*"          # a signal name, or a prefix ending in ".*"
      verdict: malicious
  decision_stages: [decision]   # the stages whose analyzers count; unset: every stage

The decision, in order:

  1. The sender lists. A decisive_signals rule that matches sender_list.blocked or sender_list.allowed, the signals core adds to an email whose sender is listed (Sender lists), sets the verdict before any analyzer's: score 100 when the domain is blocked, 0 when it is allowed; decided_by is signal:<name>. The blocked signal is read first. Without such a rule the lists' signals are shown and never counted.
  2. Decisive signals. The first signal that a decisive_signals rule matches, emitted by any analyzer that ran in any stage, sets the verdict. An analyzer's signal named sender_list.blocked or sender_list.allowed is shown, never decisive: only core's lists give those. Signals are read in workflow order: the stages in their order, each stage's analyzers as listed. The score is the highest score of the analyzers that emitted it; decided_by is signal:<name>.
  3. The analyzers that count are those of decision_stages, less the unknown ones and those of weight 0. A required analyzer of those stages that is unknown (a timeout included, not one skipped by a condition) makes the verdict unknown (required_unknown). With no analyzer left, it is unknown too (all_unknown). An unknown verdict has score 0.
  4. The method decides the rest; decided_by is method:<name>:
  5. most_confident: the analyzer with the highest confidence gives its own verdict and score; ties go to the higher weight, then the higher score.
  6. weighted_mean, mean, max: the score is the mean weighted by weight, the plain mean, or the highest score. It is malicious at or above malicious_threshold and suspicious at or above suspicious_threshold. Below, the verdict is spam when at least half of the counting weight says spam, and benign otherwise.
  7. majority: the verdict with the most weight wins; ties go to the more severe (malicious, suspicious, spam, benign). The score is the weighted mean of the scores of the analyzers that chose it.

Means are rounded half up.

The decision stages must hold an analyzer of weight above 0, or the workflow is refused. A new workflow's policy defaults to weighted_mean over every stage.

Responders of a workflow

on_verdict maps a verdict to the responders to call. Whether they act is the job's mode, its connection's and its route's: see Observe and enforce and Responders.

Workflows written in contract 1

The workflow API (create, update, validate) refuses a contract 1 workflow with 400, naming where its contract 1 field is. Workflows stored in contract 1, in spot.yaml or a workflows file, are converted when read, each change logged, and the api-gateway writes them back once when spot.yaml is writable:

  • final_stage_name becomes verdict_policy: {method: most_confident, decision_stages: [<that stage>]}, which keeps contract 1's verdicts; the last stage when none was named. A workflow that already has a verdict_policy keeps it;
  • <stage>.weighted_confidence > X becomes <stage>.score > 100X. A score measures how malicious the email is, not how confident the analyzers were: check every converted condition;
  • failure_strategy: fallback becomes skip;
  • a stage's aggregation_method, continue_on_failure and min_successful_analyzers are dropped.

A workflow of either contract that still holds continue_on_failure, min_successful_analyzers or max_parallel_analyzers, which SDK 2 removed, loses them the same way, each removal logged. The workflow API accepts a body with them and ignores them.

Complete Workflow Example

The default workflow that ships in config/defaults/spot.yaml:

workflows:
  - id: "default-workflow"
    name: "Default Phishing Detection Workflow"
    version: 1
    description: "Standard workflow: parallel analysis (NLP, LLM) followed by decision"
    stages:
      - name: "parallel-analysis"
        type: "parallel"
        depends_on: []
        analyzers:
          - id: "analyzer-nlp"
            weight: 0.5
            timeout_ms: 30000
            required: false
            failure_strategy: "skip"
            retry_config:
              max_attempts: 2
              backoff_ms: 1000
              max_backoff_ms: 5000
              exponential_backoff: false
          - id: "analyzer-llm"
            weight: 0.5
            timeout_ms: 45000
            required: false
            failure_strategy: "skip"
            retry_config:
              max_attempts: 2
              backoff_ms: 2000
              max_backoff_ms: 10000
              exponential_backoff: true

      - name: "decision"
        type: "sequential"
        depends_on: ["parallel-analysis"]
        analyzers:
          - id: "analyzer-llm"
            weight: 1.0
            timeout_ms: 60000
            required: true
            failure_strategy: "retry"
            retry_config:
              max_attempts: 3
              backoff_ms: 2000
              max_backoff_ms: 15000
              exponential_backoff: true

    timeout_ms: 300000
    verdict_policy:
      method: "most_confident"
      decision_stages: ["decision"]
    created_by: "system"

The three workflows of the defaults (default-workflow, nlp-only-workflow and fast-workflow) hold analyzer-rules in their first stage as an observer: weight 0, not required, failure strategy skip. Only its decisive signals count, so nothing in its score moves a verdict, and where the plugin is not installed it is unknown and the workflow decides without it. Every shipped workflow, those three and the three of spot-cli setup, sets suspicious on the signal attachment.bidi_override, an attachment name that hides its real extension behind a bidi control (invoice<U+202E>fdp.exe shows as invoiceexe.pdf). The example above omits the observer and the signal for brevity.

A workflow that weighs every analyzer, overrides on a MISP hit, and sends malicious emails to a responder:

workflows:
  - id: "weighted-workflow"
    name: "Weighted analysis"
    stages:
      - name: "analysis"
        type: "parallel"
        analyzers:
          - id: "analyzer-rules"
            weight: 1.0
            failure_strategy: "skip"
          - id: "analyzer-nlp"
            weight: 2.0
            failure_strategy: "skip"
          - id: "analyzer-misp"
            weight: 0             # observes: only its decisive signals count
            failure_strategy: "skip"
    verdict_policy:
      method: "weighted_mean"
      malicious_threshold: 80
      suspicious_threshold: 50
      decisive_signals:
        - signal: "misp.ioc_match"
          verdict: "malicious"
    on_verdict:
      malicious: ["responder-ticket"]

Retention (spot.yaml)

A received email is kept for data_retention_days, and an entry of the audit log for audit_retention_days, platform settings of config/spot.yaml:

platform:
  data_retention_days: 365    # 1 to 36500
  audit_retention_days: 365   # 30 to 3650
Setting Default Description
platform.data_retention_days 365 Days a received email is kept, with its raw copy and analyses. DATA_RETENTION_DAYS in the environment overrides it
platform.audit_retention_days 365 Days an entry of the audit log is kept. AUDIT_RETENTION_DAYS in the environment overrides it

Each email expires data_retention_days after it is stored; an email stored before its expiry was recorded (migration 002) expires data_retention_days after it was created. The api-gateway purges the expired emails when it starts and then once a day, a thousand at a time, each batch a transaction of its own; the database deletes their raw copies, analyses and signals with them. A purge that fails is logged and tried again an hour later. The gateway reads the setting for each message it stores and for each purge: a change (PUT /api/v1/config/platform, or an edit of spot.yaml and a config reload) applies to the emails stored afterwards, and every email keeps the expiry it was stored with. The analysis jobs' copies in Redis, which hold no content, expire on their own within 24 hours.

A dataset's copy of an email is exempt: the purge never deletes it until it is removed from its dataset or the dataset is deleted (Retention).

The same purge then deletes the audit log's entries older than audit_retention_days, read at each purge: a change applies at the next one. A failure of either purge is logged, and both are tried again an hour later.

Routing (spot.yaml)

Routes choose the workflow each email runs; the shadow runs candidate workflows beside production, to compare. Three sections of spot.yaml hold them, which GET /api/v1/routing (analysts and admins: the groups list staff addresses) answers and PUT /api/v1/routing (admins) replaces together, as spot-cli routing show and spot-cli routing apply <file> do:

recipient_groups:
  finance: ["cfo@example.com", "@finance.example.com"]

routing:
  - name: finance
    match: {recipient_groups: [finance]}
    workflow: strict-workflow
  - name: milter
    match: {sources: [retriever-milter]}
    workflow: fast-workflow
    timeout_ms: 15000
  - name: vendors
    match: {sender_domains: ["vendor.example", "*.vendor.example"]}
    workflow: default-workflow

shadow:
  - workflow: candidate-workflow
    match: {sources: [retriever-milter]}
Field Description
routing[].name Unique among the routes: lower-case letters, digits, _ and -, starting with a letter or a digit, at most 64 characters
routing[].match What an email must be for the route to take it; empty, every email
routing[].workflow The workflow the email runs
routing[].timeout_ms 1 000 to 300 000: how long a retriever that waits for the verdict waits at most
match.sources Retriever ids, or api for the submissions
match.recipient_groups Names of recipient_groups: a recipient of the email is in one
match.sender_domains example.com, or *.example.com for its subdomains alone: the From domain, when a trusted DMARC result passed for it
recipient_groups.<name> The group's entries: addresses (local@domain) or whole domains (@domain, not their subdomains)
shadow[].workflow A workflow run in the shadow; one entry per workflow
shadow[].match Which emails it runs on, as a route's

How an email is routed. The first route whose match holds chooses the workflow. Within a match, every criterion given must hold, and within a criterion any value. An email no route takes runs the default workflow (default_workflow_id). A workflow a user names, for a submission or a re-analysis, is not routed, and a dry run is never routed. A re-analysis that names no workflow is routed on the stored email, by today's routes. The analysis records its route (route in GET /api/v1/analyze/{job_id} and the job list; none for a named workflow, the default, or an analysis from before routing). A route's timeout_ms caps a waiting retriever's wait: the retriever's own, or the route's when that is shorter.

  • Recipients are the SMTP envelope's when the retriever sent one, else the To and Cc addresses; a route matches when any recipient is in a group. They are compared lower-case, their domains in their Unicode form, as the parser gives the From domain: an A-label (xn--...) is decoded. Without an envelope (a submission, a retriever that sends none), the recipients are the headers the sender wrote, and as any recipient matches and the first route wins, an address the sender adds can steer a message to an earlier route. Put the stricter routes first, and do not route a group to a laxer workflow for a source that sends no envelope.
  • Sender domains count only when a DMARC result of one of mail.trusted_authserv_ids passed for the From domain: a forged or untrusted result matches nothing. While mail.trusted_authserv_ids is empty, a route or shadow entry with sender_domains never matches: saving one through the API is refused (422), and one written by hand loads, each load warning of it once (route <name> matches sender_domains while mail.trusted_authserv_ids is empty: it never matches).
  • Domains, in group entries and sender domains, are written as mail carries them: letters, digits and hyphens, in their usual form (UTS 46, STD3 rules). _, full-width forms or a character UTS 46 ignores are refused, as no email could match them. They are stored lower-cased, their A-labels decoded.
  • A route or shadow entry to a workflow that is not loaded (an edit by hand, a workflow left out at load) is skipped: the email goes on to the next route, or the default workflow. Each load says so once: route <name> names workflow <id>, which is not loaded: it is skipped. A routing that names a workflow that is not loaded is refused when saved through the API (PUT /api/v1/routing, spot-cli routing apply: 422, naming the route), while a hand edit loads it. A workflow a route or the shadow names cannot be deleted (409): change the routing first.

A routing that breaks a rule (a route named twice, an unknown group, a malformed entry) stops the api-gateway's start and fails a config reload (400), as an invalid mail: section does; POST /api/v1/config/validate reports it. SPOT writes the three sections on every save, empty ones included, before default_workflow_id.

Retrievers that named their workflow. Up to 2.0.0-beta.7, a retriever named its workflow (its WORKFLOW_ID setting); a retriever's ingest no longer names one, and the API refuses the setting in a mail retriever's settings (422: the plugin and config routes, an install, an update). At each start, the api-gateway turns each retriever's WORKFLOW_ID into a route {sources: [<id>]} to that workflow, appended to the routes, and removes the setting; the start's commit of spot.yaml's history records it, and the log names each route. A setting written by hand after a start routes nothing until the next one.

  • The route is named after the retriever: its id made a route name (lower-cased, every run of other characters made a -, leading - and _ dropped, at most 64 characters), or the next free name (<name>-2, <name>-3, ...). IMAP.corp becomes imap-corp.
  • A setting that names a workflow that is not loaded is removed with a warning: that retriever's mail runs the routes or the default workflow.
  • When an earlier route takes all of a retriever's mail (no recipient groups or sender domains, and no sources or that retriever among them), the converted route never applies, and the log says which route takes its mail: move the routes into the order you want.
  • Rolling spot.yaml back to a commit made before the conversion restores that commit's routing, which has no such routes, and keeps today's plugins, which have no WORKFLOW_ID left: the retrievers' mail then runs the default workflow until the routes are added again.

Shadow

Each result of a production job, except a failed one, is run again through each shadow workflow whose match holds for its email, production's own workflow aside, on the stored view, as a run of its own: analyzers are not told. Where its verdict class differs from production's, the difference is stored, with the shadow workflow's result encrypted, and listed newest first at GET /api/v1/workflows/{workflow_id}/shadow (viewers: verdicts and scores only). A redelivered result stores nothing twice.

Every run that completes is also counted for its day (UTC), differing or not, for the rate of the differences the same route gives over the last days days (30 by default, up to 90), counted as Workflows says.

The load. A shadow entry adds one workflow run per matched email. The analyzer-orchestrator runs them one at a time (prefetch 1), after the email's verdict: a shadow run never delays its own email's verdict, nor drops production's work. But its analyzer calls compete with production's, so on a saturated analyzer it can slow other emails' analyses, the next waiting retriever's included. Shadow the routes whose volume the analyzers can carry twice. When the shadow cannot keep up, its queue drops its oldest work past 1 000 results (see RabbitMQ).

Sender lists (spot.yaml)

The domains analysts block or allow, from an email's label in the dashboard or through /api/v1/sender-lists (API). Each change is a commit of spot.yaml's history by the analyst. What a list does is a workflow's policy to decide (Sender lists).

sender_lists:
  block: ["evil.example", "*.evil.example"]
  allow: ["vendor.example"]
Field Description
sender_lists.block Domains whose mail gets the signal sender_list.blocked: the From domain, authenticated or not
sender_lists.allow Domains whose mail gets the signal sender_list.allowed: the From domain, when a DMARC result of mail.trusted_authserv_ids passed for it

Domains are written as a route's sender_domains are, example.com, or *.example.com for its subdomains, and stored the same way: lower-cased, their A-labels decoded. A domain is on one list at most. The allow list needs mail.trusted_authserv_ids, so a spoofed From never reaches it: while none is trusted, adding a domain to it through the API is refused (422), and one written by hand loads with a warning, as it never matches. An invalid section (a malformed domain, a domain on both lists) stops the api-gateway's start and fails a config reload; the analyzer-orchestrator loads it empty, with a warning that names the rule and its position, never the domain. SPOT writes the section on every save, empty lists included.

Mail and Intake (spot.yaml)

The api-gateway parses each received message once, in worker processes, and stores the raw message and its view, both encrypted. A plugin that fetches one decoded part of a stored message (GET /internal/emails/{email_id}/parts/{part_id}) has it parsed again, in the same workers and within the same limits. Two spot.yaml sections tune it:

mail:
  # authserv-ids of our own MTAs: only their Authentication-Results headers
  # count. The MTAs must remove the incoming headers that carry these ids
  # (RFC 8601, section 5), or a sender could forge one.
  trusted_authserv_ids: ["mx.example.com"]

intake:
  parse_timeout_s: 10      # a parse that runs longer is stopped
  parse_memory_mib: 640    # address space of one parse worker
  parse_workers: 2         # parses that run at once
  max_view_mib: 8          # defence in depth: a view past it is refused
  max_in_flight: null      # messages taken at once (null: 2 x parse_workers)
  max_content_in_flight: 2 # raw messages, parts and views served at once
  body_timeout_s: 60       # time a request body has to arrive in full
Setting Default Description
mail.trusted_authserv_ids [] Authentication results from any other authserv-id are ignored
intake.parse_timeout_s 10 Time limit of one parse, and of a worker's start; a 25 MiB ordinary message takes about 1 s
intake.parse_memory_mib 640 Address-space limit of a worker; a 25 MiB ordinary message needs about 300 MiB
intake.parse_workers 2 Worker processes, so one slow message stalls no other
intake.max_view_mib 8 A view's JSON may not be larger. Defence in depth: spot_sdk.mime.parse bounds every view to about 6.84 MiB, so none reaches this limit (an ordinary 25 MiB message gives at most about 3 MiB)
intake.max_in_flight 2 x parse_workers Messages the gateway takes at once, each from reading the body to storing the email
intake.max_content_in_flight 2 Raw messages, parts and views the gateway serves at once (GET /internal/emails/{email_id}/raw, .../parts/{part_id}, GET /api/v1/emails/{email_id}, /raw, /view and /forensics), each until its response is sent
intake.body_timeout_s 60 A body still arriving after this is answered 408, so slow uploads cannot hold every slot

A message larger than 25 MiB is refused with 413. An empty one, or one whose parse runs past a limit, is refused with 422 (parse_limit, naming the time, memory or view limit). Nothing is stored then. A part fetch past the time or memory limit is refused the same way. Past max_in_flight messages taken at once, past max_content_in_flight raw messages, parts and views served at once, or when no parse worker can take one, the gateway answers 503 with Retry-After: retry later.

A raw message, a part or a view served holds its place until its response is sent, so a client that stops reading holds one until its connection closes. That delays other content reads, never intake, which has a bound of its own. A user's read also holds its authentication's database connection until then: the bound keeps those connections within max_content_in_flight.

The gateway reads the intake section when it starts: restart it after a change. It reads mail.trusted_authserv_ids for each message, so a configuration reload is enough.

Memory. One message taken costs the gateway about 256 MiB at its peak (a 25 MiB message, its base64 JSON body, its view and their encryption), and each worker may use up to parse_memory_mib. A raw message, a part or a view served costs at most about 190 MiB (measured for 25 MiB messages: a raw read 175 to 191 MiB, a part fetch 158 to 175 MiB, a view of 7 MiB 83 MiB; decrypting dominates), plus, for a slow reader, up to the body's size in the server's send buffer: count 256 MiB. Size the api-gateway's memory limit as:

512M + max_in_flight x 256 MiB + max_content_in_flight x 256 MiB
     + parse_workers x parse_memory_mib

With the defaults: 512 + 4 x 256 + 2 x 256 + 2 x 640 = 3328 MiB.

Config Reload

Configuration can be reloaded without restart:

# Reload config via API
curl -X POST http://localhost:8001/api/v1/config/reload \
  -H "Authorization: Bearer $TOKEN"

# Response shows what changed
{
  "old_version": "abc123@20251204",
  "new_version": "def456@20251204",
  "changed": {
    "platform": false,
    "workflows": true,
    "analyzers": ["analyzer-nlp"]
  }
}

Reload behavior: - Invalid YAML/schema returns 400, previous config preserved - Version only bumps if content actually changed - Reloads run one at a time, and wait for the start's sealing step when it is still running

Editing spot.yaml by hand

Change the configuration through the API, the dashboard or spot-cli when you can. To edit spot.yaml by hand, replace it whole, in one step, then reload:

# Write the new file under a temporary name in the volume, then rename it
# into place: no service ever reads it half copied
docker compose exec -T api-gateway sh -c \
  'cat > /app/config/.spot.yaml.new && mv /app/config/.spot.yaml.new /app/config/spot.yaml' \
  < spot.yaml
curl -X POST http://localhost:8001/api/v1/config/reload -H "Authorization: Bearer $TOKEN"

A spot.yaml that does not parse or validate stops the api-gateway from starting (it never runs on the shipped defaults in its place), so docker compose exec finds no container to enter. Replace the file from a one-off container of the same image, user and volume, then start the gateway; its start records the file:

docker compose run --rm --no-deps -T api-gateway sh -c \
  'cat > /app/config/.spot.yaml.new && mv /app/config/.spot.yaml.new /app/config/spot.yaml' \
  < spot.yaml
docker compose up -d api-gateway
  • Do not write spot.yaml while a start or a reload runs. For the few seconds the sealing takes, a write is recorded in the load's commit, a plain secret it holds included.
  • Raise the version of a workflow you change past the highest it has had (spot-cli workflow history <id>). SPOT numbers each save past it, so a number names one version; an edit that keeps the number gives it a second definition.
  • The gateway rewrites the file in its own form at each start and reload: comments, unknown keys and misspelt keys do not survive. spot-cli config validate does not report an unknown key either, so check each key against this reference.
  • The environment sets no section of spot.yaml. It sets the infrastructure settings and the platform settings (Configuration Precedence); plugins, workflows, default_workflow_id, mail, intake, recipient_groups, routing and shadow come from the file alone, and a variable of the same name (MAIL, which login shells set) is ignored.

History of spot.yaml

Every change of spot.yaml is a commit of history.git, a git repository beside it: /app/config/history.git on the spot-config volume (config/history.git in development). Each commit holds the whole file. Its author is the user who made the change, or SPOT for its own, and its message says what the change did, which sections and workflows it touched, and the version of each of those workflows the file holds. The api-gateway writes it itself; the images hold no git binary, but git --git-dir history.git log reads it anywhere.

  • Changes made outside SPOT (an edit, a copy) are recorded at the gateway's next start, by SPOT, or at its next POST /api/v1/config/reload, by the reload's user. Either first seals the plain secret settings of catalog plugins, so no commit holds a plain value the catalog marks secret, as long as the catalog can be read then (Redis up): otherwise the settings are recorded as they are. Either then writes the file back in SPOT's form (Editing spot.yaml by hand).
  • A change saved during a load (the few seconds the sealing takes, longer when a plugin it seals is being installed or updated) is written, and recorded in the load's commit, under the load's author. The log keeps its own author and message: spot.yaml saved by <author> while its history is paused: <message>.
  • The history is never pruned, unlike the emails (data_retention_days). It keeps every value ever saved, each author's email and the recipient groups' staff addresses included: an address removed from a group, such as a departing employee's, stays in every earlier commit. Keep a manual plugin's secret settings in its sealed secrets (PUT /api/v1/config/plugin/{kind}/{id} with secrets), never in settings: without a catalog entry, SPOT cannot tell that they are secret, and records them as they are.
  • Backups. history.git is on the spot-config volume with spot.yaml: the deploy wrapper's spot-cli backup takes both (see the Admin guide).

Reading and rolling back. Admins read the history through the API (API). A commit's detail shows what it changed, each sealed secret reading (sealed), or (sealed, changed), and previews what a rollback to it would change in the settings in force, as SPOT writes them: a hand edit not yet reloaded shows in neither the preview nor the rollback's sections, and a rollback replaces it. A rollback (POST /api/v1/config/history/{commit}/rollback) restores every section of that commit but plugins:, as a new commit by the admin:

  • plugins: stays as SPOT holds it: a rollback never touches a plugin's entry, container, token or settings. A hand edit of spot.yaml not yet reloaded is replaced, as by any change saved through SPOT. A restored workflow that names a plugin no longer installed loads, and the readiness shows it broken.
  • A platform setting the environment sets (compose sets LOG_LEVEL, DEBUG and ENABLE_RATE_LIMITING) is written as the commit's file had it, and the environment's value still applies.
  • The analyzer-orchestrator runs the restored workflows from its next job. The platform settings apply as after a reload: those read only at start need a restart of the api-gateway, for example log_level, debug, rate limiting, allowed_hosts, allowed_origins, enable_oauth, the token lifetimes and the intake section.
  • A commit whose spot.yaml no longer validates (a rule added since) is refused with 422, naming each field and the rule it breaks; nothing is written.

When a change cannot be recorded. A change that is written but cannot be committed (a full volume, a damaged history.git) stays written: the error log names the change and the cause (spot.yaml was saved but not recorded in its history: ...), and the next commit records it. A start that cannot record the file starts anyway.

  • A lock left behind. A crash during a commit can leave history.git/refs/heads/main.lock. Every commit then fails, the log naming it (HistoryConflictError: the history's branch is locked by .../main.lock). When no gateway is saving, remove the lock: docker compose exec api-gateway rm /app/config/history.git/refs/heads/main.lock.
  • A damaged history. When an object or a ref of history.git is missing or damaged (a copy taken while a commit was written, say), the history routes, a workflow's versions and the workflow saves that number a version answer 503, naming it. Move history.git aside, as below.

When commits keep failing for another reason, move history.git aside: the next change starts a new history, whose first commit holds the whole file, and git still reads the old one. The new history knows only the workflows in force: a workflow deleted before the move is numbered from 1 again when it is created again.

docker compose exec api-gateway mv /app/config/history.git /app/config/history.git.old

Development Configuration

When APP_ENV=dev, additional variables are available:

Source Code Mounting

Variable Description
API_GATEWAY_SRC_MOUNT Path to API Gateway source
API_GATEWAY_TEST_MOUNT Path to API Gateway tests
ANALYZER_ORCHESTRATOR_SRC_MOUNT Path to Analyzer Orchestrator source
ANALYZER_ORCHESTRATOR_TEST_MOUNT Path to Analyzer Orchestrator tests
MAIL_ORCHESTRATOR_SRC_MOUNT Path to Mail Orchestrator source
MAIL_ORCHESTRATOR_TEST_MOUNT Path to Mail Orchestrator tests
SHARED_MOUNT Path to shared modules
CONFIG_MOUNT Path to config directory
MOUNT_MODE Mount mode (rw or ro)

Debug Ports

Variable Default Description
ANALYZER_ORCHESTRATOR_DEBUG_PORT (a free port) Debug port for analyzer orchestrator
KNOWLEDGE_DEBUG_PORT (a free port) Debug port for the knowledge service
MAIL_ORCHESTRATOR_DEBUG_PORT 8092 Debug port for mail orchestrator

The debug ports listen on 127.0.0.1 only. Left empty, Docker picks a free host port.

Development Tools

Variable Default Description
MAILHOG_SMTP_PORT 1025 Mailhog SMTP port
MAILHOG_WEB_PORT 8025 Mailhog web UI port
ADMINER_PORT 8080 Adminer database UI port
HOST_UID 1000 Host user ID for devtools container
HOST_GID 1000 Host group ID for devtools container

Production Configuration

When APP_ENV=prod:

Docker Registry

Variable Default Description
REGISTRY_PORT 5000 Docker registry port
CI_REGISTRY_IMAGE (CI only) Full registry path for platform services (e.g., localhost:5000/spot/platform)
CI_REGISTRY (CI only) Registry host for external analyzers (e.g., localhost:5000)
VERSION latest Image version tag
BASE_IMAGE base:latest Base image name and tag

Note: CI_REGISTRY_IMAGE and CI_REGISTRY are NOT set for local development. They are only set in .gitlab-ci-local-env for CI context.

CI/CD Configuration

GitLab-specific variables (only needed for CI/CD):

Variable Description
GITLAB_HOST GitLab hostname
GITLAB_TOKEN GitLab access token (uses CI_JOB_TOKEN if available)
CI_REGISTRY Container registry URL
GITLAB_GROUP GitLab group name

Configuration Precedence

Configuration is loaded in this order (highest to lowest priority):

  1. The environment's variables, command line included
  2. spot.yaml, over the shipped defaults (/app/defaults/spot.yaml)
  3. The .env file the settings read, in the service's working directory
  4. Code defaults (in Pydantic Settings classes)

In a compose deployment the project's .env reaches the services through the environment: compose passes its values to the containers as variables, so they rank first.

Example:

# .env file has: LOG_LEVEL=INFO
# Command-line override:
LOG_LEVEL=DEBUG make service:start    # Uses DEBUG

The environment sets the infrastructure settings and the platform settings only. A platform setting it sets (compose sets LOG_LEVEL, DEBUG and ENABLE_RATE_LIMITING) outranks spot.yaml, which keeps its own value: SPOT never writes the environment's value into the file. The sections of spot.yaml (plugins, workflows, default_workflow_id, mail, intake, recipient_groups, routing, shadow) are never read from the environment: set the default workflow with POST /api/v1/workflows/{id}/set-default, or in spot.yaml.

Configuration Format

Environment Variable Prefixes

SPOT uses standard environment variable names without a global prefix:

  • Infrastructure: POSTGRES_*, REDIS_*, RABBITMQ_*
  • Application: APP_ENV, LOG_LEVEL, SECRET_KEY
  • Plugins: analyzers, context providers, mail retrievers and responders are configured under plugins: in config/spot.yaml

Quick Reference Examples

Minimal Production .env

# Environment
APP_ENV=prod

# Security (REQUIRED - generate secure values)
SECRET_KEY=generate-with-python-secrets-module
SPOT_INTERNAL_API_KEY=generate-with-python-secrets-module
ENCRYPTION_KEY=generate-with-cryptography-fernet

# Database
POSTGRES_DB=spot
POSTGRES_USER=spot
POSTGRES_PASSWORD=secure_db_password

# Redis (optional password)
REDIS_PASSWORD=secure_redis_password

# RabbitMQ
RABBITMQ_DEFAULT_USER=spot
RABBITMQ_DEFAULT_PASS=secure_rabbitmq_password

# The host names the api-gateway answers
ALLOWED_HOSTS=localhost,127.0.0.1,api.spot.example.com

Minimal Development .env

# Environment
APP_ENV=dev

# Security
SECRET_KEY=dev-secret-key-for-testing-only
SPOT_INTERNAL_API_KEY=dev-internal-key-for-testing-only
ENCRYPTION_KEY=generate-with-cryptography-fernet

# Database (dev defaults)
POSTGRES_DB=spot
POSTGRES_USER=spot
POSTGRES_PASSWORD=spot123

# Redis (no password in dev)
REDIS_PASSWORD=

# RabbitMQ (dev defaults)
RABBITMQ_DEFAULT_USER=guest
RABBITMQ_DEFAULT_PASS=guest

Full Example with Analyzers

.env:

# Environment
APP_ENV=prod

# Security
SECRET_KEY=your-32-character-random-key-here
SPOT_INTERNAL_API_KEY=another-32-character-random-key
ENCRYPTION_KEY=your-fernet-key
ALLOWED_HOSTS=localhost,127.0.0.1,api.spot.example.com

# Infrastructure
POSTGRES_PASSWORD=secure_password
RABBITMQ_DEFAULT_PASS=secure_password

config/spot.yaml:

plugins:
  analyzers:
    analyzer-nlp:
      enabled: true
      url: "http://10.0.1.10:8000"
      settings: {}
    analyzer-llm:
      enabled: true
      url: "http://10.0.1.11:8000"
      settings: {}

Each analyzer needs a token (spot-cli plugin token <id>) before core calls it.

Validation

The platform validates configuration at startup:

  • Database URL format
  • RabbitMQ URL format
  • Analyzer URL formats
  • Port ranges (1-65535)

Invalid configuration will cause startup to fail with a descriptive error message.