Context Providers: RAG Framework for SPOT¶

Design record of 2026-04-10. The design of the context providers (POST /internal/enrich). The Knowledge Store superseded the per-stage context it describes. Internal host names are replaced by placeholders. It lists no security finding.

Context¶

SPOT's analyzers operate in isolation -- they only see the raw email. Commercial email security platforms gain a significant edge by enriching analysis with organizational context (employee data, sender history, knowledge bases, threat intelligence). SPOT needs a generic, pluggable mechanism for context retrieval that:

Is not hardcoded -- each company implements their own providers
Is optional -- workflows work with or without context providers
Makes context available to any analyzer via analysis_context
Follows the same plugin pattern as analyzers (Docker container, HTTP API, installable)
Integrates in the API and web UI

Architecture Decisions¶

Context Providers as first-class concept -- separate from analyzers in spot.yaml, with their own section, API endpoints, and UI page
Parallel execution -- providers and analyzers run together within stages. Dedicated enrichment stages can be used when sequential order is needed
Data flows via analysis_context -- already implemented, just needs population from providers
Flexible dict contract -- EnrichmentResult with free-form data: dict
Framework only -- platform-side changes, no bundled provider (separate repo if needed)

Data Flow¶

Workflow Stage (parallel):
  ├── context_providers (all run in parallel)
  │   ├── provider-A -> EnrichmentResult { data: {...} }
  │   └── provider-B -> EnrichmentResult { data: {...} }
  │
  │   Results merged into analysis_context:
  │   { "provider-A": {...}, "provider-B": {...} }
  │
  └── analyzers (all run in parallel, receive analysis_context)
      ├── analyzer-nlp  (ignores context, doesn't need it)
      └── analyzer-ml   (ignores context, doesn't need it)

Next Stage (depends_on above):
  └── analyzer-llm (reads analysis_context, uses provider data in prompt)

Changes Overview¶

1. SDK: New models¶

File: sdk/sdk/python/spot_sdk/enrichment.py (new)

class EnrichmentResult(BaseModel):
    provider_id: str
    data: dict[str, Any]        # Free-form contextual data
    source: str = ""            # Where the data came from (e.g., "company-ldap")
    confidence: float = 1.0     # 0.0-1.0, how reliable is this data
    cached: bool = False        # Whether this was served from cache
    duration_ms: int = 0        # How long retrieval took

File: sdk/sdk/python/spot_sdk/workflow.py (modify)

Add context_providers list to WorkflowStage:

class ContextProviderConfig(BaseModel):
    id: str
    timeout_ms: int = 10000
    required: bool = False      # If True, stage fails if provider fails
    failure_strategy: FailureStrategy = FailureStrategy.SKIP

class WorkflowStage(BaseModel):
    # ... existing fields ...
    context_providers: list[ContextProviderConfig] = Field(default_factory=list)

File: sdk/sdk/python/spot_sdk/__init__.py (modify) Export EnrichmentResult, ContextProviderConfig.

2. Core: spot.yaml schema¶

File: core/config/spot.yaml (modify)

Add context_providers section (same structure as analyzers minus analyzer-specific fields):

context_providers:
  provider-example:
    enabled: true
    url: http://provider-example:8000
    settings:
      SOME_KEY: value
    # Optional install metadata (same as analyzers)
    image: registry/provider-example:1.0.0
    version: 1.0.0
    container_id: ...
    container_name: ...
    installed_at: ...

3. Core: Config loader¶

File: core/shared/config/loader.py (modify)

Load context_providers section from spot.yaml alongside analyzers. Same pattern -- the orchestrator needs provider URLs.

4. Core: Orchestrator engine¶

File: core/services/analyzer-orchestrator/src/engine/engine.py (modify)

In _execute_stage(), before running analyzers: 1. Identify context providers configured for this stage 2. Execute all providers in parallel (similar to parallel analyzer execution) 3. Merge their EnrichmentResult.data into a dict keyed by provider_id 4. Inject merged context into analysis_context on the email dict 5. Then execute analyzers as before (they now have enriched context)

File: core/services/analyzer-orchestrator/src/engine/context_executor.py (new)

Similar to executor.py but calls POST /internal/enrich and expects EnrichmentResult:

class ContextExecutor:
    async def execute(self, email: dict, config: ContextProviderConfig) -> EnrichmentResult:
        response = await self.http_client.post(
            f"{url}/internal/enrich", json=email, timeout=timeout
        )
        return EnrichmentResult(**response.json())

5. Core: API Gateway¶

File: core/services/api-gateway/src/api/context_providers.py (new)

CRUD endpoints mirroring the analyzer management pattern: - GET /api/v1/context-providers -- list all - GET /api/v1/context-providers/{id} -- get details - POST /api/v1/context-providers -- register new - PUT /api/v1/context-providers/{id} -- update - DELETE /api/v1/context-providers/{id} -- remove - POST /api/v1/context-providers/{id}/enable -- enable - POST /api/v1/context-providers/{id}/disable -- disable

These read/write to the context_providers section of spot.yaml (same Caddy pattern as analyzers).

6. Core: Orchestrator routes¶

File: core/services/analyzer-orchestrator/src/routes/health.py (modify)

Include context provider connectivity in health check.

7. Web Dashboard¶

File: web-dashboard/src/routes/context_providers.py (new)

List/detail/edit pages for context providers (same pattern as analyzers page).

File: web-dashboard/src/templates/context_providers/ (new) - list.html -- table of providers with status - detail.html -- provider details

File: web-dashboard/src/templates/components/nav.html (modify)

Add "Context Providers" nav item.

File: web-dashboard/src/static/js/workflow-editor.js (modify)

Add context providers section in stage cards (similar to analyzers but simpler -- just provider ID selection + timeout + required toggle).

File: web-dashboard/src/services/api_client.py (modify)

Add context provider API methods.

8. Documentation¶

Provider development guide explaining the contract: - Endpoint: POST /internal/enrich - Input: Email JSON - Output: EnrichmentResult JSON - Health: GET /health - Settings: GET /config, POST /admin/reload-config

Context Provider HTTP Contract¶

POST /internal/enrich
Content-Type: application/json

Input: Email object (same as analyzers receive)

Output:
{
  "provider_id": "employee-dir",
  "data": {
    "sender_known": true,
    "sender_department": "Finance",
    "sender_role": "CFO",
    ...
  },
  "source": "company-ldap",
  "confidence": 0.95,
  "cached": false,
  "duration_ms": 45
}

GET /health
Output: { "status": "healthy", "service": "provider-employee-dir", ... }

What Does NOT Change¶

Analyzer interface (/internal/analyze, AnalysisResult) -- unchanged
Existing analyzers -- no modifications needed (they can ignore analysis_context)
Existing workflows -- continue working (empty context_providers list by default)
analysis_context structure -- providers are merged into ctx["<stage-name>"]["providers"], same stage-scoped structure as analyzer results

Implementation Sequence¶

SDK models -- EnrichmentResult, ContextProviderConfig, update WorkflowStage
Config loader -- load context_providers from spot.yaml
Orchestrator -- ContextExecutor, wire into _execute_stage()
API Gateway -- context provider CRUD endpoints
Web Dashboard -- context providers page + workflow editor support
Tests -- unit tests for orchestrator, integration test with mock provider
Documentation -- provider development guide

Verification¶

Add a mock context provider to spot.yaml, verify orchestrator picks it up
Create a workflow with context_providers in a stage, submit an email
Verify analysis_context contains provider data in the analyzer's request
Verify existing workflows without providers still work
Verify web UI shows context providers list and allows workflow editing with providers
Verify API CRUD operations for context providers