Skip to content

Services

Detailed description of SPOT Platform services.

Core Services

API Gateway

FastAPI-based HTTP server providing the main API interface.

Port: 8001

Responsibilities:

  • Authentication and authorization (OAuth2/JWT)
  • Request validation using Pydantic
  • Job submission to RabbitMQ (spot.analysis exchange)
  • Job status queries via RabbitMQ RPC (spot.status exchange)
  • Configuration API (read-only)
  • Health checks

Key endpoints:

  • POST /api/v1/analyze - Submit email for analysis
  • GET /api/v1/analyze/{job_id} - Check job status
  • GET /api/v1/config - Get configuration
  • GET /health - Health check

Technology:

  • FastAPI 0.109+
  • Pydantic for validation
  • PostgreSQL for persistence
  • Redis for rate limiting
  • RabbitMQ for job queue

Analyzer Orchestrator

Workflow execution engine that coordinates analyzer services.

Responsibilities:

  • DAG-based workflow execution
  • Analyzer service coordination via HTTP
  • Result aggregation from multiple analyzers
  • Retry logic and failure handling
  • Threat level determination
  • Result storage

Workflow stages:

  1. Parallel analysis - Run multiple analyzers simultaneously
  2. Decision stage - Aggregate results and determine threat level

Technology:

  • Python asyncio
  • RabbitMQ for messaging
  • spot-sdk for data models
  • PostgreSQL for result storage

Mail Orchestrator

Email ingestion routing layer.

Responsibilities:

  • Routes to configured mail retrievers
  • Email preprocessing and normalization
  • Submits emails for analysis via API Gateway

Configuration:

  • SPOT_MAIL_RETRIEVERS - Comma-separated list of retriever URLs

Technology:

  • Python asyncio
  • HTTP client for retriever communication
  • REST API integration with API Gateway

Analyzer Services

Analyzers are external HTTP services that implement the spot-sdk analyzer interface.

Common interface:

  • POST /internal/analyze - Analyze email
  • GET /health - Health check

Configuration (in config/spot.yaml):

analyzers:
  analyzer-nlp:
    enabled: true
    url: "http://analyzer-nlp:8000"
    settings: {}

NLP Analyzer

Natural language processing for spear-phishing detection.

Location: spot-analyzer-nlp repository

Capabilities:

  • Sentiment analysis
  • Named entity recognition (NER)
  • Text classification
  • Urgency detection
  • Language pattern analysis

Technology:

  • DistilBERT models
  • spaCy for NLP
  • scikit-learn for classification

LLM Analyzer

Large language model for advanced spear-phishing detection.

Location: spot-analyzer-llm repository

Capabilities:

  • Context understanding
  • Intent analysis
  • Deception detection
  • Social engineering detection

Technology:

  • Ollama for LLM inference
  • Local model deployment
  • Prompt engineering

MISP Analyzer

Threat intelligence lookup using MISP.

Location: spot-analyzer-misp repository

Capabilities:

  • IOC matching
  • Threat intelligence correlation
  • Domain/URL reputation
  • Known threat patterns

Technology:

  • PyMISP client
  • MISP server integration

Infrastructure Services

PostgreSQL

Primary database for persistent storage.

Port: 5432

Tables:

  • emails - Email records
  • analysis_results - Analysis results

Technology:

  • PostgreSQL 15+

Redis

In-memory store for rate limiting.

Port: 6379

Usage:

  • Rate limiting counters

Technology:

  • Redis 7+
  • redis-py client

RabbitMQ

Message broker for asynchronous job processing.

Ports:

  • 5672 - AMQP protocol
  • 15672 - Management UI

Exchanges:

  • spot.analysis (TOPIC) - Job routing
  • spot.status (TOPIC) - Status queries
  • spot.results (TOPIC) - Result notifications

Queues:

  • orchestrator.analysis.requests - Job consumption (bound to spot.analysis)
  • job.status.requests - Status RPC (bound to spot.status)
  • Dynamic RPC reply queues

Routing keys:

  • email.analyze - New analysis jobs
  • job.status.requests - Status queries
  • analysis.complete - Completed results

Technology:

  • RabbitMQ 3.11+
  • aio-pika (Python async client)
  • Persistent queues

Development Services

Adminer

Database management UI (dev mode only).

Port: 8080

MailHog

Email testing server (dev mode only).

Port: 8025

Devtools

Development utilities container (dev mode only).

Service Communication

API Gateway -> Analyzer Orchestrator

Communication: RabbitMQ

Flow:

  1. API Gateway receives analysis request
  2. API Gateway publishes to spot.analysis exchange with email.analyze routing key
  3. Analyzer Orchestrator consumes from orchestrator.analysis.requests queue
  4. Orchestrator executes workflow
  5. Results stored in PostgreSQL

Analyzer Orchestrator -> Analyzers

Communication: HTTP REST API

Flow:

  1. Orchestrator sends POST /internal/analyze with email data
  2. Analyzer processes and returns results
  3. Orchestrator aggregates results from all analyzers

Status Queries

Communication: RabbitMQ RPC

Flow:

  1. API Gateway publishes to spot.status exchange
  2. Orchestrator consumes from job.status.requests queue
  3. Orchestrator checks JobManager (in-memory) or PostgreSQL
  4. Response sent via RPC reply queue

Scaling Considerations

Stateless Services

Can be scaled horizontally:

  • API Gateway
  • Analyzer services (NLP, LLM, MISP)

Stateful Services

Require coordination for scaling:

  • Analyzer Orchestrator (single instance recommended)
  • Mail Orchestrator (single instance per configuration)

Infrastructure

Standard scaling approaches:

  • PostgreSQL: Read replicas, connection pooling
  • Redis: Redis Cluster for high availability
  • RabbitMQ: Clustered deployment

Health Checks

All services provide health endpoints:

# API Gateway
curl http://localhost:8001/health

# PostgreSQL
pg_isready -h localhost -p 5432

# Redis
redis-cli ping

# RabbitMQ
rabbitmq-diagnostics -q ping