Email Threat Intelligence Engine

A FastAPI and SQLAlchemy service that ingests email content, evaluates configurable detection rules, extracts links, and exposes threat analytics. It is intended as a focused backend foundation for email-security workflows.

Capabilities

• Stores normalized senders, emails, detection rules, flags, and extracted links.
• Detects keyword, sender-domain, and URL-pattern matches.
• Assigns risk points by rule severity and records an explainable flag for every match.
• Identifies suspicious links, including IP-address hosts, deceptive user-info URLs, deeply nested hosts, and selected high-risk TLDs.
• Maintains a cumulative sender reputation score based on email risk, suspicious links, rule severity, and a repeat-offender threshold.
• Provides analytics for risky senders, frequently triggered rules, and suspicious links.

Technology

• Python 3.10+
• FastAPI
• SQLAlchemy 2.x ORM
• Pydantic 2
• SQLite by default; any SQLAlchemy-supported database can be selected with DATABASE_URL

Getting started

Create a virtual environment, install dependencies, and start the application from the repository root.

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
uvicorn backend.api:app --reload

The API is available at http://127.0.0.1:8000. Interactive documentation is available at /docs.

By default, the service creates data.db in the working directory. Set DATABASE_URL before starting the server to use another database connection, for example:

$env:DATABASE_URL = "sqlite:///./email_threats.db"
uvicorn backend.api:app --reload

Seed rules and sample data

The seed script safely skips rules whose names already exist.

python -m backend.scripts.seed_rules
python -m backend.scripts.import_emails_example

API reference

Method	Path	Purpose
POST	/emails/ingest	Store an email, extract and classify links, apply rules, and update sender reputation.
POST	/rules/create	Create a keyword, domain, or link rule.
GET	/emails/{email_id}	Retrieve an email with its sender, flags, and links.
GET	/analytics/top-senders	List senders ordered by aggregate email risk.
GET	/analytics/top-rules	List rules ordered by number of generated flags.
GET	/analytics/suspicious-links	List links classified as suspicious.

The analytics endpoints accept an optional limit query parameter. Limits are validated by the API.

Create a rule

POST /rules/create
{
  "name": "Credential verification",
  "description": "Potential credential phishing language.",
  "severity": "high",
  "pattern_type": "keyword",
  "pattern_value": "verify your account"
}

severity must be low, medium, or high. pattern_type must be keyword, domain, or link.

Ingest an email

POST /emails/ingest
{
  "sender_email": "alerts@example.com",
  "subject": "Action required",
  "body": "Verify your account at https://192.0.2.1/login"
}

The response includes the resulting risk score, flag records, extracted links, and normalized sender information.

Detection and scoring

Rules are evaluated case-insensitively:

• Keyword rules search the subject and body.
• Domain rules compare the normalized sender domain.
• Link rules search each extracted URL.

Rule risk points are 1 for low, 3 for medium, and 5 for high severity. Each match creates a flag only once per email during processing.

Sender reputation is cumulative. For each processed email, it receives 10% of that email's risk score, one point per suspicious link, and an additional severity contribution of 0.5, 1, or 2 per matched low, medium, or high rule. Senders with more than five distinct flagged emails receive an additional two points for each subsequent processed flagged email.

Project layout

backend/
  api.py              FastAPI routes and dependency injection
  database.py         Engine, session factory, and session dependency
  models.py           SQLAlchemy ORM models
  schemas.py          Pydantic request and response schemas
  crud.py              Persistence and analytics queries
  rules_engine.py     Detection, risk, link, and reputation logic
  scripts/            Seed and example import commands
  utils/              Domain, link-extraction, and text helpers
  docs/               Architecture and database documentation

See the database schema and the architecture guide for implementation details.

Operational notes

Base.metadata.create_all() creates missing tables at application startup. For a production deployment, manage schema changes through a migration tool such as Alembic, configure a production database and connection pool, and add authentication, authorization, rate limits, logging, and monitoring at the API boundary.