Author: A. K. Holz (salamcast@gmail.com)
License: BSD 3-Clause
Database Engine: DuckDB
Primary Database Storage: databases/emails.duckdb
MailData is a high-performance local email intelligence, conversational threading, zero-trust security sanitization, semantic search, and archival platform. It manages large email corpuses by decomposing monolithic .mbox files into single-email archival trees, linking complex multi-party conversation threads, stripping repeated reply text, enforcing zero-trust HTML email rendering and anti-phishing link protection, running local AI multi-agents (Gemma 2 9B / Qwen 2.5 7B), and providing natural language vector similarity search.
- Key Architecture & Capabilities
- Zero-Trust Security & Privacy Armor
- Local AI Multi-Agent Intelligence System
- Database Structure (
databases/) - Quick Start & Installation
- Web Interface & Navigation
- CLI Reference Guide
- License
- Deconstructs monolithic
.mboxarchives into granular, individual single-message.mboxfiles. - Organized strictly by primary account identity and hierarchical date paths:
MailBoxData/ ├── salamcast_AT_gmail_com/ │ └── [year]/[month]/[day]/msg_[time]_[hash].mbox ├── newaeon_AT_mac_com/ │ └── [year]/[month]/[day]/msg_[time]_[hash].mbox └── binholz_AT_hotmail_com/ └── [year]/[month]/[day]/msg_[time]_[hash].mbox - Safe unlinking: original monolithic files are preserved until individual message extraction and database synchronization are committed.
- Analyzes RFC
Message-ID,In-Reply-To,References, and normalized subjects to correlate conversational exchanges into canonicalthread_idgroups. - Calculates chronological reply depth (
thread_depth), constructing complete parent-child conversation trees. - Provides interactive conversation timelines in the web UI with direct branch navigation between replies.
- Detects and strips reply headers (
On ... wrote:,-----Original Message-----, forwarded blocks, lines starting with>, and mobile signatures) intoclean_body. - Single-email views display the clean body by default, with an expandable collapsible toggle ("Show Quoted / Repeated Thread History") to view original raw context on demand.
- Natural language vector search using DuckDB's native
array_cosine_similarityon 768-dimensional embeddings. - Uses local embeddinggemma-2 via Ollama for offline privacy, with fallback to Google's Gemini API (
text-embedding-004).
- Live bidirectional IMAP synchronization with SSL and modern security:
- iCloud / .Mac / .Me (
imap.mail.me.com:993) via App-Specific Passwords. - Google Gmail (
imap.gmail.com:993) via App Passwords. - Hotmail / Outlook (
outlook.office365.com:993) via Modern Authentication (OAuth 2.0 / SASL XOAUTH2) RFC 8628 Device Flow with automatic offline token refresh.
- iCloud / .Mac / .Me (
- Granular folder selection (
INBOX,Junk,Archive, etc.) with UID tracking.
- Thread-safe sequential
JobQueueensuring non-blocking background task execution and preventing DuckDB write-lock contention. - Real-time progress monitoring via Server-Sent Events (SSE) with a slide-over drawer and navbar pulse badge.
- Live external IMAP mailbox browser directly in the web UI (
/server-browser). - Dual-row mailbox display distinguishing local on-disk storage (
MailBoxData/...) from live server mailboxes. - Checkbox-based selective and bulk archival into single-mbox files and DuckDB indexing.
- Immediate server expunge (
STORE \Deleted+EXPUNGE) upon successful archival commit to free up cloud storage immediately. - Reverse Order Sorting: 1-click toggle to list oldest emails first, prioritizing older messages for review and archival.
MailData implements a multi-layered defense-in-depth security model to safely inspect compromised emails (such as those discovered in dark web breach lists):
Incoming Email
│
┌─────────────────────────────┼─────────────────────────────┐
▼ ▼ ▼
[ Link Guard ] [ Image Privacy Shield ] [ Attachment Armor ]
maildata/security.py maildata/parser.py maildata/security.py
│ │ │
• Anchor text mismatch • Blocks tracking pixels • OCR extracts text from
• Punycode / Homoglyph by default (web beacons) image-only scam invoices
• Direct IP addresses • CSP: img-src 'self' data: • Polyglot payload strip
• Burner TLDs (.xyz, .top) • One-click "Load Remote (appended EXEs/ZIPs)
• Interstitial Warning Gate Images" UI toggle • EXIF metadata stripping
• SVG script decomposition
- Server-side AST decomposition completely strips
<script>,<iframe>,<object>,<embed>,<applet>,<meta>refresh redirects,<base>tag hijacking, and interactive phishing<form>elements. - Decomposes all inline event handlers (
onload,onerror,onclick,onmouseover, etc.) and eliminatesjavascript:,vbscript:, and unauthorizeddata:URI schemes. - Sandboxed iframe viewing uses
sandbox="allow-popups"(omittingallow-same-origin), forcing the rendered email into an opaque null origin (origin: null) with zero access to application state, cookies, or parent DOM. - Enforces strict HTTP response headers:
Content-Security-Policy: default-src 'none'; script-src 'none'; script-src-elem 'none'; script-src-attr 'none'; style-src 'unsafe-inline' https:; img-src 'self' data: cid:; font-src https: data:; frame-ancestors 'self'; X-Content-Type-Options: nosniff X-Frame-Options: SAMEORIGIN
- Senders frequently embed 1x1 transparent remote images (
<img src="https://tracker.com/pixel.gif?id=...">) to track email open times, IP addresses, location, and user-agent headers. - Default Remote Image Blocking: MailData blocks all external HTTP/HTTPS images by default, replacing them with a discrete inline placeholder and enforcing
img-src 'self' data: cid:;in the browser CSP. - "Load Remote Images" Toggle: A Privacy Shield banner reports how many tracking images were blocked, allowing you to load remote images for a specific email on demand with one click.
- Safe Local Graphics: Inline MIME attachments (
cid:) and safedata:imagegraphics are displayed safely since they reside on local disk and make zero external network requests.
- Polyglot Detection & Stripping: Detects and strips trailing executable binaries or archives appended after image terminators (JPEG
\xFF\xD9EOI marker or PNGIENDchunk) via Pillow. - EXIF Metadata Stripping: Automatically removes GPS coordinates, camera serials, and potential buffer exploit strings embedded in EXIF tags.
- SVG Vector Sanitization: Parses
.svgattachments with BeautifulSoup to strip embedded<script>,<foreignObject>, and event handlers before previewing.
- Inspects hyperlinks across all messages and flags high-risk phishing vectors:
- Anchor Text Mismatch: Link claims to be
https://paypal.com/signin, but destination points tohttp://evil-phish.xyz. - Punycode / Homoglyph Spoofing: Internationalized domain encoding (
xn--...) mimicking legitimate Latin brand names. - Direct IP Addresses: Links pointing directly to numeric IPv4/IPv6 addresses (e.g.
http://185.220.101.5/login). - High-Risk Burner TLDs: Disposable extensions heavily abused by phishing kits (
.xyz,.top,.click,.loan,.work,.fit,.gq, etc.). - Open Redirectors: Cloaked redirectors (
bit.ly,tinyurl.com, search engine redirect parameters).
- Anchor Text Mismatch: Link claims to be
- External links in emails route through the Link Gate, which displays an interstitial warning screen explaining specific risk findings before allowing navigation. Suspicious links are also visually highlighted with a wavy red underline in the email viewer.
MailData features three autonomous local AI agents operating against your local Ollama models (Gemma 2 9B or Qwen 2.5 7B). They can run individually, grouped in an offline pipeline, or as a continuous background daemon.
- Scans multi-turn conversation chains across your
.mboxarchives. - Synthesizes complex multi-party discussions into concise 2–3 sentence summaries.
- Extracts Key Takeaways (outcomes, decisions, status updates) and Action Items (pending tasks, responsible parties).
- Classifies conversation tone (
collaborative,formal,contentious,informational,urgent). - Stores results in the
thread_summariesDuckDB table, displayed directly on conversation timelines and the Thread Insights Dashboard (/intelligence/threads).
- Automatically classifies emails and groupings with strict rules:
workvs.legal(Strictly Separated):legal: Immigration legal counsel (e.g. Mahdilaw), attorneys, court notices, retainer agreements, affidavits, contracts, and NDAs.work: General employment, client projects, corporate scheduling, and AV production (GES, PSAV).
bankingvs.finance(Strictly Separated):banking: Retail bank accounts, debit cards, ATM alerts, checking/savings statements (TD, RBC, BMO, CIBC, Scotiabank, Chase).finance: Cryptocurrency/Bitcoin transactions, tax returns (CRA/IRS), accounting audits, stock portfolios, and wealth management.
- Additional Categories:
personal,receipts,newsletters,marketing_sales,subscriptions,education,security, andscam_fraud.
- Deep security audit scanning
.mboxfiles for all threat categories:- Credential phishing, deceptive login links, fake account suspensions.
- Extortion, sextortion, webcam recording claims, and Bitcoin ransom demands.
- Fake Geek Squad, Norton, and McAfee tech support renewal invoices with phone numbers.
- Tesseract OCR on Image Attachments: Extracts text from graphic-only invoice attachments to defeat image-based spam filters.
- 419 advance-fee & consignment box fraud.
- Work-from-home cashier check cashing scams.
- CEO fraud and urgent gift card / wire transfer impersonation.
- Records findings in the
threat_findingsDuckDB table, accessible on the Threat Intelligence Dashboard (/intelligence/threats).
- Connects directly to configured external mail servers (iCloud, Hotmail, Gmail) via secure SSL IMAP & OAuth 2.0.
- Bulk Archival: Ingests live server messages, converts them into the hierarchical single-mbox tree (
MailBoxData/[account]/YYYY/MM/DD/msg_*.mbox), and indexes them into DuckDB withis_archived = True. - Chronological Prioritization: Sorts messages numerically ascending by sequence ID so older emails are processed and archived first.
- Age-Based Archival Filtering: Automatically selects and archives messages older than 1, 3, 6, or 12 months.
- Domain & Sender Archival: Targets high-volume notification domains (e.g. LinkedIn, Uber, Amazon, Facebook, Twitter / X, GitHub).
- Immediate Server Purge: Safely flags (
\Deleted) and expunges (EXPUNGE) archived emails from server mailboxes immediately after successful local commit to reclaim cloud storage space. - DuckDB Archival Audit Trail: Persists complete run statistics and criteria to the
archival_runstable. - Multi-Surface Control: Run interactively via the web Server Email Browser (
/server-browser) with backgroundJobQueueSSE progress, or headless via CLI (scripts/run_ai_agents.py --archive).
All database files are stored in the dedicated databases/ directory:
databases/
├── emails.duckdb # Main columnar store (40,000+ indexed emails, threads, embeddings)
├── chat_history.db # SQLite store for assistant chat logs
├── private_vault.duckdb # Secure local vault store
└── private_auth.db # Local authentication & token store
emails: Columnar email records:id VARCHAR PRIMARY KEYmbox_source VARCHAR(Points to single.mboxinMailBoxData/)date VARCHAR,sender VARCHAR,recipient VARCHAR,subject VARCHARbody VARCHAR(Raw text)clean_body VARCHAR(Stripped of repeated reply quotes and signatures)message_id VARCHAR,in_reply_to VARCHAR,thread_id VARCHAR,thread_depth INTEGERbtc_wallets VARCHAR[](Cryptographically verified addresses: Base58Check / Bech32 / EVM)category VARCHAR,is_archived BOOLEAN,is_suspicious BOOLEANscam_type VARCHAR,scam_confidence FLOAT,scam_analysis VARCHAR
thread_summaries: AI-generated conversation insights:thread_id VARCHAR PRIMARY KEY,subject VARCHAR,email_count INTEGERfirst_date VARCHAR,last_date VARCHAR,participants VARCHAR[]summary VARCHAR,key_insights VARCHAR,action_items VARCHAR[],sentiment VARCHAR
threat_findings: Detailed security threat audit log:id VARCHAR PRIMARY KEY,email_id VARCHAR,threat_type VARCHARrisk_level VARCHAR(CRITICAL,HIGH,MEDIUM,LOW),confidence FLOATindicators VARCHAR[],summary VARCHAR,scanned_at TIMESTAMP
email_embeddings: High-dimensional vector store:email_id VARCHAR REFERENCES emails(id)embedding FLOAT[768](Native array vector for cosine similarity)model VARCHAR
email_whitelist: Persistent sender shield:id VARCHAR PRIMARY KEY,pattern VARCHAR,type VARCHAR,category VARCHAR,notes VARCHAR
archival_runs: Complete audit trail for automated and manual archival runs:run_id VARCHAR PRIMARY KEY,account_id VARCHAR,filter_type VARCHAR,criteria VARCHARscanned_count INTEGER,archived_count INTEGER,deleted_server_count INTEGERby_folder VARCHAR,by_domain VARCHAR,started_at TIMESTAMP,completed_at TIMESTAMP,status VARCHAR
accounts: IMAP account credentials and sync telemetry.
chmod +x install.sh
./install.shThe script automatically:
- Validates Python >= 3.11.
- Configures a virtual environment using
uv(if installed) or standardvenv. - Installs all required packages from
requirements.txt. - Creates your
.envconfiguration file from.env.example. - Connects to Ollama and ensures
embeddinggemma-2andgemma2:9bmodels are ready. - Initializes
databases/emails.duckdbschema and seeds default whitelist rules.
Using uv (recommended):
uv venv .venv
source .venv/bin/activate
uv pip install -r requirements.txtUsing standard pip:
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txtcp .env.example .envEdit .env to configure your settings:
# Database Path
DUCKDB_PATH=databases/emails.duckdb
# Local Ollama AI Settings
OLLAMA_HOST=http://localhost:11434
LOCAL_EMBEDDING_MODEL=embeddinggemma-2
LOCAL_CLASSIFIER_MODEL=gemma2:9b
# Optional: Cloud Gemini API Key for Fallback Embeddings
GEMINI_API_KEY=your_gemini_api_key
# Web Server
FLASK_HOST=127.0.0.1
FLASK_PORT=5000
FLASK_DEBUG=True
# IMAP Account Credentials
ICLOUD_USER=your_username@icloud.com
ICLOUD_PASS=xxxx-xxxx-xxxx-xxxx
GMAIL_USER=your_username@gmail.com
GMAIL_PASS=xxxx-xxxx-xxxx-xxxx
HOTMAIL_USER=your_username@hotmail.com
HOTMAIL_PASS=your_passwordEnsure Ollama is running (ollama serve), then pull the models:
# 768-dimensional Vector Embeddings Model (740M)
ollama pull embeddinggemma-2
# Threat Hunting & Intelligence Classifier (9B)
ollama pull gemma2:9b
# Optional Fallback Model (7B)
ollama pull qwen2.5:7bStart the web application:
python run_web.pyOr with uv:
uv run run_web.pyAccess the interface in your browser at:
http://127.0.0.1:5000
The Mailboxes page organizes email storage per account using a distinct two-row architecture:
-
Top Row (Local Archive):
- Displays the canonical on-disk storage path:
MailBoxData/[account](e.g.MailBoxData/newaeon_AT_mac_com). - Indexed email count stored locally in DuckDB.
- Local disk consumption formatted in MB / GB.
- Local date coverage range (
Earliest → Latest). - Info Icon (
(i)): Opens the metadata modal with storage details, file paths, and indexing status. - Browse Button: Opens the local Emails Browser (
/emails) filtered to that mailbox.
- Displays the canonical on-disk storage path:
-
Second Row (On Server):
- Displays live remote IMAP connection info:
<i class="fa-solid fa-server"></i> Server: username (host). - Real-time remote message count on the mail server.
- Disk size indicated as
-(remote storage). - Remote date coverage range:
Oldest → Present. - Browse Button: Directly opens the live Server Email Browser (
/server-browser?account_id=...).
- Displays live remote IMAP connection info:
The Server Email Browser provides a zero-latency, direct IMAP interface for browsing, selectively archiving, and cleaning up messages from remote mailboxes:
- Account Switcher: Seamlessly toggle between configured accounts (iCloud, Hotmail, Gmail) with automatic live reconnect.
-
Live Folder Tabs: View messages by server folder (
INBOX,Junk,Sent,Archive,Drafts, etc.) with message count badges and unread indicator dots. -
Reverse Order Sorting (Oldest First):
-
Toolbar Toggle Button: 1-click
Reverse Order (Oldest First)toggle in the top action bar to prioritize older emails so they can be archived and purged first. -
Clickable Date Header: Sort toggle on the Date column header with directional indicators (
Date (Oldest)$\uparrow$ vsDate (Newest)$\downarrow$ ). -
Persistent Sort State: Order parameter (
order=asc/order=desc) is preserved across folder switching, subject search, domain filtering, age selection, and pagination.
-
Toolbar Toggle Button: 1-click
-
Selective Checkboxes & Sticky Bulk Bar:
- Individual checkbox per email plus a master "Select All" toggle in the table header.
- Floating action bar displays dynamic selection count (
Selected: N emails) and confirmation indicator (Purge from server right away).
-
Quick Domain Presets: 1-click filter badges to immediately isolate high-volume notifications:
-
LinkedIn (
linkedin.com) -
Uber (
uber.com) -
Amazon (
amazon.com) -
Facebook (
facebookmail.com) -
Twitter / X (
x.com) -
GitHub (
github.com)
-
LinkedIn (
- Age Filtering: Quick dropdown to filter emails older than 1, 3, 6, or 12 months.
-
Immediate Server Purge: Archived emails are immediately expunged from the remote IMAP mailbox (
STORE \Deleted+EXPUNGE) right after successful local single-mbox writing and DuckDB indexing (is_archived = True), preventing duplicate downloads and instantly reclaiming cloud quota. - Automatic List Refresh: The view automatically reloads 700–800ms after archival, ensuring the displayed list always reflects the true remaining messages on the server.
- Autonomous Archival Agent Modal: Launch the background Archival Agent directly from the UI with age thresholds, domain filters, dry-run previews, and real-time SSE progress monitoring in the slide-over job drawer.
- Home (
/): MailData landing page featuring the platform architecture banner and quick navigation. - Dashboard (
/dashboard): Global metrics, indexed email counts, accounts, and recent mail. - Mailboxes (
/mailboxes): Inspect storage by account, year, and date with clean mailbox paths (MailBoxData/[account]), disk sizes, and date coverage. Includes:- Top Row (Local Archive): Displays mailbox on-disk path (
MailBoxData/newaeon_AT_mac_com), email count, disk size, and on-disk metadata popup icon beside the browse button. - Second Row (On Server): Indicates live status on remote mail server (
Server: username (host)), live email count, disk size (-), date coverage (Oldest → Present), and a Browse button opening the live Server Email Browser.
- Top Row (Local Archive): Displays mailbox on-disk path (
- Server Email Browser (
/server-browser): Browse live external IMAP mailboxes:- Reverse Order Button: 1-click toggle to sort and display oldest emails first (
Oldest First), enabling older messages to be reviewed and archived first, along with a clickable sort toggle on the Date table header. - Folder View: Toggle between
INBOX,Junk,Sent,Archive,Draftswith message count badges. - Checkbox Selection: Select individual emails or "Select All" for bulk archival.
- Quick Domain Filters: 1-click filters for
linkedin.com,uber.com,amazon.com,facebookmail.com,x.com, etc. - Age Filtering: Filter messages older than 1, 3, 6, or 12 months.
- Interactive Archiving & Immediate Purge: Ingest selected emails into single-mbox format, record to DuckDB with
is_archived = True, and purge immediately from the remote server with automatic list refresh. - Archival Agent Modal: Launch the background Archival Agent with custom filters and real-time job drawer progress.
- Reverse Order Button: 1-click toggle to sort and display oldest emails first (
- Emails Browser (
/emails): Wide-layout table with natural text wrapping and zero scrollbars. Features:- Filter by Category (
work,legal,banking,finance, etc.), Search query, Date, Mailbox, or Crypto Wallets. - Status tabs for All vs. Threats.
- View Button (
<i class="fa-regular fa-envelope-open"></i> View): Opens full message viewer. - Info Button (
<i class="fa-solid fa-circle-info"></i>): Opens full RFC 2822 header and mbox metadata popup modal.
- Filter by Category (
- Email Detail (
/emails/<id>):- Clean text view by default with collapsible quoted reply history.
- Formatted HTML view in an isolated sandbox with zero script execution.
- Privacy Guard Banner: Reports blocked tracking pixels with a one-click "Load Remote Images" toggle button.
- Conversation Thread Timeline: Displays conversation chains with depth hierarchy and the AI Thread Summary & Key Takeaways card.
- Attachment Panel: Download or preview attachments with polyglot filtering and SVG script sanitization.
- Thread Insights (
/intelligence/threads): Dedicated dashboard displaying all AI-synthesized conversation thread summaries, key decisions, action items, and sentiment. - Threat Intelligence (
/intelligence/threats): Displays all security threats identified by the Threat Hunting Agent with risk ratings and specific indicators. - Crypto Intelligence (
/intelligence): Cryptographically verified cryptocurrency wallet addresses and extortion analysis. - Timeline (
/timeline): Hierarchical drill-down explorer (Year > Month > Day) across archives. - Categories (
/categories): Visual distribution of all categories with AI reclassification tools. - Search (
/search): Natural language semantic search and keyword query matching. - Whitelist (
/whitelist): Manage domain (*@domain.com) and exact email rules to prevent false alarms. - Accounts (
/accounts): IMAP connection setup, OAuth 2.0 Device Flow authorization, and synchronization. - AI Rescan Modal: Select and trigger any combination of the local AI agents with real-time SSE progress drawer tracking.
MailData provides flexible CLI tools for offline batch operations, automation scripts, and background processing.
Run the local AI agents offline from your terminal without opening the web interface:
# Run all agents (Threat Hunting, Categorization, Thread Insights, Archival)
python scripts/run_ai_agents.py --all --limit 50
# Run specific agents
python scripts/run_ai_agents.py --threats --limit 100
python scripts/run_ai_agents.py --categorize --limit 100
python scripts/run_ai_agents.py --threads --limit 25
# Run Archival Agent: Bulk archive messages older than 3 months
python scripts/run_ai_agents.py --archive --months 3
# Run Archival Agent on specific account and folders
python scripts/run_ai_agents.py --archive --account icloud_newaeon_mac_com --folders INBOX,Junk
# Run Archival Agent: Archive by domain/email (LinkedIn, Uber, Amazon)
python scripts/run_ai_agents.py --archive --domains linkedin.com,uber.com,amazon.com
# Archive older than 12 months without deleting from server
python scripts/run_ai_agents.py --archive --months 12 --no-delete-server
# Run Archival Agent in Dry-Run mode (preview counts without downloading or deleting)
python scripts/run_ai_agents.py --archive --months 6 --dry-run
# Specify Ollama model and fallback
python scripts/run_ai_agents.py --all --model gemma2:9b --fallback-model qwen2.5:7b
# Run continuously as an offline background daemon (scans every 5 minutes)
python scripts/run_ai_agents.py --all --continuous --interval 300| Flag | Description | Default |
|---|---|---|
--all |
Runs all agents sequentially in a unified pipeline | False |
--threats |
Executes Threat Hunting Agent (scans mbox files with OCR) | False |
--categorize |
Executes Categorization Agent (work/legal & banking/finance) |
False |
--threads |
Executes Thread Insights & Summarizer Agent | False |
--archive |
Executes Archival Agent (Bulk, Age, or Domain/Email) | False |
--months <int> |
Filter messages older than 1, 3, 6, or 12 months | None |
--domains <list> |
Comma-separated domains to archive (linkedin.com,uber.com) |
None |
--folders <list> |
Comma-separated server folders to scan (INBOX,Junk,Archive) |
None |
--account <str> |
Target account ID to archive (all for all accounts) |
all |
--dry-run |
Preview match counts without downloading or deleting | False |
--no-delete-server |
Prevent purging messages from mail server after archival | False |
--limit <int> |
Maximum records/messages to process per agent | 50 |
--model <str> |
Preferred local Ollama model | gemma2:9b |
--fallback-model <str> |
Secondary Ollama model if preferred is unavailable | qwen2.5:7b |
--continuous |
Runs pipeline in a persistent background daemon loop | False |
--interval <int> |
Seconds between cycles in continuous mode | 300 |
# Start the web interface
maildata web
# Display database statistics and category breakdown
maildata stats
# Rescan unclassified emails using local Gemma 2 9B model
maildata ai-rescan --mode unprocessed --limit 100 --model gemma2:9b
# Re-evaluate flagged threats to clear false positives
maildata ai-rescan --mode suspicious --limit 50
# Authorize Microsoft / Hotmail account via OAuth 2.0 Device Flow
maildata oauth
# Priority Smart Archival to single-mbox with AI scan & server purge
maildata archive --account hotmail_binholz_hotmail_com --limit 25
# Ingest mbox archives into PostgreSQL
maildata postgres --dir ./MailBoxData --dbname maildata# Display database statistics and top mail sources
python mbox2db.py --stats
# Generate 768-dimensional vector embeddings for unindexed emails (batch of 500)
python mbox2db.py --index-vectors 500
# Trigger IMAP sync for a specific account
python mbox2db.py --sync-account icloud
# Dry-run smart archival and server cleanup
python mbox2db.py --smart-archive icloud_newaeon_mac_com --dry-run
# Execute smart archival and purge items older than 12 months
python mbox2db.py --smart-archive icloud_newaeon_mac_com --older-than 12
# Prune duplicate emails across archives
python mbox2db.py --deduplicateMigrate mbox files or partitioned MailBoxData/ directory trees into a PostgreSQL database with automated schema creation and indexing:
# Ingest entire MailBoxData archive into PostgreSQL:
python mbox2postgres.py --dir ./MailBoxData --dbname maildata --create-schema
# Ingest a single mbox file:
python mbox2postgres.py --mbox ./my_mail.mbox --dbname maildata
# With full PostgreSQL connection string:
python mbox2postgres.py --url postgresql://postgres:password@localhost:5432/maildata --dir ./MailBoxDataThis software is released under the BSD 3-Clause License.
See the full text in LICENCE.md.
Copyright (c) 2025-2026, A. K. Holz <salamcast@gmail.com>
All rights reserved.
