Skip to content

Latest commit

 

History

89 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Textractor

Structure-aware markdown extraction from documents for LLM consumption.

Textractor parses PDF, DOCX, PPTX, HTML, and EPUB files into clean GitHub-flavored markdown, preserving document structure (headings, tables, code blocks, lists, images). It can split output into chunks that respect heading boundaries, making it ideal for feeding documents into LLMs.

Features

  • Multi-format parsing -- PDF, DOCX, PPTX, HTML, EPUB
  • Structure-aware chunking -- splits at heading boundaries, never mid-paragraph
  • Image extraction -- saves images alongside markdown output
  • Vision model integration (optional) -- OCR for scanned PDFs via LightOnOCR-2-1B, image captioning via Qwen3-VL-4B
  • Post-processing -- header/footer removal, multi-page table merging, artifact cleanup (PDF)
  • LLM-friendly output -- clean markdown to stdout or separate .md files per chunk with YAML frontmatter

Installation

# Core
uv pip install .

# With vision model support (OCR + image captioning)
uv pip install ".[vision]"

Requires Python 3.11+.

Usage

# Extract to stdout
textractor extract document.pdf

# Extract to directory with chunking
textractor extract document.pdf -o output/ -t 32000

# Force format detection
textractor extract file.txt -f html

# Enable vision features (OCR, image captioning)
textractor extract scanned.pdf --use-vision

# Verbose output
textractor extract document.docx -v

Options

Option Description
-o, --output-dir DIR Write output files to directory (default: stdout)
-t, --max-tokens N Max tokens per chunk, 0 = no chunking (default: 0)
-f, --format FMT Force input format instead of auto-detect
--use-vision Enable OCR and image captioning (default: off)
-v, --verbose Verbose progress on stderr
-V, --version Show version

Output modes

  • Single output (no -o or single chunk): markdown printed to stdout
  • Chunked output (-o with -t): separate .md files per chunk + metadata.json

Supported formats

Format Extensions Library
PDF .pdf pymupdf4llm
DOCX .docx python-docx
PPTX .pptx python-pptx
HTML .html, .htm markdownify
EPUB .epub ebooklib

License

MIT

About

Structure-aware markdown extraction from PDF, DOCX, PPTX, HTML and EPUB for LLM consumption. Heading-aware chunking that never splits mid-paragraph, plus optional OCR and image captioning.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages