Skip to content
mpilhltPublic

About

Training material for a GROBID flavour that can deal with references in footnotes, which is a common practice in law and the humanities

Resources

Stars

3 stars

Watchers

3 watching

Forks

Repository files navigation

FOSSIL - Footnote-based Open-access SSH Scientific Instance Labels

This repo contains training data and other resources for building a Grobid flavour that can deal with references in footnotes, which is a common practice in law and the humanities.

Paper: https://arxiv.org/abs/2606.01109

This project is a collaboration between Christian Boulanger (mpilhlt) and Luca Foppiano (ScienciaLAB).

The repository was renamed from grobid-footnote-flavour.

Status

The dataset is actively being edited off-line and has not been published yet. This repo contains only an older and incomplete version of the dataset batches. Watch this space for updates.

Schemas

Standalone RNG schemas for validating GROBID training files are published at:

https://mpilhlt.github.io/fossil/schema

Schema Validates
grobid.training.segmentation.rng *.training.segmentation.tei.xml
grobid.training.references.rng *.training.references.tei.xml
grobid.training.references.referenceSegmenter.rng *.training.references.referenceSegmenter.tei.xml

Schema authoring

The schemas in docs/schema/ are auto-generated and must not be edited directly. The composable source lives in schema/:

schema/
  shared/
    tei-header.rng        # shared TEI header elements
    bibl-struct.rng       # shared biblStruct elements
    common-elements.rng   # lb, ptr, ref, label
  grobid.training.references.rng
  grobid.training.references.referenceSegmenter.rng
  grobid.training.segmentation.rng

After editing any file under schema/, regenerate and commit the flattened schemas:

uv sync --group dev        # first time only
uv run python scripts/build-schema.py
git add docs/schema/
git commit -m "Regenerate schemas"

The build script will abort with an error if any <define> name appears more than once in a flattened output — a sign that a shared element needs a filename-stem prefix (e.g. bibl-struct_note instead of note).

Annotation guidelines

The annotation guidelines are maintained in docs/guidelines.md. Word and PDF versions are published alongside the schemas:

Both are auto-generated and must not be edited directly. When a pull request changes docs/guidelines.md, the Build guidelines workflow regenerates them and commits them to the pull request's branch, so they are merged together with the change. Pull the branch before pushing further changes to it.

To build them locally, install pandoc and LibreOffice, then run:

uv sync --group dev        # first time only
uv run python scripts/build-guidelines-docx.py --pdf

Without --pdf, only the Word version is written, and LibreOffice is optional: it is used to fill in the page numbers of the table of contents. Without it, Word offers to update the table when the file is opened.

Data Structure

The structure of the data will be based on batches and will follow this structure:

├── batches
│   └── batch_1
│       ├── 0_input
│       │   ├── 10.12946__rg01__036-055.pdf
│       │   └── 10.5771__2699-1284-2020-1-16.pdf
│       ├── 1_generated
│       └── 2_corrected

About

Training material for a GROBID flavour that can deal with references in footnotes, which is a common practice in law and the humanities

Resources

Stars

3 stars

Watchers

3 watching

Forks

Releases

Packages

Used by

Contributors

Languages