[SergeyShk]

Description

Added anySeg to the spaCy Universe.

anySeg segments text into tokens and sentences with their positions in the source text, following the conventions of Universal Dependencies. The core is written in Rust and the text is never changed.

Its spaCy integration (pip install "anyseg[spacy]") adds the tokenizer anyseg.Tokenizer.v1 and the component anyseg_sentencizer, registered through entry points, so they are used by name in a config. The component sets sentence starts before the parser, which keeps them.

With ru_core_news_sm on the UD Russian test sets, from raw text:

• sentence F1 goes from 95.4% to 98.5% on SynTagRus and from 80.4% to 89.9% on Taiga;
• LAS rises by 1 to 3.5 points;
• the tokenizer is about ten times faster than the default one.

Docs: https://sergeyshk.github.io/anySeg/spacy/

Types of change

An enhancement to the spaCy Universe: a new entry in website/meta/universe.json.

Checklist

• I confirm that I have the right to submit this contribution under the project's MIT license.
• I ran the tests, and all new and existing tests passed. (python .github/validate_universe_json.py website/meta/universe.json; the code example runs with anySeg 0.2.0 and ru_core_news_sm 3.8.0.)
• My changes don't require a change to the documentation, or if they do, I've added all required information.