curated-tokenizers

PyPI page
Home page
Author: Explosion
License: MIT
Summary: Lightweight piece tokenization library
Latest version: 2.0.1
Required dependencies: regex

Downloads last day: 45,021
Downloads last week: 358,607
Downloads last month: 550,456
7-day smoothing