Skip to content

Pull requests: huggingface/tokenizers

Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Reviews
Assignee
Filter by who’s assigned
Assigned to nobody Loading
Sort

Pull requests list

Fix features blending into a paragraph
#1798 opened Jun 13, 2025 by bionicles Loading…
Bump brace-expansion from 1.1.11 to 1.1.12 in /bindings/node dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code
#1796 opened Jun 11, 2025 by dependabot bot Loading…
Bump webpack-dev-server from 4.10.0 to 5.2.1 in /tokenizers/examples/unstable_wasm/www dependencies Pull requests that update a dependency file javascript Pull requests that update Javascript code
#1792 opened Jun 6, 2025 by dependabot bot Loading…
add group capture to replace
#1788 opened Jun 3, 2025 by cboseak Loading…
[docs] Whitespace
#1785 opened May 27, 2025 by stevhliu Loading…
Add Truncate pre-tokenizer
#1783 opened May 27, 2025 by ArthurZucker Draft
Update decode stream api
#1780 opened May 27, 2025 by ArthurZucker Loading…
Make unigram cache optional
#1763 opened Apr 18, 2025 by wangrunji0408 Loading…
Itertools upgrade
#1756 opened Mar 25, 2025 by sftse Loading…
Implement Append normalizer
#1755 opened Mar 24, 2025 by austinleedavis Loading…
Switch to FXHash
#1752 opened Mar 19, 2025 by MeetThePatel Loading…
Add FxHash and ShortStringOptimization.
#1733 opened Feb 10, 2025 by MeetThePatel Loading…
3 of 4 tasks
Does windows aarch work ?
#1719 opened Jan 10, 2025 by Narsil Loading…
Fixed Length Pre-Tokenizer
#1713 opened Jan 5, 2025 by jonvet Loading…
Draft backtrack
#1712 opened Jan 3, 2025 by ArthurZucker Draft
Bpe clones
#1707 opened Dec 28, 2024 by sftse Loading…
Fast regex
#1605 opened Aug 8, 2024 by ArthurZucker Draft
ProTip! Find all pull requests that aren't related to any open issues with -linked:issue.