Issues

• Compact show for Tokenizer

  MurrellGroup/HuggingFaceTokenizers.jl#19 · Filter by author CarloLucibello opened on Oct 26, 2026

• Wrap the vocabulary lookups

  MurrellGroup/HuggingFaceTokenizers.jl#17 · Filter by author CarloLucibello opened on Oct 20, 2026

• Add a Julia Encoding type mirroring the Python one

  MurrellGroup/HuggingFaceTokenizers.jl#16 · Filter by author CarloLucibello opened on Oct 20, 2026

• We should change the name of encode

  MurrellGroup/HuggingFaceTokenizers.jl#5 · Filter by author murrellb opened on Feb 14, 2025
  

• encode/decode outputs.

  MurrellGroup/HuggingFaceTokenizers.jl#3 · Filter by author murrellb opened on Dec 29, 2024

• Clarify motivation

  MurrellGroup/HuggingFaceTokenizers.jl#1 · Filter by author lassepe opened on Nov 26, 2024
  2