tiktoken

toolopen source · MIT

Fast BPE tokenizer for OpenAI models: count tokens before they hit the context window.

Why is it popular?

Signals

Relationship map

No mapped relationships yet