Typescript and .NET implementation of BPE tokenizer for OpenAI LLMs.
MIT License
Bot releases are hidden (Show)
Published by shengyfu 9 months ago
You can find the published NuGet package here.
You can find the Tokenizer.exe in the attached Tokenizer.zip.
To run performance benchmark, you can download the attached PerfBenchmark.zip.
Published by shengyfu over 1 year ago
You can find the published NuGet package here.
You can find the Tokenizer.exe in the attached Tokenizer_1.3.2.zip.
To run performance benchmark, you can download the attached PerfBenchmark_1.3.2.zip.
Published by shengyfu over 1 year ago
Published by shengyfu over 1 year ago