gptme
OpenAI API-compatible server (vllm) & tokenizer #559
Unanswered
micael-git
asked this question in
Q&A
Replies: 1 comment
-
|
I found this answer on SO which answers how to deal with this generally by keeping a copy of the tokenizer locally: https://stackoverflow.com/a/76107077/965332 You could also modify this function to not use tiktoken and instead simply do |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hello,
I'm currently testing gptme with local model behind couple liteLLM/vLLM, but I'm facing an issue regarding the tokenizer.
Seems gptme wants to use the openai tokenizer:
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='openaipublic.blob.core.windows.net', port=443): Max retries exceeded with url: /encodings/cl100k_base.tiktoken (Caused by ConnectTimeoutError(<urllib3.connection.HTTPSConnection object at 0x7f5401a95450>, 'Connection to openaipublic.blob.core.windows.net timed out. (connect timeout=None)'))I search on the documentation but haven't found a way to disable the tokenizer (used for counts and costs only?) or use my own tokenizer in local.
Can anyone help, or point to me a misunderstanding?
Thank you
Beta Was this translation helpful? Give feedback.
All reactions