From b48c66b4805693e078348bc69573079395e3f7df Mon Sep 17 00:00:00 2001 From: Anton Abyzov Date: Tue, 31 Mar 2026 19:21:59 -0400 Subject: [PATCH] add local model support: Ollama, LM Studio, OpenAI via litellm proxy --- README.md | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 13d36fd..95e0212 100644 --- a/README.md +++ b/README.md @@ -49,7 +49,32 @@ export ANTHROPIC_API_KEY=your-key node cli.js ``` -> **Note:** Claude Code uses the Anthropic Messages API format. It does not support OpenAI-compatible endpoints (Ollama, LM Studio, etc.) directly — you would need an API format translator proxy like [litellm](https://github.com/BerriAI/litellm) in front. +### With Local Models (Ollama, LM Studio) + +Claude Code uses the Anthropic Messages API format. To use local models, run [litellm](https://github.com/BerriAI/litellm) as a translation proxy: + +```bash +# Terminal 1: Start litellm proxy +pip install litellm +litellm --model ollama/llama3.1:8b --port 8080 + +# Terminal 2: Point Claude Code at the proxy +export ANTHROPIC_BASE_URL=http://localhost:8080 +export ANTHROPIC_API_KEY=not-needed +node cli.js +``` + +Works with any model Ollama supports — llama3.1, codellama, deepseek-coder, mistral, etc. + +### With OpenAI / GPT Models + +```bash +# Via litellm proxy +litellm --model openai/gpt-4o --port 8080 + +# Or any OpenAI-compatible endpoint (Codex, GPT-5.4, etc.) +litellm --model openai/o3 --port 8080 +``` ---