mirrors/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2025-05-11 18:36:41 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions
ollama/runner
History
Michael Yang 74bd09652d ml/backend/ggml: load tensors in 32KiB chunks
2025-03-21 14:43:52 -07:00
..
common Runner for Ollama engine 2025-02-13 17:09:26 -08:00
llamarunner llm: remove internal subprocess req and resp types (#9324) 2025-03-14 15:21:53 -07:00
ollamarunner ml/backend/ggml: load tensors in 32KiB chunks 2025-03-21 14:43:52 -07:00
README.md Runner for Ollama engine 2025-02-13 17:09:26 -08:00
runner.go Runner for Ollama engine 2025-02-13 17:09:26 -08:00

README.md

runner

Note: this is a work in progress

A minimial runner for loading a model and running inference via a http web server.

./runner -model <model binary>

Completion

curl -X POST -H "Content-Type: application/json" -d '{"prompt": "hi"}' http://localhost:8080/completion

Embeddings

curl -X POST -H "Content-Type: application/json" -d '{"prompt": "turn me into an embedding"}' http://localhost:8080/embedding