mirrors/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2025-05-11 18:36:41 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

Introduce GPU Overhead env var (#5922)

Browse source 
Provide a mechanism for users to set aside an amount of VRAM on each GPU
to make room for other applications they want to start after Ollama, or workaround
memory prediction bugs
This commit is contained in:
Daniel Hiltgen 2024-09-05 13:46:35 -07:00 • committed by GitHub
parent a60d9b89ce
commit b05c9e83d9
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
3 changed files with 28 additions and 3 deletions
Download patch file Download diff file Expand all files Collapse all files

1
cmd/cmd.go
View file

@ -1421,6 +1421,7 @@ func NewCLI() *cobra.Command {
envVars["OLLAMA_TMPDIR"],
envVars["OLLAMA_FLASH_ATTENTION"],
envVars["OLLAMA_LLM_LIBRARY"],
envVars["OLLAMA_GPU_OVERHEAD"],
})
default:
appendEnvDocs(cmd, envs)