llm: avoid loading model if system memory is too small (#5637)

* llm: avoid loading model if system memory is too small

* update log

* Instrument swap free space

On linux and windows, expose how much swap space is available
so we can take that into consideration when scheduling models

* use `systemSwapFreeMemory` in check

---------

Co-authored-by: Daniel Hiltgen <daniel@ollama.com>
This commit is contained in:
Jeffrey Morgan 2024-07-11 16:42:57 -07:00 committed by GitHub
parent 791650ddef
commit c4cf8ad559
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
6 changed files with 22 additions and 13 deletions

View file

@ -57,6 +57,7 @@ func GetCPUMem() (memInfo, error) {
return memInfo{
TotalMemory: uint64(C.getPhysicalMemory()),
FreeMemory: uint64(C.getFreeMemory()),
// FreeSwap omitted as Darwin uses dynamic paging
}, nil
}