mirrors/ollama
1
0
Fork 0
mirror of https://github.com/ollama/ollama.git synced 2025-05-10 18:06:33 +02:00
Code Issues Projects Releases Packages Wiki Activity Actions

kvcache: Log batch size if we can't find a slot

Browse source 
In some cases, we can't find a cache slot when using sliding window
attention. It would be helpful in this (and other cases) to know what
the batch size is.

Bug #10127
This commit is contained in:
Jesse Gross 2025-05-01 13:45:32 -07:00 • committed by Jesse Gross
parent 8e8f2c6d67
commit 074bac8447
1 changed files with 1 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

2
kvcache/causal.go
View file

@ -239,7 +239,7 @@ func (c *Causal) findStartLoc() (int, error) {
}
}
return 0, fmt.Errorf("%w (length: %v)", ErrKvCacheFull, len(c.cells))
return 0, fmt.Errorf("%w (cache: %v batch: %v)", ErrKvCacheFull, len(c.cells), c.curBatchSize)
}
func (c *Causal) updateSlidingWindow() {