ollama/convert
Michael Yang 333e360422
model: handle multiple eos tokens (#10577)
* get eos_token_id from generation_config.json

* refactor

* include both ids and strings in trace

* comments

* remove special case for gemma3 special vocab (#10743)
2025-05-16 13:40:23 -07:00
..
sentencepiece chore(all): replace instances of interface with any (#10067) 2025-04-02 09:44:27 -07:00
testdata convert: import support for command-r models from safetensors (#6063) 2025-01-15 16:31:22 -08:00
convert.go model: handle multiple eos tokens (#10577) 2025-05-16 13:40:23 -07:00
convert_bert.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_commandr.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_gemma.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_gemma2.go next ollama runner (#7913) 2025-02-13 16:31:21 -08:00
convert_gemma2_adapter.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_gemma3.go fix: change default context size for gemma3 (#9744) 2025-03-13 13:59:19 -07:00
convert_llama.go fix mllama conversion (#10716) 2025-05-15 12:15:01 -07:00
convert_llama4.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_llama_adapter.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_mistral.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_mixtral.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_mllama.go chore: update mllama to use ollama engine (#10637) 2025-05-13 17:36:02 -07:00
convert_phi3.go Move quantization to new backend (#10363) 2025-05-06 11:20:48 -07:00
convert_qwen2.go model: add Qwen2.5-VL support (#10385) 2025-05-13 20:58:02 -07:00
convert_qwen25vl.go model: add Qwen2.5-VL support (#10385) 2025-05-13 20:58:02 -07:00
convert_test.go file close check and close. (#10554) 2025-05-04 15:37:59 -07:00
reader.go chore: update mllama to use ollama engine (#10637) 2025-05-13 17:36:02 -07:00
reader_safetensors.go llama4 2025-04-25 16:59:20 -07:00
reader_torch.go llama4 2025-04-25 16:59:20 -07:00
sentencepiece_model.proto all: fix typos in documentation, code, and comments (#7021) 2024-12-10 12:58:06 -08:00
tensor.go model: add Qwen2.5-VL support (#10385) 2025-05-13 20:58:02 -07:00
tokenizer.go model: handle multiple eos tokens (#10577) 2025-05-16 13:40:23 -07:00
tokenizer_spm.go temporary work around for converting spm 2025-03-11 14:49:18 -07:00
tokenizer_test.go model: handle multiple eos tokens (#10577) 2025-05-16 13:40:23 -07:00