Problem
sampling.override_preset is global: every model shares one
preset. Models with different sampling parameters (e.g. Qwen 3.8 wants top_k 20,
Gemma 4 wants top_k 64) can't be served using inline_model_loading: true.
Solution
Optional per-model preset via sampler_preset in the model directory's
tabby_config.yml (with the same fallback chain as other per-model options):
# models/qwen3.8-27b/tabby_config.yml
model:
sampler_preset: qwen3.8-27b
# models/gemma4/tabby_config.yml
model:
sampler_preset: gemma4
Resolution: request params > model preset > global preset > built-in defaults.
Problem
sampling.override_presetis global: every model shares onepreset. Models with different sampling parameters (e.g. Qwen 3.8 wants
top_k 20,Gemma 4 wants
top_k 64) can't be served using inline_model_loading: true.Solution
Optional per-model preset via
sampler_presetin the model directory'stabby_config.yml(with the same fallback chain as other per-model options):Resolution: request params > model preset > global preset > built-in defaults.