Misc. bug: kv_unified = true despite not setting --kv-unified

[wallentri88]

Name and Version

latest master (96ac5a2)

Operating systems

No response

Which llama.cpp modules do you know to be affected?

llama-server

Command line

Problem description & steps to reproduce

Try to run any model using llama-server. Documentation says that --kv-unified arg should enable unified cache and it shouldn't be enabled by default. But it's enabled by default and I didn't find a way to disable it.

First Bad Commit

probably this one? #16736

Relevant log output

llama_context: constructing llama_context
llama_context: n_seq_max     = 4
llama_context: n_ctx         = 131072
llama_context: n_ctx_seq     = 131072
llama_context: n_batch       = 2048
llama_context: n_ubatch      = 2048
llama_context: causal_attn   = 1
llama_context: flash_attn    = enabled
llama_context: kv_unified    = true
llama_context: freq_base     = 150000.0
llama_context: freq_scale    = 0.03125

[o7si]

Hi @wallentri88 , it's this piece of code that caused n_parallel and kv_unified to be modified.

llama.cpp/tools/server/server.cpp

Lines 5513 to 5522 in 4902eeb

	// TODO: should we have a separate n_parallel parameter for the server?
	// https://github.com/ggml-org/llama.cpp/pull/16736#discussion_r2483763177
	// TODO: this is a common configuration that is suitable for most local use cases
	// however, overriding the parameters is a bit confusing - figure out something more intuitive
	if (params.n_parallel == 1 && params.kv_unified == false && !params.has_speculative()) {
	LOG_WRN("%s: setting n_parallel = 4 and kv_unified = true (add -kvu to disable this)\n", __func__);
	params.n_parallel = 4;
	params.kv_unified = true;
	}