Ollama's server takes its settings from environment variables, and it reads them from the process that runs ollama serve. That explains most of the confusion: a variable you export in your SSH session does nothing to a server that systemd or Docker started, and a change does nothing until that server restarts. This page lists the settings that matter on a GPU server with their defaults in Ollama 0.40.16, released on 2026-11-09, taken from its source. Then it shows where to set them, how to check that they took effect, and the few I would change on a rented 48 GB card.
The settings and their defaults#
These are the server variables in 0.40.16 that change how a GPU server behaves. ollama serve --help prints the server variables your own version documents, CUDA_VISIBLE_DEVICES aside.
| Variable | Default | What it does |
|---|---|---|
OLLAMA_HOST | 127.0.0.1:11434. The official Docker image sets 0.0.0.0:11434 | Address and port the API listens on. The API has no login, so the loopback default is the safe one |
OLLAMA_MODELS | .ollama/models in the home of the user that runs the server | Where pulled models are stored |
OLLAMA_KEEP_ALIVE | 5m | How long a model stays in GPU memory after its last request. A duration such as 30m or 24h, or a number of seconds. A negative value keeps it loaded, and 0 unloads it straight away |
OLLAMA_NUM_PARALLEL | 1 | Requests one loaded model answers at the same time. Each slot gets a full context |
OLLAMA_MAX_LOADED_MODELS | 0, which means 3 per GPU, or 3 on a machine without one | Models kept loaded at once, as long as they fit |
OLLAMA_MAX_QUEUE | 512 | Requests allowed to wait for a model to be loaded. Past that, the API answers 503 |
OLLAMA_CONTEXT_LENGTH | 0, which means automatic: 4,096, 32,768 or 262,144 tokens by total GPU memory | Default context for every model that has no num_ctx of its own |
OLLAMA_FLASH_ATTENTION | Unset, which means automatic where the GPU supports it | 1 forces Flash Attention on, 0 turns it off |
OLLAMA_KV_CACHE_TYPE | f16 | Precision of the KV cache, for every model on the server. q8_0 takes about half the memory and q4_0 about a quarter |
OLLAMA_SCHED_SPREAD | Off | 1 spreads every model across all GPUs, even one that fits on a single GPU |
OLLAMA_GPU_OVERHEAD | 0 | Bytes to set aside on each GPU. In 0.40.16 it is only subtracted when the default context is chosen and from the available figure in the log. It does not make llama.cpp leave memory free |
LLAMA_ARG_FIT_TARGET | Unset, so llama.cpp leaves 1,024 MiB. For a model with a vision projector Ollama sets it to the projector's size plus 1 GiB | Free memory, in MiB per GPU, that llama.cpp leaves when it fits a model's layers. This is the one to set if another process needs room on the GPU |
CUDA_VISIBLE_DEVICES | Unset, so every GPU is used | Comma-separated list of the NVIDIA GPUs Ollama may use, by index or UUID. An invalid ID such as -1 forces the CPU |
OLLAMA_LOAD_TIMEOUT | 5m | How long a model load may stall before Ollama gives up. Zero or a negative value waits forever |
OLLAMA_NO_CLOUD | Off | 1 turns off Ollama's cloud models and web search |
OLLAMA_ORIGINS | Browser requests from localhost, 127.0.0.1 and 0.0.0.0 | Comma-separated list of extra web origins allowed to call the API |
OLLAMA_DEBUG | Off | 1 logs at debug level and 2 at trace level |
Ollama has no general configuration file. The one settings file the server reads, ~/.ollama/server.json, holds a single switch, disable_ollama_cloud, which does the same as OLLAMA_NO_CLOUD=1. Everything else in the table is an environment variable or nothing.
Old guides still list variables that 0.40.16 no longer reads. OLLAMA_NEW_ENGINE, OLLAMA_MULTIUSER_CACHE and OLLAMA_INTEL_GPU were in the 0.12.0 source and are gone from this one. Setting them does no harm and has no effect.
Keep alive, parallel requests and loaded models#
The three scheduling settings decide what sits in GPU memory and for how long. By default Ollama unloads a model 5 minutes after its last request, answers one request at a time per model, and keeps up to 3 models per GPU loaded if they fit.
OLLAMA_KEEP_ALIVE sets the idle time for every model, and a request can override it with its own keep_alive field. -1 keeps a model loaded until another model needs the memory or you run ollama stop with its name. On a rented GPU the 5-minute default buys nothing: the instance bills by the hour whether or not a model is loaded, and every unload costs a reload on the next request.
OLLAMA_NUM_PARALLEL has defaulted to 1 since release 0.10.0. Raising it costs memory, because Ollama starts the model with the context multiplied by the number of slots. For llama3.3:70b at 32,768 tokens, one slot is 10.0 GiB of KV cache and four slots are 40.0 GiB (our calculation). For qwen3.8:27b the same four slots are 8.0 GiB. Requests beyond the slots wait their turn. A separate limit, OLLAMA_MAX_QUEUE, caps how many requests may wait for a model to be loaded at 512, and past it the API answers 503. When many people share one model, vLLM vs Ollama explains where vLLM's batching fits better.
OLLAMA_MAX_LOADED_MODELS caps how many models are loaded together. On a GPU a second model loads next to the first only if it fits completely in the free memory. If it does not, Ollama waits for an idle model and unloads it to make room.
Flash Attention and the KV cache type#
Flash Attention is on by default wherever the GPU supports it, so most servers never need the variable. Ollama has enabled it automatically since release 0.13.4, and 0.40.16 treats every NVIDIA GPU from compute capability 6.0 as supported, with 7.2 the one exception. That covers every GPU QuantaCloud rents. OLLAMA_FLASH_ATTENTION=0 is there for the case where you need to rule it out while debugging.
OLLAMA_KV_CACHE_TYPE is the one that saves real memory. The KV cache is 16-bit by default, q8_0 stores it in about half the space and q4_0 in about a quarter. Ollama's FAQ calls the precision loss of q8_0 very small and that of q4_0 small to medium, more noticeable at long contexts. Two limits apply: it needs Flash Attention, and it is global, so every model on the server gets the same type. The KV cache guide explains what the cache holds, and Ollama context length works out its size per model.
Where models are stored#
Models live under .ollama/models in the home directory of whichever user runs the server. With the official install script that user is ollama, whose home is /usr/share/ollama, so the path is /usr/share/ollama/.ollama/models. In the official Docker image the server runs as root and the path is /root/.ollama/models, which is why Ollama's own run command mounts a volume at /root/.ollama.
OLLAMA_MODELS moves the directory. Under systemd the ollama user must be able to write there, and Ollama's FAQ gives the fix: sudo chown -R ollama:ollama on the new directory. On a QuantaCloud instance I leave the default alone. The instance has one disk, 256 GB on the single RTX A6000 offers on 2026-11-12, there are no volumes to mount, and stopping the instance deletes the disk with every model on it. Keep a list of the tags you pull instead, as the Ollama API guide shows.
Set them under systemd#
The install script runs Ollama as a systemd service, so the variables go into an override of that service.
-
Open the override:
sudo systemctl edit ollama -
Add one
Environmentline per variable under[Service], then save:[Service] Environment="OLLAMA_CONTEXT_LENGTH=32768" Environment="OLLAMA_KEEP_ALIVE=-1" -
Reload systemd and restart the server:
sudo systemctl daemon-reload sudo systemctl restart ollama
systemd keeps the override in /etc/systemd/system/ollama.service.d/override.conf. The restart unloads every model, so do it between jobs.
Set them in Docker#
In a container the variables are part of the container, so you pass them when you create it. With docker run, add one -e per variable:
sudo docker run -d --gpus=all -v ollama:/root/.ollama \
-p 127.0.0.1:11434:11434 \
-e OLLAMA_CONTEXT_LENGTH=32768 -e OLLAMA_KEEP_ALIVE=-1 \
--name ollama ollama/ollama:0.40.16
In Compose, list them under environment: on the service. Changing a variable means removing the container and creating it again, and the models survive in the volume. The image already sets OLLAMA_HOST=0.0.0.0:11434, which the container needs so that Docker can reach it. Leave that one alone and control exposure with the 127.0.0.1: prefix on the published port. Ollama in Docker has the Compose file.
The Open WebUI + Ollama template starts Ollama for you with its defaults, and the deploy page has no field for environment variables. To change server settings, run Ollama yourself on the Bare Metal template, as the two sections above do. The deploy docs cover the launch.
Check what the server read#
The server logs every setting once at startup, in a line whose message is server config. That line is the proof: it shows the value the server actually read for each variable, including the ones you did not set.
journalctl -u ollama --no-pager | grep "server config" | tail -n 1
sudo docker logs ollama 2>&1 | grep "server config" | tail -n 1
The first command is for the systemd service and the second for a container named ollama. If your value is missing, the variable was set in the wrong place or the server was not restarted. A name the server does not know never appears in the line at all.
How Ollama places a model across several GPUs#
Ollama puts a model on one GPU when it fits there, and spreads it across all of them only when it does not. Its FAQ gives the reason: a single GPU avoids moving data across the PCI bus during inference.
In 0.40.16 the scheduler estimates the model's memory as its file size plus a 16-bit KV cache for the context times the parallel slots. If that estimate is at most 80% of a GPU's free memory, the model goes on that GPU, and when several qualify, on the one with the most free memory. Otherwise Ollama hands every GPU to llama.cpp, which splits the model by layers. A log line with the message selecting single GPU for llama-server model names the GPU when one was chosen.
Three settings change that. OLLAMA_SCHED_SPREAD=1 spreads every model across all GPUs. CUDA_VISIBLE_DEVICES limits which GPUs Ollama sees, and Ollama's docs recommend UUIDs from nvidia-smi -L over index numbers, because the numbering can vary. The CUDA_VISIBLE_DEVICES guide covers the mask itself. And a longer context or more parallel slots raise the estimate, which can tip a model from one GPU to a split.
Two defaults also scale with the GPU count. The automatic context is chosen from the memory of all GPUs added together, so two 48 GB cards land in the 262,144-token tier even when each alone would not. And the loaded-model limit is 3 per GPU, so 6 on a 2-GPU VM. Multi-GPU configurations come and go in the catalog, and the GPU catalog shows what is listed now.
What I would set on a rented 48 GB card#
The rule I follow is to set the few variables whose defaults depend on luck or cost money, and leave the rest alone. On a single 48 GB card, such as an RTX A6000, that is this override:
[Service]
Environment="OLLAMA_CONTEXT_LENGTH=32768"
Environment="OLLAMA_KEEP_ALIVE=-1"
Environment="OLLAMA_NO_CLOUD=1"
| Variable | Value | Why |
|---|---|---|
OLLAMA_CONTEXT_LENGTH | 32768 | A 48 GB card sits on the line between two defaults. With ECC on it reports 46,068 MiB, under the 47 GiB threshold, and gets 32,768 tokens. With ECC off it reports 49,140 MiB and gets 262,144, capped at each model's own limit. A fixed value gives the same cache size on every instance |
OLLAMA_KEEP_ALIVE | -1 | The instance bills by the hour either way, so unloading after 5 idle minutes saves nothing and costs a reload |
OLLAMA_NO_CLOUD | 1 | If the point of the server is that prompts stay on it, this stops a cloud model tag from sending them elsewhere |
OLLAMA_NUM_PARALLEL | Leave at 1, or 2 to 4 for a small team | Each slot is a full context of KV cache. Four slots of 32,768 tokens are 8.0 GiB for qwen3.8:27b and 40.0 GiB for llama3.3:70b (our calculation) |
OLLAMA_KV_CACHE_TYPE | q8_0, only when the cache is what does not fit | Halves the cache for a very small precision loss, for every model on the server |
OLLAMA_HOST | Leave at the default | 0.0.0.0 puts an API with no login on the instance's public address. Reach it through an SSH tunnel instead |
OLLAMA_MODELS | Leave at the default | There is one disk and it is deleted on stop, so moving the directory gains nothing |
For agents and coding tools, raise the context instead of leaving it at 32,768: Ollama's docs suggest at least 64,000 tokens for that kind of work. Check the result with ollama ps after the first request. If the PROCESSOR column shows a CPU share, the model and its cache do not fit, and Ollama not using the GPU goes through the fixes in order.
Ollama settings FAQ#
Does Ollama have a config file?
Not for these settings. The server is configured with environment variables, and ~/.ollama/server.json holds only the switch that disables cloud features. Under systemd the nearest thing to a config file is the override at /etc/systemd/system/ollama.service.d/override.conf.
Where does Ollama store models on Linux?
In /usr/share/ollama/.ollama/models when it was installed with the official script, and in /root/.ollama/models inside the official Docker image. OLLAMA_MODELS changes it.
How do I keep a model loaded in Ollama?
Set OLLAMA_KEEP_ALIVE=-1 on the server, or send "keep_alive": -1 with a request. To unload a model before its time is up, run ollama stop with the model's name.
How many parallel requests can Ollama handle?
One per loaded model by default. OLLAMA_NUM_PARALLEL raises it, and the KV cache grows by the same factor, so check the memory first.
Why did my environment variable have no effect?
It was set in a shell instead of in the service or the container, or the server was not restarted. Look for your value in the server config line of the log.
My rule for Ollama's settings: set the context, the keep-alive and the cloud switch on purpose, read them back from the server config line, and leave the listen address on 127.0.0.1. Everything else is a default worth keeping until ollama ps or the log shows a reason to change it. Ollama GPU requirements sizes the models, and Open WebUI on QuantaCloud describes the template that runs Ollama with no setup.