Setting up OpenLLM with Ollama Web UI

I set up OpenLLM with Ollama Web UI on my machine. Here is how I did it.

I used these tools:

A Windows 11 or Ubuntu 22.04 PC with at least 8GB of RAM (16GB recommended)
An Nvidia GPU with the official driver installed (optional, but helps a lot)
Docker

Install Docker

Follow the official Docker install docs for your system. On Windows, install Docker Desktop with WSL 2 backend. On Linux, use the repository install method.

After installing, add your user to the docker group and restart the service.

Enable GPU support

If you have an Nvidia GPU, install the Nvidia Container Toolkit. Follow the instructions in the official docs. This lets Docker use your GPU inside containers.

Run OpenLLM and Ollama Web UI

I created a directory for the setup:

mkdir $HOME/ollama && cd $HOME/ollama

Then I ran the Ollama container. This pulls the latest image and starts a container that listens on port 11434:

docker run --rm -p 11434:11434 ollama/ollama:latest bash -c 'ollama_train --model=13b_v1.2 --dataset=https://huggingface.co/datasets/Lmsysqa/test'

Next I ran the Ollama Web UI container. It connects to the Ollama API and provides a browser interface:

docker run --rm -p 8080:8080 -v $(pwd)/ollama-webui:/app/backend/data ghcr.io/ollama-webui/ollama-webui:main bash -c 'pip install uvicorn && uvicorn main:app'

Access the web UI

Open http://localhost:8080 in your browser. The web UI connects to the Ollama API running on port 11434 and lets you interact with models through a chat interface.

Using Docker Compose (recommended for production)

Docker Compose makes it easier to run both services together. Create a file called docker-compose.yml in your project directory:

version: '3.8'
services:

  ollama:
    image: ollama/ollama:latest
    container_name: ollama
    volumes:
      - $HOME/ollama/ollama:/root/.ollama
    ports:
      - 11434:11434
    pull_policy: always
    tty: true
    restart: unless-stopped
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

  ollama-webui:
    image: ghcr.io/ollama-webui/ollama-webui:main
    container_name: ollama-webui
    volumes:
      - $HOME/ollama/ollama-webui:/app/backend/data
    ports:
      - 3000:8080
    depends_on:
      - ollama
    environment:
      - '/ollama/api=http://ollama:11434/api'
    extra_hosts:
      - host.docker.internal:host-gateway
    restart: unless-stopped

Start everything with one command:

docker compose up

The Ollama API is at http://localhost:11434/api. The web UI is at http://localhost:3000.

To stop the containers, run docker compose down.

Shutting down manual containers

If you ran the containers without Compose, stop them with:

docker stop $(docker ps --format "{{.ID}}")

That covers the setup. OpenLLM and Ollama Web UI are running on your machine. Use the browser interface to chat with models.