# Self Host Llama 3 On Arch Linux

Technology is evolving very fast. Now a person without knowledge of how AI works can easily setup and host a [ChatGPT](https://chatgpt.com/)\-like service on personal computer. Llama is a large language model released by [Meta AI](https://ai.meta.com/meta-ai/). Meta has recently released Llama 3, making it available for download. I have set it up in my Arch Linux home lab, but the instruction is not distro-specific and be performed on other distros.

# How It Works

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1715268591588/0c6acd2e-5467-46f9-b921-9818a5374f10.png align="center")

[Ollama](https://ollama.com/) is an application that download different language models and provides CLI/API web service for interacting with the models. I am using it to download Llama 3.

Open WebUI provides a web UI for interacting with the model. It achieves this by integrating Ollama API.

**Prerequisite**: Make sure [docker](https://wiki.archlinux.org/title/docker) is installed, I use it to install & run Open WebUI container.

# Setup Ollama

Run the installer script.

```bash
curl -fsSL https://ollama.com/install.sh | sudo sh
```

The script will create an `ollama.service` systemd service and start it immediately. It also creates a group and user, both named "ollama". Ollama will be running as "ollama" user. You can check if Ollama service is running with `systemctl status ollama` command.

Reference: [https://github.com/ollama/ollama](https://github.com/ollama/ollama)

Download Llama 3.

```bash
ollama pull llama3
```

This downloads the 8B version. If you prefer 70B version, pull "llama3:70b" instead.

Run the model.

```bash
ollama run llama3
```

This launches a CLI for interacting with the model. You can start asking questions or request it to write a poem.

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1715270263897/2862ee3a-39cb-4ddf-bc40-af839aab1b64.png align="center")

`ollama.service` launches an API web service, which listens on port 11434 by default. Let's setup a web UI to interact with it.

# Web UI

```bash
docker run -d --network=host -v open-webui:/app/backend/data -e PORT=8080 -e OLLAMA_BASE_URL=http://127.0.0.1:11434 --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```

It downloads docker image and launches the container. `-e PORT=8080` is optional as it is running on port 8080 by default. You can change it to other port number if you want.

## Networking Issue

```bash
# Does not work for me
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
```

Bridged networking with port mapping does not work for me. (*If you insist on using bridged networking, check out*[*https://github.com/open-webui/open-webui/issues/209*](https://github.com/open-webui/open-webui/issues/209))

Now you can load the web UI on your browser: http://<server-ip>:8080. You have to signup a new user account. The first created user account will be assigned administrator role. Upon login, you can disable new signups.

In *Settings > Connections > Ollama Base URL*, make sure the value is "http://127.0.0.1:11434".

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1715271818073/98514789-0971-472a-94cd-abed71596f14.png align="center")

Start chatting...

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1715272084295/b110155b-c2d4-41da-836e-7583148f0388.png align="center")

# To Infinity And Beyond!

Ollama makes it easy to try out different models. This is a [list of supported models](https://ollama.com/library).

If you have a domain name and want to make it publicly accessible, you can setup a reverse proxy. This is a sample Apache configuration:

```apache
# Ensure these 2 modules are loaded: mod_proxy, mod_proxy_http
<VirtualHost *:80>
	ServerName example.com

    ProxyPreserveHost On
    ProxyPass / http://127.0.0.1:8080/
    ProxyPassReverse / http://127.0.0.1:8080/

    RewriteEngine on
    RewriteCond %{HTTP:Upgrade} websocket [NC]
    RewriteCond %{HTTP:Connection} upgrade [NC]
    RewriteRule ^/?(.*) "ws://127.0.0.1:8080/$1" [P,L]

    ErrorLog /var/log/httpd/error_log
    CustomLog /var/log/httpd/access_log common
</VirtualHost>
```