In this procedure we install and configure an OpenHands agent connected to an LLM inference service. The OpenHands agent is hosted on a virtual machine (VM) under virt-manager and is connected to an inference service backend hosted by DeepInfra, serving the model Qwen3.5 397B-A17B, via OpenHands’ built-in LiteLLM integration. Tested on Ubuntu Desktop 24.04 LTS (Noble Numbat).

Note re Ubuntu Desktop 26.04: At the time of this writing there was a bug in the remote desktop server, making Ubuntu Desktop 26.04 unsuitable for this exercise.

Understanding OpenHands

OpenHands is a coding agent designed for software engineering tasks: reading and editing files, running shell commands, writing and running tests, opening pull requests (PRs). It can also be given a procedure document and a target VM and asked to execute the installation via SSH from inside its sandbox.

OpenHands web interface

The architecture has two components. The controller process manages the agent loop and handles LLM abstraction via its built-in LiteLLM. The sandbox is a Docker container spawned per task: the agent’s code, shell commands, and file writes all run inside the sandbox, isolated from the host VM.

KVM host          →  openhands01  (Ubuntu Desktop VM)
                       ├─ OpenHands controller  (port 3000, web UI)
                       │    └─ Built-in LiteLLM → DeepInfra → Qwen3.5-397B-A17B
                       └─ Docker sandbox containers  (spawned per task)

OpenHands vs OpenClaw vs Hermes

OpenClaw: sysadmin agent, browser automation, SSH deployments, approval gates.
Hermes: sysadmin agent, cron scheduling, persistent memory, SSH backends, reusable skills.
OpenHands: coding agent, file editing, test running, SSH deployments from sandbox. Docker required. LiteLLM built in: no separate proxy needed. No skills system. No persistent memory across tasks. Reusable procedures are documents in the workspace directory.

Creating a DeepInfra account

OpenHands connects directly to DeepInfra through its built-in LiteLLM. You only need one API key.
Navigate to https://deepinfra.com and create an account.
Go to Dashboard → Billing / Credits → Add Credit. Add US $10.
Navigate to Dashboard → API Keys → Create API Key. Copy the key.

Verifying the key with curl

curl https://api.deepinfra.com/v1/openai/chat/completions \
  -H 'Authorization: Bearer YOUR_DEEPINFRA_KEY' \
  -H 'Content-Type: application/json' \
  -d '{
    "model": "Qwen/Qwen3.5-397B-A17B",
    "messages": [{"role": "user", "content": "Say hello in French."}]
  }'

Expected: valid JSON with a choices array. Do not proceed until this returns a valid response.

Creating the virtual machine

In this procedure, OpenHands runs inside an Ubuntu Desktop 24.04 Noble Numbat) VM on a server running virt-manager/KVM/Qemu. You can use other virtualization platforms, such as VMware, Hyper-V, Proxmox, Parallels, or a cloud platform (VirtualBox is not recommended for performance reasons). Docker runs inside the VM. Ubuntu Desktop is used so you can watch the web UI on screen. The virtual machine is accessed by SSH and remote desktop software such as remmina.

Specifications for the virtual machine

Ubuntu Desktop 24.04 ISO.
RAM: 16384 MB, CPU: 4 vCPU, Disk: 100 GB.
Hardware: CPU model host-passthrough, Disk bus VirtIO, NIC VirtIO bridged, Display Spice or QXL.

Notes: Configure the virtual machine with a static IP address, enable SSH and remote desktop access. enable auto-login and disable screen lock and screen timeouts. Hint: enter the command sudo ufw allow 3389/tcp to expose the remote desktop through the virtual machine’s firewall.

Creating a snapshot (“fresh-install”)

Create a snapshot called fresh-install. for virt-manager, the commands from the hypervisor host are:

(from the hypervisor host)

virsh shutdown openhands01 && virsh domstate openhands01
virsh snapshot-create-as openhands01 fresh-install \
  --description 'Clean Ubuntu Desktop, no packages' --atomic
virsh start openhands01

Setting up the base OS

Updating the system

sudo apt update
sudo apt full-upgrade -y
sudo reboot

Installing dependencies

sudo apt install -y \
  curl git jq htop tmux \
  build-essential ca-certificates gnupg \
  python3 python3-pip python3.12-venv \
  openssh-client openssh-server \
  qemu-guest-agent

Installing Google Chrome

Do not install chromium-browser via apt: it installs a snap stub. Install the official Chrome deb:

wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
sudo dpkg -i google-chrome-stable_current_amd64.deb
sudo apt --fix-broken install -y
google-chrome --version

Enabling services and rebooting

sudo systemctl enable --now qemu-guest-agent
sudo systemctl enable --now ssh
sudo reboot

Creating a snapshot (“base-os”)

Create a snapshot called base-os. For virt-manager, the commands from the hypervisor host are:

(from the hypervisor)

virsh shutdown openhands01 && virsh domstate openhands01
virsh snapshot-create-as openhands01 base-os \
  --description 'Base OS packages installed, pre-Docker' --atomic
virsh start openhands01

Installing Docker

Docker is required. OpenHands spawns sandbox containers for each task via the Docker socket. Without it the agent loop fails immediately.

Installing Docker Engine

sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | \
  sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
 
echo \
  "deb [arch=$(dpkg --print-architecture) \
  signed-by=/etc/apt/keyrings/docker.gpg] \
  https://download.docker.com/linux/ubuntu \
  $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
 
sudo apt update
sudo apt install -y docker-ce docker-ce-cli containerd.io docker-buildx-plugin
sudo systemctl enable --now docker
sudo docker run hello-world

Adding desktop to the docker group

sudo usermod -aG docker desktop

Log out and back in is required

Docker group membership does not take effect in existing shell sessions. Log out of the desktop session completely and log back in. Verify with: docker run hello-world (no sudo). If you skip this step openhands serve will fail with permission denied on the Docker socket.

Creating a snapshot (“docker-installed”)

Create a snapshot called fresh-install. for virt-manager, the commands from the hypervisor host are:

(from the hypervisor host)

virsh shutdown openhands01 && virsh domstate openhands01
virsh snapshot-create-as openhands01 docker-installed \
  --description 'Docker installed and verified' --atomic
virsh start openhands01

Installing uv and OpenHands

OpenHands is installed via uv. Do not use pip install openhands-ai that is the legacy V0 SDK and will not work.

Installing uv

curl -LsSf https://astral.sh/uv/install.sh | sh
source ~/.bashrc
uv --version

Installing OpenHands

uv tool install openhands --python 3.12
openhands --version

Creating workspace and SSH directories

mkdir -p ~/openhands-workspace
mkdir -p ~/.openhands
mkdir -p ~/openhands-workspace/.ssh

Creating a snapshot (“post-install”)

Create a snapshot called base-os. For virt-manager, the commands from the hypervisor host are:

(from the hypervisor)

virsh shutdown openhands01 && virsh domstate openhands01
virsh snapshot-create-as openhands01 openhands-installed \
  --description 'OpenHands installed via uv, pre-config' --atomic
virsh start openhands01

Starting OpenHands for the first time

Run openhands serve once manually to pull the Docker images. This must happen in an interactive terminal session before the autostart is configured.

cd ~/openhands-workspace
openhands serve

On first run this pulls the openhands Docker image: several gigabytes. It will appear to hang at Pulling for several minutes. Wait for:

✅ Starting OpenHands GUI server...
The server will be available at: http://localhost:3000

Do not kill the process during image pull

The first run pulls several GB of Docker images. If it appears to hang at Pulling for 5-10 minutes this is normal. Wait for the startup confirmation before opening the web UI.

Open the web UI on the openhands01 desktop:

http://localhost:3000

Configuring the model

The Settings page appears on first launch. Navigate to Settings → LLM → click the Advanced tab.

How to access Settings

The gear icon in the sidebar navigates to a broken route (/automations) in this version: ignore it. Access settings directly at: http://localhost:3000/settings

Entering the model details

Click Save Changes. Settings are stored in ~/.openhands/settings.json.

The openai/ prefix is required

The openai/ prefix tells LiteLLM to use the OpenAI-compatible request format. Without it, LiteLLM auto-detects the wrong format and produces cryptic 400 errors. Always enter openai/Qwen/Qwen3.5-397B-A17B.
The Basic tab shows named providers only: DeepInfra is not listed. The Advanced tab is the correct path.

Testing the connection

Start a new conversation and send:

Say hello in French.

Expected: Bonjour. The first response may take 30-60 seconds while the sandbox container starts. If the agent produces no response at all, see next section.

Streaming troubleshooting

If the agent produces no response or an empty response, DeepInfra’s streaming implementation may be causing empty SSE responses: the same issue that affected OpenClaw and Hermes. Add disable_streaming = true to ~/.openhands/config.toml:

nano ~/.openhands/config.toml

[llm]
disable_streaming = true

Restart OpenHands and test again. If disable_streaming = true resolves it, leave it in place, it has no effect on agent capability.

To restart:

# Ctrl+C in the terminal running openhands serve, then:
openhands serve

Persisting across reboots

openhands serve calls docker run -it which requires an interactive TTY. Neither rc.local nor systemd services provide a TTY at boot time: both fail with ‘cannot attach stdin to a TTY-enabled container’. The reliable solution for a Desktop VM accessed via RDP is a GNOME desktop autostart entry, which fires when the desktop session starts and has a proper TTY context.

Creating the autostart entry

Stop openhands serve first (Ctrl+C), then:

mkdir -p ~/.config/autostart
nano ~/.config/autostart/openhands.desktop

Paste the following:

[Desktop Entry]
Type=Application
Name=OpenHands
Exec=/home/desktop/.local/bin/openhands serve
Hidden=false
NoDisplay=false
X-GNOME-Autostart-enabled=true

Why desktop autostart and not rc.local or systemd

openhands serve uses docker run -it which requires a controlling terminal. rc.local and systemd both start without a terminal and fail immediately. The GNOME autostart mechanism runs after the desktop session initialises: it has a full terminal environment and docker run -it succeeds.

Rebooting and verifying

sudo reboot

Log back into openhands01 via RDP. Wait one minute for OpenHands to start automatically, then verify:

docker ps | grep openhands
curl http://localhost:3000

Open the web UI at http://localhost:3000 and confirm the model responds.

Creating a snapshot (“post-rebooting”)

Create a snapshot called base-os. For virt-manager, the commands from the hypervisor host are:

(frm the hypervisor)

virsh shutdown openhands01 && virsh domstate openhands01
virsh snapshot-create-as openhands01 openhands-configured \
  --description 'OpenHands configured, working, reboot-verified' --atomic
virsh start openhands01

Watching the browser work

OpenHands browser automation runs via Playwright inside the Docker sandbox container. There is no visible Chrome window on the openhands01 desktop: the sandbox has no display. All browser activity is observable through the web UI event log, where every navigation, click, and screenshot appears inline in the conversation.

There is no browser toggle in Settings, browser capability is built into the sandbox and always available in this version.

Testing browser automation

Start a new conversation in the web UI and submit:

Go to https://example.com and tell me the page title and main heading.

Watch the event log on the right side of the UI. You should see browser_navigate, browser_snapshot, and the page content returned inline.

No visible Chrome window: this is expected

OpenHands browser automation is headless inside the Docker sandbox. This is different from Hermes and OpenClaw where Chrome opens visibly on the desktop. The web UI event log and inline screenshots give full visibility into what the agent is doing.

Giving the sandbox SSH access to a remote VM

When OpenHands SSHes to a remote VM, the connection runs from inside the sandbox container. The sandbox needs a key that the target trusts as root. Place the key inside the workspace directory: mounted into the sandbox at /workspace: and authorize it on the target before starting the task.

Generating a dedicated key pair

ssh-keygen -t ed25519 -C 'openhands-sandbox' \
  -f ~/openhands-workspace/.ssh/openhands_sandbox
# Press Enter twice: no passphrase

Authorizing the key on the target VM

ssh-copy-id -i ~/openhands-workspace/.ssh/openhands_sandbox.pub root@192.168.x.x
 
# Verify passwordless root access
ssh -i ~/openhands-workspace/.ssh/openhands_sandbox root@192.168.x.x 'uname -a'

Root access via SSH key

The sandbox needs root on the target to install packages, configure services, and write to system directories. This key is scoped to OpenHands sandbox use: revoke it from authorized_keys on the target when the deployment is complete.

Pre-populating known_hosts

Pre-populate to prevent SSH host key prompts stalling the agent mid-task:

ssh-keyscan -H 192.168.x.x >> ~/openhands-workspace/.ssh/known_hosts
chmod 600 ~/openhands-workspace/.ssh/known_hosts

Running a deployment task

Provide to OpenHands a procedure document and a target VM and let it execute the installation.

Placing the procedure in the workspace

cp /path/to/procedure.md ~/openhands-workspace/

The task prompt

Submit this in the web UI: the prompt must be entirely self-contained since each sandbox starts fresh with no memory:

Read the procedure in /workspace/procedure.md.
 
Connect to the target VM:
  Host: 192.168.x.x
  User: root
  SSH key: /workspace/.ssh/openhands_sandbox
  Known hosts: /workspace/.ssh/known_hosts
 
Execute the procedure exactly as written.
Verify each step succeeds before continuing.
If a step fails, stop and report the error.
 
When complete, provide a full status report:
what was installed, current state of each service,
and any issues encountered.

Self-contained prompts are mandatory

Each OpenHands task starts in a fresh sandbox with no memory of prior sessions. Include the target host, SSH key path, and procedure reference in every prompt.

Running a first coding task

Create a test file and give the agent a scoped coding task to verify the full stack:

cat > ~/openhands-workspace/utils.py << 'EOF'
def add(a, b):
    return a + b
 
def multiply(a, b):
    return a * b
EOF

In the web UI, submit:

Add docstrings to both functions in utils.py.
Write a test file test_utils.py with pytest tests for both functions.
Run the tests and confirm they pass.

Watch the event log. You should see the agent read utils.py, write docstrings, create test_utils.py, and run pytest inside the sandbox. Files are visible in ~/openhands-workspace on the host after the task completes.

Verifying that the system survives a reboot

After the autostart is configured (section 9), reboot survival is automatic: log in via RDP and OpenHands starts. Confirm:

# Docker starts on boot
systemctl is-enabled docker   # Expected: enabled
 
# Autostart entry is in place
cat ~/.config/autostart/openhands.desktop

After any reboot, log back in via RDP, wait one minute, then:

docker ps | grep openhands
curl http://localhost:3000

Known issues and fixes

Snapshot reference

Maintaining the system ongoing

Daily usage

OpenHands starts automatically when the gnome desktop session is started at bootup with autologin. Open a browser on the openhands01 desktop and navigate to http://localhost:3000.

To hand the agent a procedure document and a target VM:

# Place the procedure in the workspace
cp procedure.md ~/openhands-workspace/
 
# Pre-populate known_hosts for the target
ssh-keyscan -H 192.168.x.x >> ~/openhands-workspace/.ssh/known_hosts
 
# Open http://localhost:3000 and submit the task prompt

For a project session:

cd ~/myproject
openhands serve --mount-cwd

–mount-cwd for quick sessions

openhands serve –mount-cwd mounts the current directory into the sandbox as /workspace. Use this for quick one-off sessions with a specific project without copying files into the workspace directory.

In this procedure, we create a Python script that connects to a large language model (LLM) to facilitate the creation of Ansible playbooks using natural language queries. We begin by creating the Ansible deployment environment on a staging server, with a hosts file and inventory.ini file that correspond to a group of servers. We enable passwordless entry from the staging server to the servers, and ensure that visudo is configured for passwordless escalation to root using sudo.

The system does not run the generated playbook; the human operator must review it and then execute it manually.

A preview of the system in operation

To use the system, an operator starts the playbook generator at the command line:

python3 playbook_generator.py

The operator enters a natural language query, such as:

CLI> install net-tools on node03

The system generates a playbook:

📌 Saving Playbook: playbook_20250320123755.yml
✅ Playbook saved: playbook_20250320123755.yml
CLI> quit

The operator runs the playbook:

(llmansible_env) root@node01:/opt/llmansible# ansible-playbook -i inventory.ini playbooks/playbook_20250320123755.yml

PLAY [Install net-tools on node03] ***

TASK [Gathering Facts] *
ok: [node03]

TASK [Install net-tools] *
changed: [node03]

PLAY RECAP *
node03 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

Escalating to root using sudo su

Enter the following command:

sudo su

Configuring the staging server

On the staging server, enter the following commands:

apt clean && update && sudo apt upgrade -y<br>reboot

Enter the following commands:

sudo su
apt install -y ansible python3-pip tmux python3-venv \
openssh-client pkg-config

cd /etc
notepad hosts

Use the nano editor to add the following text to the file. Modify IP address values as appropriate for your setup:

127.0.1.1 node01
192.168.56.82 node02
192.168.56.83 node03
192.168.56.84 node04
192.168.56.85 node05

Save and exit the file.

Preparing the nodes for automation

On each node, ensure that the server has a host name, static IP address, and has visudo configured for passwordless escalation to root with sudo su.

For each node:

Enter the following commands:

sudo su
visudo

Use the nano editor to add the following text to the bottom of the file (substitute your login name):

desktop ALL=(ALL:ALL) NOPASSWD:ALL

Save and exit the file.

Generating SSH keys

On the staging server, enter the following commands:

sudo su
ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rsa -N ""

Copying the public SSH key to all nodes

Enter the following commands (substitute login username and server name as appropriate for your installation.

ssh-copy-id -i ~/.ssh/id_rsa.pub desktop@node02
ssh-copy-id -i ~/.ssh/id_rsa.pub desktop@node03
ssh-copy-id -i ~/.ssh/id_rsa.pub desktop@node04
ssh-copy-id -i ~/.ssh/id_rsa.pub desktop@node05

Verifying that passwordless login via SSH is working

Enter the following commands (modify login user name and host name as appropriate to match your installation):

ssh desktop@node02
exit

Creating the inventory.ini file

Enter the following commands:

cd /opt
mkdir llmansible
cd llmansible
nano inventory.ini

Use the nano editor to add the following text to the file:

[all]
node02 ansible_host=192.168.56.82 ansible_user=desktop
node03 ansible_host=192.168.56.83 ansible_user=desktop
node04 ansible_host=192.168.56.84 ansible_user=desktop
node05 ansible_host=192.168.56.85 ansible_user=desktop
[all:vars]
ansible_ssh_private_key_file=~/.ssh/id_rsa
ansible_python_interpreter=/usr/bin/python3

Save and exit the file.

Creating the config.ini file

Enter the following command:

nano config.ini

Use the nano editor to add the following text to the file (modify values as appropriate to match your installation):

[LLM]
system_prompt = You are an AI that generates structured Ansible playbooks. Ensure: The playbook is idempotent. It installs required packages using apt (for Ubuntu) or yum (for CentOS). It does not execute shell commands directly. It follows proper YAML formatting. assume that OS is ubuntu unless otherwise stated. only respond with the contents of the ansible playbook, nothing more. do not offer multiple playbooks. do not add commentary.
api_url = https://api.lemonfox.ai/v1/chat/completions
api_token = your-api-token
model_name = llama-8b-chat

Save and exit the file.

Creating a Python virtual environment (venv), and adding Python dependencies

Enter the following commands:

cd /opt/llmansible
python3 -m venv llmansible_env
source llmansible_env/bin/activate

Adding Python dependencies using pip

Enter the following command:

pip install PyYAML requests

Creating the playbook_generator.py file

Enter the following command:

nano playbook_generator.py

Use the nano editor to add the following text to the file:

# MIT license Gordon Buchan 2025
# see https://opensource.org/license/mit
# some of the code was generated with the assistance of AI tools.

import requests
import configparser
import os
import re
import yaml
from datetime import datetime

# Load Configuration
config = configparser.ConfigParser()
config.read("config.ini")

LLM_API_URL = config.get("LLM", "api_url")
LLM_API_TOKEN = config.get("LLM", "api_token")
LLM_MODEL = config.get("LLM", "model_name")
SYSTEM_PROMPT = config.get("LLM", "system_prompt")

PLAYBOOK_DIR = "/opt/llmansible/playbooks"

def extract_yaml(text):
    """Extracts valid YAML content and removes explanations or malformed sections."""

    # Remove Markdown-style code block markers (e.g., ```yaml)
    text = re.sub(r"```(yaml|yml)?", "", text, flags=re.IGNORECASE).strip()

    # Capture the first YAML block (ensuring it's well-formed)
    match = re.search(r"(?s)(---\n.+?)(?=\n\S|\Z)", text)

    if match:
        yaml_content = match.group(1).strip()

        # Remove trailing incomplete YAML lines or explanations
        yaml_content = re.sub(r"\n\w+:\s*\"?[^\n]*$", "", yaml_content).strip()

        # Validate extracted YAML before returning
        try:
            yaml.safe_load(yaml_content)  # If this fails, the YAML is invalid
            return yaml_content
        except yaml.YAMLError as e:
            print(f"❌ YAML Validation Error: {e}")
            return ""

    print("❌ Error: No valid YAML found.")
    return ""

def query_llm(prompt):
    """Queries the LLM API and extracts a valid Ansible playbook."""
    payload = {
        "model": LLM_MODEL,
        "messages": [
            {"role": "system", "content": SYSTEM_PROMPT},
            {"role": "user", "content": prompt}
        ],
        "max_tokens": 2048
    }

    headers = {
        "Authorization": f"Bearer {LLM_API_TOKEN}",
        "Content-Type": "application/json"
    }

    response = requests.post(LLM_API_URL, json=payload, headers=headers)

    print("\n🔍 API RAW RESPONSE:\n", response.text)

    try:
        response_json = response.json()
        llm_response = response_json["choices"][0]["message"]["content"]
        yaml_content = extract_yaml(llm_response)

        if not yaml_content:
            print("❌ Error: No valid YAML extracted.")
            return ""

        print("\n✅ Extracted YAML Playbook:\n", yaml_content)
        return yaml_content

    except (KeyError, IndexError):
        print("❌ Error: Unexpected API response format.")
        return ""

def save_playbook(machine, command, playbook_content):
    """Saves the extracted YAML playbook if it's valid."""
    if not playbook_content.strip().startswith("---"):
        print("❌ Error: Extracted content is not a valid Ansible playbook. Skipping save.")
        return None

    timestamp = datetime.now().strftime("%Y%m%d%H%M%S")
    playbook_name = f"playbook_{timestamp}.yml"
    playbook_path = os.path.join(PLAYBOOK_DIR, playbook_name)

    os.makedirs(PLAYBOOK_DIR, exist_ok=True)

    print(f"\n📌 Saving Playbook: {playbook_name}")

    with open(playbook_path, "w") as f:
        f.write(playbook_content)

    return playbook_name

def main():
    """CLI Operator Console."""
    while True:
        user_input = input("CLI> ")
        if user_input.lower() in ["exit", "quit"]:
            break

        llm_response = query_llm(user_input)
        if llm_response:
            playbook_name = save_playbook("all", user_input, llm_response)
            if playbook_name:
                print(f"✅ Playbook saved: {playbook_name}")

if __name__ == "__main__":
    main()

Save and exit the file.

Creating an Ansible playbook using the playbook generator

Ensuring that you are in the Python venv

Ensure that you are already in the Python venv. If not, enter the following commands:

cd /opt/llmansible
source llmansible_env/bin/activate

Enter the following command:

python3 playbook_generator.py

The operator enters a natural language query, such as:

CLI> install net-tools on node04
🔍 API RAW RESPONSE:
{"id":"chatcmpl-923543777fb7a294","object":"chat.completion","created":1742474275313,"model":"llama-8b-chat","choices":[{"index":0,"message":{"content":"---\n- name: Install net-tools on node04\n hosts: node04\n become: yes\n tasks:\n - name: Install net-tools\n apt:\n name: net-tools\n state: present","role":"assistant"},"finish_reason":"stop"}],"usage":{"prompt_tokens":130,"completion_tokens":47,"total_tokens":177}}

✅ Extracted YAML Playbook:

name: Install net-tools on node04
hosts: node04
become: yes
tasks:

name: Install net-tools
apt:
name: net-tools
state: present

📌 Saving Playbook: playbook_20250320135023.yml
✅ Playbook saved: playbook_20250320135023.yml
CLI> quit

The operator runs the playbook:

(llmansible_env) root@node01:/opt/llmansible# ansible-playbook -i inventory.ini playbooks/playbook_20250320135023.yml

PLAY [Install net-tools on node04] ***

TASK [Gathering Facts] *
ok: [node04]

TASK [Install net-tools] *
changed: [node04]

PLAY RECAP *
node04 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0

In this post, we install the Ollama LLM hosting software, and load a large language model (LLM), a 5GB file produced by a company called Mistral. We then test local inference, interacting with the model at the command line. We send test queries to the application protocol interface (API) server. We install an application called Open-WebUI that enables a web chat interface to the LLM.

Note: this procedure references the mistral model. however, you can specify other models, such as dolphin-mistral. Consult the following page for available models. Try to limit your choices to 7B complexity, unless you have a GPU.

https://ollama.com/library?sort=newest

Using the CPU servers we have now

Until 2023, graphical processing units (GPUs) were only of interest to video gamers, animators, and mechanical designers. There is now an imperative for GPU resources on most new servers going forward, for local inference and retrieval augmented generation (RAG). However we will need to devise an interim approach to use the CPU-centric servers we have, even for some AI inference tasks, until the capex cycles have refreshed in 3-4 years from now. On a CPU-only system, the system response time for a query can range from 2-5 seconds to 30-40 seconds. This level of performance may be acceptable for some use cases, including scripted tasks for which a 40 second delay is not material. Deploying this solution on a system with even a modest Nvidia GPU will result in dramatic increases in performance.

Why host an LLM locally

• To learn how LLMs are built
• To achieve data sovereignty by operating on a private system
• To save expense by avoiding the need for external LLM vendors

Preparing a computer for deployment

This procedure was tested on Ubuntu Server 24.04. Baremetal is better than a virtual machine for this use case, allowing the software to access all of the resources of the host system. In terms of resources, you will need a relatively powerful CPU, like an i7, and 16-32GB of RAM.

Note: the version of Python required by Open-WebUI is Python 3.12, which is supported by default in Ubuntu Server 24.04 LTS. If you are on an older version of the operating, you can install a newer version of Python using a PPA.

Do you need a GPU?

No, but a GPU will make your inference noticeably faster. If you have an Nvidia GPU, ensure that you have Nvidia CUDA drivers enabled. If you have an AMD GPU, ensure that you have AMD ROCM drivers. There is some talk of support for Intel GPUs but none of it is yet practical.

Note: if you have an Nvidia GPU, you may want to consider vLLM.

Ollama is able to work on a CPU-only system

Ollama is able to work on a CPU-only system, and that is what we will implement in this post. Ollama is able to achieve performance that may be acceptable for certain kinds of operations. For example, large batch operations that run overnight, that can accept a 30-60 second delay versus 2-10 seconds for a GPU-driven solution. For some questions, like “why is the sky blue?’ an answer will start immediately. For more complex questions, there may be a 5-10 second delay before answering, and the text will arrive slowly enough to remind you of 300 baud modems (for those of you who get that reference). The wonder of a dancing bear is not in how well it dances, but that it dances at all. This level of performance may be acceptable for some use cases, in particular batched operations and programmatic access via a custom function invoking commands sent to the API server.

Escalating to root using sudo

From a shell, enter the following command:

sudo su

(enter the password when requested)

Opening ports in the UFW firewall

You may need to open ports on the UFW firewall to enable the chat client.

Enter the following commands:

ufw allow 11434/tcp
ufw allow 8080/tcp

Ensuring that the system is up to date

Enter the following commands:

apt clean
apt update
apt upgrade
apt install curl python3-venv python3-pip ffmpeg

A note re RHEL and variants like Fedora and AlmaLinux

Although this procedure has not been tested on RHEL and variants like Fedora and AlmaLinux, I looked at the installation script and those platforms are supported. In theory, you could configure an RHEL-type system by using equivalent firewall-cmd and dnf commands.

Installing Ollama using the installation script

Ollama provides an installation script that automates the installation. From a shell as root, enter the following command:

curl -fsSL https://ollama.com/install.sh | sh

Pulling the mistral image

Enter the following command:

ollama pull mistral

Listing the images available

Enter the following command:

ollama list

Testing Ollama and the LLM using the command line

Enter the following command. Test the chat interface on the command line in the shell:

ollama run mistral

Testing the API server using curl

Enter the following commands:

systemctl restart ollama
systemctl status ollama
systemctl enable ollama
curl http://localhost:11434/api/generate -d '{
"model": "mistral",
"prompt":"Why is the sky blue?"
}'

Enter the following command:

curl http://localhost:11434/api/chat -d '{
"model": "mistral",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'

Preparing the system for Open-WebUI

To prepare the system for Open-WebUI, we must create a working directory, and create a venv (virtual python environment).

Enter the following commands:

cd ~
pwd
mkdir ollamatmp
cd ollamatmp
python3 -m venv ollama_env
source ollama_env/bin/activate
pip install open-webui

Starting the open-webui serve process

Enter the following command:

open-webui serve

Visiting the Open-WebUI web page interface

Using a web browser, visit this address:

http://127.0.0.1:8080

You will be prompted to create an admin account:

Using the Open-WebUI chat interface

This window took 30 seconds to begin showing its answer, then another 20 seconds to complete generating the answer:

Using nginx as a proxy to expose the API port to the local network

By default, the Ollama API server answers on port 11434 but only on the local address 127.0.0.1. You can use nginx as a proxy to expose the API to the local network. Enter the following commands:

ufw allow 8085/tcp
apt install nginx
cd /etc/nginx/sites-enabled
nano default

Use the nano editor to add the following text:

server {
    listen 8085;

    location / {
        proxy_pass http://127.0.0.1:11434;  # Replace with your Ollama API port
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_cache_bypass $http_upgrade;

        # Optional: Add timeout settings for long-running API calls
        proxy_connect_timeout 60s;
        proxy_read_timeout 60s;
        proxy_send_timeout 60s;
    }
}

Save and exit the file.

Enter this command:

systemctl restart nginx

Testing the exposed API port from another computer

From another computer, enter the command (where xxx.xxx.xxx.xxx is the IP address of the computer hosting the Ollama API server):

curl http://xxx.xxx.xxx.xxx:8085/api/chat -d '{
"model": "mistral",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'

Creating a systemd service to start the Open-WebUI chat interface automatically

Enter the following command:

nano /etc/systemd/system/open-webui.service

Use the nano editor to add the following text:

[Unit]
Description=Open-WebUI Service
After=network.target

[Service]
User=root
WorkingDirectory=/root/ollamatmp/ollama_env
ExecStart=/usr/bin/bash -c "source /root/ollamatmp/ollama_env/bin/activate && open-webui serve"
Restart=always
Environment=PYTHONUNBUFFERED=1
StandardOutput=journal
StandardError=journal

[Install]
WantedBy=multi-user.target

Save and exit the file.

Enter the following commands:

systemctl daemon-reload
systemctl start open-webui
systemctl enable open-webui
systemctl status open-webui

Conclusion

You now have an LLM API server, and a web chat for interactive access.

References

https://ollama.com

https://github.com/ollama/ollama

A related post

You may find the following post to be of interest: Creating a script that analyzes email messages messages using a large language model (LLM), and where appropriate escalates messages to the attention of an operator.

In this post we install and configure an instance of Ubuntu Server under WSL2 for Linux. Many developers are going to choose to use WSL2, so we guide them to install the Ubuntu Server version of WSL2. This provides a more standard environment referenced by many howto procedures on the Internet.

WSL2 enables a computer running Windows 11 Pro to host a guest instance of Ubuntu Server:

WSL2 offers some advantages for a developer experience

WSL2 offers some advantages for a local developer in terms of networking: if they install a service that opens a port, like port 22/tcp for SSH, that same port is opened on localhost of the Windows machine hosting the WSL2 Ubuntu Server instance. A developer can SSH to port 22 on localhost, without configuring a static IP address or port forwarding. This eliminates the need to configure virtual switches, static IP addresses, and port forwarding. In addition, WSL2 provides a file mounts of the Windows file system within Linux, and a file mount of the Linux filesystem within Windows.

Other ways to install Ubuntu Server on a Windows 11 Pro computer

There are other ways to install Ubuntu Server on a Windows 11 Pro computer, including Windows Hyper-V. If you need to host multiple instances of Ubuntu Server with static IP addresses and subnet routing, consider using Hyper-V instead.

Installing WSL2

Start a CMD window as Administrator. At the Start menu, type the letters “cmd” then right-click on the app icon to run the command prompt as root:

Enter the following command:

powershell

Enter the following command:

wsl --install -d Ubuntu-24.04

Reboot your computer.

Enter values for username and password:

Enter the following command from a CMD window running as Administrator:

wsl --list --verbose

Enter the following command:

wsl --setdefault Ubuntu-24.04

Enter the following command:

wsl

Enter the following command:

lsb_release -a

Enter the following command:

sudo su

Updating apt packages on the Linux system

Enter the following commands:

apt clean
apt update

Enter the following command:

apt upgrade

Enter y for yes:

Enter the following command:

reboot

from a CMD window running as Administrator, enter the following command:

wsl

Installing some utilities

Enter the following commands:

sudo su
apt install net-tools git build-essential

Installing openssh-server

Enter the following command:

apt install openssh-server

From a CMD window running as Administrator, enter the following command:

ssh localhost

From KiTTY SSH terminal:

Create and save a session called localhost pointing to the host localhost:

Click on “Accept”:

Accessing the Windows file system within Linux

Look at the mountpoint:

/mnt/c

Accessing the Linux file system within Windows

From File Explore on Windows, look for the Linux device icon:

Exposing the SSH port on the public IP address of the Windows computer

From a CMD window run as Administrator, enter the following command:

ipconfig /all | more

Look through the listing to find the public IP address of the Windows computer:

From a shell on the Linux instance, enter the following command:

ifconfig | less

Look through the listing to find the public IP address of the Linux instance:

Opening a firewall exception for port 22/tcp (SSH) inbound on the Windows computer

Click on “New Rule…”:

Select “Port”, click on “Next”:

Select “TCP”. Select “Specify local ports”, enter the value 22. Click Next:

Select “Allow the connection”, click “Next”:

Ensure that “Domain”,”Private”,”Public” are selected. Click “Next”:

For “Name” enter the value: in22tcp

For “Description (Optional):” enter the value: in22tcp

Click “Finish”:

Allowing connections to port 22/tcp of the Linux instance via the public IP address of the Windows Computer

From a CMD window running as Administrator, enter the following commands (subtitute appropriate values for windowsip and wsl2ip:

netsh interface portproxy add v4tov4 listenaddress=windowsip listenport=22 connectaddress=wsl2ip connectport=22

netsh interface portproxy show all

In this post, we build a home server using Ubuntu Desktop Linux that includes a Samba network file server, an OpenVPN virtual private network (VPN), and a KVM hypervisor hosting virtual machine (VM) guests including a Linux/Apache/MySQL/PHP web server. In addition, we build an offsite backup server using Fedora Server Linux, and link the offsite backup server to the home server via a WireGuard secure network tunnel.

Tasks for the home server

The home server will perform the following tasks:

• Samba network file server
• OpenVPN server
• KVM hypervisor to host virtual machine (VM) guests
• LAMP web server in a VM
• Wireguard public-facing host to client connection to offsite backup server

Hardware for the home server

As the home server will host virtual machine guests, I need a certain level of performance, so I bought a refurbished circa 2017 computer for C$403 (US$294). CPU performance can affect OpenVPN performance, so the VPN server will benefit from a stronger CPU as well.

The home server is a small form factor (SFF) desktop circa 2017:

• Dell OptiPlex 5050 SFF (circa 2017)
• i7-7700 CPU
• 32GB DDR4 RAM
• 1TB SATA SSD

Formatting and configuring the home server with Ubuntu Desktop 22.04 LTS

I formatted the home server with Ubuntu Desktop 22.04 LTS.

Why choose Ubuntu Desktop instead of Ubuntu Server?

For the home server, I wanted the option of a graphical user interface (GUI) desktop for use at console, and via remote desktop. A GUI desktop is also more convenient for the creation and management of KVM virtual machine guests using the virt-manager GUI, (and avoids the need for SSH tunnel forwarding and an X11 server to reach a headless server).

Formatting in UEFI mode

With modern hardware, I like to use UEFI mode for disk booting. Although we do not need a multiple-boot menu for this server, it is easier to construct a multiple-boot menu using grub when booting in UEFI mode. This is the default on a post-2016 motherboard, but it is worth looking at the BIOS when you first lay hands on a machine.

Connecting using wired Ethernet

We need a wired Ethernet connection for the home server, as we want to create a bridge mode adapter (br0) so that virtual machine (VM) guests can have IP addresses in the host networking subnet.

Downloading Ubuntu desktop Linux

https://ubuntu.com/download/desktop

Using the Rufus USB utility under Windows to write the installer

If you are writing the installer to a USB using Windows, consider using Rufus:

https://rufus.ie

Installing Ubuntu on the home server

Click on “Install Ubuntu”:

Click on “Continue”:

Click on “Continue”:

Click on “Install Now”:

Click on “Continue”:

Select a time zone. Click “Continue”:

Complete the fields as needed, then click on “Continue”:

Click on “Restart Now”:

Press the ENTER key on your keyboard:

Using the nmcli command to create a bridge mode adapter (br0)

Because we are working on an Ubuntu desktop, we will use the nmcli command to create a bridge mode adapter (br0).

Open a terminal window. Enter the following commands:

sudo su
apt install net-tools bridge-utils
ifconfig

Look at the information displayed by the ifconfig command. Identify the name of the wired Ethernet connection. The name may be “eth0” or a string such as “enp0s31f6”

Use the value you identified above and use it in place of ethernet_name.

Enter the following commands:

nmcli con add ifname br0 type bridge con-name br0
nmcli con add type ethernet ifname ethernet_name master br0
nmcli con up br0
nmcli con show
brctl show

Using the nmcli command to set a static IP address on the bridge mode adapter (br0)

Although the br0 adapter appears in the Gnome Settings control panel, its IP address cannot be set using this graphical user interface (GUI). We can set the IP address and other IPV4 values of a br0 adapter using the nmcli command.

Enter the following commands:

nmcli con modify br0 ipv4.addresses 192.168.56.40/24 ipv4.gateway 192.168.56.1 ipv4.method manual
nmcli con modify br0 ipv4.dns "8.8.8.8 8.8.4.4"
nmcli con down br0 && sudo nmcli con up br0
con show br0

Understanding the bridge networking device (br0) and its relationship with the Ethernet adapter

The bridge networking device (br0) is a wrapper around the Ethernet adapter. The br0 adapter replaces the Ethernet adapter.

Configuring the desktop user to login automatically

From the Ubuntu Desktop, Start the Settings application. Click on the search icon and search for “users”:

Click on “Unlock…”:

When prompted, enter the password for the user that owns the desktop session:

Enable “Automatic Login”:

Setting Blank Screen Delay to Never and Disabling Automatic Screen Lock

In the Settings application, go to Privacy, then Screen. Change “Blank Screen Delay” to “Never”. Disable “Automatic Screen Lock”:

Enabling Remote Desktop Sharing

In the Settings application, go to Sharing, then go to “Remote Desktop”. Enable “Remote Desktop”. Enable “Remote Control”. Provide values for “User Name” and “Password”

Creating a firewall exception for the remote desktop port

Open a terminal window. Enter the following commands:

sudo su
ufw allow 3389/tcp

Testing Remote Desktop access to the home server from a Linux desktop

Use the Remmina program and select the RDP protocol. Complete the fields as necessary for your installation, then click on “Save and Connect”:

Testing Remote Desktop Sharing from a Windows 11 Pro desktop

Click on the Start button. Enter the text “remote desktop”. Click on the icon for “Remote Desktop Connection”:

Enter the IP address of the home server. Click “Connect”:

Enter the username and password you specified in the Settings application on the home server under Sharing | Remote Desktop:

Check the box “Don’t ask me again for connection to this computer”. Click on “Yes”:

Considering VNC as an alternative to Remote Desktop (RDP)

If you have difficulty connecting to the home server using a Windows remote desktop client, consider using VNC:

Installing x11vnc to replace broken screen sharing on Ubuntu 21.04

Creating a network file share using the Files (Nautilus) program

From the home server’s desktop, start the Files (Nautilus) program:

Right-click on “Documents”. Click on “Properties”:

Click on “Local Network Share”. Check the box “Share this folder”:

Click on “Install service”:

Click on “Install”:

Enter the password for the user that owns the desktop on the home server. Click Authenticate:

Check the box “Share this folder”. Enter a value for the “Comment” field. Click on “Create Share”:

Installing the Samba program on the home server

Open a terminal window. Enter the following command:

apt install samba

Creating a network file share using Samba

Open a terminal window. Enter the following commands:

sudo su
cd /etc/samba
nano smb.conf

Use the nano text editor to modify the Samba configuration file:

[global]
workgroup = WORKGROUP
security = user
passdb backend = tdbsam
map to guest = Bad User
log file = /var/log/samba/%m.log
max log size = 50
dns proxy = no
[share01]
path = /mount2/share01
create mask = 0644
directory mask = 0755
writable = yes
browseable = yes
valid users = @share01
force group = share01
[share02]
path = /mount2/share02
create mask = 0644
directory mask = 0755
writable = yes
browseable = yes
valid users = @share02
force group = share02

Save and exit the file.

Restarting Samba

Enter the following command:

systemctl restart samba

Adding users to the groups share01 and share02

groupadd share01
groupadd share02
usermod -aG share01 username
usermod -aG share02 username

Using the smbpasswd command to create a Samba username to match the desktop username

Open a terminal window. Enter the following commands. Replace username with the user that owns the desktop on the home server. When prompted, provide a value for the password:

sudo su
smbpasswd -a username

Creating a firewall exception for the network file sharing (CIFS) port

Enter the following command:

ufw allow 137,138,139,445/tcp

Testing the network file share using the Files (Nautilus) program

In the Files (Nautilus) application, click on “+ Other Locations”:

Select “Registered User”. Provide a value for “Username”. For Domain, put “WORKGROUP”. Provide a value for “Password”. Click on “Connect”:

Testing the network file share using File Explorer in Windows 11 Pro

From the File Explorer application in Windows 11 Pro, enter the address of the server in the address bar. Prefix the address with “\\” as in “\\192.168.56.40” for the following example. Enter the IP address of your home server:

Advanced applications of Samba including Active Directory authentication

For a detailed discussion about Samba and advanced topics including Active Directory authentication, refer to Integrating open source software in the enterprise Chapter 1: Creating a network file share with Linux and Samba authenticating against Active Directory

Installing a few utilities on the home server

Open a terminal window. Enter the following commands:

sudo su
apt install iptraf-ng finger wireguard virt-manager build-essential

Registering a persistent host name with noip.com

Visit the website noip.com

Create a free account. Create a hostname. Click on “Dynamic Update Client”:

Installing the noip dynamic update client (DUC)

Follow the instructions provided by noip.com to install the noip dynamic update client (DUC):

Enter the following commands in the terminal window. Use the version number in place of x.xx:

cd /usr/local/src
tar xzf noip-duc-linux.tar.gz
cd no-ip-x.xx
make
make install

Creating the /etc/rc.local startup script and adding the noip DUC command to the /etc/rc.local startup script

Open a terminal window. Enter the following commands:

sudo su
cd /etc
nano rc.local

Use the nano text editor to add the following text:

#!/usr/bin/bash
# persistent host name
/usr/local/bin/noip2
exit 0

Save and exit the file.

Enter the following commands:

chmod 755 rc.local
systemctl start rc-local
systemctl enable rc-local

Declaring a CNAME record in DNS to map a subdomain to the IP address of the persistent host name

If you have a registered domain name, and you have access to the DNS control panel for that domain, you can declare a CNAME record in DNS to map a subdomain to the ip address of the persistent hostname. For example, the GoDaddy DNS control panel allows the following kind of CNAME declaration:

This creates the subdomain servername.example.com, which will ping to the same IP address as persistenthostname.ddns.net

In this case we have set the time-to-live (TTL) value to 1 hour, so the IP address of the CNAME host would be updated once per hour. Many DNS providers block the option of declaring a CNAME to the apex (@) host of a domain. You can still host a subdomain, for example:

https://servername.example.com

If you need to declare the @ host as a CNAME consider pobox.com

If you need to declare the @ host of a domain as a CNAME associated with a persistent host name, consider using pobox.com as your DNS provider.

Using a script to automate the installation of OpenVPN

The openvpn-install.sh from Nyr automates the installation of the OpenVPN server application:

https://github.com/Nyr/openvpn-install

Downloading the OpenVPN installation script

To download the openvpn-install.sh script, enter the following commands:

sudo su
cd /root
mkdir openvpn
cd openvpn
wget https://git.io/vpn -O openvpn-install.sh
chmod +x openvpn-install.sh

Modifying the OpenVPN installation script to use a non-default subnet

Enter the following command:

nano openvpn-install.sh

Use the nano text editor to modify the file openvpn-install.sh by using nano’s search-and-replace function:

Press Control-| for search-and-replace

search for: “10.8.”

replace with: “10.4”

(replace all occurrences)

Running the OpenVPN installation script

cd /root/openvpn
./openvpn-install.sh

When prompted, choose the following options:

protocol: TCP
port: 10443

Modifying the OpenVPN server installation script file

From a root shell, enter the following commands:

cd /etc/openvpn/server
nano server.conf

Locate the following line:

push "redirect-gateway def1 bypass-dhcp"

Change the line to:

push "redirect-gateway def bypass-dhcp"

Press Ctrl-X to save and exit the file.

Modifying the OpenVPN client profile

Use a text editor to load the OpenVPN client profile. Add the following text to the bottom of the file:

Modifying the /etc/openvpn/server/client-common.txt file

Enter the following commands

cd /etc/openvpn/server
nano client-common.txt

Use the nano text editor to modify the file.

Replace the line:

remote xxx.xxx.xxx.xxx 10443

with the line:

remote persistenthostname.ddns.net 10443

Save and exit the file.

Restarting the OpenVPN server

From a root shell, enter the following command:

systemctl start openvpn-server@server
systemctl enable openvpn-server@server

Creating a firewall exception for the OpenVPN server port

Enter the following commands:

sudo su
ufw allow 10443/tcp

Modifying the /etc/sysctl.conf file to enable network forwarding

From a root shell, enter the following commands:

cd /etc
nano sysctl.conf

Add the following text to the bottom of the file:

net.ipv4.ip_forward=1

Press Ctrl-X to save and exit the file.

Enter the following command to reload the sysctl settings:

sysctl -a

Creating the /etc/rc.local file

Enter the following commands:

cd /etc
nano rc.local

Add the following text. Provide a value for adaptername that matches your installation:

!/usr/bin/bash
iptables -t nat -A POSTROUTING -s 10.4.0.0/24 -o adaptername -j MASQUERADE

If you are using the no-ip.com dynamic update client (DUC), add the following text:

/usr/local/bin/noip2

Add the following text:

exit 0

Press Ctrl-X to save and exit the file.

Enter the following command

chmod 755 rc.local

Starting the rc-local service

Enter the following command:

systemctl restart rc-local
systemctl enable rc-local

Forwarding ports from the public-facing IP address to the internal IP address of the VPN host

Use the control panel of your router to forward a port from the public-facing IP address to the internal IP address of the VPN host.

As an example, for the server described in this procedure, we are using the TCP port 10443 to host the connection:

Creating an OpenVPN client adapter profile

Enter the following command and follow the instructions:

./openvpn.sh

Select an IP address from the list:

Which IPv4 address should be used?
1) xxx.xxx.xxx.xxx

IPv4 address [1]: 1

Enter “2” for “2) TCP”:

Which protocol should OpenVPN use?

1) UDP (recommended)
2) TCP
Protocol [1]: 2

Enter “10443”:

What port should OpenVPN listen to?
Port [1194]: 10443

Downloading the OpenVPN client profile

Use the FileZilla file transfer client to download the OpenVPN client profile:

https://blog.gordonbuchan.com/blog/index.php/2021/03/07/web-presence-step-by-step-chapter-7-configuring-the-ssh-server-on-an-ubuntu-linux-cloud-server-to-limit-sftp-directory-visibility-within-chroot-jail-directories/#:~:text=Obtaining%20the%20FileZilla%20file%20transfer%20program

Importing an OpenVPN client profile

Import the OpenVPN client profile into the OpenVPN client application.

Connecting to the OpenVPN server

Tip: connect a computer to your phone’s hotspot, so that you are testing a connection from outside the network.

An example of a Windows client connecting to the OpenVPN server:

For “Username” enter the username of the VPN connection. For “Password” enter the one-time password (OTP) displayed by the Google Authenticator app:

A successful connection:

Advanced applications of OpenVPN including two-factor authentication

For a detailed discussion about OpenVPN and advanced topics including two-factor authentication, refer to Integrating open source software in the enterprise Chapter 2: Using Linux and OpenVPN to create a virtual private network (VPN) server with two-factor authentication (2FA) enabled using Google Authenticator

Installing and configuring virt-manager and KVM virtual machine (VM) hypervisor

We will install some programs, then run virt-manager.

Installing programs to support KVM and virt-manager

Open a terminal window. Enter the following commands:

sudo su
apt install qemu-system qemu-utils python3 python3-pip git 

Modifying the KVM hypervisor to use a non-default subnet

Enter the following commands:

cd /etc/libvirt/qemu/networks/
nano default.xml

Use the nano text editor to modify the default.xml file. Change the value “122” to “162”:

<ip address='192.168.162.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='192.168.162.2' end='192.168.162.254'/>
    </dhcp>
  </ip>

Save and exit the file. Enter the following command:

systemctl restart libvirtd

Downloading an ISO file of Ubuntu Server

Visit the following website:

https://ubuntu.com/download/server

Click on “Download Ubuntu Server xx.xx.x LTS”:

Starting the virt-manager program

From the desktop of the home server, open a terminal window. Enter the following commands:

virt-manager

Verifying that the virtual machine (VM) is set for bridge mode with the br0 device

Click on the “i” icon on the VM. Select “NIC”:

Determining the current IP address of the VM

Enter the following commands:

sudo su
apt install net-tools
ifconfig

Note the name (ie enp1so) and IP address of the first adapter:

Connecting to the server with SSH

Open a terminal window on the desktop of the home server. Enter the following command, substituting values for username and ipaddress to match your installation:

ssh username@ipaddress

Creating a netplan for a static IP address for the VM

As the VM is running Ubuntu Server, we will use netplan to create a static IP address.

From the SSH terminal window, enter the following commands:

sudo su
cd /etc/netplan
cp 00-installer-config.yaml 00-installer-config.yaml.b4
nano 00-installer-config.yaml

Use the nano text editor to modify the 00-installer-config.yaml file. Change the value of adaptername as needed ie “enp1s0”:

network:
  version: 2
  renderer: networkd
  ethernets:
    adaptername:
      dhcp4: no
      addresses:
        - 192.168.56.23/24
      gateway4: 192.168.56.1
      nameservers:
        addresses: [8.8.8.8, 8.8.4.4]

From the VM console on the desktop of the home server, enter the following commands:

sudo su
ifconfig
cd /etc/netplan
netplan try

Verify that the new IP address has taken effect:

From the VM console on the desktop of the home server, enter the following command:

ifconfig

Updating the Ubuntu Server software

Enter the following commands:

apt clean
apt update
apt upgrade
ufw allow 80/tcp
ufw allow 443/tcp
ufw allow 22/tcp
apt install net-tools iptraf-ng
reboot

Creating a LAMP web server in a virtual machine (VM) guest

Open an SSH terminal window to the home server. Substitute values for username and ipaddress to match your installation:

ssh username@ipaddress

Enter the following commands:

sudo su
apt install lamp-server^
cd /etc/apache2/mods-enabled
nano dir.conf

Use the nano text editor to modify the dir.conf file. Modify the line so that index.php is the first entry in the DirectoryIndex line:

DirectoryIndex index.php index.html index.cgi index.pl index.xhtml index.htm

Save and exit the file.

Enter the following commands:

nano apache2.conf

Use the nano text editor to modify the apache2.conf file. Find the “<Directory /var/www/>” section. Change “AlllowOverride None” to “AllowOverride All”:

<Directory /var/www/html>
    Options Indexes FollowSymLinks
    AllowOverride All
    Require all granted
</Directory>

Enter the following commands:

a2enmod rewrite
systemctl restart apache2

Running the mysql_secure_installation command

Enter the following command.

mysql_secure_installation

Answer the prompts as follows:

Testing the web server on port 80

From the desktop of the home server, start a web browser. Visit the IP address of the VM that hosts the LAMP web server:

Forwarding the LAMP web server ports from the public-facing router to the bridge mode IP address of the VM hosting the LAMP web server

Testing the web server from a public address

Using your cell phone: switch to LTE data mode. Visit the URL of your persistent hostname. If you have a CNAME declared for a subdomain host in DNS, visit that URL as well.

Creating virtual hosts for Apache

Open an SSH terminal window to the VM hosting the LAMP web server:

ssh desktop@192.168.56.23

Enter the following commands:

sudo su
cd /etc/apache2/sites-available
nano persistenthostname.ddns.net.conf

Use the nano text editor to edit the persistenthostname.ddns.net.conf file:

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    ServerName persistenthostname.ddns.net
    DocumentRoot /var/www/html
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Save and exit the file.

Enter the following commands

nano subdomain.example.com.conf

Use the nano text editor to edit the subdomain.example.com.conf file:

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    ServerName subdomain.example.com
    DocumentRoot /var/www/html
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Save and exit the file.

Enter the following commands:

a2ensite persistenthostname.ddns.net.conf
a2ensite subdomain.example.com.conf
systemctl restart apache2

Using Let’s Encrypt to create an SSL certificate for https

Open an SSH terminal window to the VM hosting the LAMP web server. Enter the following commands:

sudo su
apt install python3-certbot-apache
certbot --apache
systemctl restart apache2

Enabling Wireguard on the home server

Open an SSH terminal window to the home server. Provide values for username and ipaddress to match your installation:

ssh username@ipadress

Creating public and private WireGuard keys

Enter the following commands:

sudo su
cd /etc/wireguard
umask 077
wg genkey > privatekey
wg pubkey < privatekey > publickey

Creating a firewall exception for the WireGuard port on the home server

ufw allow 55555/udp

Enter the following commands:

nano wg0.conf

Creating the wg0.conf file

Use the nano text editor to modify the wg0.conf file. Provide a value for privatekey matching the privatekey of the home server, generated above. (Provide a value for publickey of the peer system (the offsite backup server) when the value becomes available, then restart the wg-quick@wg0 service):

[Interface]
# home server
Address = 10.5.0.1/24
PrivateKey = privatekeyofhomeserver
ListenPort = 55555

[Peer]
# offsite backup server
PublicKey = publickeyofoffsitebackupserver
AllowedIPs = 10.5.0.0/24, 192.168.1.0/24

Starting the wg-quick service

Enter the following command:

systemctl restart wg-quick@wg0
systemctl enable wg-quick@wg0

Forwarding the WireGuard port from the public-facing router to the home server

Tasks for the offsite backup server

The offsite backup server will perform the following tasks

• Samba network file server
• Wireguard client connection to home server

Hardware for the backup server

My brother donated a computer to the project, a computer that was headed for a dumpster. This is an example of a hacker living his principles.

This machine could not address more than 1.5GB RAM of the RAM we found in our junkpiles. This machine has a 20GB mechanical hard drive — we could certainly upgrade that with a 120GB SSD, but we decided to see what was possible with the mechanical drive. We will be attaching an SSD drive to the computer. Because the taskings are Samba network file sharing and a Wiregurd tunnel to the home server, it may not be necessary to upgrade the mechanical drive.

The offsite backup server is a small form factor (SFF) desktop circa 2005:

• HP HSTNC-008P-SF (circa 2005)
• Pentium(R) D CPU
• 1.5GB DDR RAM
• 20GB mechanical drive (presumably 5400RPM)

Formatting and configuring the offsite backup server with Fedora Server 38

My brother formatted the offsite backup server with Fedora Server 38. This server will have a text-only console. This will allow us to conserve about 1.1GB RAM, ie 3/4 of the 1.5GB RAM we have available in the system.

Why choose Fedora Server instead of Fedora Desktop?

For the offsite backup server, as the hardware is limited, we will use Fedora Server to conserve CPU and RAM resources.

Formatting in Legacy Mode

With older, pre-2016 hardware, it is simpler to format in Legacy Mode. In this case, the system literally is legacy, this is the only mode available.

Connecting using wired Ethernet

We will connect the offsite backup server using wired Ethernet. This simplifies some kinds of networking, including WireGuard, which we will use later in this procedure to create a secure tunnel to the home server.

Installing a few utilities on the offsite backup server

Log in at the console of the offsite backup server. Enter the following commands:

sudo su
dnf install net-tools iptraf-ng finger wireguard
ifconfig

Examine the output of the ifconfig command. Find the name of the Ethernet adapter, it may be something like “enp0s25” or “eth0” — take note of this value.

Using the nmcli command to configure a static IP address for the offsite backup server

Enter the following commands. Provide values for adaptername and ipv4.gateway that match your installation:

nmcli con modify adaptername ipv4.addresses 192.168.1.95/24 ipv4.gateway 192.168.1.1 ipv4.method manual
nmcli con modify br0 ipv4.dns "8.8.8.8 8.8.4.4"
nmcli con down br0 && sudo nmcli con up br0
con show br0
reboot

Installing the Samba program on the offsite backup server

Open an SSH terminal window to the offsite backup server. Enter the following command:

dnf install samba

Creating a network file share using Samba on the offsite backup server

Refer to the section above “Creating a network file share using Samba“

Enabling Wireguard on the offsite backup server

Open an SSH terminal window to the offsite server. Provide values for username and ipaddress to match your installation:

ssh username@ipadress

Creating public and private WireGuard keys

Enter the following commands:

sudo su
cd /etc/wireguard
umask 077
wg genkey > privatekey
wg pubkey < privatekey > publickey

Creating a firewall exception for the WireGuard port on the offsite backup server

firewall-cmd --zone=public --add-port=55555/udp --permanent

Enter the following commands:

nano wg0.conf

Creating the wg0.conf file

Use the nano text editor to modify the wg0.conf file. Provide a value for privatekey matching the privatekey of the home server, generated above. Provide a value for publickey matching the private key of the offsite backup server:

[Interface]
# offsite backup server
Address = 10.5.0.2/24
PrivateKey = privatekeyofoffsitebackupserver
ListenPort = 55555

[Peer]
# home server
PublicKey = publickeyofhomeserver
AllowedIPs = 10.5.0.0/24, 192.168.56.0/24
Endpoint = persistenthostnameofhomeserver.ddns.net:55555
PersistentKeepalive = 25

Starting the wg-quick service

Enter the following command:

systemctl restart wg-quick@wg0
systemctl enable wg-quick@wg0

Testing the WireGuard secure tunnel between the offsite backup server and the home server

From the offsite backup server, enter the following command:

ping 10.5.0.1

If the ping is successful, the offsite backup server has a working WireGuard connection to the home server.

From the home server, enter the following command:

ping 10.5.0.2

If the ping is successful, the home server has a working Wireguard connection to the offsite backup server.