As of early 2025, large language models (LLMs) are primarily accessed through web interfaces offered by companies like OpenAI, Anthropic (Perplexity/Claude), and Google (Gemini). Alongside these proprietary offerings, a “second tier” of open-source LLM models has emerged, including Meta’s LLaMA 3.1, Mistral, DeepSeek, and others. These open-source models are becoming increasingly viable for self-hosting, offering significant advantages in data sovereignty, confidentiality, and cost savings. For many use cases, they are roughly on par with proprietary models, making them an appealing alternative.
While web interfaces are the most visible way to interact with LLMs, they are largely loss leaders, designed to promote application programming interface (API) services. APIs are the backbone of the LLM ecosystem, enabling developers to integrate LLM capabilities into their own software. Through APIs, businesses can pass data and instructions to an LLM and retrieve outputs tailored to their needs. These APIs are central to the value proposition of LLMs, powering applications like retrieval-augmented generation (RAG) workflows for the scanning of document collections, automated form processing, and natural language interfaces for structured databases.
The growing market for LLM APIs
OpenAI was the first major player to offer an API for its LLMs, and its design has become a de facto standard, with many other LLM providers emulating its structure. This compatibility has paved the way for a competitive LLM inference hosting market. Applications leveraging APIs can often switch between providers with minimal effort, simply by changing the host address and API key. This interoperability is fostering a dynamic market for LLM inferencing, where cost, performance, and data privacy are key differentiators.
Example of an LLM API call
Here’s an example of a basic API call using curl. This same structure is supported by most LLM APIs:
curl https://api.lemonfox.ai/v1/chat/completions \
-H "Authorization: Bearer YOUR_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "mixtral-chat",
"messages": [
{ "role": "system", "content": "You are a helpful assistant." },
{ "role": "user", "content": "Why is the sky blue?" }
]
}'
This straightforward interface makes it easy for developers to integrate LLM capabilities into their applications, whether for natural language understanding, data extraction, or other advanced AI tasks.
Note: you may notice differences between this API call and the API calls we used with Ollama and Open-WebUI in previous blog posts. Ollama and Open-WebUI use a simplified protocol using a prompt field. The example above uses a messages array, compatible with the chat completions API, used by OpenAI and implemented by third parties such as lemonfox.ai
A historical parallel: LLM hosting and the web hosting market of the 2000s
The current trajectory of LLM inference hosting bears striking similarities to the early days of web hosting in the late 1990s and early 2000s. Back then, the advent of open-source technologies like Linux, Apache, MySQL, and PHP enabled hobbyists and businesses to build industrial-grade web servers on consumer hardware. While some opted to host websites themselves, most turned to professional web hosting providers, creating a competitive market that eventually drove down prices and established commoditized hosting as the norm.
Similarly, the LLM inference hosting market is evolving into a spectrum of options:
Self-hosting: Organizations can invest in high-performance hardware like NVIDIA’s H100 GPUs (priced at around US$30,000) or more modest setups using GPUs like the RTX 4090 or RTX 5090 (priced at around US$5,000). This option offers full control but requires significant upfront investment and technical expertise.
Leased GPU services: Cloud providers offer GPU resources on an hourly basis, making it possible to run LLMs without committing to physical hardware. For example, renting an H100 GPU typically costs around US$3 per hour.
Hosted inference services: Many providers offer LLM inference as a service, where customers pay per transaction or token. This model eliminates the need for infrastructure management, appealing to businesses that prioritize simplicity.
The economics of LLM hosting
The emergence of open-source models and interoperable APIs is driving fierce competition in the LLM hosting market. This competition has already led to dramatic price differences between providers. For example:
lemonfox.ai Mistral 7B: US$5 per 10 million tokens (using open-source models)
These disparities highlight the potential cost savings of opting for open-source models hosted by third-party providers or self-hosting solutions.
Renting GPUs vs. buying inference services
For businesses and developers, choosing between renting GPU time, self-hosting, or using inference services depends on several factors:
Scalability: Hosted inference services are ideal for unpredictable or spiky workloads, as they scale effortlessly.
Cost efficiency: For steady, high-volume workloads, self-hosting may be more economical in the long run.
Data control: Organizations with strict confidentiality requirements may prefer self-hosting to ensure data never leaves their infrastructure.
Open source software is free as in freedom, and free as in free beer. Although there are significant hardware costs for GPU capability, in general an enterprise can self-host AI without incurring software licensing fees.
Price competition from vendors using open source solutions no doubt has the effect of constraining the pricing power of closed source vendors.
For example, a small startup building a chatbot might initially use an inference provider like lemonfox.ai to minimize costs and complexity. As their user base grows, they might transition to leased GPU services or invest in dedicated hardware to optimize expenses.
A law firm or medical practice may begin with an air-gapped cloud instance with non-disclosure (NDA) and data protection (DPA) agreements. At some point, the business case may justify taking the service in-house with a self-hosted inference server with GPU hardware.
Conclusion: the road ahead for LLM inference hosting
As LLMs continue to gain traction, the LLM inference hosting market will likely follow the trajectory of web hosting two decades ago—moving toward commoditization and low-margin competition. Businesses and individuals will increasingly weigh the trade-offs between cost, control, and convenience when deciding how to deploy LLM capabilities. The availability of open-source models and interoperable APIs ensures that options will continue to expand, empowering developers to choose the solution that best meets their needs.
In this post, we create a Python script that connects to a Gmail inbox, extracts the text of the subject and body of each message, submits that text with a prompt to a large language model (LLM), then if conditions are met that match the prompt, escalates the message to the attention of an operator, based on a prompt.
Create a new app password. Take note of the password, it will not be visible again.
Note: Google adds spaces to the app password for readability. You should remove the spaces from the app password and use that value.
Escalating to the root user
In this procedure we run as the root user. Enter the following command:
sudo su
Adding utilities to the operating system
Enter the following command:
apt install python3-venv python3-pip sqlite3
Creating a virtual environment and installing required packages with pip
Enter the following commands:
cd ~
mkdir doicareworkdir
cd doicareworkdir
python3 -m venv doicare_env
source doicare_env/bin/activate
pip install requests imaplib2
Creating the configuration file (config.json)
Enter the following command:
nano config.json
Use the nano editor to add the following text:
{
"gmail_user": "xxxxxxxxxxxx@xxxxx.xxx",
"gmail_app_password": "xxxxxxxxxxxxxxxx",
"api_base_url": "http://xxx.xxx.xxx.xxx:8085",
"openai_api_key": "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx",
"database": "doicare.db",
"scanasof": "18-Jan-2025",
"alert_recipients": [
"xxxxx@xxxxx.com"
],
"smtp_server": "smtp.gmail.com",
"smtp_port": 587,
"smtp_user": "xxxxxx@xxxxx.xxxxx",
"smtp_password": "xxxxxxxxxxxxxxxx",
"analysis_prompt": "Analyze the email below. If it needs escalation (urgent, sender upset, or critical issue), return 'Escalation Reason:' followed by one short sentence explaining why. If no escalation is needed, return exactly 'DOESNOTAPPLY'. Always provide either 'DOESNOTAPPLY' or a reason.",
"model": "mistral"
}
Save and exit the file.
Creating a Python script called doicare that connects to a Gmail inbox, submits messages to an LLM, and escalates messages based on a prompt (Ollama version)
Enter the following command:
nano doicare_gmail.py
import imaplib
import email
import sqlite3
import requests
import smtplib
import json
from datetime import datetime
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from email.header import decode_header, make_header
# MIT license 2025 Gordon Buchan
# see https://opensource.org/licenses/MIT
# Some of this code was generated with the assistance of AI tools.
# --------------------------------------------------------------------
# 1. LOAD CONFIG
# --------------------------------------------------------------------
with open("config.json", "r") as cfg:
config = json.load(cfg)
GMAIL_USER = config["gmail_user"]
GMAIL_APP_PASSWORD = config["gmail_app_password"]
API_BASE_URL = config["api_base_url"]
OPENAI_API_KEY = config["openai_api_key"]
DATABASE = config["database"]
SCAN_ASOF = config["scanasof"]
ALERT_RECIPIENTS = config.get("alert_recipients", [])
SMTP_SERVER = config["smtp_server"]
SMTP_PORT = config["smtp_port"]
SMTP_USER = config["smtp_user"]
SMTP_PASSWORD = config["smtp_password"]
ANALYSIS_PROMPT = config["analysis_prompt"]
MODEL = config["model"]
# --------------------------------------------------------------------
# 2. DATABASE SETUP
# --------------------------------------------------------------------
def setup_database():
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
cur.execute("""
CREATE TABLE IF NOT EXISTS escalations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
email_date TEXT,
from_address TEXT,
to_address TEXT,
cc_address TEXT,
subject TEXT,
body TEXT,
reason TEXT,
created_at TEXT
)
""")
cur.execute("""
CREATE TABLE IF NOT EXISTS scan_info (
id INTEGER PRIMARY KEY AUTOINCREMENT,
last_scanned_uid INTEGER
)
""")
conn.commit()
conn.close()
def get_last_scanned_uid():
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
cur.execute("SELECT last_scanned_uid FROM scan_info ORDER BY id DESC LIMIT 1")
row = cur.fetchone()
conn.close()
return row[0] if (row and row[0]) else 0
def update_last_scanned_uid(uid_val):
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
cur.execute("INSERT INTO scan_info (last_scanned_uid) VALUES (?)", (uid_val,))
conn.commit()
conn.close()
def is_already_processed(uid_val):
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
cur.execute("SELECT 1 FROM scan_info WHERE last_scanned_uid = ?", (uid_val,))
row = cur.fetchone()
conn.close()
return bool(row)
# --------------------------------------------------------------------
# 3. ANALYSIS & ALERTING
# --------------------------------------------------------------------
def analyze_with_openai(subject, body):
prompt = f"{ANALYSIS_PROMPT}\n\nSubject: {subject}\nBody: {body}"
url = f"{API_BASE_URL}/v1/completions"
headers = {"Content-Type": "application/json"}
if OPENAI_API_KEY:
headers["Authorization"] = f"Bearer {OPENAI_API_KEY}"
payload = {
"model": MODEL,
"prompt": prompt,
"max_tokens": 300,
"temperature": 0.7
}
try:
response = requests.post(url, headers=headers, json=payload, timeout=60)
data = response.json()
if "error" in data:
print(f"[DEBUG] API Error: {data['error']['message']}")
return "DOESNOTAPPLY"
if "choices" in data and data["choices"]:
raw_text = data["choices"][0]["text"].strip()
return raw_text
return "DOESNOTAPPLY"
except Exception as e:
print(f"[DEBUG] Exception during API call: {e}")
return "DOESNOTAPPLY"
def send_alerts(reason, email_date, from_addr, to_addr, cc_addr, subject, body):
for recipient in ALERT_RECIPIENTS:
msg = MIMEMultipart()
msg["From"] = SMTP_USER
msg["To"] = recipient
msg["Subject"] = "Escalation Alert"
alert_text = f"""
Escalation Triggered
Date: {email_date}
From: {from_addr}
To: {to_addr}
CC: {cc_addr}
Subject: {subject}
Body: {body}
Reason: {reason}
"""
msg.attach(MIMEText(alert_text, "plain"))
try:
with smtplib.SMTP(SMTP_SERVER, SMTP_PORT) as server:
server.starttls()
server.login(SMTP_USER, SMTP_PASSWORD)
server.sendmail(SMTP_USER, recipient, msg.as_string())
print(f"Alert sent to {recipient}")
except Exception as ex:
print(f"Failed to send alert to {recipient}: {ex}")
def save_escalation(email_date, from_addr, to_addr, cc_addr, subject, body, reason):
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
cur.execute("""
INSERT INTO escalations (
email_date, from_address, to_address, cc_address,
subject, body, reason, created_at
) VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""", (
email_date, from_addr, to_addr, cc_addr,
subject, body, reason, datetime.now().isoformat()
))
conn.commit()
conn.close()
# --------------------------------------------------------------------
# 4. MAIN LOGIC
# --------------------------------------------------------------------
def process_message(raw_email, uid_val):
parsed_msg = email.message_from_bytes(raw_email)
date_str = parsed_msg.get("Date", "")
from_addr = parsed_msg.get("From", "")
to_addr = parsed_msg.get("To", "")
cc_addr = parsed_msg.get("Cc", "")
subject_header = parsed_msg.get("Subject", "")
subject_decoded = str(make_header(decode_header(subject_header)))
body_text = ""
if parsed_msg.is_multipart():
for part in parsed_msg.walk():
ctype = part.get_content_type()
disposition = str(part.get("Content-Disposition"))
if ctype == "text/plain" and "attachment" not in disposition:
charset = part.get_content_charset() or "utf-8"
body_text += part.get_payload(decode=True).decode(charset, errors="replace")
else:
charset = parsed_msg.get_content_charset() or "utf-8"
body_text = parsed_msg.get_payload(decode=True).decode(charset, errors="replace")
reason = analyze_with_openai(subject_decoded, body_text)
if "DOESNOTAPPLY" in reason:
print(f"[UID {uid_val}] No escalation: {reason}")
return
print(f"[UID {uid_val}] Escalation triggered: {subject_decoded[:50]}")
save_escalation(date_str, from_addr, to_addr, cc_addr, subject_decoded, body_text, reason)
send_alerts(reason, date_str, from_addr, to_addr, cc_addr, subject_decoded, body_text)
def main():
setup_database()
last_uid = get_last_scanned_uid()
print(f"[DEBUG] Retrieved last UID: {last_uid}")
try:
mail = imaplib.IMAP4_SSL("imap.gmail.com")
mail.login(GMAIL_USER, GMAIL_APP_PASSWORD)
print("IMAP login successful.")
except Exception as e:
print(f"Error logging into Gmail: {e}")
return
mail.select("INBOX")
if last_uid == 0:
print(f"[DEBUG] First run: scanning since date {SCAN_ASOF}")
r1, d1 = mail.search(None, f'(SINCE {SCAN_ASOF})')
else:
print(f"[DEBUG] Subsequent run: scanning for UIDs > {last_uid}")
r1, d1 = mail.uid('SEARCH', None, f'UID {last_uid + 1}:*')
if r1 != "OK":
print("[DEBUG] Search failed.")
mail.logout()
return
seq_nums = d1[0].split()
print(f"[DEBUG] Found {len(seq_nums)} messages to process: {seq_nums}")
if not seq_nums:
print("[DEBUG] No messages to process.")
mail.logout()
return
highest_uid_seen = last_uid
for seq_num in seq_nums:
if is_already_processed(seq_num.decode()):
print(f"[DEBUG] UID {seq_num.decode()} already processed, skipping.")
continue
print(f"[DEBUG] Processing sequence number: {seq_num}")
r2, d2 = mail.uid('FETCH', seq_num.decode(), '(RFC822)')
if r2 != "OK" or not d2 or len(d2) < 1 or not d2[0]:
print(f"[DEBUG] Failed to fetch message for UID {seq_num.decode()}")
continue
print(f"[DEBUG] Successfully fetched message for UID {seq_num.decode()}")
raw_email = d2[0][1]
try:
process_message(raw_email, int(seq_num.decode()))
mail.uid('STORE', seq_num.decode(), '+FLAGS', '\\Seen')
if int(seq_num.decode()) > highest_uid_seen:
highest_uid_seen = int(seq_num.decode())
except Exception as e:
print(f"[DEBUG] Error processing message UID {seq_num.decode()}: {e}")
if highest_uid_seen > last_uid:
print(f"[DEBUG] Updating last scanned UID to {highest_uid_seen}")
update_last_scanned_uid(highest_uid_seen)
mail.logout()
if __name__ == "__main__":
main()
Save and exit the file.
Creating a Python script called doicare that connects to a Gmail inbox, submits messages to an LLM, and escalates messages based on a prompt (OpenAI-compatible version)
Enter the following command:
nano doicare_gmail.py
import imaplib
import email
import sqlite3
import requests
import smtplib
import json
from datetime import datetime
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
from email.header import decode_header, make_header
# MIT license 2025 Gordon Buchan
# see https://opensource.org/licenses/MIT
# Some of this code was generated with the assistance of AI tools.
# --------------------------------------------------------------------
# 1. LOAD CONFIG
# --------------------------------------------------------------------
with open("config.json", "r") as cfg:
config = json.load(cfg)
GMAIL_USER = config["gmail_user"]
GMAIL_APP_PASSWORD = config["gmail_app_password"]
API_BASE_URL = config["api_base_url"]
OPENAI_API_KEY = config["openai_api_key"]
DATABASE = config["database"]
SCAN_ASOF = config["scanasof"]
ALERT_RECIPIENTS = config.get("alert_recipients", [])
SMTP_SERVER = config["smtp_server"]
SMTP_PORT = config["smtp_port"]
SMTP_USER = config["smtp_user"]
SMTP_PASSWORD = config["smtp_password"]
ANALYSIS_PROMPT = config["analysis_prompt"]
MODEL = config["model"]
# --------------------------------------------------------------------
# 2. DATABASE SETUP
# --------------------------------------------------------------------
def setup_database():
""" Ensure the database and necessary tables exist. """
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
print("[DEBUG] Ensuring database tables exist...")
cur.execute("""
CREATE TABLE IF NOT EXISTS escalations (
id INTEGER PRIMARY KEY AUTOINCREMENT,
email_date TEXT,
from_address TEXT,
to_address TEXT,
cc_address TEXT,
subject TEXT,
body TEXT,
reason TEXT,
created_at TEXT
)
""")
cur.execute("""
CREATE TABLE IF NOT EXISTS scan_info (
id INTEGER PRIMARY KEY AUTOINCREMENT,
last_scanned_uid INTEGER UNIQUE
)
""")
# Ensure at least one row exists in scan_info
cur.execute("SELECT COUNT(*) FROM scan_info")
if cur.fetchone()[0] == 0:
cur.execute("INSERT INTO scan_info (last_scanned_uid) VALUES (0)")
conn.commit()
conn.close()
print("[DEBUG] Database setup complete.")
def get_last_scanned_uid():
""" Retrieve the last scanned UID from the database """
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
cur.execute("SELECT last_scanned_uid FROM scan_info ORDER BY id DESC LIMIT 1")
row = cur.fetchone()
conn.close()
return int(row[0]) if (row and row[0]) else 0
def update_last_scanned_uid(uid_val):
""" Update the last scanned UID in the database """
conn = sqlite3.connect(DATABASE)
cur = conn.cursor()
cur.execute("""
INSERT INTO scan_info (id, last_scanned_uid)
VALUES (1, ?)
ON CONFLICT(id) DO UPDATE SET last_scanned_uid = excluded.last_scanned_uid
""", (uid_val,))
conn.commit()
conn.close()
# --------------------------------------------------------------------
# 3. ANALYSIS & ALERTING
# --------------------------------------------------------------------
def analyze_with_openai(subject, body):
""" Send email content to OpenAI API for analysis """
prompt = f"{ANALYSIS_PROMPT}\n\nSubject: {subject}\nBody: {body}"
url = f"{API_BASE_URL}/v1/chat/completions"
headers = {
"Content-Type": "application/json",
"Authorization": f"Bearer {OPENAI_API_KEY}" if OPENAI_API_KEY else "",
}
payload = {
"model": MODEL,
"messages": [
{"role": "system", "content": "You are a helpful assistant"},
{"role": "user", "content": prompt}
],
"max_tokens": 300,
"temperature": 0.7
}
try:
response = requests.post(url, headers=headers, json=payload, timeout=60)
data = response.json()
if "error" in data:
print(f"[DEBUG] API Error: {data['error']['message']}")
return "DOESNOTAPPLY"
if "choices" in data and data["choices"]:
return data["choices"][0]["message"]["content"].strip()
return "DOESNOTAPPLY"
except Exception as e:
print(f"[DEBUG] Exception during API call: {e}")
return "DOESNOTAPPLY"
# --------------------------------------------------------------------
# 4. MAIN LOGIC
# --------------------------------------------------------------------
def process_message(raw_email, uid_val):
""" Process a single email message """
parsed_msg = email.message_from_bytes(raw_email)
date_str = parsed_msg.get("Date", "")
from_addr = parsed_msg.get("From", "")
to_addr = parsed_msg.get("To", "")
cc_addr = parsed_msg.get("Cc", "")
subject_header = parsed_msg.get("Subject", "")
subject_decoded = str(make_header(decode_header(subject_header)))
body_text = ""
if parsed_msg.is_multipart():
for part in parsed_msg.walk():
ctype = part.get_content_type()
disposition = str(part.get("Content-Disposition"))
if ctype == "text/plain" and "attachment" not in disposition:
charset = part.get_content_charset() or "utf-8"
body_text += part.get_payload(decode=True).decode(charset, errors="replace")
else:
charset = parsed_msg.get_content_charset() or "utf-8"
body_text = parsed_msg.get_payload(decode=True).decode(charset, errors="replace")
reason = analyze_with_openai(subject_decoded, body_text)
if "DOESNOTAPPLY" in reason:
print(f"[UID {uid_val}] No escalation: {reason}")
return
print(f"[UID {uid_val}] Escalation triggered: {subject_decoded[:50]}")
update_last_scanned_uid(uid_val)
def main():
""" Main function to fetch and process emails """
setup_database()
last_uid = get_last_scanned_uid()
print(f"[DEBUG] Retrieved last UID: {last_uid}")
try:
mail = imaplib.IMAP4_SSL("imap.gmail.com")
mail.login(GMAIL_USER, GMAIL_APP_PASSWORD)
print("IMAP login successful.")
except Exception as e:
print(f"Error logging into Gmail: {e}")
return
mail.select("INBOX")
search_query = f'UID {last_uid + 1}:*' if last_uid > 0 else f'SINCE {SCAN_ASOF}'
print(f"[DEBUG] Running IMAP search: {search_query}")
r1, d1 = mail.uid('SEARCH', None, search_query)
if r1 != "OK":
print("[DEBUG] Search failed.")
mail.logout()
return
seq_nums = d1[0].split()
seq_nums = [seq.decode() for seq in seq_nums]
print(f"[DEBUG] Found {len(seq_nums)} new messages: {seq_nums}")
if not seq_nums:
print("[DEBUG] No new messages found, exiting.")
mail.logout()
return
highest_uid_seen = last_uid
for seq_num in seq_nums:
numeric_uid = int(seq_num)
if numeric_uid <= last_uid:
print(f"[DEBUG] UID {numeric_uid} already processed, skipping.")
continue
print(f"[DEBUG] Processing UID: {numeric_uid}")
r2, d2 = mail.uid('FETCH', seq_num, '(RFC822)')
if r2 != "OK" or not d2 or len(d2) < 1 or not d2[0]:
print(f"[DEBUG] Failed to fetch message for UID {numeric_uid}")
continue
raw_email = d2[0][1]
process_message(raw_email, numeric_uid)
highest_uid_seen = max(highest_uid_seen, numeric_uid)
if highest_uid_seen > last_uid:
print(f"[DEBUG] Updating last scanned UID to {highest_uid_seen}")
update_last_scanned_uid(highest_uid_seen)
mail.logout()
if __name__ == "__main__":
main()
Save and exit the file.
Running the doicare_gmail.py script
Enter the following command:
python3 doicare_gmail.py
Sample output
(doicare_env) root@xxxxx:/home/desktop/doicareworkingdir# python3 doicare_gmail.py
[DEBUG] Retrieved last UID: 0
IMAP login successful.
[DEBUG] First run: scanning since date 18-Jan-2025
[DEBUG] Found 23 messages to process: [b'49146', b'49147', b'49148', b'49149', b'49150', b'49151', b'49152', b'49153', b'49154', b'49155', b'49156', b'49157', b'49158', b'49159', b'49160', b'49161', b'49162', b'49163', b'49164', b'49165', b'49166', b'49167', b'49168']
[DEBUG] Processing sequence number: b'49146'
[DEBUG] FETCH response: b'49146 (UID 50196)'
[DEBUG] FETCH line to parse: 49146 (UID 50196)
[DEBUG] Parsed UID: 50196
[DEBUG] Valid UID Found: 50196
[DEBUG] Successfully fetched message for UID 50196
[UID 50196] No escalation: DOESNOTAPPLY. The email does not contain any urgent matter, sender is not upset, and there does not seem to be a critical issue mentioned.
[DEBUG] Processing sequence number: b'49147'
[DEBUG] FETCH response: b'49147 (UID 50197)'
[DEBUG] FETCH line to parse: 49147 (UID 50197)
[DEBUG] Parsed UID: 50197
[DEBUG] Valid UID Found: 50197
[DEBUG] Successfully fetched message for UID 50197
[UID 50197] No escalation: DOESNOTAPPLY
[DEBUG] Processing sequence number: b'49148'
[DEBUG] FETCH response: b'49148 (UID 50198)'
[DEBUG] FETCH line to parse: 49148 (UID 50198)
[DEBUG] Parsed UID: 50198
[DEBUG] Valid UID Found: 50198
[DEBUG] Successfully fetched message for UID 50198
[UID 50198] No escalation: DOESNOTAPPLY. The email does not contain any urgent matter, sender is not upset, and there doesn't seem to be a critical issue presented in the content.
[DEBUG] Processing sequence number: b'49149'
[DEBUG] FETCH response: b'49149 (UID 50199)'
[DEBUG] FETCH line to parse: 49149 (UID 50199)
[DEBUG] Parsed UID: 50199
[DEBUG] Valid UID Found: 50199
[DEBUG] Successfully fetched message for UID 50199
[UID 50199] No escalation: DOESNOTAPPLY. The email does not contain any urgent matter, the sender is not upset, and there is no critical issue mentioned in the message.
[DEBUG] Processing sequence number: b'49150'
[DEBUG] FETCH response: b'49150 (UID 50200)'
[DEBUG] FETCH line to parse: 49150 (UID 50200)
[DEBUG] Parsed UID: 50200
[DEBUG] Valid UID Found: 50200
[DEBUG] Successfully fetched message for UID 50200
[UID 50200] No escalation: DOESNOTAPPLY. The email lacks sufficient content for an escalation.
[DEBUG] Processing sequence number: b'49151'
[DEBUG] FETCH response: b'49151 (UID 50201)'
[DEBUG] FETCH line to parse: 49151 (UID 50201)
[DEBUG] Parsed UID: 50201
[DEBUG] Valid UID Found: 50201
[DEBUG] Successfully fetched message for UID 50201
[UID 50201] Escalation triggered: Security alert
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49152'
[DEBUG] FETCH response: b'49152 (UID 50202)'
[DEBUG] FETCH line to parse: 49152 (UID 50202)
[DEBUG] Parsed UID: 50202
[DEBUG] Valid UID Found: 50202
[DEBUG] Successfully fetched message for UID 50202
[UID 50202] Escalation triggered: Delivery Status Notification (Failure)
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49153'
[DEBUG] FETCH response: b'49153 (UID 50203)'
[DEBUG] FETCH line to parse: 49153 (UID 50203)
[DEBUG] Parsed UID: 50203
[DEBUG] Valid UID Found: 50203
[DEBUG] Successfully fetched message for UID 50203
[UID 50203] No escalation: DOESNOTAPPLY
[DEBUG] Processing sequence number: b'49154'
[DEBUG] FETCH response: b'49154 (UID 50204)'
[DEBUG] FETCH line to parse: 49154 (UID 50204)
[DEBUG] Parsed UID: 50204
[DEBUG] Valid UID Found: 50204
[DEBUG] Successfully fetched message for UID 50204
[UID 50204] Escalation triggered: my server lollipop is down
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49155'
[DEBUG] FETCH response: b'49155 (UID 50205)'
[DEBUG] FETCH line to parse: 49155 (UID 50205)
[DEBUG] Parsed UID: 50205
[DEBUG] Valid UID Found: 50205
[DEBUG] Successfully fetched message for UID 50205
[UID 50205] No escalation: DOESNOTAPPLY
[DEBUG] Processing sequence number: b'49156'
[DEBUG] FETCH response: b'49156 (UID 50206)'
[DEBUG] FETCH line to parse: 49156 (UID 50206)
[DEBUG] Parsed UID: 50206
[DEBUG] Valid UID Found: 50206
[DEBUG] Successfully fetched message for UID 50206
[UID 50206] Escalation triggered: now doomfire is down too!
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49157'
[DEBUG] FETCH response: b'49157 (UID 50207)'
[DEBUG] FETCH line to parse: 49157 (UID 50207)
[DEBUG] Parsed UID: 50207
[DEBUG] Valid UID Found: 50207
[DEBUG] Successfully fetched message for UID 50207
[UID 50207] No escalation: DOESNOTAPPLY
[DEBUG] Processing sequence number: b'49158'
[DEBUG] FETCH response: b'49158 (UID 50208)'
[DEBUG] FETCH line to parse: 49158 (UID 50208)
[DEBUG] Parsed UID: 50208
[DEBUG] Valid UID Found: 50208
[DEBUG] Successfully fetched message for UID 50208
[UID 50208] Escalation triggered: pants is down now
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49159'
[DEBUG] FETCH response: b'49159 (UID 50209)'
[DEBUG] FETCH line to parse: 49159 (UID 50209)
[DEBUG] Parsed UID: 50209
[DEBUG] Valid UID Found: 50209
[DEBUG] Successfully fetched message for UID 50209
[UID 50209] Escalation triggered: server05 down
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49160'
[DEBUG] FETCH response: b'49160 (UID 50210)'
[DEBUG] FETCH line to parse: 49160 (UID 50210)
[DEBUG] Parsed UID: 50210
[DEBUG] Valid UID Found: 50210
[DEBUG] Successfully fetched message for UID 50210
[UID 50210] No escalation: DOESNOTAPPLY (The sender has asked for a phone call instead of specifying the issue in detail, so it doesn't appear to be urgent or critical at first glance.)
[DEBUG] Processing sequence number: b'49161'
[DEBUG] FETCH response: b'49161 (UID 50211)'
[DEBUG] FETCH line to parse: 49161 (UID 50211)
[DEBUG] Parsed UID: 50211
[DEBUG] Valid UID Found: 50211
[DEBUG] Successfully fetched message for UID 50211
[UID 50211] Escalation triggered: my server is down
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49162'
[DEBUG] FETCH response: b'49162 (UID 50212)'
[DEBUG] FETCH line to parse: 49162 (UID 50212)
[DEBUG] Parsed UID: 50212
[DEBUG] Valid UID Found: 50212
[DEBUG] Successfully fetched message for UID 50212
[UID 50212] No escalation: DOESNOTAPPLY
[DEBUG] Processing sequence number: b'49163'
[DEBUG] FETCH response: b'49163 (UID 50213)'
[DEBUG] FETCH line to parse: 49163 (UID 50213)
[DEBUG] Parsed UID: 50213
[DEBUG] Valid UID Found: 50213
[DEBUG] Successfully fetched message for UID 50213
[UID 50213] Escalation triggered: this is getting bad
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49164'
[DEBUG] FETCH response: b'49164 (UID 50214)'
[DEBUG] FETCH line to parse: 49164 (UID 50214)
[DEBUG] Parsed UID: 50214
[DEBUG] Valid UID Found: 50214
[DEBUG] Successfully fetched message for UID 50214
[UID 50214] No escalation: DOESNOTAPPLY
[DEBUG] Processing sequence number: b'49165'
[DEBUG] FETCH response: b'49165 (UID 50215)'
[DEBUG] FETCH line to parse: 49165 (UID 50215)
[DEBUG] Parsed UID: 50215
[DEBUG] Valid UID Found: 50215
[DEBUG] Successfully fetched message for UID 50215
[UID 50215] Escalation triggered: server zebra 05 is down
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49166'
[DEBUG] FETCH response: b'49166 (UID 50216)'
[DEBUG] FETCH line to parse: 49166 (UID 50216)
[DEBUG] Parsed UID: 50216
[DEBUG] Valid UID Found: 50216
[DEBUG] Successfully fetched message for UID 50216
[UID 50216] No escalation: DOESNOTAPPLY
[DEBUG] Processing sequence number: b'49167'
[DEBUG] FETCH response: b'49167 (UID 50217)'
[DEBUG] FETCH line to parse: 49167 (UID 50217)
[DEBUG] Parsed UID: 50217
[DEBUG] Valid UID Found: 50217
[DEBUG] Successfully fetched message for UID 50217
[UID 50217] Escalation triggered: help
Alert sent to xxxx@hotmail.com
[DEBUG] Processing sequence number: b'49168'
[DEBUG] FETCH response: b'49168 (UID 50218)'
[DEBUG] FETCH line to parse: 49168 (UID 50218)
[DEBUG] Parsed UID: 50218
[DEBUG] Valid UID Found: 50218
[DEBUG] Successfully fetched message for UID 50218
[UID 50218] Escalation triggered: server is down
Alert sent to xxxx@hotmail.com
[DEBUG] Updating last scanned UID to 50218
[DEBUG] Attempting to update last scanned UID to 50218
[DEBUG] Last scanned UID successfully updated to 50218
Example of an alert message
Escalation Triggered
Date: Sat, 18 Jan 2025 21:00:16 +0000
From: Gordon Buchan <gordonhbuchan@hotmail.com>
To: "gordonhbuchan@gmail.com" <gordonhbuchan@gmail.com>
CC:
Subject: server is down
Body: server down help please
Reason: Escalation Reason: This email indicates that there is a critical issue (server downtime).
Creating a systemd service to run the doicare script automatically
Enter the following command:
nano /etc/systemd/system/doicare.service
Use the nano editor to add the following text (change values to match your path):
In this post, we install the Ollama LLM hosting software, and load a large language model (LLM), a 5GB file produced by a company called Mistral. We then test local inference, interacting with the model at the command line. We send test queries to the application protocol interface (API) server. We install an application called Open-WebUI that enables a web chat interface to the LLM.
Note: this procedure references the mistral model. however, you can specify other models, such as dolphin-mistral. Consult the following page for available models. Try to limit your choices to 7B complexity, unless you have a GPU.
Until 2023, graphical processing units (GPUs) were only of interest to video gamers, animators, and mechanical designers. There is now an imperative for GPU resources on most new servers going forward, for local inference and retrieval augmented generation (RAG). However we will need to devise an interim approach to use the CPU-centric servers we have, even for some AI inference tasks, until the capex cycles have refreshed in 3-4 years from now. On a CPU-only system, the system response time for a query can range from 2-5 seconds to 30-40 seconds. This level of performance may be acceptable for some use cases, including scripted tasks for which a 40 second delay is not material. Deploying this solution on a system with even a modest Nvidia GPU will result in dramatic increases in performance.
Why host an LLM locally
To learn how LLMs are built
To achieve data sovereignty by operating on a private system
To save expense by avoiding the need for external LLM vendors
Preparing a computer for deployment
This procedure was tested on Ubuntu Server 24.04. Baremetal is better than a virtual machine for this use case, allowing the software to access all of the resources of the host system. In terms of resources, you will need a relatively powerful CPU, like an i7, and 16-32GB of RAM.
Note: the version of Python required by Open-WebUI is Python 3.12, which is supported by default in Ubuntu Server 24.04 LTS. If you are on an older version of the operating, you can install a newer version of Python using a PPA.
Do you need a GPU?
No, but a GPU will make your inference noticeably faster. If you have an Nvidia GPU, ensure that you have Nvidia CUDA drivers enabled. If you have an AMD GPU, ensure that you have AMD ROCM drivers. There is some talk of support for Intel GPUs but none of it is yet practical.
Note: if you have an Nvidia GPU, you may want to consider vLLM.
Ollama is able to work on a CPU-only system
Ollama is able to work on a CPU-only system, and that is what we will implement in this post. Ollama is able to achieve performance that may be acceptable for certain kinds of operations. For example, large batch operations that run overnight, that can accept a 30-60 second delay versus 2-10 seconds for a GPU-driven solution. For some questions, like “why is the sky blue?’ an answer will start immediately. For more complex questions, there may be a 5-10 second delay before answering, and the text will arrive slowly enough to remind you of 300 baud modems (for those of you who get that reference). The wonder of a dancing bear is not in how well it dances, but that it dances at all. This level of performance may be acceptable for some use cases, in particular batched operations and programmatic access via a custom function invoking commands sent to the API server.
Escalating to root using sudo
From a shell, enter the following command:
sudo su
(enter the password when requested)
Opening ports in the UFW firewall
You may need to open ports on the UFW firewall to enable the chat client.
A note re RHEL and variants like Fedora and AlmaLinux
Although this procedure has not been tested on RHEL and variants like Fedora and AlmaLinux, I looked at the installation script and those platforms are supported. In theory, you could configure an RHEL-type system by using equivalent firewall-cmd and dnf commands.
Installing Ollama using the installation script
Ollama provides an installation script that automates the installation. From a shell as root, enter the following command:
curl -fsSL https://ollama.com/install.sh | sh
Pulling the mistral image
Enter the following command:
ollama pull mistral
Listing the images available
Enter the following command:
ollama list
Testing Ollama and the LLM using the command line
Enter the following command. Test the chat interface on the command line in the shell:
ollama run mistral
Testing the API server using curl
Enter the following commands:
systemctl restart ollama
systemctl status ollama
systemctl enable ollama
curl http://localhost:11434/api/generate -d '{
"model": "mistral",
"prompt":"Why is the sky blue?"
}'
Enter the following command:
curl http://localhost:11434/api/chat -d '{
"model": "mistral",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'
Preparing the system for Open-WebUI
To prepare the system for Open-WebUI, we must create a working directory, and create a venv (virtual python environment).
Enter the following commands:
cd ~
pwd
mkdir ollamatmp
cd ollamatmp
python3 -m venv ollama_env
source ollama_env/bin/activate
pip install open-webui
This window took 30 seconds to begin showing its answer, then another 20 seconds to complete generating the answer:
Using nginx as a proxy to expose the API port to the local network
By default, the Ollama API server answers on port 11434 but only on the local address 127.0.0.1. You can use nginx as a proxy to expose the API to the local network. Enter the following commands:
ufw allow 8085/tcp
apt install nginx
cd /etc/nginx/sites-enabled
nano default
Use the nano editor to add the following text:
server {
listen 8085;
location / {
proxy_pass http://127.0.0.1:11434; # Replace with your Ollama API port
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
# Optional: Add timeout settings for long-running API calls
proxy_connect_timeout 60s;
proxy_read_timeout 60s;
proxy_send_timeout 60s;
}
}
Save and exit the file.
Enter this command:
systemctl restart nginx
Testing the exposed API port from another computer
From another computer, enter the command (where xxx.xxx.xxx.xxx is the IP address of the computer hosting the Ollama API server):
curl http://xxx.xxx.xxx.xxx:8085/api/chat -d '{
"model": "mistral",
"messages": [
{ "role": "user", "content": "why is the sky blue?" }
]
}'
Creating a systemd service to start the Open-WebUI chat interface automatically
In this post we install and configure an instance of Ubuntu Server under WSL2 for Linux. Many developers are going to choose to use WSL2, so we guide them to install the Ubuntu Server version of WSL2. This provides a more standard environment referenced by many howto procedures on the Internet.
WSL2 enables a computer running Windows 11 Pro to host a guest instance of Ubuntu Server:
WSL2 offers some advantages for a developer experience
WSL2 offers some advantages for a local developer in terms of networking: if they install a service that opens a port, like port 22/tcp for SSH, that same port is opened on localhost of the Windows machine hosting the WSL2 Ubuntu Server instance. A developer can SSH to port 22 on localhost, without configuring a static IP address or port forwarding. This eliminates the need to configure virtual switches, static IP addresses, and port forwarding. In addition, WSL2 provides a file mounts of the Windows file system within Linux, and a file mount of the Linux filesystem within Windows.
Other ways to install Ubuntu Server on a Windows 11 Pro computer
There are other ways to install Ubuntu Server on a Windows 11 Pro computer, including Windows Hyper-V. If you need to host multiple instances of Ubuntu Server with static IP addresses and subnet routing, consider using Hyper-V instead.
Installing WSL2
Start a CMD window as Administrator. At the Start menu, type the letters “cmd” then right-click on the app icon to run the command prompt as root:
Enter the following command:
powershell
Enter the following command:
wsl --install -d Ubuntu-24.04
Reboot your computer.
Enter values for username and password:
Enter the following command from a CMD window running as Administrator:
wsl --list --verbose
Enter the following command:
wsl --setdefault Ubuntu-24.04
Enter the following command:
wsl
Enter the following command:
lsb_release -a
Enter the following command:
sudo su
Updating apt packages on the Linux system
Enter the following commands:
apt clean
apt update
Enter the following command:
apt upgrade
Enter y for yes:
Enter the following command:
reboot
from a CMD window running as Administrator, enter the following command:
wsl
Installing some utilities
Enter the following commands:
sudo su
apt install net-tools git build-essential
Installing openssh-server
Enter the following command:
apt install openssh-server
From a CMD window running as Administrator, enter the following command:
In this post, we build a home server using Ubuntu Desktop Linux that includes a Samba network file server, an OpenVPN virtual private network (VPN), and a KVM hypervisor hosting virtual machine (VM) guests including a Linux/Apache/MySQL/PHP web server. In addition, we build an offsite backup server using Fedora Server Linux, and link the offsite backup server to the home server via a WireGuard secure network tunnel.
Tasks for the home server
The home server will perform the following tasks:
Samba network file server
OpenVPN server
KVM hypervisor to host virtual machine (VM) guests
LAMP web server in a VM
Wireguard public-facing host to client connection to offsite backup server
Hardware for the home server
As the home server will host virtual machine guests, I need a certain level of performance, so I bought a refurbished circa 2017 computer for C$403 (US$294). CPU performance can affect OpenVPN performance, so the VPN server will benefit from a stronger CPU as well.
The home server is a small form factor (SFF) desktop circa 2017:
Dell OptiPlex 5050 SFF (circa 2017)
i7-7700 CPU
32GB DDR4 RAM
1TB SATA SSD
Formatting and configuring the home server with Ubuntu Desktop 22.04 LTS
I formatted the home server with Ubuntu Desktop 22.04 LTS.
Why choose Ubuntu Desktop instead of Ubuntu Server?
For the home server, I wanted the option of a graphical user interface (GUI) desktop for use at console, and via remote desktop. A GUI desktop is also more convenient for the creation and management of KVM virtual machine guests using the virt-manager GUI, (and avoids the need for SSH tunnel forwarding and an X11 server to reach a headless server).
Formatting in UEFI mode
With modern hardware, I like to use UEFI mode for disk booting. Although we do not need a multiple-boot menu for this server, it is easier to construct a multiple-boot menu using grub when booting in UEFI mode. This is the default on a post-2016 motherboard, but it is worth looking at the BIOS when you first lay hands on a machine.
Connecting using wired Ethernet
We need a wired Ethernet connection for the home server, as we want to create a bridge mode adapter (br0) so that virtual machine (VM) guests can have IP addresses in the host networking subnet.
Complete the fields as needed, then click on “Continue”:
Click on “Restart Now”:
Press the ENTER key on your keyboard:
Using the nmcli command to create a bridge mode adapter (br0)
Because we are working on an Ubuntu desktop, we will use the nmcli command to create a bridge mode adapter (br0).
Open a terminal window. Enter the following commands:
sudo su
apt install net-tools bridge-utils
ifconfig
Look at the information displayed by the ifconfig command. Identify the name of the wired Ethernet connection. The name may be “eth0” or a string such as “enp0s31f6”
Use the value you identified above and use it in place of ethernet_name.
Enter the following commands:
nmcli con add ifname br0 type bridge con-name br0
nmcli con add type ethernet ifname ethernet_name master br0
nmcli con up br0
nmcli con show
brctl show
Using the nmcli command to set a static IP address on the bridge mode adapter (br0)
Although the br0 adapter appears in the Gnome Settings control panel, its IP address cannot be set using this graphical user interface (GUI). We can set the IP address and other IPV4 values of a br0 adapter using the nmcli command.
Enter the following commands:
nmcli con modify br0 ipv4.addresses 192.168.56.40/24 ipv4.gateway 192.168.56.1 ipv4.method manual
nmcli con modify br0 ipv4.dns "8.8.8.8 8.8.4.4"
nmcli con down br0 && sudo nmcli con up br0
con show br0
Understanding the bridge networking device (br0) and its relationship with the Ethernet adapter
The bridge networking device (br0) is a wrapper around the Ethernet adapter. The br0 adapter replaces the Ethernet adapter.
Configuring the desktop user to login automatically
From the Ubuntu Desktop, Start the Settings application. Click on the search icon and search for “users”:
Click on “Unlock…”:
When prompted, enter the password for the user that owns the desktop session:
Enable “Automatic Login”:
Setting Blank Screen Delay to Never and Disabling Automatic Screen Lock
In the Settings application, go to Privacy, then Screen. Change “Blank Screen Delay” to “Never”. Disable “Automatic Screen Lock”:
Enabling Remote Desktop Sharing
In the Settings application, go to Sharing, then go to “Remote Desktop”. Enable “Remote Desktop”. Enable “Remote Control”. Provide values for “User Name” and “Password”
Creating a firewall exception for the remote desktop port
Open a terminal window. Enter the following commands:
sudo su
ufw allow 3389/tcp
Testing Remote Desktop access to the home server from a Linux desktop
Use the Remmina program and select the RDP protocol. Complete the fields as necessary for your installation, then click on “Save and Connect”:
Testing Remote Desktop Sharing from a Windows 11 Pro desktop
Click on the Start button. Enter the text “remote desktop”. Click on the icon for “Remote Desktop Connection”:
Enter the IP address of the home server. Click “Connect”:
Enter the username and password you specified in the Settings application on the home server under Sharing | Remote Desktop:
Check the box “Don’t ask me again for connection to this computer”. Click on “Yes”:
Considering VNC as an alternative to Remote Desktop (RDP)
If you have difficulty connecting to the home server using a Windows remote desktop client, consider using VNC:
Using the smbpasswd command to create a Samba username to match the desktop username
Open a terminal window. Enter the following commands. Replace username with the user that owns the desktop on the home server. When prompted, provide a value for the password:
sudo su
smbpasswd -a username
Creating a firewall exception for the network file sharing (CIFS) port
Enter the following command:
ufw allow 137,138,139,445/tcp
Testing the network file share using the Files (Nautilus) program
In the Files (Nautilus) application, click on “+ Other Locations”:
Select “Registered User”. Provide a value for “Username”. For Domain, put “WORKGROUP”. Provide a value for “Password”. Click on “Connect”:
Testing the network file share using File Explorer in Windows 11 Pro
From the File Explorer application in Windows 11 Pro, enter the address of the server in the address bar. Prefix the address with “\\” as in “\\192.168.56.40” for the following example. Enter the IP address of your home server:
Advanced applications of Samba including Active Directory authentication
Declaring a CNAME record in DNS to map a subdomain to the IP address of the persistent host name
If you have a registered domain name, and you have access to the DNS control panel for that domain, you can declare a CNAME record in DNS to map a subdomain to the ip address of the persistent hostname. For example, the GoDaddy DNS control panel allows the following kind of CNAME declaration:
This creates the subdomain servername.example.com, which will ping to the same IP address as persistenthostname.ddns.net
In this case we have set the time-to-live (TTL) value to 1 hour, so the IP address of the CNAME host would be updated once per hour. Many DNS providers block the option of declaring a CNAME to the apex (@) host of a domain. You can still host a subdomain, for example:
https://servername.example.com
If you need to declare the @ host as a CNAME consider pobox.com
If you need to declare the @ host of a domain as a CNAME associated with a persistent host name, consider using pobox.com as your DNS provider.
Using a script to automate the installation of OpenVPN
The openvpn-install.sh from Nyr automates the installation of the OpenVPN server application:
From the desktop of the home server, open a terminal window. Enter the following commands:
virt-manager
Verifying that the virtual machine (VM) is set for bridge mode with the br0 device
Click on the “i” icon on the VM. Select “NIC”:
Determining the current IP address of the VM
Enter the following commands:
sudo su
apt install net-tools
ifconfig
Note the name (ie enp1so) and IP address of the first adapter:
Connecting to the server with SSH
Open a terminal window on the desktop of the home server. Enter the following command, substituting values for username and ipaddress to match your installation:
ssh username@ipaddress
Creating a netplan for a static IP address for the VM
As the VM is running Ubuntu Server, we will use netplan to create a static IP address.
From the SSH terminal window, enter the following commands:
sudo su
cd /etc/netplan
cp 00-installer-config.yaml 00-installer-config.yaml.b4
nano 00-installer-config.yaml
Use the nano text editor to modify the 00-installer-config.yaml file. Change the value of adaptername as needed ie “enp1s0”:
Use the nano text editor to modify the apache2.conf file. Find the “<Directory /var/www/>” section. Change “AlllowOverride None” to “AllowOverride All”:
<Directory /var/www/html>
Options Indexes FollowSymLinks
AllowOverride All
Require all granted
</Directory>
Enter the following commands:
a2enmod rewrite
systemctl restart apache2
Running the mysql_secure_installation command
Enter the following command.
mysql_secure_installation
Answer the prompts as follows:
Testing the web server on port 80
From the desktop of the home server, start a web browser. Visit the IP address of the VM that hosts the LAMP web server:
Forwarding the LAMP web server ports from the public-facing router to the bridge mode IP address of the VM hosting the LAMP web server
Testing the web server from a public address
Using your cell phone: switch to LTE data mode. Visit the URL of your persistent hostname. If you have a CNAME declared for a subdomain host in DNS, visit that URL as well.
Creating virtual hosts for Apache
Open an SSH terminal window to the VM hosting the LAMP web server:
ssh desktop@192.168.56.23
Enter the following commands:
sudo su
cd /etc/apache2/sites-available
nano persistenthostname.ddns.net.conf
Use the nano text editor to edit the persistenthostname.ddns.net.conf file:
Using Let’s Encrypt to create an SSL certificate for https
Open an SSH terminal window to the VM hosting the LAMP web server. Enter the following commands:
sudo su
apt install python3-certbot-apache
certbot --apache
systemctl restart apache2
Enabling Wireguard on the home server
Open an SSH terminal window to the home server. Provide values for username and ipaddress to match your installation:
ssh username@ipadress
Creating public and private WireGuard keys
Enter the following commands:
sudo su
cd /etc/wireguard
umask 077
wg genkey > privatekey
wg pubkey < privatekey > publickey
Creating a firewall exception for the WireGuard port on the home server
ufw allow 55555/udp
Enter the following commands:
nano wg0.conf
Creating the wg0.conf file
Use the nano text editor to modify the wg0.conf file. Provide a value for privatekey matching the privatekey of the home server, generated above. (Provide a value for publickey of the peer system (the offsite backup server) when the value becomes available, then restart the wg-quick@wg0 service):
[Interface]
# home server
Address = 10.5.0.1/24
PrivateKey = privatekeyofhomeserver
ListenPort = 55555
[Peer]
# offsite backup server
PublicKey = publickeyofoffsitebackupserver
AllowedIPs = 10.5.0.0/24, 192.168.1.0/24
Forwarding the WireGuard port from the public-facing router to the home server
Tasks for the offsite backup server
The offsite backup server will perform the following tasks
Samba network file server
Wireguard client connection to home server
Hardware for the backup server
My brother donated a computer to the project, a computer that was headed for a dumpster. This is an example of a hacker living his principles.
This machine could not address more than 1.5GB RAM of the RAM we found in our junkpiles. This machine has a 20GB mechanical hard drive — we could certainly upgrade that with a 120GB SSD, but we decided to see what was possible with the mechanical drive. We will be attaching an SSD drive to the computer. Because the taskings are Samba network file sharing and a Wiregurd tunnel to the home server, it may not be necessary to upgrade the mechanical drive.
The offsite backup server is a small form factor (SFF) desktop circa 2005:
HP HSTNC-008P-SF (circa 2005)
Pentium(R) D CPU
1.5GB DDR RAM
20GB mechanical drive (presumably 5400RPM)
Formatting and configuring the offsite backup server with Fedora Server 38
My brother formatted the offsite backup server with Fedora Server 38. This server will have a text-only console. This will allow us to conserve about 1.1GB RAM, ie 3/4 of the 1.5GB RAM we have available in the system.
Why choose Fedora Server instead of Fedora Desktop?
For the offsite backup server, as the hardware is limited, we will use Fedora Server to conserve CPU and RAM resources.
Formatting in Legacy Mode
With older, pre-2016 hardware, it is simpler to format in Legacy Mode. In this case, the system literally is legacy, this is the only mode available.
Connecting using wired Ethernet
We will connect the offsite backup server using wired Ethernet. This simplifies some kinds of networking, including WireGuard, which we will use later in this procedure to create a secure tunnel to the home server.
Installing a few utilities on the offsite backup server
Log in at the console of the offsite backup server. Enter the following commands:
sudo su
dnf install net-tools iptraf-ng finger wireguard
ifconfig
Examine the output of the ifconfig command. Find the name of the Ethernet adapter, it may be something like “enp0s25” or “eth0” — take note of this value.
Using the nmcli command to configure a static IP address for the offsite backup server
Enter the following commands. Provide values for adaptername and ipv4.gateway that match your installation:
nmcli con modify adaptername ipv4.addresses 192.168.1.95/24 ipv4.gateway 192.168.1.1 ipv4.method manual
nmcli con modify br0 ipv4.dns "8.8.8.8 8.8.4.4"
nmcli con down br0 && sudo nmcli con up br0
con show br0
reboot
Installing the Samba program on the offsite backup server
Open an SSH terminal window to the offsite backup server. Enter the following command:
dnf install samba
Creating a network file share using Samba on the offsite backup server
Use the nano text editor to modify the wg0.conf file. Provide a value for privatekey matching the privatekey of the home server, generated above. Provide a value for publickey matching the private key of the offsite backup server:
Web presence step by step is a series of posts that show you to how to build a web presence.
In this post, we learn how to declare multiple subdomain names under a single domain name for use as virtual host names, for multiple separate websites.
Understanding virtual hosts and their relation to domain and subdomain names
A shared LAMP web server can host multiple websites, or “virtual hosts,” named after domain and subdomain names.
You may wish to host a web-based application like MyBB or Nextcloud as its own website, rather than as a subdirectory of an existing website, without purchasing an additional domain name.
Multiple subdomains can be hosted on the same web server, or on different web servers.
The bare domain and the www subdomain
A virtual host can be identified by a “bare” domain name like “webpresencestepbystep.com,” or by a subdomain name, like “www.webpresencestepbystep.com” — by convention, the www subdomain prefix points to the same content as the “bare” domain name.
Additional subdomains like “community” and “media”
Additional subdomain names can be declared like “community.webpresencestepbystep.com” and “media.webpresencestepbystep.com” – these subdomains can point to separate websites on the same web server, or on different web servers.
Understanding domain and subdomain names and their relation to Domain Name System (DNS) zone files
As we can see in the DNS zone for the domain, the host names “@” (“bare domain”) and “community” are A declarations associated with the IP address of web server A.
The host name “www” is a CNAME declaration associated with the host name “@” so implicitly is associated with the IP address of web server A.
The host name “media” is an A declaration associated with the IP address of web server B.
2 websites on web server A, 1 website on web server B, all as subdomains of a single domain name
By using multiple subdomains of the same domain name, 3 separate websites can be declared, with 2 websites hosted on web server A, and 1 website hosted on web server B, without the need to purchase additional domain names.
Bare domain and subdomain www on web server A
webpresencestepbystep.com and www.webpresencestepbystep.com on web server A
A note about www and CNAME
the subdomain “www” host name is a canononical name (“CNAME”) of the @ host name, which identifies the “bare” domain. This means that www.webpresencestepbystep.com will resolve to the same IP address as webpresencestepbystep.com
Web presence step by step is a series of posts that show you to how to build a web presence.
In this chapter we install and use a script to automate the creation of a virtual host on an Apache web server.
A PHP script that automates the creation of a virtual host under Apache
This script collects and validates inputs, then executes the commands to create a virtual host under Apache.
A note about the source code view below
For formatting reasons, the text is limited to a fixed width. To fully view the text, you can scroll to the right to see the ends of lines, or use the print view for this blog post.
To view the source code in an another text editor, download and uncompress the zip file described below, or select and copy the text from the source code example below, and paste the text into a file on your computer called “addvhost.php”
Consider copying the file to your Apache web server’s /usr/bin directory with a chmod of 755 so that it can be executed from the system path. Steps to do so are included in the procedure below.
Saving the PHP script to a file called addvhost.php
Uncompress the zip file to extract the file “addvhost.php” then copy the file to your Apache web server.
Source code of the script
Scroll right to see the ends of lines.
#!/usr/bin/php
<?PHP
// addvhost.php
// v0102
// updated to variable-ize vhostip as a base setting
// creates a virtual host under Apache
// Gordon Buchan 20210512 https://gordonbuchan.com
// MIT License https://mit-license.org
// tested on Ubuntu 20.04, may work on Debian
// directory structure allows for chroot jails for SFTP:
// in a jail you do not own your home directory, only your webdir
// tip: apt install finger whois
// ////////////////////////////////////////////////////////////////
// start summary
// initialize base settings in variables ie bvhwb
// ask for vhostsubdomain, vhostusername, vhostpassword
// infer vhosthomedir, vhostwebdir by convention
// create user, home directory, password
// create directory
// create index.php document
// chown vhosthomedir as root:root
// chown vhostwebdir as vhostusername:vhostusername
// chmod vhostwebdir
// create virtual host file
// enable virtual host
// echo suggestion that client restart apache, run certbot --apache, restart apache
// end summary
// ////////////////////////////////////////////////////////////////
// start base settings
$bvhostconfdir = "/etc/apache2/sites-available";
$bvhwb = "/usr/web";
$restartcommandstr = "systemctl apache2 restart";
$vhostenablecommandstr = "a2enmod";
$echoplaintextpasswords = TRUE;
$logplaintextpassword = TRUE;
$vhostserveradmin = "info@yourdomain.com";
// tip: could be "xxx.xxx.xxx.xxx"
$vhostip = "*";
// ////////////////////////////////////////////////////////////////
// end base settings
// ////////////////////////////////////////////////////////////////
// start function sink
// start polyfill
// str_contains() polyfill for pre PHP8: code for this function taken from php.net
if (!function_exists('str_contains')) {
function str_contains(string $haystack, string $needle): bool
{
return '' === $needle || false !== strpos($haystack, $needle);
} // end function str_contains()
}
// ////////////////////////////////////////////////////////////////
// end polyfill
// validate functions
// We will be using the readline() function to ask questions on the command line.
// These functions allow us to do rich validation within a while statement to trap
// the readline in a loop until our conditions are satisfied.
// We will also echo text to the console with reasons for rejection to assist the client.
// For example: bad string format, vhost appears to exist already, etc.
// ////////////////////////////////////////////////////////////////
function vhostsubdomainverify($vhostsubdomainstr) {
global $bvhwb;
global $bvhostconfdir;
//assume true until proven false
$returnval = TRUE;
// is the string clean?
// note that "-" hyphen character is permitted, not part of symbol sieve
if ( preg_match('/[\'^£$%&*()}{@#~?><>,|=_+¬]/', $vhostsubdomainstr) ) {
$returnval = FALSE;
echo "string has special character that is not permitted\n";
}
// string does not contain a period symbol
if (!str_contains($vhostsubdomainstr,".") ) {
$returnval = FALSE;
echo "string does not contain a \".\" period symbol.\n";
}
// string contains two period symbols in a row
if (str_contains($vhostsubdomainstr,"..") ) {
$returnval = FALSE;
echo "string contain two \"..\" period symbols in a row.\n";
}
// string contains leading period symbol
$strlen = strlen($vhostsubdomainstr);
$begsample = substr($vhostsubdomainstr,0,1);
if ($begsample == ".") {
$returnval = FALSE;
echo "string begins with a \".\" period symbol.\n";
}
// string contains trailing period symbol
$endlen = strlen($vhostsubdomainstr);
$endsample = substr($vhostsubdomainstr,($endlen - 1),1);
if ($endsample == ".") {
$returnval = FALSE;
echo "string ends with a \".\" period symbol.\n";
}
// does the vhostsubdomain already exist?
$vhostsubdomainstrund = str_replace(".","_",$vhostsubdomainstr);
clearstatcache();
if (is_dir("$bvhwb/$vhostsubdomainstrund") ) {
$returnval = FALSE;
echo "webdir for proposed vhost already exists.\n";
} else {
} // end if (is_dir()
$grepforvhost1str = "grep -i 'ServerName $vhostsubdomainstr' $bvhostconfdir/*";
$grepforvhost2str = "grep -i 'ServerAlias $vhostsubdomainstr' $bvhostconfdir/*";
$grepforvhost1res = shell_exec($grepforvhost1str);
$grepforvhost2res = shell_exec($grepforvhost2str);
// if the string has contents something was there for the grep to find
if ($grepforvhost1res || $grepforvhost2res) {
echo "subdomain appears to be part of an existing virtual host\n";
$returnval = FALSE;
}
return $returnval;
} // end function vhostsubdomainverify()
// ////////////////////////////////////////////////////////////////
function prependverify($prependverify) {
// let us make our tests and comparisons case-insensitive
$lowerpv = strtolower($prependverify);
if ( ( $lowerpv == "n") || ($lowerpv == "no") || ($lowerpv == "y") || ($lowerpv == "yes") ) {
$returnval = TRUE;
} else {
echo "please indicate n or no, y or yes\n";
$returnval = FALSE;
}
return $returnval;
} // end function prependverify()
// ////////////////////////////////////////////////////////////////
function usernameverify($vhostusername) {
// force to lower-case
$vhostusername = strtolower($vhostusername);
// assume TRUE until proven false
$returnval = TRUE;
// is the string clean?
// note that "-" hyphen character is permitted, as is the "_" underscore character, not part of symbol sieve
if ( preg_match('/[\'^£$%&*()}{@#~?><>,|=+¬]/', $vhostusername) ) {
$returnval = FALSE;
echo "string has special character that is not permitted\n";
}
$vhunstrlen = strlen($vhostusername);
if ($vhunstrlen < 2) {
echo "username should be minimum 2 characters\n";
$returnval = FALSE;
}
if ($vhunstrlen > 32) {
echo "username should be maximum 32 characters\n";
$returnval = FALSE;
}
// what does finger return?
$fingerstr = shell_exec("finger $vhostusername 2>&1");
if (!str_contains("$fingerstr","no such user") ) {
echo "finger found this username to already be in use\n";
$returnval = FALSE;
}
return $returnval;
} // end function usernameverify()
// ////////////////////////////////////////////////////////////////
function passwordplainverify($passwordplain) {
// assume TRUE until proven false
$returnval = TRUE;
// we should do some tests here
// but mostly just for length, not all that fancy stuff.
// but: we will want to offer to auto-generate a plaintext password
$ppstrlen = strlen($passwordplain);
if ($ppstrlen < 8) {
echo "password should be at least 8 characters\n";
$returnval = FALSE;
}
return $returnval;
} // end function passwordplainverify()
// ////////////////////////////////////////////////////////////////
function genpassverify($genpassverify) {
// let us make our tests and comparisons case-insensitive
$lowergpv = strtolower($genpassverify);
if ( ( $lowergpv == "n") || ($lowergpv == "no") || ($lowergpv == "y") || ($lowergpv == "yes") ) {
$returnval = TRUE;
} else {
echo "please indicate n or no, y or yes\n";
$returnval = FALSE;
}
return $returnval;
} // end function genpassverify()
// ////////////////////////////////////////////////////////////////
function genuserverify($genuserverify) {
// let us make our tests and comparisons case-insensitive
$lowerguv = strtolower($genuserverify);
if ( ( $lowerguv == "n") || ($lowerguv == "no") || ($lowerguv == "y") || ($lowerguv == "yes") ) {
$returnval = TRUE;
} else {
echo "please indicate n or no, y or yes\n";
$returnval = FALSE;
}
return $returnval;
} // end function genuserverify()
// end function sink
// ////////////////////////////////////////////////////////////////
// start get information at command line: vhostsubdomain, vhostusername, vhostpassword
// also, generate and derive values
echo "\naddvhost.php\n";
echo "Add a virtual host to Apache\n\n";
// ask and validate inputs
// the readline is trapped in a loop until vhostsubdomainverify() is satisfied
// function will also echo text to the console with reasons for rejection to assist the client
// bad string format or vhost appears to exist already, etc.
// ////////////////////////////////////////////////////////////////
// vhostsubdomain
$vhostsubdomain = "";
while (!$vhostsubdomain || !vhostsubdomainverify($vhostsubdomain) ) {
$vhostsubdomain = readline("Enter domain xxxxxxxx.xxx or subdomain xxxxxxxx.xxxxxxxx.xxx: ");
}
// putting this here because it is right after we have the $vhostsubdomain string, and just before we need it for $genuseranswer
// will also need this later for derived values like the $vhostwebdir
$vhostsubdomainund = str_replace(".","_",$vhostsubdomain);
// should we prepend with www. as well?
$prependanswer = "";
while (!$prependanswer || !prependverify($prependanswer) ) {
$prependanswer = readline("Do you wish to prepend the subdomain www.$vhostsubdomain as well (n/y)? ");
}
$prependanswer = strtolower($prependanswer);
// ////////////////////////////////////////////////////////////////
// vhostusername
// default username
// should we offer to automatically generate a username based on the subdomain host name?
$genuseranswer = "";
while (!$genuseranswer || !genuserverify($genuseranswer) ) {
$genuseranswer = readline("Generate a username? ");
}
$genuseranswer = strtolower($genuseranswer);
if ( ($genuseranswer=="yes") || ($genuseranswer=="y") ) {
// generate a username
// we are counting on the novel construction of this name with _ modeled on subdomain
$vhostusername = $vhostsubdomainund;
$vhostusernamestrlen = strlen($vhostusername);
// the unique stuff is closer to the front
// so we will truncate to first 32 characters
if ($vhostusernamestrlen > 32) {
$vhostusername = substr($vhostusername,0,32);
}
// what does finger return?
$fingerstr2 = shell_exec("finger $vhostusername 2>&1");
if (!str_contains("$fingerstr2","no such user") ) {
echo "finger found this username to already be in use\n";
exit();
}
} else {
// the client said no to automatic generation of username so we will ask for one
$vhostusername = "";
while (!$vhostusername || !usernameverify($vhostusername) ) {
$vhostusername = readline("Enter username: ");
}
} // end if ($genuseranswer=="yes")
// ////////////////////////////////////////////////////////////////
// vhostpasswordplain
// should we offer to automatically generate a plaintext password?
$genpassanswer = "";
while (!$genpassanswer || !genpassverify($genpassanswer) ) {
$genpassanswer = readline("Generate a plaintext password? ");
}
$genpassanswer = strtolower($genpassanswer);
if ( ($genpassanswer=="yes") || ($genpassanswer=="y") ) {
// generate a random plaintext password
$vhostpasswordplain = "";
$passwordlength = "8";
$posscharsplain = '0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ';
$posscharssymbols = '!@#%*?';
$posscharspstrlen = strlen($posscharsplain);
$posscharssstrlen = strlen($posscharssymbols);
// first the plain characters
for ($i=0;$i<($passwordlength-1);$i++) {
$randomint = random_int(0,$posscharspstrlen-1);
$randomchar = substr($posscharsplain,$randomint,1);
$vhostpasswordplain .= $randomchar;
} //end for $i
// now the symbol character
$randomint = random_int(0,$posscharssstrlen-1);
$randomchar = substr($posscharssymbols,$randomint,1);
$vhostpasswordplain .= $randomchar;
// now shuffle the string so the symbol position moves and as bonus the string is different
$vhostpasswordplain = str_shuffle($vhostpasswordplain);
} else {
// the client said no to automatic generation of plaintext, so we will ask for one
$vhostpasswordplain = "";
while (!$vhostpasswordplain || !passwordplainverify($vhostpasswordplain) ) {
$vhostpasswordplain = readline("Enter plaintext password: ");
}
} // end if ($genpassanswer=="yes")
// ////////////////////////////////////////////////////////////////
// vhostpasswordhashed (transformation)
// yes, i tried password_hash() -- it did not work for SHA512, this does.
// tip: apt install whois to get mkpasswd command
$vhostpasswordhashed = shell_exec("mkpasswd -m sha-512 $vhostpasswordplain");
// remove linefeed from the string
$vhostpasswordhashed = str_replace("\n","",$vhostpasswordhashed);
// ////////////////////////////////////////////////////////////////
// end get information at command line: vhostsubdomain, vhostusername, vhostpassword
// ////////////////////////////////////////////////////////////////
// start print collected values
$vhosthomedir = "$bvhwb/$vhostusername";
$vhostwebdir = "$bvhwb/$vhostusername/$vhostsubdomain";
echo "\nvalues collected, generated, and derived\n\n";
echo "vhostsubdomain: $vhostsubdomain\n";
echo "prependanswer: $prependanswer\n";
echo "vhostusername: $vhostusername\n";
echo "genpassanswer: $genpassanswer\n";
if ($echoplaintextpasswords) {
echo "vhostpasswordplain: $vhostpasswordplain\n";
}
echo "vhostpasswordhashed: $vhostpasswordhashed\n";
echo "vhosthomedir: $vhosthomedir\n";
echo "vhostwebdir: $vhostwebdir\n";
// ////////////////////////////////////////////////////////////////
// end print collected values
// ////////////////////////////////////////////////////////////////
// start engine section
// ////////////////////////////////////////////////////////////////
// create the $vhostusername with $vhosthomedir and $vhostpasswordhashed
// build the string, look at the string, then maybe do a shell_exec of the string
$shelluseraddstr = "useradd -m -d '$vhosthomedir' '$vhostusername' -s '/usr/bin/bash' -p '$vhostpasswordhashed'";
// disable for production
// echo "shelluseraddstr: $shelluseraddstr\n";
// so it will always be declared
$shelluseraddret = "";
// disable for testing other conditions without committing to this
$shelluseraddret = shell_exec($shelluseraddstr);
//echo "shelluseraddret: $shelluseraddret\n";
// non-null (non-0) exit value from shell indicates an error
if ($shelluseraddret) {
echo "ERROR: there was a problem executing the shell command to create the vhostusername $vhostusername. Stopping.\n";
exit();
} else {
//echo "SUCCESS: the vhostusername $vhostusername was created\n";
}
echo "\n";
// ////////////////////////////////////////////////////////////////
// mkdir $vhostwebdir
$mkdirvhostwebdirret = mkdir($vhostwebdir,0775,TRUE);
if (!$mkdirvhostwebdirret) {
echo "ERROR: there was a problem creating the vhostwebdir $vhostwebdir. Stopping\n";
exit();
} else {
//echo "SUCCESS: the vhostwebdir $vhostwebdir was created.\n";
}
// ////////////////////////////////////////////////////////////////
// fwrite $vhostwebdir/index.php
$indexfilecontents = "<?PHP\n\necho \"<p>$vhostsubdomain</p>\\n\";\n";
$fh1 = fopen("$vhostwebdir/index.php","w");
$filesuccess1 = fwrite($fh1,$indexfilecontents);
fclose($fh1);
if ($filesuccess1) {
//chown root $vhosthomedir
$vhosthomedirownretu1 = chown("$vhosthomedir","root");
if ($vhosthomedirownretu1) {
//echo "SUCCESS chown root $vhosthomedir\n";
} else {
echo "ERROR chown root $vhosthomedirdir not successful\n";
exit();
}
//chgrp root $vhosthomedir
$vhosthomedirownretg1 = chgrp("$vhosthomedir","root");
if ($vhosthomedirownretg1) {
//echo "SUCCESS chgrp root $vhosthomedir\n";
} else {
echo "ERROR chgrp root $vhosthomedirdir not successful\n";
exit();
}
//echo "SUCCESS indexfile written to file: $vhostwebdir/index.php\n";
// chmod the $vhostwebdir
chmod("$vhostwebdir", 0755);
$vhostwebdirperms = substr(sprintf('%o', fileperms("$vhostwebdir")), -4);
//echo "vhostwebdirperms: $vhostwebdirperms\n";
if ($vhostwebdirperms == "0755") {
//echo "SUCCESS chmod 755 $vhostwebdir\n";
} else {
echo "ERROR chmod 755 $vhostwebdir not successful. Stopping.\n";
exit();
}
// chown $vhostusername $vhostwebdir
$vhostwebdirownretu1 = chown("$vhostwebdir",$vhostusername);
if ($vhostwebdirownretu1) {
//echo "SUCCESS chown $vhostusername $vhostwebdir\n";
} else {
echo "ERROR chown $vhostusername $vhostwebdir not successful\n";
exit();
}
// chgrp $vhostusername $vhostwebdir
$vhostwebdirownretg1 = chgrp("$vhostwebdir",$vhostusername);
if ($vhostwebdirownretu1) {
//echo "SUCCESS chgrp $vhostusername $vhostwebdir\n";
} else {
echo "ERROR chgrp $vhostusername $vhostwebdir not successful\n";
exit();
}
// chmod the $vhostwebdir/index.php
chmod("$vhostwebdir/index.php", 0755);
$vhostindexperms = substr(sprintf('%o', fileperms("$vhostwebdir/index.php")), -4);
//echo "vhostindexperms: $vhostindexperms\n";
if ($vhostindexperms == "0755") {
//echo "SUCCESS chmod 755 $vhostwebdir/index.php\n";
} else {
echo "ERROR chmod 755 $vhostwebdir/index.php not successful. Stopping.\n";
exit();
}
// chown $vhostusername $vhostwebdir/index.php
$vhostindexownretu1 = chown("$vhostwebdir/index.php",$vhostusername);
if ($vhostindexownretu1) {
//echo "SUCCESS chown $vhostusername $vhostwebdir/index.php\n";
} else {
echo "ERROR chown $vhostusername $vhostwebdir/index.php not successful\n";
exit();
}
// chgrp $vhostusername $vhostwebdir/index.php
$vhostindexownretg1 = chgrp("$vhostwebdir/index.php",$vhostusername);
if ($vhostindexownretu1) {
//echo "SUCCESS chgrp $vhostusername $vhostwebdir/index.php\n";
} else {
echo "ERROR chgrp $vhostusername $vhostwebdir/index.php not successful\n";
exit();
}
} else {
echo "ERROR indexfile not written to file: $vhostwebdir/index.php\n";
exit();
}
// ////////////////////////////////////////////////////////////////
// fwrite $bvhostconfdir/$vhostsubdomain.conf
$timestring = date("Y/m/d H:i:s T");
$vhostconffilecontents = "# generated $timestring by addvhost.php\n";
$vhostconffilecontents .= "<VirtualHost $vhostip:80>\n";
$vhostconffilecontents .= "<IfModule mpm_itk_module>\n";
$vhostconffilecontents .= "\tAssignUserID $vhostusername $vhostusername\n";
$vhostconffilecontents .= "</IfModule>\n";
$vhostconffilecontents .= "ServerName $vhostsubdomain\n";
if ( ($prependanswer == "y") || ($prependanswer == "yes") ) {
$vhostconffilecontents .= "ServerAlias www.$vhostsubdomain\n";
}
$vhostconffilecontents .= "DocumentRoot $vhostwebdir\n";
$vhostconffilecontents .= "ServerAdmin $vhostserveradmin\n";
$vhostconffilecontents .= "CustomLog /var/log/apache2/$vhostsubdomain-access_log combined\n";
$vhostconffilecontents .= "ErrorLog /var/log/apache2/$vhostsubdomain-error_log\n";
$vhostconffilecontents .= "</VirtualHost>\n";
// disable in production
// echo "vhostconffilecontents = \n$vhostconffilecontents\n";
// write the text file
$fh2 = fopen("$bvhostconfdir/$vhostsubdomain.conf","w");
$filesuccess2 = fwrite($fh2,$vhostconffilecontents);
fclose($fh2);
if ($filesuccess2) {
//echo "SUCCESS virtual host config written to file: $bvhostconfdir/$vhostsubdomain.conf\n";
} else {
echo "ERROR virtual host config not written to file: $bvhostconfdir/$vhostsubdomain.conf\n";
exit();
}
// ////////////////////////////////////////////////////////////////
// shell_exec a2ensite $bvhostconfdir/$vhostsubdomain.conf
// so it will always be declared
$shella2enret = "";
// disable for testing other conditions without committing to this
$shella2enret = shell_exec("a2ensite $vhostsubdomain.conf");
//echo "shella2enret: $shella2enret\n";
// non-null (non-0) exit value from shell indicates an error
if ( str_contains($shella2enret,"ERROR") ) {
//echo "ERROR: there was a problem executing the shell command to enable the vhostsubdomain $vhostsubdomain. Stopping.\n";
exit();
} else {
echo "SUCCESS: enabled vhostsubdomain $vhostsubdomain\n";
}
// ////////////////////////////////////////////////////////////////
// echo to console suggestion that systemctl restart apache2 be executed
echo "\n";
echo "Next steps:\n";
echo "This script did not restart apache2. That is up to you.\n";
echo "systemctl restart apache2\n";
echo "validate site on port 80\n";
echo "run certbot --apache to expand ssl cert\n";
echo "systemctl restart apache2\n";
echo "validate site on port 443\n";
echo "\n";
// ////////////////////////////////////////////////////////////////
// end engine section
Declaring the host name in DNS
Declare the host name in the DNS zone file for the domain:
Testing that the host name loads a “park page”
Use a web browser to visit the host name:
Executing commands as root
This chapter assumes that you are logged in as the root user. If you are not already root, escalate using this command:
sudo su
Installing the finger, whois, and unzip utilities
This script depends on the finger, whois, and unzip utilities.
Enter this command:
apt install finger whois unzip
Downloading and uncompressing the addvhost.zip file
Completing the addvhost.php questionnaire, entering a username and a plaintext password
In this example, we decline the option to generate a username, and enter a value for the username. We also decline the option to generate a plaintext password, and enter a value for the plaintext password.
root@server01:~# addvhost.php
addvhost.php
Add a virtual host to Apache
Enter domain xxxxxxxx.xxx or subdomain xxxxxxxx.xxxxxxxx.xxx: webpresencestepbystep.ca
Do you wish to prepend the subdomain www.webpresencestepbystep.ca as well (n/y)? y
Generate a username? n
Enter username: webuserca
Generate a plaintext password? n
Enter plaintext password: password
values collected, generated, and derived
vhostsubdomain: webpresencestepbystep.ca
prependanswer: y
vhostusername: webuserca
genpassanswer: n
vhostpasswordplain: password
vhostpasswordhashed: $6$IdLp5YrW.Z3Tvnm$hlRvIBour47UcZrVm0QA2YgLp2z3C3e5W7PwiS3o.KbZz.mtFeCvWdew/eemdec3Wz9t.WEIuIm3Q2EKTuXYd1
vhosthomedir: /usr/web/webuserca
vhostwebdir: /usr/web/webuserca/webpresencestepbystep.ca
SUCCESS: enabled vhostsubdomain webpresencestepbystep.ca
Next steps:
This script did not restart apache2. That is up to you.
systemctl restart apache2
validate site on port 80
run certbot --apache to expand ssl cert
systemctl restart apache2
validate site on port 443
Enter this command:
systemctl restart apache2
Enter this command:
certbot --apache
Enter this command:
systemctl restart apache2
Visiting the virtual host
Use a web browser to visit the host name:
Completing the addvhost.php questionnaire, accepting a generated username and plaintext password
In this example, we accept the option to generate a username. We also accept the option to generate a plaintext password.
Take careful note of the plaintext password value, as shown in the “vhostpasswordplain” field.
root@server01:~# addvhost.php
addvhost.php
Add a virtual host to Apache
Enter domain xxxxxxxx.xxx or subdomain xxxxxxxx.xxxxxxxx.xxx: webpresencestepbystep.com
Do you wish to prepend the subdomain www.webpresencestepbystep.com as well (n/y)? y
Generate a username? y
Generate a plaintext password? y
values collected, generated, and derived
vhostsubdomain: webpresencestepbystep.com
prependanswer: y
vhostusername: webpresencestepbystep_com
genpassanswer: y
vhostpasswordplain: NQeQ2%VT
vhostpasswordhashed: $6$Woe9pPUwnXqUP$9RW60p6SSNfqLJSi4BeAyhe89mBpyTELk2/at7eJcKqou5Q9Y6Nti4P7EoyTV0CBfin6SxlvNHvkZjrpEGxxX0
vhosthomedir: /usr/web/webpresencestepbystep_com
vhostwebdir: /usr/web/webpresencestepbystep_com/webpresencestepbystep.com
SUCCESS: enabled vhostsubdomain webpresencestepbystep.com
Next steps:
This script did not restart apache2. That is up to you.
systemctl restart apache2
validate site on port 80
run certbot --apache to expand ssl cert
systemctl restart apache2
validate site on port 443
Web presence step by step is a series of posts that show you to how to build a web presence.
In this chapter we install and configure dwservice.net to provide remote technical support as an alternative to TeamViewer.
dwservice.net allows a client to share their computer’s desktop so you can provide technical support
When you build and maintain technical systems, you need to support the clients of those systems, including customers, colleagues, and contractors. dwservice.net allows you to invite a client to share their computer’s desktop so you can provide technical support.
Enter the username and password displayed on the Windows workstation running the dwservice.net client, click on “Sign in”:
Resources available on the Windows workstation
This page shows the resources available on the Windows workstation:
Screen (remote desktop)
Remote control of the desktop of the Windows workstation:
Files and Folders
Access to the filesystem on the Windows workstation:
Shell
Access to the operating system shell prompt on the Windows workstation:
Installing the dwservice.net client on a Windows workstation to enable unattended access
Right-click on the “dwagent.exe” file. Select “Run as administrator”:
Select “Install,” click on “Next”:
Select “Yes,” click on “Next”:
Select “Creating a new agent, click on “Next”:
Enter the username and password of a valid dwservice.net account in the “DWS user” and DWS password” fields. Enter a name to describe the workstation in the “Agent name” field. Click on “Next”:
Enter the username and password displayed on the MacOS workstation running the dwservice.net client, click on “Sign in”:
Resources available on the MacOS workstation
This page shows the resources available on the MacOS workstation:
Screen (remote desktop)
Remote control of the desktop of the MacOS workstation:
Files and Folders
Access to the filesystem on the MacOS workstation:
Shell
Access to the operating system shell prompt on the MacOS workstation:
Installing the dwservice.net client on a MacOS workstation to enable unattended access
Click on “Open”:
Select “Install,” click on “Next”:
enter the “User Name” and “Password” for the MacOS workstation, click on “OK”:
Click on “Next”:
Select “Yes,” click on “Next”:
Select “Creating a new agent,” click on “Next”:
Enter the username and password of a valid dwservice.net account in the “DWS user” and DWS password” fields. Enter a name to describe the workstation in the “Agent name” field. Click on “Next”:
Enter the username and password displayed on the Linux workstation running the dwservice.net client, click on “Sign in”:
Resources available on the Linux workstation
This page shows the resources available on the Linux workstation:
Screen (remote desktop)
Remote control of the desktop of the Linux workstation:
Files and Folders
Access to the filesystem on the Linux workstation:
Shell
Access to the operating system shell prompt on the Linux workstation:
Installing the dwservice.net client on a Linux workstation to enable unattended access
Open a terminal window on the Linux desktop. Change to the folder where the file was downloaded.
Use “sudo su” to become root if you have not already done so. enter these commands:
Enter this command:
bash dwagent.sh
Select “Install,” click on “Next”:
Click on “Next”:
Select “Yes,” click on “Next”:
Enter the username and password of a valid dwservice.net account in the “DWS user” and DWS password” fields. Enter a name to describe the workstation in the “Agent name” field. Click on “Next”:
Creating the MySQL database that will store data for the Live Helper Chat software
Enter this command:
mysql -u root -p
In the MySQL console, enter these commands (where ‘xxxxxx’ is your password):
create user lhc01@localhost;
set password for lhc01@localhost = 'xxxxxx';
create database lhc01;
use lhc01;
grant all privileges on * to lhc01@localhost;
quit
Downloading the zip file containing the Live Helper Chat software
Using Filezilla, connect to the Ubuntu Linux cloud server you created in Chapter 3: Buying an Ubuntu Linux cloud server from Digital Ocean. On the right (remote) side, select the directory that contains the documents for your website. On the left (local) side, select the directory that contains the “lhc_web” directory. Right-click or command-click on the directory, select Upload:
The transfer completes:
Installing the Live Helper Chat software using the Live Helper Chat installation wizard
Complete the field as shown below. Click “Finish installation”:
Click “Login here”:
Generating the Live Helper Chat JavaScript text to add to an HTML header
(Note: for our example domain, we can visit this site at https://linuxstepbyste.com/lhc_web/index.php/site_admin)
Enter your username and password. Click “Login”:
Click on “Settings”:
Under “Mobile,” click on “Settings”:
Select the checkbox “Enable notifications,” click on “Save”:
Click on “Settings”:
Click on “Embed code”:
Click on “Widget embed code (new):
Complete the fields and select the options as required for your site. Copy the code from the text area to the page where you would like the web chat window to appear:
Embedding Live Helper Chat in a WordPress site
To embed the Live Helper Chat application in a WordPress site, we must first install the Insert Headers and Footers plugin for WordPress.
Installing the Insert Headers and Footers WordPress plugin
In your WordPress control panel, click on “Plugins,” then click on “Add New”:
In the search window, enter the text “insert headers and footers,” locate the plugin “Insert Headers and Footers,” click on “Install Now”:
Click on “Activate”:
Click on “Settings,” then click on “Insert Headers and Footers”:
Adding the JavaScript code using the Insert Headers and Footers WordPress plugin
Complete the fields as shown.
Scroll to the bottom of the page. Paste the JavaScript code you generated earlier in the section “Generating the Live Help Chat JavaScript text to add to an HTML header” to the field “Scripts in Footer,” click “Save”:
Cick on “Visit Site”:
The chat window is visible on the WordPress site:
Texting between the site’s visitors and the site’s operators:
Embedding Live Helper Chat in a simple web page
We can add the Live Helper Chat text chat window to a simple web page.
Using an SSH terminal program, connect to your Ubuntu Linux cloud server. Change to the directory containing the documents for your website. Enter the command:
nano test.html
Paste the JavaScript code you generated in the section “Generating the Live Help Chat JavaScript text to add to an HTML header,” press Control-X to save and exit the file:
Enter these commands (where username is the username that owns the documents for the website):
From the MySQL console prompt, enter these commands (use a unique password in place of ‘xxxxxx’):
create user bb01@localhost;
set password for bb01@localhost = 'xxxxxx';
create database bb01;
use bb01;
grant all privileges on * to bb01@localhost;
quit
Installing the php-gd extension
In order to generate graphics representing captchas, we need to install the php-gd extension.
Enter the command:
apt install php-gd
Enter the command:
systemctl restart apache2
Configuring a Gmail account to allow MyBB to send email messages
Uncompressing the zip file extracts 2 directories: “Documentation,” and “Upload”:
Renaming the directory named “Upload” to “community”
Rename the “Upload” directory to “community”:
Using FileZilla to upload the community directory to the directory containing the documents for your website
We will use the FileZilla file transfer program to upload the directory renamed as “community” to the directory containing the documents for your website.
If you need detailed instructions on how to install and use FileZilla on Windows, MacOS, or Linux
Using Filezilla, connect to the Ubuntu Linux cloud server you created in Chapter 3: Buying an Ubuntu Linux cloud server from Digital Ocean. On the right (remote) side, select the directory that contains the documents for your website. On the left (local) side, select the directory that contains the “community” directory. Right-click or command-click on the directory, select Upload:
The transfer completes:
Installing the MyBB software using the MyBB installation wizard
For our example domain, we will visit:
https://linuxstepbystep.com/community
(substitute your domain name)
Click “Next”:
Click “Next”:
Click “Next”:
Complete the fields as show below. Click “Next”:
Scroll to the bottom of the web page. Click “Next”:
Click “Next”:
Click “Next”:
Complete the fields as show below. Click “Next”:
Complete the fields as show below. Click “Next”:
This screen will display when we have finished:
Visiting the community forum site
For our example domain, we will visit:
https://linuxstepbystep.com/community
(substitute your domain name)
Logging into the admin account
Enter the username and password for the admin user. Click “Login”:
Click on “Admin CP”:
Enter the username and password for the admin user. Enter the secret PIN. Click “Login”:
Configuring the MyBB community forum site
Click on “Configuration”:
Scroll towards the bottom of the web page:
Click on “Mail Settings”:
Configure settings for “Mail handler”
Mail handler: select "SMTP mail" SMTP hostname: smtp.gmail.com SMTP port: 587 SMTP username: the address of the gmail account you selected or created earlier in this procedure SMTP Encryption Mode: select "TLS encryption"
Click “Save Settings”:
Click on “Login and Registration Options”:
For “Registration Method,” select “Send Email Verification”:
Scroll to the bottom of the web page. Click on “Save Settings”:
Registering as a new user on the community forum site
Use a web browser to visit the community forum site. Click on “Register.” On the page titled “Registration Agreement,” consider then if you decide click “I agree”:
Complete the fields as show in the example below. Click “Submit Registration!”:
MyBB displays this message:
Check the email account you provided during registration. View the message sent by MyBB community forum site:
To activate your account, click on the link in the email:
Welcome as a member:
We will create a post. Click on “My Forum”:
Click on “Post Thread”:
In the field “Thread Subject:” enter a subject.
In the field “Your Message:” enter the text of a message.
