Severed/Severed-Blog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

refactor + added more notes

Browse Source
This commit is contained in:
wboughattas
2026-01-02 15:55:39 -05:00
parent 72871e3840
commit 74a8527e2c
2 changed files with 173 additions and 78 deletions
Download Patch File Download Diff File Expand all files Collapse all files

68
_posts/homelab/2025-12-31-homelab-part1.md
View File

@@ -31,53 +31,52 @@ vPro and ThinkCentre BIOS, which is essential for the advanced BIOS settings tha
For this setup, we will not take advantage of vPro's features such as accessing the BIOS over the network, but we will
take advantage of features that come with ThinkCentre BIOS such as the auto-power on after a power loss.
# ThinkCentre M720q: Node Configuration
I already created bootable USB drive for debian 13 ISO using rufus.
## 1. Initial Verification
# ThinkCentre M720q: Node Configuration
- **Enter BIOS:** Press repeatedly the **F1** key.
- **Hardware Audit:** Confirm the CPU, RAM, and Storage match what is expected. Reset BIOS to default settings if it's
the first time.
- **Set the Standard:** Set the system clock to **UTC** if it's not the case. This is necessary for Kubernetes because
it ensures that logs from different nodes align perfectly regardless of local time zones.
## 2. Hardware Decoupling
We want to strip away desktop features to save power and reduce the potential attack surface.
| Menu Path | Setting | Action | Why |
| :-------------------------- | :------------------ | :-------------------------- | :--------------------------------------------------------- |
| **Devices > Audio Setup** | Integrated Audio | **Disabled** | Servers don't need sound |
| **Devices > Network Setup** | Wi-Fi / BT / PXE | **Disabled** | Forces the node to rely on the Onboard Ethernet. |
| **Devices > USB Setup** | USB Legacy Support | **Disabled** | Prevents the use of less secure USB protocols during boot. |
| **Advanced > CPU Setup** | Virtualization | **Enabled** | Needed for container runtimes. |
| **Advanced > CPU Setup** | VT-d | **Enabled** | IO magic. |
| **Power** | After Power Loss | **Power On** | The Auto-Restart rule. |
| **Power** | Intelligent Cooling | **Performance or Acoustic** | Either to prevent thermal throttling or lower noise. |
| Menu Path | Setting | Action | Why |
| :-------------------------- | :--------------------------------- | :------------------------------------------------------------------------------------------------------------ | :--------------------------------------------------------------- |
| **Devices > Audio Setup** | Integrated Audio | **Disabled** | Servers don't need sound |
| **Devices > Network Setup** | Wi-Fi / PXE | **Disabled** | Forces the node to rely on the Onboard Ethernet. |
| **Devices > USB Setup** | USB Legacy Support | **Disabled** | Prevents the use of less secure USB protocols during boot. |
| **Advanced > CPU Setup** | Virtualization | **Enabled** | Needed for container runtimes. |
| **Advanced > CPU Setup** | VT-d | **Enabled** | IO magic. |
| **Power** | After Power Loss | **Power On** | The Auto-Restart rule. |
| **Power** | Intelligent Cooling | **Performance or Acoustic** | Either to prevent thermal throttling or lower noise. |
| **Security** | Administrator Password | **Set** | Prevents tampering with the BIOS settings. |
| **Security** | Windows UEFI Update | **Disabled** | We are replacing Windows with Debian. |
| **Security** | Password for F1/F12 | **Yes** | Requires your admin password to boot from an unauthorized USB. |
| **Security** | Require POP on System Boot/Restart | **No** | Overkill. |
| **Security** | POP Changeable by User | **No** | Requires your admin to change his password. |
| **Security** | Secure Boot | **Enabled** | Verifies the Debian kernel signature before allowing it to boot. |
| **Startup** | Boot Sequence | **Move the drive(s) to the #1 spot (prioritize the one storing the OS Bootloader). Exclude everything else.** | |
## 3. Security Governance
Debian installation is straightforward. Set most settings to default.
| Menu Path | Setting | Action | Why |
| :----------- | :--------------------- | :----------- | :--------------------------------------------------------------- |
| **Security** | Administrator Password | **Set** | Prevents tampering with the BIOS settings. |
| **Security** | Windows UEFI Update | **Disabled** | We are replacing Windows with Debian. |
| **Security** | Password for F1/F12 | **Yes** | Requires your admin password to boot from an unauthorized USB. |
| **Security** | POP Changeable by User | **No** | Requires your admin to change his password. |
| **Security** | Secure Boot | **Enabled** | Verifies the Debian kernel signature before allowing it to boot. |
## 4. Boot Sequence
**Startup > Boot Sequence:** Move the drive(s) to the #1 spot (prioritize the one storing the OS Bootloader). Exclude
everything else.
**Startup > CSM:** must be disabled to restrict non-UEFI operating systems.
## 5. Post-Install
## Post-Install
Once Debian is running, we need to tell the OS to take ownership of the hardware management. In a better world, we
prefer to take full ownership ourselves using CoreBoot. But we cannot because Intel Boot Guard is supported on the
hardware level. Since Lenovo ships these with Intel Boot Guard (Verified Boot) enabled, we must work within the
manufacturer's ecosystem.
### Disable the GUI on boot
```bash
# Switch to root
su -
sudo systemctl set-default multi-user.target
reboot
```
### Synchronize the Hardware Clock
This command tells Debian to treat the motherboard's clock as UTC (the server standard).
@@ -86,6 +85,13 @@ This command tells Debian to treat the motherboard's clock as UTC (the server st
sudo timedatectl set-local-rtc 0
```
Add missing locales. In my case, `en_CA.UTF-8`.
```bash
sudo dpkg-reconfigure locales
# Select en_CA.UTF-8 (or your preference) -> OK
```
### Enable Linux-Native Firmware Updates
We disabled the Windows update hook in the BIOS; now we replace it with the **Linux Vendor Firmware Service (LVFS)**.

183
_posts/homelab/2026-01-01-homelab-part2.md
View File

@@ -14,11 +14,11 @@ our ISP does not provide a Static IP, and we want to avoid exposing ports on our
DDNS services. So we implement a VPN, specifically a hub-and-spoke VPN using an EC2 server (with a static IPV4):
- **The Hub (EC2):** A tiny Amazon Linux instance (`t4g.nano`) acts as a Relay and a static Public IP.
- **The Spokes (Nodes):** Home nodes initiate _outbound_ connections to the EC2 Hub. we bypass the need for home
- **The Spokes (Nodes):** Home nodes initiate outbound connections to the EC2 Hub. we bypass the need for home
port forwarding.
- **The Client (Personal Mac):** Connects to the EC2 Hub to reach the home nodes.
- **Security:** Zero trust. SSH ports on home nodes are closed to the local network and only listen on the VPN
interface. Access requires both the specific WireGuard Private Key and the SSH Key.
- **Security:** SSH ports on home nodes are closed to the local network and only listen on the VPN interface. Access
requires both the specific WireGuard Private Key and the SSH Key.
## 2. Network Map
@@ -30,16 +30,14 @@ DDNS services. So we implement a VPN, specifically a hub-and-spoke VPN using an
| **Node 2** | Server | `10.100.0.11` | `192.168.2.251` | (Hidden) |
| **Node 3** | Server | `10.100.0.12` | `192.168.2.252` | (Hidden) |
---
## 3. Phase 1: EC2 Setup (The Hub)
## 3. Phase 1: The EC2 Relay (The Hub)
_OS: Amazon Linux 2023_
OS: Amazon Linux 2023
### 3.1 Initial Setup
1. Launch a `t4g.nano` instance.
2. **Security Group:** Allow UDP Port `51820` from `0.0.0.0/0`.
2. **Security Group:** Allow inbound UDP Port `51820` from `0.0.0.0/0` . All traffic for outbound.
3. **Source/Dest Check:** Go to EC2 Console > Actions > Networking > **Change Source/Destination check** > **Stop** (
Required for routing).
@@ -58,9 +56,13 @@ sudo sysctl -p /etc/sysctl.d/99-wireguard-forward.conf
priv=$(wg genkey); pub=$(echo "$priv" | wg pubkey); echo "Private: $priv"; echo "Public: $pub"
```
### 3.3 Configure WireGuard (`/etc/wireguard/wg0.conf`) (EC2 proxy)
### 3.3 Configure WireGuard (EC2 proxy)
_We will replace `<KEYS>` with actual values as we set up the nodes._
```bash
sudo nano /etc/wireguard/wg0.conf
```
We will replace `<KEYS>` with actual values as we set up the nodes.
```ini
[Interface]
@@ -93,13 +95,13 @@ PublicKey = <NODE_3_PUBLIC_KEY>
AllowedIPs = 10.100.0.12/32
```
**Start Service:** `sudo wg-quick up wg0`
Start Service: `sudo wg-quick up wg0`
---
## 4. Phase 2: Node Setup (The Spokes)
_OS: Debian (Headless). Repeat for each node._
OS: Debian (Headless). Repeat for each node.
### 4.1 Root Access & Users
@@ -107,29 +109,42 @@ _OS: Debian (Headless). Repeat for each node._
# Switch to root
su -
# Create generic user if needed (optional) or just rely on root
# Verify sudo is installed
apt update && apt install sudo -y
apt update
```
### 4.2 Temporary IP Setup
_Avoids conflicts with Home Router DHCP immediately._
Even though we will set a static IP on the Debian node itself, your router might still try to claim that IP for another
device via DHCP. Ensure you go into your router settings and reserve those IPs by marking them as "Static" in the
device list or by shrinking the DHCP IPs range to .. Reboot the nodes to apply the changes.
```bash
ip link show
```
Remember the interface name, in my case `enp0s31f6`.
```bash
# Example for Node 2 (.251) - Adjust for .250 or .252
ip addr flush dev enp0s31f6
ip addr add 192.168.2.251/24 dev enp0s31f6
ip link set enp0s31f6 up
ip route add default via 192.168.2.1
sudo ip addr flush dev enp0s31f6
sudo ip link set enp0s31f6 up
sudo ip addr add 192.168.2.251/24 dev enp0s31f6
sudo ip route add default via 192.168.2.1
```
### 4.3 SSH Hardening & Firewall (Temporary Access)
```bash
ping -c 3 192.168.2.1
ip addr show
```
_We allow local SSH temporarily to copy-paste keys._
### 4.3 SSH Setup & Firewall (Temporary Access)
We allow temporary local SSH temporarily to copy-paste keys.
```bash
apt install ufw openssh-server -y
apt install ufw openssh-server resolvconf -y
# ssh server auto started
# we will update ssh config in the next step
# 1. Firewall Basics
ufw default deny incoming
@@ -139,36 +154,76 @@ ufw default allow outgoing
# Replace 192.168.2.147 with your Mac's current local IP
ufw allow from 192.168.2.147 to any port 22 proto tcp
ufw enable
# 3. SSH Config
nano /etc/ssh/sshd_config
# Change:
# PermitRootLogin yes
# PasswordAuthentication no
systemctl restart ssh
```
### 4.4 Keys & Permanent Access (On Mac)
1. **Generate Key:** `ssh-keygen -t ed25519 -f ~/.ssh/home-server`
2. **Copy to Node:** `ssh-copy-id -i ~/.ssh/home-server root@192.168.2.251`
3. **Test:** `ssh root@192.168.2.251`
1. Generate Key: `ssh-keygen -t ed25519 -f ~/.ssh/home-server`
2. Copy to Node: `ssh-copy-id -i ~/.ssh/home-server root@192.168.2.251`
3. Update SSH rules:
```bash
# 3. SSH Config
nano /etc/ssh/sshd_config
# Change:
# PermitRootLogin yes
# PasswordAuthentication no
# Find the line AcceptEnv LANG LC_* and comment it out
systemctl restart ssh
```
4. Test: `ssh root@192.168.2.251` which should not work without ssh key and
`ssh -i ~/.ssh/home-server root@192.168.2.251` which should work.
### 4.5 Permanent Network Config
_Make the Static IP survive a reboot._
`nano /etc/network/interfaces`
Make the Static IP survive a reboot. Before that, we must disable `NetworkManager` to prevent it from fighting with our
manual configuration, and enable `resolvconf` to handle DNS.
```bash
# 1. Stop the automated manager
sudo systemctl stop NetworkManager
sudo systemctl disable NetworkManager
# 2. Enable the DNS helper
sudo systemctl enable --now resolvconf
```
Now, let's edit the interface file to make the IP survive a reboot,
```bash
sudo nano /etc/network/interfaces
```
Append at the end,
```text
# The Primary Network Interface
auto enp0s31f6
iface enp0s31f6 inet static
address 192.168.2.251 # <--- change per node
address 192.168.2.251
netmask 255.255.255.0
gateway 192.168.2.1
dns-nameservers 1.1.1.1 8.8.8.8
dns-nameservers 1.1.1.1 8.8.8.8 192.168.2.1
```
Test the syntax before rebooting,
```bash
sudo ifup -v --no-act enp0s31f6
```
and reboot to apply the changes,
```bash
reboot
```
On mac,
```bash
ping 192.168.2.251
```
### 4.6 WireGuard Setup (nodes)
@@ -180,7 +235,7 @@ iface enp0s31f6 inet static
priv=$(wg genkey); pub=$(echo "$priv" | wg pubkey); echo "Private: $priv"; echo "Public: $pub"
```
3. **Configure (`/etc/wireguard/wg0.conf`):**
3. **Configure (`sudo nano /etc/wireguard/wg0.conf`):**
```ini
[Interface]
@@ -299,12 +354,45 @@ Host node3
## 6. Troubleshooting
### 1. Check the Handshakes
### Check the Handshakes
On every machine (Mac, EC2, Nodes), run:
On every node, run:
```bash
sudo wg show
# must enabled and can see latest handshake and transfer
sudo systemctl status networking
# must be enabled
sudo systemctl status NetworkManager
# must be disabled
sudo systemctl status ssh
# must be enabled
sudo systemctl status resolvconf
# must be enabled
cat /etc/resolv.conf
# nameserver 1.1.1.1
# nameserver 8.8.8.8
# nameserver 192.168.2.1
ufw numbered
# Status: active
#
# To Action From
# -- ------ ----
# 22/tcp on wg0 ALLOW Anywhere
# 22/tcp (v6) on wg0 ALLOW Anywhere (v6)
```
On mac and EC2, run
```bash
sudo wg show
# must enabled and can see latest handshake and transfer
```
We want to see `latest handshake: X seconds ago`. If "Handshake: None":
@@ -313,7 +401,7 @@ We want to see `latest handshake: X seconds ago`. If "Handshake: None":
- Ensure AWS Security Group allows inbound **UDP 51820** from `0.0.0.0/0` and all traffic for outbound.
- Ensure the Nodes have the correct EC2 Public IP in their `Endpoint` field.
### 2. "Destination Host Unreachable"
### Destination Host Unreachable
If you can ping EC2 (`10.100.0.1`) but not the Nodes (`10.100.0.11`):
@@ -323,13 +411,13 @@ If you can ping EC2 (`10.100.0.1`) but not the Nodes (`10.100.0.11`):
- Ensure `iptables` is installed on Amazon Linux (`sudo dnf install iptables -y`) and that your `PostUp` rules in
`wg0.conf` are active.
### 3. Connection Stalls (MTU Issues)
### Connection Stalls (MTU Issues)
If SSH connects but "hangs" or freezes, or if ping works but SSH doesn't, you likely have an MTU (Packet Size) issue.
This is common with home ISPs (PPPoE).
- **Fix:** Lower the MTU on the Node interface.
- Edit `/etc/wireguard/wg0.conf` on the Node:
- Edit `sudo nano /etc/wireguard/wg0.conf` on the Node:
```ini
[Interface]
@@ -339,7 +427,7 @@ MTU = 1280
- Restart WireGuard: `sudo systemctl restart wg-quick@wg0`
### 4. SSH "Permission Denied" (Public Key)
### SSH "Permission Denied" (Public Key)
If you see a handshake but SSH rejects you:
@@ -347,7 +435,7 @@ If you see a handshake but SSH rejects you:
- On the Node, run `tail -f /var/log/ufw.log`. If you see blocks on Port 22, your UFW rule `ufw allow in on wg0` might
be missing.
### 5. "Remote Host Identification Has Changed"
### "Remote Host Identification Has Changed"
Since we moved from a direct connection to a VPN, your Mac might think the IP `10.100.0.x` is being spoofed.
@@ -357,4 +445,5 @@ Since we moved from a direct connection to a VPN, your Mac might think the IP `1
ssh-keygen -R 10.100.0.1
ssh-keygen -R 10.100.0.10
ssh-keygen -R 10.100.0.11
ssh-keygen -R 10.100.0.12
```