Skip to main content

Prerequisites

  • Internet Gateway. (for apt and opkg)
  • DHCP static reservation. (OpenWRT)
  • DNS Server with a domain name. (OpenWRT) # Optional
  • Static IP on each host using NetworkManager or dhcpcd. Install network-manager or dhcpcd package, comment out the interface from /etc/network/interfaces and /etc/network/interfaces.d. Set the network configuration using NetworkManager system connections(nmcli/nmtui).
  • Time sync using systemd-timesyncd, make sure to remove ntp package.
  • hostname and domain name set on each host. # Optional
  • Make sure contrib non-free non-free-firmware added to http://deb.debian.org/debian repository for few proprietary packages like nvidia-driver.
  • For old nvidia cards (like NVIDIA GeForce MX250) manually pin the version in /etc/apt/preferences.d/nvidia like bellow
# Basic driver packages, includes foreign architectures
Package: src:nvidia-graphics-drivers:any src:nvidia-kmod-open:any
Pin: version 550*
Pin-Priority: 1000

# Basic driver packages, only in the native architectures
Package: src:nvidia-modprobe src:nvidia-modprobe src:nvidia-persistenced src:nvidia-settings src:nvidia-xconfig
Pin: version 550*
Pin-Priority: 1000

# Meta packages, includes foreign architectures
# cuda-compat, built from cuda-drivers must not be part of the pinning
Package: cuda-drivers* src:nvidia-open:any
Pin: version 550*
Pin-Priority: 1000

# Extra driver packages, only in the native architectures
Package: src:libnvidia-nscq src:libnvsdm src:nvidia-fabricmanager src:nvidia-imex src:nvlink5
Pin: version 550*
Pin-Priority: 1000

Router port forwarding​

Forward the following ports on the router to the appropriate hosts. Only services meant to be internet-facing are listed; services reverse-proxied internally (Collabora, Whiteboard, Imaginary, Nextcloud Talk's signaling port, etc.) must not be forwarded.

ServiceProtocolPortNotes
Reverse Proxy (Caddy)TCP80, 443HTTP/HTTPS
Reverse Proxy (Caddy)UDP443HTTP/3
Reverse Proxy (Nginx Proxy Manager)TCP80, 443HTTP/HTTPS; only one reverse proxy backend runs at a time
Nextcloud TalkTCPcs_nc_talk_turn_port (Vault: talk_turn_port)coturn TURN relay port
Nextcloud TalkUDPcs_nc_talk_turn_port (Vault: talk_turn_port)coturn TURN relay port
WireGuard VPNUDPits configured listen port (Vault)WireGuard listen port

Services:

  • Set up a DevOps Server (any server providing containers, PyPI/generic artifacts, CI/CD runners, and code hosting).

Tools:

  • Install uv and dependencies.

Localhost variables​

OptionTypeDescriptionDefault
ansible_connectionstringAnsible connection typelocal
ansible_python_interpreterstringPath to the Python interpreter for Ansible on local./.venv/bin/python
ansible_hoststringThe name/IP of the host to connect to for localhostlocalhost

Global variables​

These variables are defined in the all.vars section of inventory.yml or in group_vars/all/all.yml.

Every file inside group_vars/all/ (all.yml, patchservers.yml, etc.) applies to every host in the inventory: Ansible merges all of them for the all group regardless of filename. The filenames only organize variables by topic (cluster-wide config in all.yml, patch-stage bootstrapping in patchservers.yml); they do not scope a variable to hosts in the patchservers group.

OptionTypeDescriptionDefault
cs_cluster_namestringName of the clusterblr-home
cs_project_codestringProject code for organization/secretsCS_PROJECT_CODE env variable (mandatory)
cs_env_debugbooleanShow task output (from the DEBUG environment variable)false
cs_no_logbooleanDisable logging of sensitive taskstrue
cs_force_rotate_leaf_certsboolForce rotate leaf private keys, CSRs, and certificatestrue

The cluster's domain name, network CIDRs, DNS servers, SMTP configuration, and root CA material are all read from Vault at runtime rather than set here; see the Vault data source section for each secret's key path and schema.

Environment Variables:

CS_PROJECT_CODE="Project code, mandatory. Resolves to the Ansible variable cs_project_code."
DEBUG="Optional. Set to true to show task output; resolves to cs_env_debug, and cs_no_log is its negation."
DEVOPS_SERVER_INVENTORY_HOSTNAME="Inventory hostname of the DevOps server host, mandatory. Selects the DevOps server's Vault path, which supplies the artifact registry used by every play that pulls images or the Restic binary."
VAULT_API_ENDPOINT="API Endpoint, https://127.0.0.1:8083"
VAULT_API_KEY="API Key"
VAULT_CA_CERT_BASE64="Base64 encoded Vault CA certificate."
VAULT_CLIENT_CERT_BASE64="Base64 encoded Vault mTLS client certificate."
VAULT_CLIENT_KEY_BASE64="Base64 encoded Vault mTLS client private key."

These variables can also be placed in a .env file in the project root. The vault_kv_get lookup plugin, the vault_kv_mod action plugin, and every filter plugin in filter_plugins/ load it via python-dotenv at import time, so .env values are picked up automatically without exporting them manually.