QNAP TS-473A with CentOS Stream 9
Table of Contents
In happy distro hopping, I installed CentOS Stream 9 on my QNAP TS-473A.
These are my installation notes. They are similar to my RHEL8 notes and my Fedora Server notes.
Used Hardware
- QNAP TS-473 4-bay NAS with AMD Ryzen Embedded V1500B 4-core/8-thread @ 2.2 GHz CPU
- QNAP QXG-10G2SF-CX4 2x 10 GbE SFP+ network card
- ASUS GeForce GT 1030 BRK 2.0 GB GPU
- 500GB Samsung SSD 980 NVMe M.2 2280 PCIe 3.0 V-NAND MLC
- 2TB Crucial P2 M.2 NVMe
- 64 GiB RAM, G.Skill F4-3200C22D-64GRS (that is a kit containing 2x 32G SO-DIMMs that each report as F4-3200C22-32GRS)
- 4x 1TB HDD, for now, these might be replaced later with something larger
Firmware Settings
- ensure you have added a GPU to the TS-x73A
- connect screen and keyboard
- enter firmware setup by pressing Del or Esc during power on self test (POST)
- Boot / Quiet Boot: Disabled (simply so I get shown on screen which key to press during POST to enter UEFI)
- Boot / Boot Option Priorities: as you see fit. I disabled USB DISK MODULE PMAP and reordered the others to my liking.
- Save & Exit: Save Changes and Exit
Note that if you ever want to return to QTS, you must re-enable the USB DISK MODULE PMAP to be able to successfully boot from it by selecting it at Save & Exit / Boot Override.
Firmware Details
As of 2021-12-18 I have Aptio Setup Utility Version 2.20.1274:
| description | value |
|---|---|
| BIOS Vendor | American Megatrends |
| Core Version | 5.14 |
| Compliancy | UEFI 2.7; PI 1.6 |
| Project Version | Q07DAR12 |
| Build Date and Time | 05/03/2021 10:59:15 |
| Total Memory | Total Memory 65536 MB (DDR4) |
| Memory Frequency | 2400 MHz |
| EC Version | Q07DE008 |
Kickstart Install of CentOS Stream 9
Much like previous installs, I started my kickstart from a previous linux install.
While the QNAP TS-473A boots from a Fedora Server USB stick just fine, like it does from a RHEL stick, and one can instyyall interactively just fine, I prefer to automatically install with kickstart. While I could just modify the boot entry when starting from a stick, I find it easier to simply put the kernel and initrd from Fedora Everything onto the QNAP’s /boot/ partition and add a custom menu entry to grub.
While I do this with Ansible and my local Fedora Everything mirro, any method is fine.
The Ansible tasks should be self explanatory.
Create the file, run grub2-mkconfig -o …, reboot, choose the kickstart target.
- name: "GRUB | ensure initrd for CentOS Stream 9 kickstart is present"
get_url:
url: "http://fileserver.internal.pcfe.net/ftp/distributions/CentOS/9-stream/DVD/x86_64/images/pxeboot/initrd.img"
dest: "/boot/initrd-kickstart-cos9.img"
mode: "0600"
- name: "GRUB | ensure kernel for CentOS Stream 9 kickstart is present"
get_url:
url: "http://fileserver.internal.pcfe.net/ftp/distributions/CentOS/9-stream/DVD/x86_64/images/pxeboot/vmlinuz"
dest: "/boot/vmlinuz-kickstart-cos9"
mode: "0755"
- name: "GRUB | ensure kickstarting CentOS Stream 9 entry is present"
copy:
dest: "/etc/grub.d/12_cos9_kickstart"
owner: "root"
group: "root"
mode: 0755
content: |
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
menuentry "WARNING Kickstart this box with CentOS Stream 9 as a TS-473A ceph node WARNING" {
linuxefi /vmlinuz-kickstart-cos9 inst.kdump_addon=on ip=enp6s0:dhcp inst.repo=http://fileserver.internal.pcfe.net/ftp/distributions/CentOS/9-stream/DVD/x86_64/ inst.ks=http://fileserver.internal.pcfe.net/ftp/kickstart/CentOSstream9-x86_64-QNAP-TS-473A-ks.cfg
initrdefi /initrd-kickstart-cos9.img
}
notify: grub2-mkconfig | run
My Kickstart file F35-QNAP-TS-473A-ks.cfg (click the triangle to expand)
# Generated by Anaconda 35.22.2
# Generated by pykickstart v3.34
# changed by pcfe, 2022-01-29
#version=DEVEL
# avoid using half arsed names like sda, sdb, etc
# TS-473A User Guide, page 10, says
# top is M.2 SSD slot 1
# lower is M.2 SSD slot 2
# Disks bays are numbered starting from 1, bay furthest away from the power button.
# for PCIe slots, the user guide says top is slot 1, bottom is slot 2
#
# NVMe slot 1 /dev/disk/by-path/pci-0000:03:00.0-nvme-1 (the top slot, contains a Samsung 980 500GB)
# NVMe slot 2 /dev/disk/by-path/pci-0000:04:00.0-nvme-1 (the bottom slot, contains a Crucial P2 2TB)
# HDD bay 1 /dev/disk/by-path/pci-0000:07:00.0-ata-1 (bay furthest away from the power button)
# HDD bay 2 /dev/disk/by-path/pci-0000:07:00.0-ata-2
# HDD bay 3 /dev/disk/by-path/pci-0000:09:00.0-ata-1
# HDD bay 4 /dev/disk/by-path/pci-0000:09:00.0-ata-2 (bay closest to the power button)
# reboot after installation is complete?
reboot
# Use graphical install
graphical
# Keyboard layouts
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_US.UTF-8 --addsupport=de_DE.UTF-8,de_LU.UTF-8,en_DK.UTF-8,en_GB.UTF-8,en_IE.UTF-8,fr_FR.UTF-8,fr_LU.UTF-8
# Network information
# all switch ports have the respective VLAN as native
# 2.5 Gig on-board 1 ('access' network)
network --bootproto=dhcp --device=enp6s0 --ipv6=auto --activate
# 2.5 Gig on-board 2 (will go on 'storage' via ansible)
network --bootproto=dhcp --device=enp5s0 --onboot=off --ipv6=auto --no-activate
# 10 Gig on PCIe (will go on 'ceph' via ansible)
network --bootproto=dhcp --device=enp2s0f0np0 --onboot=off --ipv6=auto --no-activate
# 10 Gig on PCIe slot 2 (PCIe 3.0 x4), currently unused
network --bootproto=dhcp --device=enp2s0f1np0 --onboot=off --ipv6=auto --no-activate
# Use network installation
url --url="http://fileserver.internal.pcfe.net/ftp/distributions/CentOS/9-stream/DVD/x86_64"
# The AppStream repo
repo --name="AppStream" --baseurl="http://fileserver.internal.pcfe.net/ftp/distributions/CentOS/9-stream/DVD/x86_64/AppStream"
# Package groups to install
# see https://docs.fedoraproject.org/en-US/fedora/f35/install-guide/appendixes/Kickstart_Syntax_Reference/#sect-kickstart-packages
# For Ceph use, '@^server-product-environment' should be enough. The Ceph installer pulls in what is needed.
# For general Fedora Server use, I also had '@container-management' and '@domain-client'.
%packages
@^server-product-environment
%end
# Run the Setup Agent on first boot
firstboot --enable
# we only install to the 500GB Samsung NVMe, that is in _M.2 SSD slot 1_, the top slot.
ignoredisk --only-use=/dev/disk/by-path/pci-0000:03:00.0-nvme-1
# Partition clearing information
# note that OS goes on a small portion os the device in bay 1, the rest will be allocated to Ceph in a separtate VG.
# so kickstarting with the below clearpart line will nuke the Ceph bits on SSD !!!
clearpart --all --initlabel --drives=/dev/disk/by-path/pci-0000:03:00.0-nvme-1
# Disk partitioning information
# the 500GB Samsung NVMe in slot 1 will be fully used for the OS
# the 2TB Crucial NVMe in slot 2 and the HDDs in slots 1 through 4
# will be fed to ceph-ansible as devices
# c.f. https://docs.ceph.com/ceph-ansible/master/osds/scenarios.html
part /boot --fstype="ext4" --ondisk=/dev/disk/by-path/pci-0000:03:00.0-nvme-1 --size=1024
part /boot/efi --fstype="efi" --ondisk=/dev/disk/by-path/pci-0000:03:00.0-nvme-1 --size=200 --fsoptions="umask=0077,shortname=winnt"
part pv.01 --fstype="lvmpv" --ondisk=/dev/disk/by-path/pci-0000:03:00.0-nvme-1 --size=18500 --grow
volgroup VG_OS --pesize=4096 pv.01
logvol / --fstype="xfs" --size=4096 --name=LV_root --vgname=VG_OS
logvol swap --fstype="swap" --size=1024 --name=LV_swap --vgname=VG_OS
logvol /var --fstype="xfs" --size=4096 --name=LV_var --vgname=VG_OS
logvol /var/crash --fstype="xfs" --size=1024 --name=LV_var_crash --vgname=VG_OS
logvol /var/log --fstype="xfs" --size=2048 --name=LV_var_log --vgname=VG_OS
logvol /var/lib/ceph --fstype="xfs" --size=1024 --name=LV_cephd_data --vgname=VG_OS
logvol /var/lib/containers --fstype="xfs" --size=4096 --name=LV_containers --vgname=VG_OS
logvol /home --fstype="xfs" --size=1024 --name=LV_home --vgname=VG_OS
timesource --ntp-server=epyc.internal.pcfe.net
timesource --ntp-server=edgerouter-6p.internal.pcfe.net
# System timezone
timezone Europe/Berlin --utc
# Root password
rootpw --iscrypted $6$tojQNyCl8NsOXJ0h$YiKGtd1StvpOxVoAFtGGwFv9.ArUAuZ4v0J1KXfi2pSNji9Wm.Q.B4C8rANM5i/zvWSegIVz8oduzccVnslU/.
# Ansible user
user --uid=1100 --gid=1100 --name=ansible --lock --gecos="Ansible User"
# pcfe user
user --uid=1000 --gid=1000 --groups=wheel --name=pcfe --password=$y$j9T$ZWDidv6BLl.N4DxKVv0aY1$ct5WbCcT5e/hVBlW0u/mqCDyWwRPB6B5/jWGGPtCPF4 --iscrypted --gecos="Patrick C. F. Ernzer"
# no kdump
%addon com_redhat_kdump --disable
%end
%post --log=/root/ks-post.log
# dump pcfe's ssh key to the root user
# obviously change this to your own pubkey unless you want to grant me root access
mkdir /root/.ssh
chown root.root /root/.ssh
chmod 700 /root/.ssh
cat <>/root/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAgEAvNDSbbViufkQdqHfI4lrF3utwd028ndTJspdiOZ2JtdIVBjUokQRoVFY8+DXjTpBIBKWd/WciqMc02gYUXG94pDZkxHe9Z0xD/SdpoGng2XodVUVEnNWImVSbpPFDwqFZWOqyC8QVEp7AMhj+4AdWp9JmTQUeYWLssrmvnY9m0dB3K2CL6G532y7cZ9cEl73kBIxPVOHkdRZGUyTC05fZL7Ldd2eepi/oWRpDXmfWn/rN6zl1vKaYq5TaOcnATCL1tmP/t8yOdodMCgqYHRbhh8zuFcsMxl7b+eenjhlsh87V/pdKrWZFcfeWxamj7CdEQA79r3Sw/7h6Y2OGvKYKzofGtnjPzJnu63Hzdu7oQcQTXQpuMgoSMkhS+MbJOfiJUONK1tfTKiN29NJZ90biSonu7XpOpemIRAlx/vhpVXkKcN2PY12fRy7wL0A9yghb6M1Hkw1bHK7tlw/cpQiHhEPJuTbBWTZJ3OWSLXx+EMRfdn8cHx1yckaqXzMLoGh52OkgVbNeN52bbrwDrelOc237zknPnSzbnB7wIwZwmRE0GDvl/Ta+AM5A8N7FMC5K9wbOgP9qObTbUQGwP0hwg/Xai2kR/7QUwSB3/y2ja2wZNCSP5aSGszLkJd3X5M0yALcQFVzNyqUKy5wQhQEpUKnteAvwbwpmUmuU6WQPNk= private key 2008-05-22
EOF
chown root.root /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
restorecon /root/.ssh/authorized_keys
cat <>/etc/udev/rules.d/75-disable-5GB-on-board-stick.rules
# The on-board 5GB stick should be disabled
# I currently have no use for it and leaving it untouched allows a reset to the shipped state
# by choosing the USB stick as boot target during POST
# c.f. https://projectgus.com/2014/09/blacklisting-a-single-usb-device-from-linux/
SUBSYSTEM=="usb", ATTRS{idVendor}=="1005", ATTRS{idProduct}=="b155", ATTR{authorized}="0"
EOF
chown root.root /etc/udev/rules.d/75-disable-5GB-on-board-stick.rules
chmod 644 /etc/udev/rules.d/75-disable-5GB-on-board-stick.rules
restorecon /etc/udev/rules.d/75-disable-5GB-on-board-stick.rules
# pull check-mk-agent from my monitoring server (checkmk Raw edition)
dnf -y install http://check-mk.internal.pcfe.net/HouseNet/check_mk/agents/check-mk-agent-2.0.0p17-1.noarch.rpm
echo "check-mk-agent installed from monitoring server" >> /etc/motd
# disable Red Hat graphical boot (rhgb)
sed --in-place "s/rhgb//g" /etc/default/grub
echo "removed graphical boot from grub defaults" >> /etc/motd
echo "kickstarted at `date` for CentOS Stream 9 and Ceph on QNAP TS-473A" >> /etc/motd
%end
Interactive Installation
Alternatively, create a bootable installation medium to install CentOS Stream interactively.
All other Setup via Ansible
All other configuration tasks were done with the same Ansible Playbooks shown in this post: