TerraMaster F5-422 with RHEL8
Table of Contents
In further happy distro hopping (since Ansible makes this quite painless), I installed RHEL8 on my four TerraMaster F5-422.
Please also see the earlier posts TerraMaster F5-422 with CentOS Stream and Install four TerraMaster F5-422 with CentOS Stream for more info.
Upgrade RAM
All my F5-422 had their RAM upgraded to 12 GiB. Proceeded as follows;
- remove 6 screws on the back
- carefully remove back cover (mind the 2 cables connecting the the fans to the motherboard)
- add a Crucial CT102464BF160B 8GB SO-DIMM or similar.
- close case in reverse order
Install SSD for OS and Ceph use
- I put a Samsung SSD 860 EVO 500GB, SATA in drive bay 1
- I only less than 100GiB for base OS, I keep the rest as unpartitioned space for Ceph use.
Firmware Settings
As so often, I was not happy with the vendor’s UEFI settings and made the following changes (it is probably a good idea to reset to defaults before you start).
- connect screen, keyboard, mouse (and USB boot medium)
- enter firmware setup
- Advanced / Power Management Features: enable Wake on LAN
- Advanced / Power Management Features: disable Wake Up on Alarm
- Chipset / South Bridge / OS Selection: change from Windows to Intel Linux
- Boot / Setup Prompt Timeout: change to 5 seconds or more
- Boot / Boot Mode: change from Legcy+UEFI to UEFI Only
- save and exit
Install RHEL8
- create USB boot medium as per docs with
dd if=rhel-8.1-x86_64-boot.iso of=/dev/sdd status=progress - connect screen, keyboard, mouse and USB boot medium
- enter firmware setup
- change OS type from Windows to Intel Linux
- set boot to UEFI only
- save and exit
- boot from USB
- do a minimal install or a kickstart install
Create Kickstart Entry on USB Stick
[root@t7910 iso]# mount /dev/sdd2 /mnt/import/
[root@t7910 iso]# vim /mnt/import/EFI/BOOT/grub.cfg
[root@t7910 iso]# grep -A 3 Kickstart /mnt/import/EFI/BOOT/grub.cfg
menuentry 'Kickstart a F5-422 with Red Hat Enterprise Linux 8.1.0' --class fedora --class gnu-linux --class gnu --class os {
linuxefi /images/pxeboot/vmlinuz ip=dhcp inst.repo=ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-8.1/Server/x86_64/os inst.ks=ftp://fileserver.internal.pcfe.net/pub/kickstart/RHEL81-TerraMaster-F5-422-ks.cfg
initrdefi /images/pxeboot/initrd.img
}
<Del>to enter Setup- Go to Save & Exit / Boot Override and select UEFI: stickname, Partition 1
- Selected the entry Kickstart a F5-422 with Red Hat Enterprise Linux 8.1.0'
Used Kickstart File
#version=RHEL8
# avoid using half arsed names like sda, sdb, etc
# mapping seems to be:
# bay 1 /dev/disk/by-path/pci-0000:00:12.0-ata-1
# bay 2 /dev/disk/by-path/pci-0000:00:12.0-ata-2
# bay 3 /dev/disk/by-path/pci-0000:03:00.0-ata-1
# bay 4 /dev/disk/by-path/pci-0000:03:00.0-ata-2
# bay 5 /dev/disk/by-path/pci-0000:04:00.0-ata-1
# reboot after installation is complete?
#reboot
# we only use the SATA SSD in bay 1
ignoredisk --only-use=/dev/disk/by-path/pci-0000:00:12.0-ata-1
# Partition clearing information
# note that OS goes on a small portion os the device in bay 1, the rest will be allocated to Ceph in a separtate VG.
# so kickstarting with the below clearpart line will nuke the Ceph bits on SSD !!!
clearpart --all --initlabel --drives=/dev/disk/by-path/pci-0000:00:12.0-ata-1
# Use graphical install
graphical
# Keyboard layouts
keyboard --vckeymap=us --xlayouts='us'
# System language
lang en_US.UTF-8
# Network information all switch ports have the respective VLAN as native
# 1 Gig on PCIe (will go on 'storage' via ansible)
network --bootproto=dhcp --device=enp2s0 --onboot=off --ipv6=auto --no-activate
# 10 Gig on PCIe (will go on 'ceph' via ansible)
network --bootproto=dhcp --device=enp1s0 --onboot=off --ipv6=auto --no-activate
# 1 Gig on USB ('access' network)
network --bootproto=dhcp --device=enp0s21f0u3 --ipv6=auto --activate
network --hostname=localhost.localdomain
# Use network installation
url --url="ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-8.2/Server/x86_64/os/BaseOS"
repo --name="AppStream" --baseurl=ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-8.2/Server/x86_64/os/AppStream
# Root password
rootpw --iscrypted $6$X55cCQsXt3xeCukD$qJhkHV0x8cHjBc8zBbRldKq7YdMIf52lfM2GvaNW13HEPBIllwsXfIzmD1s4ZxstUTO5u4JzxAuLjz27SSmy6/
# Run the Setup Agent on first boot?
firstboot --disable
# Do not configure the X Window System
skipx
# System services
services --enabled="chronyd"
# Intended system purpose
syspurpose --role="Red Hat Enterprise Linux Server" --sla="Self-Support" --usage="Development/Test"
# System timezone
timezone Europe/Berlin --isUtc --ntpservers=epyc.internal.pcfe.net,edgerouter-6p.internal.pcfe.net
# Ansible user
user --groups=wheel --name=ansible --password=$6$1XAtqKKCAXHi17l/$Yosu3OEblUaM2SvrbeBH6wlzV8TAv7eO.3jI9CpTG3gbRfGnXlbuLPmgw.IBacL3km/ArjEjKbd9Ki2LTxYpu. --iscrypted --gecos="ansible"
# Disk partitioning information
# the 120 GB SATA SSD in slot 1 will be fully used for the OS
# the 500 GB SATA SSD in slot 2 and the HDDs in slots 3 through 5
# will be fed top ceph-ansible as devices
# c.f. https://docs.ceph.com/ceph-ansible/master/osds/scenarios.html
part /boot --fstype="ext4" --ondisk=/dev/disk/by-path/pci-0000:00:12.0-ata-1 --size=1024
part /boot/efi --fstype="efi" --ondisk=/dev/disk/by-path/pci-0000:00:12.0-ata-1 --size=512 --fsoptions="umask=0077,shortname=winnt"
part pv.01 --fstype="lvmpv" --ondisk=/dev/disk/by-path/pci-0000:00:12.0-ata-1 --size=61440 --grow
volgroup VG_OS_SSD_bay1 --pesize=4096 pv.01
logvol / --fstype="xfs" --size=4096 --name=LV_root --vgname=VG_OS_SSD_bay1
logvol swap --fstype="swap" --size=4096 --name=LV_swap --vgname=VG_OS_SSD_bay1
logvol /var --fstype="xfs" --size=4096 --name=LV_var --vgname=VG_OS_SSD_bay1
logvol /var/log --fstype="xfs" --size=4096 --name=LV_var_log --vgname=VG_OS_SSD_bay1
logvol /var/crash --fstype="xfs" --size=15360 --name=LV_var_crash --vgname=VG_OS_SSD_bay1
logvol /var/lib/containers --fstype="xfs" --size=8192 --name=LV_containers --vgname=VG_OS_SSD_bay1
logvol /home --fstype="xfs" --size=1024 --name=LV_home --vgname=VG_OS_SSD_bay1
%packages
@^minimal-environment
chrony
kexec-tools
%end
%addon com_redhat_kdump --enable --reserve-mb='auto'
%end
%anaconda
pwpolicy root --minlen=6 --minquality=1 --notstrict --nochanges --notempty
pwpolicy user --minlen=6 --minquality=1 --notstrict --nochanges --emptyok
pwpolicy luks --minlen=6 --minquality=1 --notstrict --nochanges --notempty
%end
%post --log=/root/ks-post.log
# dump pcfe's ssh key to the root user
# obviously change this to your own pubkey unless you want to grant me root access
mkdir /root/.ssh
chown root.root /root/.ssh
chmod 700 /root/.ssh
cat <>/root/.ssh/authorized_keys
ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAgEAvNDSbbViufkQdqHfI4lrF3utwd028ndTJspdiOZ2JtdIVBjUokQRoVFY8+DXjTpBIBKWd/WciqMc02gYUXG94pDZkxHe9Z0xD/SdpoGng2XodVUVEnNWImVSbpPFDwqFZWOqyC8QVEp7AMhj+4AdWp9JmTQUeYWLssrmvnY9m0dB3K2CL6G532y7cZ9cEl73kBIxPVOHkdRZGUyTC05fZL7Ldd2eepi/oWRpDXmfWn/rN6zl1vKaYq5TaOcnATCL1tmP/t8yOdodMCgqYHRbhh8zuFcsMxl7b+eenjhlsh87V/pdKrWZFcfeWxamj7CdEQA79r3Sw/7h6Y2OGvKYKzofGtnjPzJnu63Hzdu7oQcQTXQpuMgoSMkhS+MbJOfiJUONK1tfTKiN29NJZ90biSonu7XpOpemIRAlx/vhpVXkKcN2PY12fRy7wL0A9yghb6M1Hkw1bHK7tlw/cpQiHhEPJuTbBWTZJ3OWSLXx+EMRfdn8cHx1yckaqXzMLoGh52OkgVbNeN52bbrwDrelOc237zknPnSzbnB7wIwZwmRE0GDvl/Ta+AM5A8N7FMC5K9wbOgP9qObTbUQGwP0hwg/Xai2kR/7QUwSB3/y2ja2wZNCSP5aSGszLkJd3X5M0yALcQFVzNyqUKy5wQhQEpUKnteAvwbwpmUmuU6WQPNk= private key 2008-05-22
EOF
chown root.root /root/.ssh/authorized_keys
chmod 600 /root/.ssh/authorized_keys
restorecon /root/.ssh/authorized_keys
cat <>/etc/udev/rules.d/75-disable-120MB-on-board-stick.rules
# The on-board 120MB stick should be disabled
# it's tiny and leaving it untouched allows a reset to the shipped state
# by choosing the USB stick as boot target during POST and booting UEFI+Legacy mode
# c.f. https://projectgus.com/2014/09/blacklisting-a-single-usb-device-from-linux/
SUBSYSTEM=="usb", ATTRS{idVendor}=="13fe", ATTRS{idProduct}=="4100", ATTR{authorized}="0"
EOF
chown root.root /etc/udev/rules.d/75-disable-120MB-on-board-stick.rules
chmod 644 /etc/udev/rules.d/75-disable-120MB-on-board-stick.rules
restorecon /etc/udev/rules.d/75-disable-120MB-on-board-stick.rules
# Since Ceph and EPEL should not be mixed,
# pull check-mk-agent from my monitoring server (checkmk Raw edition)
dnf -y install http://check-mk.internal.pcfe.net/HouseNet/check_mk/agents/check-mk-agent-2.0.0p17-1.noarch.rpm
echo "check-mk-agent installed from monitoring server" >> /etc/motd
# seems I can NOT specify a connection name for the network setup
# https://pykickstart.readthedocs.io/en/latest/kickstart-docs.html#network
# so just remove the line setting the name "System enp…" and then move the files
# linux-system-roles.network will do the rest later
# I do this because I find speaking names so much more pleasant
sed --in-place "s/^NAME//g" /etc/sysconfig/network-scripts/ifcfg-enp*
mv /etc/sysconfig/network-scripts/ifcfg-enp0s21f0u3 /etc/sysconfig/network-scripts/ifcfg-1G_USB
mv /etc/sysconfig/network-scripts/ifcfg-enp1s0 /etc/sysconfig/network-scripts/ifcfg-10G_PCIe
mv /etc/sysconfig/network-scripts/ifcfg-enp2s0 /etc/sysconfig/network-scripts/ifcfg-1G_PCIe
# disable Red Hat graphical boot (rhgb)
sed --in-place "s/ rhgb//g" /etc/default/grub
echo "kickstarted at `date` for RHEL 8.2 on TerraMaster F5-422" >> /etc/motd
%end
Since the machine is supposed to be mainly a Ceph OSD, initially (der RHEL 8.1) I only used some 50GiB of the SSD for the OS.
But since then I’ve gone to
bay 1: 120 GB SSD used completely for OS bay 2: 500 GB HDD split up decided by cep-ansible bays 3 through 5: assorted HDD I had in my homelab
Everything on top of this minimal kickstart install is handled via Ansible. You could also do a manual minimal install, I simply could not be asked to click around interactively for 4 installs.
ToDo
the following still need to be added to the playbook.
Set Up smartd
TODO: still need to set up smartd to send alerts to my sms gateway.
Disable the 120 MB USB stick
note: the NAS has a 120MB stick in an on-board connector, I decided to not touch it for now. In case I ever need to go back to the vendor OS. Note that enabling EUFI only boot mode makes the 120MB stick not show up as a valid boot target with the shipped image. Seems someone forgot to get their boot medium signed.
I disabled it with a blacklist entry.
[root@f5-422-01 ~]# cat /etc/udev/rules.d/75-disable-120MB-on-board-stick.rules
# The on-board 120MB stick should be disabled
# it's tiny and I have 0 use for it
# c.f. https://projectgus.com/2014/09/blacklisting-a-single-usb-device-from-linux/
SUBSYSTEM=="usb", ATTRS{idVendor}=="13fe", ATTRS{idProduct}=="4100", ATTR{authorized}="0"
TODO: put the above cleanly in a playbook, at the moment it’s handled in kickstart.
After the next boot, as expected the tiny flash drive is no longer visible
[root@f5-422-01 ~]# lsusb
Bus 002 Device 003: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 13fe:4100 Kingston Technology Company Inc. Flash drive
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
[root@f5-422-01 ~]# lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/7p, 5000M
|__ Port 3: Dev 3, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 480M
Drive Bay Mapping
To avoid using half arsed names like sda, sdb, etc
I address the drive bays by their /dev/disk/by-path/ entries.
FIXME: double check. The mapping seems to be
| bay number | /dev/disk/by-path/ entry |
|---|---|
| bay 1 | /dev/disk/by-path/pci-0000:00:12.0-ata-1 |
| bay 2 | /dev/disk/by-path/pci-0000:00:12.0-ata-2 |
| bay 3 | /dev/disk/by-path/pci-0000:03:00.0-ata-1 |
| bay 4 | /dev/disk/by-path/pci-0000:03:00.0-ata-2 |
| bay 5 | /dev/disk/by-path/pci-0000:04:00.0-ata-1 |
Initial Setup
check-mk-agent
Since Ceph and EPEL are not meant to be mixed, I install check-mk-agent from my monitoring server.
For now I just hammered it into the F5-422 kickstart as all other boxes use the RPM from EPEL or Fedora.
terramaster-initial-setup-centos8.yml
takes care of
- creating an ansible user
- basic-security-setup
- local settings (mirros etc)
- adding
consoleblank=0to my default boot options
It’s the same as terramaster-initial-setup-centos8.yml but with Subscription handling added.
As user, on my F31 x96_64 workstation.
ansible-playbook -i ../inventories/ceph-F5-422-cluster.ini terramaster-initial-setup-rhel8.yml
---
# sets up a RHEL 8 minimal install to be ready for ceph-ansible
#
# this is for my home setup, not for production!
- hosts:
- TerraMaster_boxes
become: false
roles:
- pcfe.user_owner
- pcfe.basic_security_setup
- pcfe.housenet
#- pcfe.checkmk
# note though that check-mk-agent is in EPEL but RHCS 4 docs say EPEL needs to be off
# for now I manually installed the rpm with
# dnf install http://check-mk.internal.pcfe.net/HouseNet/check_mk/agents/check-mk-agent-1.6.0p5-1.noarch.rpm
# no need for double indirect if you are OK with checking in ak details into git
# this is OK to do if you use an in-house Satellite server and your security policies allow it
# this is not a good idea if you register your systems directly to redhat.com and cannot guarantee that your git remains private
vars_files:
- "vars/subscription-manager-autoattach-ak-secrets.yml"
vars:
ansible_user: root
user_owner: ansible
common_timezone: Europe/Berlin
rhsm_activationkey: "{{ vaulted_rhsm_activationkey }}"
rhsm_org_id: "{{ vaulted_rhsm_org_id }}"
rhsm_pool_ids: "{{ vaulted_rhsm_pool_ids }}"
pre_tasks:
# work around identical UUID on each host as per https://theforeman.org/plugins/katello/nightly/troubleshooting/content_hosts.html
# The TerraMaster F5-422 boxes all have the same system-uuid :-(
# RHSM does not like that, so override to avoid both
# "HTTP error (409 - Conflict): Request failed due to concurrent modification, please re-try.\n"
# and all 4 boxes overriding each other in insights inventory
- name: "RHSM | ensure uuid override is derived from fqdn, F5-422 all have idential uuid in DMI"
copy:
dest: "/etc/rhsm/facts/uuid_override.facts"
owner: "root"
group: "root"
mode: 0644
content: |
{"dmi.system.uuid": "{{ ansible_fqdn | to_uuid }}"}
# https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/4/html-single/installation_guide/index#enabling-the-red-hat-ceph-storage-repositories-install
- name: "RHSM on the RHEL8 boxes"
block:
- name: "RHSM | ensure system is registered with my activation key"
redhat_subscription:
activationkey: "{{ rhsm_activationkey }}"
org_id: "{{ rhsm_org_id }}"
syspurpose:
usage: "Development/Test"
role: ""
service_level_agreement: "Self-Support"
- name: "RHSM | ensure RHEL8 BaseOS repos needed for Ceph are enabled"
rhsm_repository:
name:
- rhel-8-for-x86_64-baseos-rpms
- rhel-8-for-x86_64-appstream-rpms
state: enabled
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "8"
tags: do_subsmgr_register
- name: "RHSM on the RHEL7 boxes"
block:
- name: "RHSM | ensure system is registered with my activation key"
redhat_subscription:
activationkey: "{{ rhsm_activationkey }}"
org_id: "{{ rhsm_org_id }}"
- name: "RHSM | disable all repositories, next task will enable needed repos"
rhsm_repository:
name: '*'
state: disabled
- name: "RHSM | ensure RHEL7 BaseOS repos needed for Ceph are enabled"
rhsm_repository:
name:
- rhel-7-server-rpms
- rhel-7-server-extras-rpms
state: enabled
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "7"
tags: do_subsmgr_register
tasks:
# !!!
#
# if I ever enable EPEL, then I MUST exclude
# - ansible
# - ceph
# in the EPEL repo files to ensure no newer versions of those packages are pulled in from EPEL
#
# exclude = *ceph* nfs-ganesha-rgw rbd-mirror *ansible*
#
# !!!
#
# https://access.redhat.com/documentation/en-us/red_hat_ceph_storage/4/html-single/installation_guide/index#enabling-the-red-hat-ceph-storage-repositories-install
- name: "REPOS | ensure EPEL is disabled"
yum_repository:
name: epel
state: absent
# start by enabling time sync, RHSM operations will fail on too large time delta
# note that this uses chronyd, not ntpd.
- name: "CHRONYD | ensure chrony is installed"
package:
name: chrony
state: present
- name: "CHRONYD | ensure chrony-wait is enabled"
service:
name: chrony-wait
enabled: true
- name: "CHRONYD | ensure chronyd is enabled and running"
service:
name: chronyd
enabled: true
state: started
# enable persistent journal
- name: "JOURNAL | ensure persistent logging for the systemd journal is possible"
file:
path: /var/log/journal
state: directory
owner: root
group: systemd-journal
mode: u=rwx,g=rs,o=rx
# 2.10. Enabling Password-less SSH for Ansible
- name: "SUDO | enable passwordless sudo for ansible user"
copy:
dest: /etc/sudoers.d/ansible
content: |
ansible ALL=NOPASSWD: ALL
owner: root
group: root
mode: 0440
# I admit, the regexp is a search engine hit
- name: "GRUB | ensure console blanking is disabled in defaults file"
lineinfile:
state: present
dest: /etc/default/grub
backrefs: yes
regexp: '^(GRUB_CMDLINE_LINUX=(?!.* consoleblank)\"[^\"]+)(\".*)'
line: '\1 consoleblank=0\2'
terramaster-general-setup.yml
Besides running some of my roles, it takes care of
- ensuring Wake on LAN is set up for the 1Gig PCIe network interface
- set up watchdog, rngd, tuned
- ensuring cockbit is installed but disabled
- ensuring libvirtd is not running
- ensuring kdump is on
- ensuring I can monitor the host with my Check MK instance
- ensuring
consoleblank=0is set in my default boot options - ensuring I can kickstart the node again via a target in grub
- ensuring the system has the latest updates applied
As user, on my F31 x96_64 workstation.
ansible-playbook -i ../inventories/ceph-F5-422-cluster.ini terramaster-general-setup.yml
---
- name: General setup of my Terramaster F5-422 Ceph nodes
hosts:
- TerraMaster_boxes
become: true
roles:
- fedora.linux_system_roles.network
- pcfe.user_owner
- pcfe.basic_security_setup
- pcfe.housenet
- pcfe.comfort
# - pcfe.checkmk
handlers:
- name: Handle running grub2-mkconfig for dist RedHat major vers 9
ansible.builtin.command: grub2-mkconfig -o /boot/grub2/grub.cfg
changed_when: true
- name: Handle running grub2-mkconfig for dist RedHat major vers 8
ansible.builtin.command: grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg
changed_when: true
tasks:
# work around identical UUID on each host as per https://theforeman.org/plugins/katello/nightly/troubleshooting/content_hosts.html
# The TerraMaster F5-422 boxes all have the same system-uuid :-(
# RHSM does not like that, so override to avoid both
# "HTTP error (409 - Conflict): Request failed due to concurrent modification, please re-try.\n"
# and all 4 boxes overriding each other in insights inventory
- name: "RHSM | ensure uuid override is derived from fqdn, F5-422 all have idential uuid in DMI"
ansible.builtin.copy:
dest: /etc/rhsm/facts/uuid_override.facts
owner: root
group: root
mode: u=rw,g=r,o=r
content: |
{"dmi.system.uuid": "{{ ansible_fqdn | to_uuid }}"}
# Ensure the packages that the RHEL8 only preflight play from cephadm-ansible would install
# also ensure ceph-selinux is installed, docs say that's in the default set
- name: Ensure needed packages for RHCS5 on RHEL9 are installed
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "9"
ansible.builtin.package:
name:
- chrony
- cephadm
- podman
- ceph-common
- ceph-selinux
state: present
update_cache: false
# Install some tools
- name: "PACKAGE | tool installation, el8 and el9"
ansible.builtin.package:
name:
- pciutils
- usbutils
- fio
- powertop
- tuned
- tuned-utils
- numactl
- iperf3
- tcpdump
- hwloc-gui
- fwupd
state: present
update_cache: false
- name: "PACKAGE | tool installation el9"
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "9"
ansible.builtin.package:
name:
- s-nail
state: present
update_cache: false
- name: "PACKAGE | tool installation el8"
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "8"
ansible.builtin.package:
name:
- teamd
- NetworkManager-team
- mailx
state: present
update_cache: false
# linux-system-roles.network sets static network config (from host_vars)
# but I want the static hostname nailed down too
#
# FIXME; ansible_fqdn is a fact, so that does not help when it's a freshly kickstarted localhost.localdomain
- name: Ensure static hostname is set to FQDN
ansible.builtin.hostname:
name: "{{ ansible_fqdn }}"
use: systemd
# enable WOL manually until https://github.com/linux-system-roles/network/issues/150 is fixed
- name: Ensure Wake On LAN is enable for the upper network card
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "8"
ansible.builtin.lineinfile:
path: /etc/sysconfig/network-scripts/ifcfg-1G_PCIe
create: false
regexp: '^ETHTOOL_OPTS= '
insertafter: '^TYPE=Ethernet'
line: 'ETHTOOL_OPTS="wol g"'
# enable watchdog
# it's a Jun 22 13:12:09 localhost kernel: sbsa-gwdt e0bb0000.gwdt: Initialized with 10s timeout @ 250000000 Hz, action=0.
- name: "WATCHDOG | ensure kernel module wdat_wdt has correct options configured"
ansible.builtin.lineinfile:
path: /etc/modprobe.d/wdat_wdt.conf
create: true
regexp: '^options '
insertafter: '^#options'
line: 'options wdat_wdt nowayout=0'
mode: u=rw,g=r,o=r
# configure both watchdog.service and systemd watchdog, but only use the latter
- name: "PACKAGE | ensure watchdog package is installed"
ansible.builtin.package:
name: watchdog
state: present
update_cache: false
- name: "WATCHDOG | ensure correct watchdog-device is used by watchdog.service"
ansible.builtin.lineinfile:
path: /etc/watchdog.conf
regexp: '^watchdog-device'
insertafter: '^#watchdog-device'
line: 'watchdog-device = /dev/watchdog'
- name: "WATCHDOG | ensure timeout is set to 30 seconds for watchdog.service"
ansible.builtin.lineinfile:
path: /etc/watchdog.conf
regexp: '^watchdog-timeout'
insertafter: '^#watchdog-timeout'
line: 'watchdog-timeout = 30'
# Using systemd watchdog rather than watchdog.service
- name: "WATCHDOG | ensure watchdog.service is disabled"
ansible.builtin.systemd:
name: watchdog.service
state: stopped
enabled: false
# configure systemd watchdog
# c.f. http://0pointer.de/blog/projects/watchdog.html
- name: "SYSTEMD | ensure systemd watchdog is enabled"
ansible.builtin.lineinfile:
path: /etc/systemd/system.conf
regexp: '^RuntimeWatchdogSec'
insertafter: 'EOF'
line: 'RuntimeWatchdogSec=30'
- name: "SYSTEMD | ensure systemd shutdown watchdog is enabled"
ansible.builtin.lineinfile:
path: /etc/systemd/system.conf
regexp: '^ShutdownWatchdogSec'
insertafter: 'EOF'
line: 'ShutdownWatchdogSec=30'
# install and enable rngd
- name: "PACKAGE | ensure rng-tools package is installed"
ansible.builtin.package:
name: rng-tools
state: present
update_cache: false
- name: "RNGD | ensure rngd.service is enabled and started"
ansible.builtin.systemd:
name: rngd.service
state: started
enabled: true
# ensure tuned is set up as I wish
- name: "TUNED | ensure tuned.service is enabled and running"
ansible.builtin.systemd:
name: tuned.service
state: started
enabled: true
- name: "TUNED | check which tuned profile is active"
ansible.builtin.command: tuned-adm active
register: tuned_active_profile
ignore_errors: true
changed_when: false
- name: "TUNED | activate tuned profile {{ tuned_profile }}"
ansible.builtin.command: "tuned-adm profile {{ tuned_profile }}"
when: not tuned_active_profile.stdout is search('Current active profile:' ~ ' ' ~ tuned_profile)
changed_when: true
# install cockpit, but disabled for now
- name: "COCKPIT | ensure packages for https://cockpit-project.org/ are installed"
ansible.builtin.package:
name:
- cockpit
- cockpit-selinux
- cockpit-kdump
- cockpit-storaged
- cockpit-system
state: present
update_cache: false
- name: "COCKPIT | ensure cockpit.socket is stopped and disabled"
ansible.builtin.systemd:
name: cockpit.socket
state: stopped
enabled: false
- name: "COCKPIT | ensure firewalld forbids service cockpit in zone public"
ansible.posix.firewalld:
service: cockpit
zone: public
permanent: true
state: disabled
immediate: true
# disable libvirtd
- name: "Ensure libvirtd.service is disabled and stopped"
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "8"
ansible.builtin.systemd:
name: libvirtd.service
state: stopped
enabled: false
# enable kdump.service since kickstart now creates a sufficiently large /var/crash
# alternatively, you could set up netdump
- name: "Ensure kdump.service is enabled and started"
ansible.builtin.systemd:
name: kdump.service
state: started
enabled: true
# podman
- name: "PACKAGE | ensure podman is installed"
ansible.builtin.package:
name:
- podman
- podman-docker
state: present
# setroubleshoot, see also https://danwalsh.livejournal.com/20931.html
- name: "PACKAGE | ensure setroubleshoot for headless server is installed"
ansible.builtin.package:
name:
- setroubleshoot-server
- setroubleshoot-plugins
state: present
- name: "MONITORING | ensure packages for monitoring are installed"
ansible.builtin.package:
name:
- smartmontools
- hdparm
- check-mk-agent
- lm_sensors
state: present
- name: "MONITORING | ensure firewalld permits 6556 in zone public for check-mk-agent"
ansible.posix.firewalld:
port: 6556/tcp
permanent: true
state: enabled
immediate: true
zone: public
- name: "MONITORING | ensure tarsnap cache is in fileinfo"
ansible.builtin.lineinfile:
path: /etc/check_mk/fileinfo.cfg
line: "/usr/local/tarsnap-cache/cache"
create: true
mode: u=rw,g=r,o=r
- name: "MONITORING | ensure entropy_avail plugin for Check_MK is present"
ansible.builtin.template:
src: templates/check-mk-agent-plugin-entropy_avail.j2
dest: /usr/lib/check_mk_agent/plugins/entropy_avail
mode: u=rwx,g=rx,o=rx
group: root
owner: root
- name: "MONITORING | plugins from running CEE instance"
ansible.builtin.get_url:
url: "http://check-mk.internal.pcfe.net/HouseNet/check_mk/agents/plugins/{{ item }}"
dest: "/usr/lib/check_mk_agent/plugins/{{ item }}"
mode: u=rwx,g=rx,o=rx
loop:
- smart
- lvm
- name: "MONITORING | ensure check_mk.socket is started and enabled"
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "8"
ansible.builtin.systemd:
name: check_mk.socket
state: started
enabled: true
- name: "MONITORING | ensure check-mk-agent.socket is started and enabled"
when: ansible_distribution == "RedHat" and ansible_distribution_major_version == "9"
ansible.builtin.systemd:
name: check-mk-agent.socket
state: started
enabled: true
# I admit, the regexp is a search engine hit
# maybe using grubby(8) would be more readable
# yamllint disable rule:line-length
# - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/8/html/managing_monitoring_and_updating_the_kernel/configuring-kernel-command-line-parameters_managing-monitoring-and-updating-the-kernel#what-is-grubby_configuring-kernel-command-line-parameters
# - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/system_administrators_guide/sec-Making_Persistent_Changes_to_a_GRUB_2_Menu_Using_the_grubby_Tool
# yamllint enable rule:line-length
- name: "GRUB | ensure console blanking is disabled in defaults file plus handler"
ansible.builtin.lineinfile:
state: present
dest: /etc/default/grub
backrefs: true
regexp: '^(GRUB_CMDLINE_LINUX=(?!.* consoleblank)\"[^\"]+)(\".*)'
line: '\1 consoleblank=0\2'
notify: "Handle running grub2-mkconfig for dist {{ ansible_distribution }} major vers {{ ansible_distribution_major_version }}"
# Since I do not manage to get these F5-422 to PXE boot, add an entry into grub
# so that I can kickstart the box after this without fiddling with a USB stick
- name: "GRUB | ensure initrd for RHEL 8.9 kickstart is present"
ansible.builtin.get_url:
url: "ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-8.9/Server/x86_64/os/images/pxeboot/initrd.img"
dest: "/boot/initrd-kickstart-rhel89.img"
mode: u=rw
- name: "GRUB | ensure kernel for RHEL 8.9 kickstart is present"
ansible.builtin.get_url:
url: "ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-8.9/Server/x86_64/os/images/pxeboot/vmlinuz"
dest: "/boot/vmlinuz-kickstart-rhel89"
mode: u=rwx,g=rx,o=rx
# yamllint disable rule:line-length
- name: "GRUB | ensure kickstarting RHEL 8.9 entry is present"
ansible.builtin.copy:
dest: "/etc/grub.d/11_RHEL8x_kickstart"
owner: "root"
group: "root"
mode: u=rwx,g=rx,o=rx
content: |
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
menuentry "WARNING Kickstart this box with RHEL 8.9 as a F5-422 ceph node WARNING" {
linuxefi /vmlinuz-kickstart-rhel89 ip=dhcp inst.repo=ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-8.9/Server/x86_64/os inst.ks=ftp://fileserver.internal.pcfe.net/pub/kickstart/RHEL89-TerraMaster-F5-422-ks.cfg
initrdefi /initrd-kickstart-rhel89.img
}
notify: "Handle running grub2-mkconfig for dist {{ ansible_distribution }} major vers {{ ansible_distribution_major_version }}"
# yamllint enable rule:line-length
- name: "GRUB | ensure initrd for RHEL 9.3 kickstart is present"
ansible.builtin.get_url:
url: "ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-9.3/Server/x86_64/os/images/pxeboot/initrd.img"
dest: "/boot/initrd-kickstart-rhel93.img"
mode: u=rw
- name: "GRUB | ensure kernel for RHEL 9.3 kickstart is present"
ansible.builtin.get_url:
url: "ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-9.3/Server/x86_64/os/images/pxeboot/vmlinuz"
dest: "/boot/vmlinuz-kickstart-rhel93"
mode: u=rwx,g=rx,o=rx
# yamllint disable rule:line-length
- name: "GRUB | ensure kickstarting RHEL 9.3 entry is present"
ansible.builtin.copy:
dest: "/etc/grub.d/12_RHEL9x_kickstart"
owner: "root"
group: "root"
mode: u=rwx,g=rx,o=rx
content: |
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries. Simply type the
# menu entries you want to add after this comment. Be careful not to change
# the 'exec tail' line above.
menuentry "WARNING Kickstart this box with RHEL 9.3 as a F5-422 ceph node WARNING" {
linuxefi /vmlinuz-kickstart-rhel93 ip=dhcp inst.repo=ftp://fileserver.internal.pcfe.net/pub/redhat/RHEL/RHEL-9.3/Server/x86_64/os inst.ks=ftp://fileserver.internal.pcfe.net/pub/kickstart/RHEL93-TerraMaster-F5-422-ks.cfg
initrdefi /initrd-kickstart-rhel93.img
}
notify: "Handle running grub2-mkconfig for dist {{ ansible_distribution }} major vers {{ ansible_distribution_major_version }}"
# yamllint enable rule:line-length
# upgrade the box
- name: Package | ensure all updates are applied
ansible.builtin.package:
update_cache: true
name: '*'
state: latest # noqa package-latest
tags: apply_errata
test forcefully crashing the box.
Verify that the watchdog works as expected.
As root, on a F5-422:
echo '1' > /proc/sys/kernel/sysrq
echo 'c' > /proc/sysrq-trigger
As expected, I get an oops output on the console after the c and,
as previously tested, the box reboots itself after 30 seconds.
Hardware Info
dmidecode -t baseboard
[root@f5-422-01 ~]# dmidecode -t baseboard
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.0.0 present.
Handle 0x0002, DMI type 2, 15 bytes
Base Board Information
Manufacturer: GIFA Industrial Control
Product Name: TM-J3455-4G3L
Version: Default string
Serial Number: Default string
Asset Tag: Default string
Features:
Board is a hosting board
Board is replaceable
Location In Chassis: Default string
Chassis Handle: 0x0003
Type: Motherboard
Contained Object Handles: 0
Handle 0x0025, DMI type 10, 6 bytes
On Board Device Information
Type: Video
Status: Enabled
Description: To Be Filled By O.E.M.
Handle 0x002D, DMI type 41, 11 bytes
Onboard Device
Reference Designation: Onboard IGD
Type: Video
Status: Enabled
Type Instance: 1
Bus Address: 0000:00:02.0
Handle 0x002E, DMI type 41, 11 bytes
Onboard Device
Reference Designation: Onboard LAN
Type: Ethernet
Status: Enabled
Type Instance: 1
Bus Address: 0000:00:19.0
Handle 0x002F, DMI type 41, 11 bytes
Onboard Device
Reference Designation: Onboard 1394
Type: Other
Status: Enabled
Type Instance: 1
Bus Address: 0000:03:1c.2
lspci
[root@f5-422-04 ~]# lspci
00:00.0 Host bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Host Bridge (rev 0b)
00:02.0 VGA compatible controller: Intel Corporation Device 5a85 (rev 0b)
00:0e.0 Audio device: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Audio Cluster (rev 0b)
00:0f.0 Communication controller: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Trusted Execution Engine (rev 0b)
00:12.0 SATA controller: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SATA AHCI Controller (rev 0b)
00:13.0 PCI bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series PCI Express Port A #1 (rev fb)
00:13.2 PCI bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series PCI Express Port A #3 (rev fb)
00:13.3 PCI bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series PCI Express Port A #4 (rev fb)
00:14.0 PCI bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series PCI Express Port B #1 (rev fb)
00:14.1 PCI bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series PCI Express Port B #2 (rev fb)
00:15.0 USB controller: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series USB xHCI (rev 0b)
00:1a.0 Serial bus controller [0c80]: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series PWM Pin Controller (rev 0b)
00:1f.0 ISA bridge: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series Low Pin Count Interface (rev 0b)
00:1f.1 SMBus: Intel Corporation Celeron N3350/Pentium N4200/Atom E3900 Series SMBus Controller (rev 0b)
01:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
03:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
04:00.0 SATA controller: ASMedia Technology Inc. ASM1062 Serial ATA Controller (rev 02)
lsusb
[root@f5-422-04 ~]# lsusb
Bus 002 Device 003: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 13fe:4100 Kingston Technology Company Inc. Flash drive
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
lsusb -t
[root@f5-422-04 ~]# lsusb -t
/: Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/7p, 5000M
|__ Port 3: Dev 3, If 0, Class=Vendor Specific Class, Driver=r8152, 5000M
/: Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/8p, 480M
lscpu
[root@f5-422-04 ~]# lscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Byte Order: Little Endian
CPU(s): 4
On-line CPU(s) list: 0-3
Thread(s) per core: 1
Core(s) per socket: 4
Socket(s): 1
NUMA node(s): 1
Vendor ID: GenuineIntel
CPU family: 6
Model: 92
Model name: Intel(R) Celeron(R) CPU J3455 @ 1.50GHz
Stepping: 9
CPU MHz: 892.928
CPU max MHz: 2300.0000
CPU min MHz: 800.0000
BogoMIPS: 2995.20
Virtualization: VT-x
L1d cache: 24K
L1i cache: 32K
L2 cache: 1024K
NUMA node0 CPU(s): 0-3
Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc art arch_perfmon pebs bts rep_good nopl xtopology tsc_reliable nonstop_tsc cpuid aperfmperf tsc_known_freq pni pclmulqdq dtes64 ds_cpl vmx est tm2 ssse3 sdbg cx16 xtpr pdcm sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave rdrand lahf_lm 3dnowprefetch cpuid_fault cat_l2 ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase tsc_adjust smep erms mpx rdt_a rdseed smap clflushopt intel_pt sha_ni xsaveopt xsavec xgetbv1 xsaves dtherm ida arat pln pts md_clear arch_capabilities
free -m and free -h
[root@f5-422-04 ~]# free -m
total used free shared buff/cache available
Mem: 11691 429 9855 1 1406 10980
Swap: 4095 0 4095
[root@f5-422-04 ~]# free -h
total used free shared buff/cache available
Mem: 11Gi 430Mi 9.6Gi 1.0Mi 1.4Gi 10Gi
Swap: 4.0Gi 0B 4.0Gi
dmidecode -t memory
[root@f5-422-04 ~]# dmidecode -t memory
# dmidecode 3.2
Getting SMBIOS data from sysfs.
SMBIOS 3.0.0 present.
Handle 0x0030, DMI type 16, 23 bytes
Physical Memory Array
Location: System Board Or Motherboard
Use: System Memory
Error Correction Type: None
Maximum Capacity: 32 GB
Error Information Handle: Not Provided
Number Of Devices: 4
Handle 0x0031, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0030
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 4096 MB
Form Factor: SODIMM
Set: None
Locator: ChannelA-DIMM0
Bank Locator: BANK 0
Type: DDR3
Type Detail: Synchronous
Speed: 1600 MT/s
Manufacturer: Undefi
Serial Number: 88000000
Asset Tag: 9876543210
Part Number: /
Rank: Unknown
Configured Memory Speed: 1600 MT/s
Minimum Voltage: 44.975 V
Maximum Voltage: 44.975 V
Configured Voltage: 1.5 V
Handle 0x0032, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0030
Error Information Handle: Not Provided
Total Width: 64 bits
Data Width: 64 bits
Size: 8192 MB
Form Factor: SODIMM
Set: None
Locator: ChannelB-DIMM0
Bank Locator: BANK 1
Type: DDR3
Type Detail: Synchronous
Speed: 1600 MT/s
Manufacturer: Unde
Serial Number: [REDACTED]
Asset Tag: 9876543210
Part Number: CT102464BF160B.C16
Rank: Unknown
Configured Memory Speed: 1600 MT/s
Minimum Voltage: 1.35 V
Maximum Voltage: 1.5 V
Configured Voltage: 1.5 V
Handle 0x0033, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0030
Error Information Handle: Not Provided
Total Width: Unknown
Data Width: Unknown
Size: No Module Installed
Form Factor: DIMM
Set: None
Locator: ChannelC-DIMM0
Bank Locator: BANK 2
Type: Unknown
Type Detail: None
Speed: Unknown
Manufacturer: Not Specified
Serial Number: Not Specified
Asset Tag: 9876543210
Part Number: Not Specified
Rank: Unknown
Configured Memory Speed: 1600 MT/s
Minimum Voltage: Unknown
Maximum Voltage: Unknown
Configured Voltage: Unknown
Handle 0x0034, DMI type 17, 40 bytes
Memory Device
Array Handle: 0x0030
Error Information Handle: Not Provided
Total Width: Unknown
Data Width: Unknown
Size: No Module Installed
Form Factor: DIMM
Set: None
Locator: ChannelD-DIMM0
Bank Locator: BANK 3
Type: Unknown
Type Detail: None
Speed: Unknown
Manufacturer: Not Specified
Serial Number: Not Specified
Asset Tag: 9876543210
Part Number: Not Specified
Rank: Unknown
Configured Memory Speed: 1600 MT/s
Minimum Voltage: Unknown
Maximum Voltage: Unknown
Configured Voltage: Unknown
The vendor’s filling of the DMI registers does leave to be desired…
Minimum Voltage: 44.975 V
Maximum Voltage: 44.975 V
Configured Voltage: 1.5 V
yeah, right. But well, Configured Voltage is right.
Network Devices
upper NIC, 1Gig, PCIe
[root@f5-422-04 ~]# ethtool -i enp2s0
driver: r8169
version:
firmware-version: rtl8168h-2_0.0.2 02/26/15
expansion-rom-version:
bus-info: 0000:02:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
[root@f5-422-04 ~]# lspci -s 02:00.0
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller (rev 15)
middle NIC, 1Gig, USB
[root@f5-422-04 ~]# ethtool -i enp0s21f0u3
driver: r8152
version: v1.09.9
firmware-version:
expansion-rom-version:
bus-info: usb-0000:00:15.0-3
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: no
supports-priv-flags: no
[root@f5-422-04 ~]# lsusb -s 002:003
Bus 002 Device 003: ID 0bda:8153 Realtek Semiconductor Corp. RTL8153 Gigabit Ethernet Adapter
lower NIC, 10Gig, PCIe
[root@f5-422-04 ~]# ethtool -i enp1s0
driver: atlantic
version: 2.0.4.0-kern
firmware-version: 3.1.57
expansion-rom-version:
bus-info: 0000:01:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no
[root@f5-422-04 ~]# lspci -s 01:00.0
01:00.0 Ethernet controller: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] (rev 02)
iperf3 between 2 10G NICs
FIXME: get new measurements. These are from CentOS Stream
In c’t 02/2020 S. 102 they were measured at roughly 7 GBit/s (Article here but paywalled. As a subscriber I just read it on paper.)
7,0 gigabit are 6,07 gibibit, so I seem to be pretty close with FIXME on jumbo frames.
Setup of iperf3
[root@f5-422-01 ~]# firewall-cmd --add-port 5201/tcp
success
[root@f5-422-01 ~]# iperf3 --server --bind 192.168.30.181
MTU 1500
With the default MTU, I get 5.4 Gbits/sec out of the interfaces.
iperf3 -c 192.168.30.181 -i 1 -t 20 -P 4 | tee $(date +%F)-iperf3-mtu1500-4parallel.log
Connecting to host 192.168.30.181, port 5201
[ 5] local 192.168.30.182 port 39590 connected to 192.168.30.181 port 5201
[ 7] local 192.168.30.182 port 39592 connected to 192.168.30.181 port 5201
[ 9] local 192.168.30.182 port 39594 connected to 192.168.30.181 port 5201
[ 11] local 192.168.30.182 port 39596 connected to 192.168.30.181 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 113 MBytes 949 Mbits/sec 0 585 KBytes
[ 7] 0.00-1.00 sec 215 MBytes 1.80 Gbits/sec 0 642 KBytes
[ 9] 0.00-1.00 sec 214 MBytes 1.80 Gbits/sec 0 634 KBytes
[ 11] 0.00-1.00 sec 115 MBytes 961 Mbits/sec 0 665 KBytes
[SUM] 0.00-1.00 sec 657 MBytes 5.51 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 119 MBytes 995 Mbits/sec 0 643 KBytes
[ 7] 1.00-2.00 sec 204 MBytes 1.71 Gbits/sec 0 679 KBytes
[ 9] 1.00-2.00 sec 202 MBytes 1.70 Gbits/sec 0 672 KBytes
[ 11] 1.00-2.00 sec 120 MBytes 1.01 Gbits/sec 0 696 KBytes
[SUM] 1.00-2.00 sec 645 MBytes 5.41 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 0 643 KBytes
[ 7] 2.00-3.00 sec 215 MBytes 1.80 Gbits/sec 0 713 KBytes
[ 9] 2.00-3.00 sec 212 MBytes 1.78 Gbits/sec 0 707 KBytes
[ 11] 2.00-3.00 sec 111 MBytes 933 Mbits/sec 0 696 KBytes
[SUM] 2.00-3.00 sec 650 MBytes 5.45 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 108 MBytes 902 Mbits/sec 0 643 KBytes
[ 7] 3.00-4.00 sec 216 MBytes 1.81 Gbits/sec 0 713 KBytes
[ 9] 3.00-4.00 sec 216 MBytes 1.81 Gbits/sec 0 707 KBytes
[ 11] 3.00-4.00 sec 108 MBytes 902 Mbits/sec 0 696 KBytes
[SUM] 3.00-4.00 sec 648 MBytes 5.43 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 109 MBytes 912 Mbits/sec 0 643 KBytes
[ 7] 4.00-5.00 sec 216 MBytes 1.81 Gbits/sec 0 713 KBytes
[ 9] 4.00-5.00 sec 216 MBytes 1.81 Gbits/sec 0 707 KBytes
[ 11] 4.00-5.00 sec 109 MBytes 912 Mbits/sec 0 696 KBytes
[SUM] 4.00-5.00 sec 650 MBytes 5.45 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 110 MBytes 922 Mbits/sec 0 710 KBytes
[ 7] 5.00-6.00 sec 210 MBytes 1.76 Gbits/sec 0 789 KBytes
[ 9] 5.00-6.00 sec 210 MBytes 1.76 Gbits/sec 0 786 KBytes
[ 11] 5.00-6.00 sec 110 MBytes 922 Mbits/sec 0 776 KBytes
[SUM] 5.00-6.00 sec 640 MBytes 5.37 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 108 MBytes 902 Mbits/sec 0 803 KBytes
[ 7] 6.00-7.00 sec 215 MBytes 1.80 Gbits/sec 0 885 KBytes
[ 9] 6.00-7.00 sec 214 MBytes 1.79 Gbits/sec 0 875 KBytes
[ 11] 6.00-7.00 sec 108 MBytes 902 Mbits/sec 0 875 KBytes
[SUM] 6.00-7.00 sec 644 MBytes 5.40 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 109 MBytes 913 Mbits/sec 0 803 KBytes
[ 7] 7.00-8.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 7.00-8.00 sec 216 MBytes 1.81 Gbits/sec 0 875 KBytes
[ 11] 7.00-8.00 sec 109 MBytes 913 Mbits/sec 0 875 KBytes
[SUM] 7.00-8.00 sec 650 MBytes 5.46 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 109 MBytes 912 Mbits/sec 0 803 KBytes
[ 7] 8.00-9.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 8.00-9.00 sec 216 MBytes 1.81 Gbits/sec 0 875 KBytes
[ 11] 8.00-9.00 sec 109 MBytes 912 Mbits/sec 0 875 KBytes
[SUM] 8.00-9.00 sec 650 MBytes 5.45 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 108 MBytes 902 Mbits/sec 0 803 KBytes
[ 7] 9.00-10.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 9.00-10.00 sec 218 MBytes 1.82 Gbits/sec 0 875 KBytes
[ 11] 9.00-10.00 sec 108 MBytes 902 Mbits/sec 0 875 KBytes
[SUM] 9.00-10.00 sec 649 MBytes 5.44 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 10.00-11.00 sec 109 MBytes 912 Mbits/sec 0 803 KBytes
[ 7] 10.00-11.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 10.00-11.00 sec 216 MBytes 1.81 Gbits/sec 0 875 KBytes
[ 11] 10.00-11.00 sec 109 MBytes 912 Mbits/sec 0 875 KBytes
[SUM] 10.00-11.00 sec 650 MBytes 5.45 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 11.00-12.00 sec 108 MBytes 902 Mbits/sec 0 803 KBytes
[ 7] 11.00-12.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 11.00-12.00 sec 216 MBytes 1.81 Gbits/sec 0 875 KBytes
[ 11] 11.00-12.00 sec 108 MBytes 902 Mbits/sec 0 875 KBytes
[SUM] 11.00-12.00 sec 648 MBytes 5.43 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 12.00-13.00 sec 109 MBytes 912 Mbits/sec 0 803 KBytes
[ 7] 12.00-13.00 sec 218 MBytes 1.82 Gbits/sec 0 885 KBytes
[ 9] 12.00-13.00 sec 216 MBytes 1.81 Gbits/sec 0 875 KBytes
[ 11] 12.00-13.00 sec 109 MBytes 912 Mbits/sec 0 875 KBytes
[SUM] 12.00-13.00 sec 651 MBytes 5.46 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 13.00-14.00 sec 108 MBytes 902 Mbits/sec 0 803 KBytes
[ 7] 13.00-14.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 13.00-14.00 sec 216 MBytes 1.81 Gbits/sec 0 875 KBytes
[ 11] 13.00-14.00 sec 108 MBytes 902 Mbits/sec 0 875 KBytes
[SUM] 13.00-14.00 sec 648 MBytes 5.43 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 14.00-15.00 sec 109 MBytes 912 Mbits/sec 0 803 KBytes
[ 7] 14.00-15.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 14.00-15.00 sec 216 MBytes 1.81 Gbits/sec 0 1.28 MBytes
[ 11] 14.00-15.00 sec 109 MBytes 912 Mbits/sec 0 875 KBytes
[SUM] 14.00-15.00 sec 650 MBytes 5.45 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 15.00-16.00 sec 109 MBytes 912 Mbits/sec 0 803 KBytes
[ 7] 15.00-16.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 15.00-16.00 sec 218 MBytes 1.82 Gbits/sec 0 1.28 MBytes
[ 11] 15.00-16.00 sec 109 MBytes 912 Mbits/sec 0 875 KBytes
[SUM] 15.00-16.00 sec 651 MBytes 5.46 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 16.00-17.00 sec 108 MBytes 902 Mbits/sec 0 803 KBytes
[ 7] 16.00-17.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 16.00-17.00 sec 216 MBytes 1.81 Gbits/sec 0 1.28 MBytes
[ 11] 16.00-17.00 sec 108 MBytes 902 Mbits/sec 0 875 KBytes
[SUM] 16.00-17.00 sec 648 MBytes 5.43 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 17.00-18.00 sec 109 MBytes 912 Mbits/sec 0 803 KBytes
[ 7] 17.00-18.00 sec 218 MBytes 1.82 Gbits/sec 0 885 KBytes
[ 9] 17.00-18.00 sec 216 MBytes 1.81 Gbits/sec 0 1.28 MBytes
[ 11] 17.00-18.00 sec 109 MBytes 912 Mbits/sec 0 875 KBytes
[SUM] 17.00-18.00 sec 651 MBytes 5.46 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 18.00-19.00 sec 109 MBytes 913 Mbits/sec 0 803 KBytes
[ 7] 18.00-19.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 18.00-19.00 sec 216 MBytes 1.81 Gbits/sec 0 1.28 MBytes
[ 11] 18.00-19.00 sec 109 MBytes 913 Mbits/sec 0 875 KBytes
[SUM] 18.00-19.00 sec 650 MBytes 5.45 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 19.00-20.00 sec 108 MBytes 902 Mbits/sec 0 803 KBytes
[ 7] 19.00-20.00 sec 216 MBytes 1.81 Gbits/sec 0 885 KBytes
[ 9] 19.00-20.00 sec 216 MBytes 1.81 Gbits/sec 0 1.28 MBytes
[ 11] 19.00-20.00 sec 108 MBytes 902 Mbits/sec 0 875 KBytes
[SUM] 19.00-20.00 sec 648 MBytes 5.43 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-20.00 sec 2.13 GBytes 916 Mbits/sec 0 sender
[ 5] 0.00-20.02 sec 2.13 GBytes 914 Mbits/sec receiver
[ 7] 0.00-20.00 sec 4.20 GBytes 1.81 Gbits/sec 0 sender
[ 7] 0.00-20.02 sec 4.20 GBytes 1.80 Gbits/sec receiver
[ 9] 0.00-20.00 sec 4.20 GBytes 1.80 Gbits/sec 0 sender
[ 9] 0.00-20.02 sec 4.20 GBytes 1.80 Gbits/sec receiver
[ 11] 0.00-20.00 sec 2.14 GBytes 917 Mbits/sec 0 sender
[ 11] 0.00-20.02 sec 2.13 GBytes 915 Mbits/sec receiver
[SUM] 0.00-20.00 sec 12.7 GBytes 5.44 Gbits/sec 0 sender
[SUM] 0.00-20.02 sec 12.7 GBytes 5.43 Gbits/sec receiver
iperf Done.
MTU 9000
With jumbo frames, I get 5.8 Gbits/sec out of the interfaces.
iperf3 -c 192.168.30.181 -i 1 -t 20 -P 4 | tee $(date +%F)-iperf3-mtu9000-4parallel.log
Connecting to host 192.168.30.181, port 5201
[ 5] local 192.168.30.182 port 39576 connected to 192.168.30.181 port 5201
[ 7] local 192.168.30.182 port 39578 connected to 192.168.30.181 port 5201
[ 9] local 192.168.30.182 port 39580 connected to 192.168.30.181 port 5201
[ 11] local 192.168.30.182 port 39582 connected to 192.168.30.181 port 5201
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 114 MBytes 953 Mbits/sec 0 585 KBytes
[ 7] 0.00-1.00 sec 234 MBytes 1.96 Gbits/sec 0 673 KBytes
[ 9] 0.00-1.00 sec 233 MBytes 1.96 Gbits/sec 0 699 KBytes
[ 11] 0.00-1.00 sec 121 MBytes 1.01 Gbits/sec 0 559 KBytes
[SUM] 0.00-1.00 sec 701 MBytes 5.88 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 1.00-2.00 sec 121 MBytes 1.02 Gbits/sec 0 585 KBytes
[ 7] 1.00-2.00 sec 231 MBytes 1.94 Gbits/sec 0 673 KBytes
[ 9] 1.00-2.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 1.00-2.00 sec 111 MBytes 929 Mbits/sec 0 612 KBytes
[SUM] 1.00-2.00 sec 694 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 2.00-3.00 sec 119 MBytes 1000 Mbits/sec 0 673 KBytes
[ 7] 2.00-3.00 sec 229 MBytes 1.92 Gbits/sec 0 708 KBytes
[ 9] 2.00-3.00 sec 229 MBytes 1.92 Gbits/sec 0 699 KBytes
[ 11] 2.00-3.00 sec 118 MBytes 988 Mbits/sec 0 699 KBytes
[SUM] 2.00-3.00 sec 695 MBytes 5.83 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 3.00-4.00 sec 135 MBytes 1.13 Gbits/sec 0 673 KBytes
[ 7] 3.00-4.00 sec 212 MBytes 1.78 Gbits/sec 0 708 KBytes
[ 9] 3.00-4.00 sec 213 MBytes 1.78 Gbits/sec 0 699 KBytes
[ 11] 3.00-4.00 sec 136 MBytes 1.14 Gbits/sec 0 699 KBytes
[SUM] 3.00-4.00 sec 695 MBytes 5.83 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 4.00-5.00 sec 115 MBytes 968 Mbits/sec 0 673 KBytes
[ 7] 4.00-5.00 sec 232 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 4.00-5.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 4.00-5.00 sec 115 MBytes 967 Mbits/sec 0 699 KBytes
[SUM] 4.00-5.00 sec 694 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 5.00-6.00 sec 116 MBytes 972 Mbits/sec 0 673 KBytes
[ 7] 5.00-6.00 sec 230 MBytes 1.93 Gbits/sec 0 708 KBytes
[ 9] 5.00-6.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 5.00-6.00 sec 116 MBytes 969 Mbits/sec 0 699 KBytes
[SUM] 5.00-6.00 sec 693 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 6.00-7.00 sec 116 MBytes 970 Mbits/sec 0 673 KBytes
[ 7] 6.00-7.00 sec 231 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 6.00-7.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 6.00-7.00 sec 115 MBytes 967 Mbits/sec 0 699 KBytes
[SUM] 6.00-7.00 sec 693 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 7.00-8.00 sec 116 MBytes 972 Mbits/sec 0 673 KBytes
[ 7] 7.00-8.00 sec 231 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 7.00-8.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 7.00-8.00 sec 116 MBytes 973 Mbits/sec 0 699 KBytes
[SUM] 7.00-8.00 sec 694 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 8.00-9.00 sec 115 MBytes 969 Mbits/sec 0 673 KBytes
[ 7] 8.00-9.00 sec 231 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 8.00-9.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 8.00-9.00 sec 115 MBytes 964 Mbits/sec 0 699 KBytes
[SUM] 8.00-9.00 sec 692 MBytes 5.81 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 9.00-10.00 sec 115 MBytes 968 Mbits/sec 0 673 KBytes
[ 7] 9.00-10.00 sec 231 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 9.00-10.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 9.00-10.00 sec 116 MBytes 970 Mbits/sec 0 699 KBytes
[SUM] 9.00-10.00 sec 693 MBytes 5.81 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 10.00-11.00 sec 116 MBytes 972 Mbits/sec 0 673 KBytes
[ 7] 10.00-11.00 sec 232 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 10.00-11.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 10.00-11.00 sec 116 MBytes 974 Mbits/sec 0 699 KBytes
[SUM] 10.00-11.00 sec 695 MBytes 5.83 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 11.00-12.00 sec 115 MBytes 962 Mbits/sec 0 673 KBytes
[ 7] 11.00-12.00 sec 230 MBytes 1.93 Gbits/sec 0 708 KBytes
[ 9] 11.00-12.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 11.00-12.00 sec 115 MBytes 967 Mbits/sec 0 699 KBytes
[SUM] 11.00-12.00 sec 691 MBytes 5.80 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 12.00-13.00 sec 116 MBytes 970 Mbits/sec 0 673 KBytes
[ 7] 12.00-13.00 sec 232 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 12.00-13.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 12.00-13.00 sec 116 MBytes 969 Mbits/sec 0 699 KBytes
[SUM] 12.00-13.00 sec 694 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 13.00-14.00 sec 116 MBytes 974 Mbits/sec 0 673 KBytes
[ 7] 13.00-14.00 sec 231 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 13.00-14.00 sec 232 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 13.00-14.00 sec 116 MBytes 970 Mbits/sec 0 699 KBytes
[SUM] 13.00-14.00 sec 694 MBytes 5.83 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 14.00-15.00 sec 115 MBytes 967 Mbits/sec 0 673 KBytes
[ 7] 14.00-15.00 sec 232 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 14.00-15.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 14.00-15.00 sec 115 MBytes 966 Mbits/sec 0 699 KBytes
[SUM] 14.00-15.00 sec 693 MBytes 5.81 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 15.00-16.00 sec 117 MBytes 978 Mbits/sec 0 673 KBytes
[ 7] 15.00-16.00 sec 231 MBytes 1.93 Gbits/sec 0 708 KBytes
[ 9] 15.00-16.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 15.00-16.00 sec 116 MBytes 974 Mbits/sec 0 699 KBytes
[SUM] 15.00-16.00 sec 694 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 16.00-17.00 sec 115 MBytes 968 Mbits/sec 0 673 KBytes
[ 7] 16.00-17.00 sec 231 MBytes 1.93 Gbits/sec 0 708 KBytes
[ 9] 16.00-17.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 16.00-17.00 sec 116 MBytes 974 Mbits/sec 0 699 KBytes
[SUM] 16.00-17.00 sec 694 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 17.00-18.00 sec 116 MBytes 972 Mbits/sec 0 673 KBytes
[ 7] 17.00-18.00 sec 232 MBytes 1.95 Gbits/sec 0 708 KBytes
[ 9] 17.00-18.00 sec 231 MBytes 1.93 Gbits/sec 0 699 KBytes
[ 11] 17.00-18.00 sec 115 MBytes 968 Mbits/sec 0 699 KBytes
[SUM] 17.00-18.00 sec 694 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 18.00-19.00 sec 116 MBytes 970 Mbits/sec 0 673 KBytes
[ 7] 18.00-19.00 sec 230 MBytes 1.93 Gbits/sec 0 708 KBytes
[ 9] 18.00-19.00 sec 232 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 18.00-19.00 sec 116 MBytes 970 Mbits/sec 0 699 KBytes
[SUM] 18.00-19.00 sec 693 MBytes 5.82 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ 5] 19.00-20.00 sec 115 MBytes 968 Mbits/sec 0 673 KBytes
[ 7] 19.00-20.00 sec 231 MBytes 1.94 Gbits/sec 0 708 KBytes
[ 9] 19.00-20.00 sec 231 MBytes 1.94 Gbits/sec 0 699 KBytes
[ 11] 19.00-20.00 sec 116 MBytes 971 Mbits/sec 0 699 KBytes
[SUM] 19.00-20.00 sec 693 MBytes 5.81 Gbits/sec 0
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-20.00 sec 2.28 GBytes 981 Mbits/sec 0 sender
[ 5] 0.00-20.02 sec 2.28 GBytes 979 Mbits/sec receiver
[ 7] 0.00-20.00 sec 4.50 GBytes 1.93 Gbits/sec 0 sender
[ 7] 0.00-20.02 sec 4.49 GBytes 1.93 Gbits/sec receiver
[ 9] 0.00-20.00 sec 4.50 GBytes 1.93 Gbits/sec 0 sender
[ 9] 0.00-20.02 sec 4.49 GBytes 1.93 Gbits/sec receiver
[ 11] 0.00-20.00 sec 2.28 GBytes 979 Mbits/sec 0 sender
[ 11] 0.00-20.02 sec 2.28 GBytes 977 Mbits/sec receiver
[SUM] 0.00-20.00 sec 13.6 GBytes 5.82 Gbits/sec 0 sender
[SUM] 0.00-20.02 sec 13.5 GBytes 5.81 Gbits/sec receiver
iperf Done.