Going all-in with a Redhat 8 Home setup

Going all-in with a Redhat 8 Home setup

When CentOS went Stream there was outcry, personally however it didn't cause me much concern as I was using Ubuntu in my HomeLab. What's happening with CentOS is a shame, I understand the problems.. they were just not my issues..

DISCLAIMER

A few points of note for the uninitiated on my blog posts.

  1. I don't write these for you, I write them for me and make them public as they may help someone.
  2. I can't spell, my grammar is terrible.. Move past it.
  3. I'm not interested in a Linux Distro flame war.. I'm over it, its a tool, choose the right one for you, and be happy with it.
  4. I can't stress enough.. I'm not a master at any of this, i'm probably making poor choices. If you wnt to help me improve them. Thank you, If you want to troll me over them.. Jog on.. You're opinion isn't worth worrying about.

Lets move on..

Then at the start of Feb 2021 something happened. I couldn't tell if it was planned or a reaction to the CentOS backlash. RedHat allowed 16 RedHat Enterprise Linux servers "free" under thier Developer Accounts and specifically stated it was ok to use RHEL 8 on HomeLab setups, up to 16 devices.'

FAQs for no-cost Red Hat Enterprise Linux | Red Hat Developer
Frequently asked questions about no-cost RHEL

This information changed things for me.

At the time of writing I'm using RHEL at work, we have been moving to Ansible and its plainly obvious that while Ansible is cross linux compatible its strenght lies with RedHat.

I'm not looking for latest and greatest on my Home setup, I'm looking for stable. There are other technology challenges i'd like to get on top of and managing the OS on my Home setup isn't one of them.

So I made the decision to migrate my home setup from Ubuntu Server to RHEL8

Why?

Mainly for the challenge, the learning and the fun of it if i'm brutally honest. I like Red Hat as a server and I like ansible as a technology and combining the two of them seemed to be a good start of year project.

Get my Home setup 100% Red Hat based

How?

Having made the choice to do this there were some things I wanted to do on this setup.

  1. I wanted to create the OS Build in code using Packer
  2. I wanted to run on ProxMox
  3. I wanted to setup the servers using Ansible with as close to zero manual effort as possible.
  4. I wanted to run a container platform

I had a slight headstart as I have been using/learning Ansible over 2020 on and off so had lots of playbooks however this was a different challenge.

This post is about the technology stack i've chosen,  and fun I've had over the last 2 weeks migrating off Ubuntu onto Red Hat

Proxmox

I've based the builds virtually on Proxmox running on an old Thinkpad Laptop running a 1Tb SSD, 16Gb Ram and CPU(s) 4 x Intel(R) Core(TM) i5-3320M CPU @ 2.60GHz (1 Socket)

I'm not one of those people with a rack and servers with loads of Ram at home, my home network is based on lots of old hardware repurposed with RAM and Disk updates

I have a QNAP Nas which provides iSCSI to the Proxmox server (I was running this as a cluster of 3 Proxmox servers and the iSCSI was for failover on the cluster. I repurposed 2 of the 3 servers as described below)

Note
My next learning step is to understand how to use Terraform and Proxmox

Packer

I created the core VM build for RHEL using Packer, Packer now has a native builder for Proxmox

https://www.packer.io/docs/builders/proxmox

I like being able to create my Images consistantly from code, it means I can ensure each time i deploy the packer code I know I'm getting the same thing output.

I purposly kept the RHEL8 image very light, I didn't install any software, used the minimal server image, didn't enable subscription-manager as I can control this later using the Ansible code.

Bootstrap Script

In order to start using Ansible on the RHEL8 boxes i'd stood up I wanted to get each box setup with some specific settings. To do this I created a hacky bash script

Note
This should NOT be used in production environments.
#!/bin/bash
 ##setup consistent variables
YUM_CMD=$(which yum)
APT_GET_CMD=$(which apt-get)
PACKAGE_NAME=sshpass

##pull variables from file
source creds
echo "Rootuser is $ROOTUSER"
echo "Password is $ROOTPASS"

##check and install sshpass
echo "Checking if $PACKAGE_NAME is installed"
ISINSTALLED=$(which $PACKAGE_NAME)
echo "Performing appropriate install action"
if [[ -z $ISINSTALLED ]]; then
##If package isnt installed attept to installed using the
##appropriate package manager
echo "Checking for appropriate installer (Yum/Deb)"
    if [[ ! -z $YUM_CMD ]]; then
        yum install -y $PACKAGE_NAME
    elif [[ ! -z $APT_GET_CMD ]]; then
        apt-get install -y $PACKAGE_NAME
    else
        echo "error can't install package $PACKAGE_NAME"
        exit 1;
    fi
else
    echo "Package $PACKAGE_NAME is installed"
fi

##SCRIPT
#A loop will be run over a CSV file which pulls the contents in and 
#turns them into variables
INPUT=bootstrap.csv
OLDIFS=$IFS
IFS=','
[ ! -f $INPUT ] && { echo "$INPUT file not found"; exit 99; }
while read USERNAME PASSWD IPADDRESS SERVERNAME VMID
 do
echo "---------------------------------------------------------"
    echo "Name : $USERNAME"
    echo "Password : $PASSWD"
    echo "IP Address: $IPADDRESS"
    echo "Hostname : $SERVERNAME"
echo "Proxmox VM ID : $VMID" 
##Updates on the Remote Server
echo "Run Hostname and Sudo on Remote server"
sshpass -p $ROOTPASS ssh -o StrictHostKeyChecking=no $ROOTUSER@$IPADDRESS << EOF
uname -mrs
echo "Update Hostname"
hostnamectl set-hostname $SERVERNAME
echo "Updated Sodoers"
echo "$USERNAME   ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers
EOF
##Setup passwordless ssh for user
echo "Run ssh-copy-id"
sshpass -p $PASSWD ssh-copy-id $USERNAME@$IPADDRESS
echo "Copy Additional Certs over"
sshpass -p $PASSWD scp ~/.ssh/* $USERNAME@$IPADDRESS:/home/$USERNAME/.ssh/
echo "Remote Command Done"
 ##Enable QEMU Guest Agent on Boot on proxmox server
sshpass -p $ROOTPASS ssh -o StrictHostKeyChecking=no $ROOTUSER@$PROXMOX << EOF
qm set $VMID --agent enabled=1
EOF

done < $INPUT
IFS=$OLDIFS

What this script toes is take input from a CSV file which has the details of the server and the creds I setup on the packer image to login.

Once run it will

  • Set the hostname
  • Setup sudoers with nopasswd
  • copy over the installation ssh certs for passwordless ssh
  • Enable the QEMU Guest agent support on Proxmox

Having done this I had a cluster of machines ready to get started provisioning with Ansible.

AWX

I used AWX to manage running the Ansible Scripts I've setup, I wrote a post a while back covering how to setup AWX.

Setting up AWX to run Ansible Jobs from Gitlab at home
Back in June 2020 I wrote a post going over how to install AWX on Ubuntu 18.04.It turns out the instructions quite usefully work on Ubuntu 20.04 as well. Following many months of successfully running Rundeck[/rundeck-3-install-setup-and-an-example-project/] to complete the backgroundautomation t…
Note:
the look and feel of AWX has become more inline with the other Red Hat software, however the guide still works.

I like the simplicity of AWX and since the latest upgrade i've performed its also taken on the look and feel of other Redhat sites and apps like Cockpit. Its a simple easy location to manage and schedule Ansible jobs.

RHEL8

As previously mentioned, while the use of RHEL8 was to provide stability. Being an enterprise solution it also has some perks as well as being a well rounded and thought out ecosystem. As well as the OS there are some services which make using and monitoring the platform easier than some other platforms.

Subscription Manager

RedHat are providing 16 machines to use per subscription, while that isn't a huge number to count. Redhat do have subscription manager to "manage" the machines.

The installed RH boxes will use the command

subscription-manager --register

To connect the OS to the subscription which will display them on the Subscription page as systems and enable updates and access to the Red Hat repos.

Red Hat

This link will take you to the the dashboard which provides an overview of your Subscriptions.

Under Systems a list of the systems you've installed RHEL and subscribed appears, the list shows the name, the type, last time the system checked in and if there is any outstanding errata for the server.

Drlling down on each of the listed servers provides a wealth of information about the server, whats installed, facts which can be used by Ansible, errata and much more.

Heading over to Subscription Watch

Subscription Watch

will provide a more graphical display of this information

As well as controlling access to the updates and providing information surrounding the subscription RedHat also has..

Insights

Insights provide detailed information around the security and compliance of your servers. It does this in a dashboard format where its possible to drill down and find things like detailed advisories

and specific CVE details relating to machines on your network.

Using insights its quick and easy to roll out updates for security issues across an entire platform, record that this has been done and provide management reports on these actions.

Cockpit

Another item of software provided by Red Hat is Cockpit, this unline subscription manager and Insights runs locally on the server and provides a web interface to manage the server from.

I wrote about some of the features of Cockpit last year.

Using Red Hat Cockpit to centrally manage my Homelab
As I’ve been redesigning how i use my homelab, having a central console I can goto to manage the servers and get viability has become more of a requirement formy homelab setup. I have the key based ssh setup, I have software sat on each server formanagement of tools, what I was looking for was a…

While I do most of my work from the command line and am comfortable using the command line its nice to have some form of GUI.

Ansible

I've been using ansible on ubuntu for about a year and moving over to it on Red Hat has made life so mch easier. While the initial outlay is a longer one, being able to deploy each of the servers I use as its own Ansible playbook with an over arching playbook for shared (core) services which i can then assign to the serviers through AWX has made the deploying, breaking, redeploying of the home lab so so much easier.

When creating playbooks, in my own self taught way i've run throught waht I want to do from the command line, noted the steps and then looked into how to do this the Ansible way.

Being a service which is designed to ensure that the state of a server is the same each time the playbook is run and the outcome should be expected to be the same every time, little things like copng files are frowned upon by the community, however a little lateral thinking usually gets round the issue.

Following the rule of doing it the Ansible way and reverting to command or shell only when needed is also a good method of finding a resolution to an issue.

Homelab Management Software

AdGuard not PiHole

AdGuard Home | Network-wide software for any OS: Windows, macOS, Linux
AdGuard Home is a network-wide software for blocking ads & tracking. After you set it up, it’ll cover ALL your home devices, and you don’t need any client-side software for that.

I was a late entry to the Pi Hole stable, and love it. I love its blocking hooky links, its providing me with better control over my local Lan DNS far better than the GoogleWifi I was using.

Installing (at the time of writing) PiHole on RHEL8 however isn't a simple task, and when the hoops are jumped through the resulting install felt flakey at best.

This was a problem as I wanted a similar setup.

The result of a google search was ADGuard.

So what is AdGuard?

Its prudent to state here that i'm referring to AdGuard Home and if you are a PiHole user ill where I can explain the differences.

Pi-hole has been around for over five years, having initially been released on June 15, 2015. AdGuard Home, on the other hand, hasn’t even turned two yet. But ever since its announcement on October 16, 2018, it has gradually been attracting users away from Pi-hole.

AdGuard Home and Pi-hole can both be hosted locally, for example on a Raspberry Pi, and don’t require any additional software on your devices. Pihole however does seem to be tied tightly to debian based systems.

They are both Ad blockers and act as a DNS sinkholes and cover any device connected to your local network. Its the DNS setting of the device pointing to the AdGuard or Pi-hole server thats doing the work not the need for an agent on the device, so its transparent to the user.

You smart televisions, smartphones, tablets, and PCs are all included. That is why AdGuard Home and Pi-hole are described as network-level advertisement and internet tracker blocking applications.

So obviously the primary obbjective is to block unwanted ads, malware and that nasty stuff you could do on your laptop with any half decent ad blocking software. In todays home however there are many internet connected devices which you can't install software of this type on. So you push all the draffic to a server.

Aesthetically I personally find AdGuard to be much better laid out, the menu items are well thought out and I didn't have to go digging around quite as much as I did with Pi-hole. the graphine layouts are also a bit nicer looking than the blocky in your face pie charts on Pi-hole.

Its easy to both add additioanl filters onto the platform and where needed (as i did with itv.com which refuses to work unless there are no ad blockers) disable an entire domain.

As well as blocking Ad's I'm also using the Custom Filters section of AdGuard to provide the local DNS on my home network. Its very basic but easy to setup.

AdGuard Home supports DNS-over-HTTPS and DNS-over-TLS out of the box. Pi-hole does not have this feature out of the box. In order to use AdGuard on the more recent Android phones as DNS, this needs to be enabled and its easy to do so.

The upgrade process is also pretty painless and involves clicking an update button on the web interface and 2 minutes later (in my case) it was updated.

Finally there are the built in parental controls as well as having a global catch all ad blocking its possible to do fine grained controls per device and apply parental controls to groups of or individual devices. Handy

AdGuard Home vs. Pi-hole (2020) - Two ad blockers compared
AdGuard Home and Pi-hole both cover the basics. They block ads and trackers. But which one is the better choice? Find out in this comparison.

All in all I'm a big fan of the simplicity of setup of AdGuard, the interface and its DNS over HTTPS support..

AXIGEN Mailserver

On my previous setup I was using Ubuntu as a mail relay though my Office365 account to send emails out of the services I was running. While this worked well this time round I was looking to host an internal mail server which could recived all the system related mails into one place.

there are a myriad of options out there for doing this from howtoforge posts on building your own mail servers, iredmail to mailinabox. All of which are great systems. I plumped for Axigen mail mainly because I wanted to see what it was like and if it worked as a general enterprise level mail server.

Basic installation is very simple, in this case its an RPM based install (Deb and Windows available) and once setup and the appropriate firewalls are opened. You'll need to register for a free licence in the Axigen site and apply it.

There is an Admin Web interface and a Mail webgui, support for letsencrypt directly for both. I used a reverse proxy for the web interface.

As a fully functioning enterpise mail server it offers everrything you'd expect, and for a home network server its pretty useful.

The features list is here..

Free Mail Server | Axigen
Axigen Free Mail Server is a great alternative to open source. Runs on Linux and Windows and offers free email server users with calendars, WebMail, and mobile access.

Kubernetes Cluster

This is where the fun really started..

I originally had a 3 node Docker swarm running on RHEL8, however due to circumstances I've decieded to learn Kubernetes.

How to Install a Kubernetes Cluster on CentOS 8
In this article, we will run through the process of installing Kubernetes cluster on CentOS 8 platform, running on Docker-CE (Community Edition).

Using thee instructions I was able to manually install K8 on a Master and 2 node cluster.

This was then torn down and over a couple of days I've transformed this into Ansible code which is un under a kubernets Inventory on AWX.

Now I just need to brush up on how to stand something up in Kubernetes..

Thoughts

This is a hodge podge of a post, its not a howto guide or a software list. It doesn't really tell you that much. Its been an interesting journey. I've found Red hats tools to be really useful from a patching and security point of view and Ansible a bit easier to use on the platform I'm guessing its developed on.

While the latest and greatest versions of things may not always be there, because Red Hat is so well known and supported from an application standpoint most things have an RPM and or troubleshooting/howto's are available for most problems at an OS level.

This as someone who wasn't affected directly by the CentOS change is a good move and I do think companies like Ubuntu do need to do more especially with supporting services moving forward.


Share Tweet Send
0 Comments
Loading...