Linux and a little rant

Allow me to start with the rant bit first. I took my first steps in the world of Linux around 1993, 1994. Back then, Linux was still pretty new and very few distributions existed. Debian just spawned into life. RedHat wouldn’t come to life until mid 1995. There were no big Internet forums or user communities. Most people using Linux were computer engineers that knew how to build a Linux system from scratch and didn’t take newbie kid questions all too well. N00b questions were typically quickly and swiftly dealt with by a grump “RTFM” and you were left to do a lot of reading and figuring it out yourself. None to hold your hand and do the hard work for you. Annoying at times, but when you did figure it out it gave you a real sense of accomplishment. This is how I learned my way around Linux.

Fast forward to today, and you have hundreds of distributions. Some are touted as ‘beginner distros’, others as ‘advanced distros’. Let me shoot that down immediately. There are only user-friendly distros and user-unfriendly distros. Some extremely user-unfriendly. Functionally, they can all to the same. There is not some magical advanced functionality in these so-called ‘advanced distros’ not available in the ‘beginner distros’. The only difference between the two is that the ‘beginner’ distros are as easy to initially set up and run as your typical Windows or MacOS install and don’t offer too many customization choices to confuse you and provide you with a default set of applications you may or may not use, to get you started, while the ‘advanced’ distros pretty much require you to be a masochist and think and decide about every step. Once they are up and running, they work pretty much the same and can do the same. There is literally nothing one of these ‘advanced’ distros can do you cannot also do on one of these ‘beginner’ distros.

This gets me to the users… Some of these ‘advanced’ distro users feel they are sooo smart. They consider everybody that uses a ‘beginner’ distro to be a n00b that should be pitied as they cannot begin to grasp the advancedness of their knowledge and their choice of distribution. This holds particularly true to a significant portion of the Arch Linux community. They think they are so smart for using a distribution that only installs a basic system and dumps you to the command-line so you can use their package manager to install the stuff you actually want on your system. Don’t get me wrong, it is a very valid philosophy that avoids cluttering up your disk with GB’s of stuff you’re never going to be looking at. But it is not advanced.

Back in the day, you had to download the sources for just about everything because if you wanted something that wasn’t installed by default, it probably didn’t have a binary available for your system. You’d have go through the documentation, find and fix all the dependencies yourself (often also by compiling and installing the right versions, in order) and then compile your application from source, hoping it actually would compile without errors you’d then have to debug and fix before trying again. And when it was finally installed, it probably didn’t work until you build your configuration files manually.

Telling a package manager which software you want to install only for the package manager to download it for you from a central repository, fix any dependencies automatically and install your selected package so it works is not anything advanced. It’s just manual work. It’s typing dumb commands on the command prompt. It does not make you some Linux Guru. It doesn’t even give you any usable extra knowledge. If you enjoy doing things that way, all the more power to you. But don’t be some cocky arrogant SOB that belittles others for not wanting to do that. It doesn’t make your install better, just leaner, it doesn’t make you smarter and behaving like that only makes you a prick.

There are a lot of people that have heard enough about Linux to be curious and wanting to try it, that are permanently put-off by the arrogance and belittlement of the Linux community. People should try to remember they too had to learn at some point and realize there are no stupid questions, only stupid answers. More competition means more and better choices for us as users, but for that to happen new users that are willing to learn something new should be encouraged, not belittled for being new and put down until they give up to never return.

If you want to try Linux, just do it. Pick a distro you like and stick with it. Don’t be fooled into believing the nonsense about beginner and advanced distributions, thinking at some point you need to upgrade to get a more advanced version. They can all be customized to look like whatever you want, they can all have the same functionality, there is not a single one that is better than the rest. There is only personal preference of the users using them. That’s not to say it cannot be fun to do some distro hopping to find the one that really suits you. Just as long as you remember its preference, not functionality that makes the difference.

Accessing WSL on Windows 11

WSL filesystems in Windows Explorer on Windows 11

Moving files around between WSL instances and Windows just got a whole lot easier on Windows 11. All you need to do is scroll down in the left pane until you see Linux and can access the filesystem from any WSL instance to copy files to it, grab files from it and whatever it is you need.

Previously on Windows 10, you could access the special UNC path \\wsl$ to access the filesystem of a running WSL instance, but in Windows 11 you can access any WSL instance without the need to have it running.

Windows 11 on a unsupported computer

Yes, you can run the new Windows 11 beta or development versions on unsupported hardware. Yes, you will have heard all about it already. No, not every method is the same or works as well.

That said, most of what you see online are in my opinion methods that are not guaranteed to work or can in some cases be problematic.

This method described below always works, on every computer that can run and install Windows 10. Most likely, it will work with the release version of Windows 11, but we will have to wait to be sure obviously.

Step 1: Creating a Windows 10 ISO

Download the Media Creation Tool from the Microsoft site, run it and accept the license terms. It will get some things ready, or so it says so just wait.

When it is done, you are presented with two options:
o Upgrade this PC
o Create installation media (USB flash drive, DVD, or ISO file) for another PC

Select the second option and click next.

Select your language, edition and architecture and click next.

Select ISO image and click next. It will prompt you for a filename and location to save to. Then it should start downloading and generating your ISO file. Just let it run its course.

Step 2: Creating a Windows 11 ISO

Go to the UUP Dump website. This site will help you generate scripts to download and compile an ISO for the Windows software of your choice.

Note: These scripts download the necessary files directly from Microsoft, so it is safe and not some unknown source.

As of this writing, the latest version is “Windows 11 Insider Preview 10.0.22000.100 (co_release) amd64“. Click the release of your choice.

Select your language and click next.

Select your edition(s) and click next. I would choice just Windows 11 Pro here, however it is up to you.

Select Download and convert to ISO and below that, check all but component cleanup under conversion options. Click Create download package.

It will create and download a small zip file of a few kilobytes that contains scripts to create your ISO on Windows, MacOS or Linux.

Open an command prompt as Administrator and run the appropriate script. On Windows that would be uup_download_windows.cmd. Running this will take a while. Let it run. Go have a coffee. When it is done, you will have a Windows 11 ISO.

This ISO can be used on a supported machine to upgrade or install as normal.

Step 3: Creating a hybrid W10/W11 ISO

First we need to extract the Windows 10 ISO we created under Step 1. If you have a program like WinRAR, you can just right click the ISO file and extract or you can double click the ISO file to mount it, and then just copy all files and directories to a place on your hard disk.

When you have the contents of the Windows 10 ISO on your hard disk, navigate to the sources folder in the location you extracted or copied the ISO files to and find the file called install.esd. Delete it.

Now double click the Windows 11 ISO you created and navigate to the sources folder here as well. Again, find the file called install.esd. Copy it, and paste inside the sources folder of your Windows 10 files on your hard disk.

Now you basically have a Windows 10 DVD structure on your hard disk with Windows 11 content. We can now convert this file structure into a new, hybrid W10/W11 ISO.

First, you will need to download and install ImgBurn. Start the program and select Create image file from files/folders. It’s the middle option on the right side.

Select your source by selecting the location of your extracted or copied Windows 10 ISO on your hard disk where we just copied the Windows 11 install.esd to.

Select a destination where you want to save your newly created hybrid ISO to.

Under the advanced tab on the right side of the window, select “Bootable Disc” and make sure to check “Make Image Bootable”.

For the Boot image, browse to boot folder where you extracted your ISO on your hard disk and select the file etfsboot.com.

Lastly, under “Sectors To Load”, change the default of 4 to 8.

Now click the big button on the left bottom to create your new Hybrid ISO. This will be very quick.

Final words

What you have just created is a Windows 10 install disk that contains the Windows 11 files. Because of this, no pre-install checks on TPM, Secure boot or CPU are done and Windows 11 will be installed just like you are used to for Windows 10. After the install is finish, Windows does not do any post-install checks and it just boots and runs.

However, if you are like me and don’t meet most of the hardware requirements, you cannot turn on the Windows Insider option so you can download and install updates as they become available. To fix this, visit this GitHub site to download a small script to enable the Insider channel of your choice (mine is Beta) without even having to log in.

Monitoring your network or homelab using Zabbix and Grafana

For the longest time, I have been monitoring my network and homelab using Observium. This worked and does work very well. Observium is very good at what it does. However, there are a couple of things that do not work so well for me using Observium:

  • Observium does not let me add applications to monitor very easily or at all
  • Observium is limited to what can be offered through SNMP
  • Observium is not open source and as such it cannot be modified or changed to your needs
  • Observium is not an application I have come across in my professional life, so knowing how it works does not help me professionally.

That last bit is obviously not a necessity, however I do feel it’s always a nice thing to be able to apply things you have learned in your homelab into your professional environment.

After lots of investigations and trial & error, I have settled on using Zabbix for my monitoring needs. Zabbix is open source product that is used a lot in corporate environments and it is very flexible and extensible. Obviously you could add or change code as it is open source, but you don’t really need to. As Zabbix is template driven, its functionality can be extended by adding templates and there is a plethora of templates available for Zabbix, both on the Zabbix site itself (Zabbix Share) as well as places like GitHub. Also, Zabbix can use an agent installed on the system to collect the data you want to monitor, or you can use SNMP if you can’t or don’t want to install an agent on a device (for instance a network router).

The one thing I do not like about Zabbix is that the historic view is not easy to get to, nor as pretty displayed in a dashboard-like view as it is in Observium. However, that is not a blocker as we can use Grafana, another open source tool that is used quite a bit in Corporate Land to create dashboards and display relevant historic data.

Installing Zabbix

Installing Zabbix is no more complicated than installing most other software on Linux. I installed Zabbix on my Raspberry Pi 2B and the short version is this:

a. Install the Zabbix repository:
# wget https://repo.zabbix.com/zabbix/5.4/raspbian/pool/main/z/zabbix-release/zabbix-release_5.4-1+debian10_all.deb
# dpkg -i zabbix-release_5.4-1+debian10_all.deb
# apt update

b. Install the Zabbix server, frontend and agent on the server machine
# apt install zabbix-server-mysql zabbix-frontend-php zabbix-apache-conf zabbix-sql-scripts zabbix-agent

c. Create the database for Zabbix
# mysql -uroot -p
password
mysql> create database zabbix character set utf8 collate utf8_bin;
mysql> create user zabbix@localhost identified by 'password';
mysql> grant all privileges on zabbix.* to zabbix@localhost;
mysql> quit;

And import the database schema
# zcat /usr/share/doc/zabbix-sql-scripts/mysql/create.sql.gz | mysql -uzabbix -p zabbix
If your database is not residing on your Zabbix server, you can add -h <server> to the above to make sure you are connecting to the remote database server. Also, when creating the user and you are using a remote database, make sure the Zabbix user is allowed to connect over the network to the database.

d. Configure the Zabbix server to use the database you have created by filling out the relevant fields in /etc/zabbix/zabbix_server.conf

e. Start your brand-new Zabbix server:
# systemctl restart zabbix-server zabbix-agent apache2
# systemctl enable zabbix-server zabbix-agent apache2

f. You are done. You can now login to your server on http://<server>:3000 and log in with user Admin and password zabbix and configure everything else using the web frontend.

Installing Grafana

Installing Grafana is equally simple.
a. Install the pre-requisites and add the repository key
sudo apt-get install -y apt-transport-https
sudo apt-get install -y software-properties-common wget
wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -

b. Add the repository
echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list

c. Install Grafana
apt update
apt install grafana

d. Then it is just a matter of starting the Grafana server and making sure it starts at boot.
systemctl daemon-reload
systemctl start grafana-server
systemctl status grafana-server
systemctl enable grafan-server

That’s it! You can now log in using the default username/password of admin/admin.

Note: if you want to apply specific configurations, like for instance database (mysql, postgres, sqlite3) beyond the default, you should refer the Grafana manuals as that would be a bit beyond the scope of this page.

Tying things together…

First you will need to install the Zabbix app into Grafana. We can do this using the commandline interface for Grafana:

grafana-cli plugins install alexanderzobnin-zabbix-app

After you do this, you can configure the Zabbix datasource. Go to Configuration -> Data sources and click “add data source”. Scroll down to the bottom, and you will see one called ‘Zabbix’.

Fill in the details of your Zabbix installation, and make sure you add the api_jsonrpc.php to the end of your URL. Check ‘With credentials’ under auth and under Zabbix API details add your username and password. Click “Save & test” and if all is ok, it will give you a green checkmark while saying data source updated.

You are now ready to add a dashboard to your Grafana and start monitoring your Zabbix data. I use this dashboard from Paulo Paim. You can add it by going to Dashboards -> Manage and then clicking the import button. In the box saying import from grafana.com, type the ID of the dashboard. In this case 5363 and click load.

That’s it!

Links:
Zabbix manual
Grafana documentation
Zabbix templates and add-ons
Grafana dashboards and plugins

Prettify your fish shell

Windows Terminal running Ubuntu in WSL2 with Fish

I am using the fish shell these days. Fish stands for Friendly Interactive Shell. I like how programming the shell works and it has some very nice features like syntax and command highlighting you will not find in regular shells.

Note: important note if you are considering to use fish, fish is not a POSIX compliant shell. As such, for shell scripts that will need to work on any system you will still need to know how to script in sh/bash/ksh/etc. I do not consider fish to be a replacement for bash and it should not be the default shell on any system. It’s great as your interactive shell when working the console though.

With that out of the way, using a different or alternate shell can be fun but you probably also want it to look good. Or you want it to look the same or similar to your previous shell because that is what you are used to. As did I. I had a pretty nice prompt setup in bash, so I really wanted my fish shell to look and feel similar. This was the result:

fish prompt for root and regular user

To get this prompt, I created the following fish_prompt.fish under ~/.config/fish/functions.

# This theme is based on Bira theme from oh-my-fish (https://github.com/oh-my-fish/theme-bira)
# This theme also based on the default bash prompt of Kali Linux. (https://www.kali.org/)
# Created, modified and where possible bluntly stolen by throttlemeister.
#
# Bira theme from oh-my-fish listed abouve, based on:
# Theme based on Bira theme from oh-my-zsh: https://github.com/robbyrussell/oh-my-zsh/blob/master/themes/bira.zsh-theme
# Some code stolen from oh-my-fish clearance theme: https://github.com/bpinto/oh-my-fish/blob/master/themes/clearance/

function __user_host
  set fqdn (hostname -f)
  set -l content 
  if [ (id -u) = "0" ];
    echo -n (set_color --bold yellow)\((set_color --bold red)$USER(set_color --bold yellow)💀(set_color --bold red)$fqdn(set_color --bold yellow)\) (set color normal)
  else
    echo -n (set_color --bold blue)\((set_color --bold white)$USER(set_color --bold blue)웃(set_color --bold white)$fqdn(set_color --bold blue)\) (set color normal)
  end
end

function __current_path
  if [ (id -u) = "0" ];
    echo -n (set_color --bold yellow) [(set_color --bold white)(prompt_pwd)(set_color --bold yellow)] (set_color normal)
  else
    echo -n (set_color --bold blue) [(set_color --bold white)(prompt_pwd)(set_color --bold blue)] (set_color normal) 
  end
end

function _git_branch_name
  echo (command git symbolic-ref HEAD 2> /dev/null | sed -e 's|^refs/heads/||')
end

function _git_is_dirty
  echo (command git status -s --ignore-submodules=dirty 2> /dev/null)
end

function __git_status
  if [ (_git_branch_name) ]
    set -l git_branch (_git_branch_name)

    if [ (_git_is_dirty) ]
      set git_info '<'$git_branch"*"'>'
    else
      set git_info '<'$git_branch'>'
    end

    echo -n (set_color yellow) $git_info (set_color normal) 
  end
end

function fish_prompt
  if [ (id -u) = "0" ];
    echo -n (set_color --bold yellow)"╭─"(set_color normal)
  else
    echo -n (set_color --bold blue)"╭─"(set_color normal)
  end
  __user_host
  __current_path
  __git_status
  echo -e ''
  if [ (id -u) = "0" ];
    echo (set_color --bold yellow)"╰─""# "(set_color normal)
  else
    echo (set_color --bold blue)"╰─""\$ "(set_color normal)
  end
end

function fish_right_prompt
  set -l st $status

  if [ $st != 0 ];
    echo (set_color red) ↵ $st  (set_color normal)
  end
  set_color -o 666
  date '+ %T'
  set_color normal
end

Note: the latest version of this script can always be found on my GitHub here.

This prompt was, as the top comments indicate, not all my own and heavily borrowed and modified other prompt and resources. This exercise greatly helped me to understand the fishshell scripting language. This helped me to create some more scripts and functions to make my life in fish easier and simpler.

Mind you, sometimes bash is a lot simpler. For the same prompt, including checks for color support etc, the full bash equivalent for this in my .bashrc is this:

if [ -n "$force_color_prompt" ]; then
    if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then
        # We have color support; assume it's compliant with Ecma-48
        # (ISO/IEC-6429). (Lack of such support is extremely rare, and such
        # a case would tend to support setf rather than setaf.)
        color_prompt=yes
    else
        color_prompt=yes
    fi
fi

if [ "$color_prompt" = yes ]; then
    PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '
else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi

host=`hostname -f`

if [ "$color_prompt" = yes ]; then
    prompt_color='\[\033[;94m\]'
    info_color='\[\033[0;1m\]'
    prompt_symbol=웃
    p_col='\[\033[1;33m\]'
    if [ "$EUID" -eq 0 ]; then # Change prompt colors for root user
        prompt_color='\[\033[1;33m\]'
        info_color='\[\033[1;31m\]'
        prompt_symbol=💀
        p_col='\[\033[0;1m\]'
    fi
    PS1=$prompt_color'┌──${debian_chroot:+($debian_chroot)──}('$info_color'\u'$p_col'${prompt_symbol}'$info_color'$host'$prompt_color')-[\[\033[0;1m\]\w'$prompt_color']\n'$prompt_color'└─\$\[\033[0m\] '
    # BackTrack red prompt

else
    PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ '
fi

Where basically only the last bit is what makes the prompt. And even that can be concatenated into a single line if you insert the colors directly instead of setting variables.

As short as possible: application containers, system containers, virtualization

Sometimes, there is a bit of confusion as to what the difference is between application containers (Docker, Podman, K8S, OpenShift), system containers (LXC/LXD) and virtualization (KVM, Vmware, Hyper-V, Xen) and when you should use them. I will try to explain the differences as short as possible.

Application containers
Application containers are created with the most minimal environment to run a specific application. This includes the OS and all dependencies for that application. Every tool or program normally present in the OS that is not needed to run the application is typically left out so that the container image is a small as possible and performs as fast as possible.

Examples of application containers are Docker, Podman, Kubernetes and OpenShift.

System containers
System containers are typically as the minimal environment needed to run a specific operating system. All the basic tools are present, including the package manager so you can set up the system as you want with all the tools you need. A system container is like a virtual machine, but without the hardware virtualization layer. A system container uses and identifies the host hardware and runs the same kernel as the host. This means it is much lighter on resources than full blown virtualization, but also that in certain cases it can be incompatible with certain software.

Examples of system containers are LXC, LXD

Virtualization
Virtualization is software that emulates the hardware so that more than one virtual machines can be installed on the same physical hardware at the same time. As the full hardware layer needs to be simulated, it has the most overhead of these options but it also is the most compatible with all software.

There are two types of virtualization:

Type 1 hypervisors: virtualization is done by the kernel, providing low-level access to the physical hardware for the virtualization software for increased performance.
Examples of Type 1 hypervisors: KVM, Vmware ESXi, Hyper-V Server

Type 2 hypervisors: virtualization is done by an application installed on an operating system. Access to hardware must be done through the OS and no direct access is possible. IT provides convenience over performance, as it can run on any sytem.
Examples of Type 2 hypervisors: Vmware Workstation, VirtualBox, Hyper-V manager

Did you know… (or, RHEL on LXC/LXD)

  • There are no RedHat Enterprise Linux (RHEL) LXC/LXD container images publicly available?
  • There are LXC/LXD container images available for CentOS and Fedora?
  • You can convert a CentOS install to RHEL using the Convert2RHEL tool?
  • This also works for a LXC/LXD container?
  • You will need to do this if you want to run RHEL on Proxmox in a LXC container?
  • You can create a tarbal of a running system to import into a vanilla LXC/LXD installation?
  • You will need to create a metafile.tar.gz with a few lines of information about the tarbal to do this?
  • You can also use an export from a Docker container to get the system?
  • And that you can also use an export from a WSL distribution for this?
  • This means you can set up a WSL environment on your Windows box (see other posts here) just the way you want with all the tools you need, and you can export it to run it independently in a LXC/LXD container as a server?

Tested distributions under WSL2

The following distributions I have installed/created, tested and used under WSL2 and Windows 10.

  1. Alpine Linux – extremely tiny; created from Docker image
  2. AlmaLinux – migrated from CentOS using AlmaLinux migration tool
  3. Arch Linux – created from virtual machine install
  4. CentOS 7 – created from rootfs image, then upgraded to latest version
  5. CentOS 8 – Upgraded from CentOS7
  6. CentOS Stream – upgraded from from CentOS8 install
  7. Deepin Linux – created from virtual machine install
  8. Debian 10 (“Buster”) – Microsoft Store
  9. Debian Testing (“Bullseye”) – upgraded from official Debian 10 release
  10. Devuan – migrated from Debian as installed from the Microsoft Store
  11. Fedora – created from virtual machine installation
  12. Gentoo – compiled from source using Stage3 tarball/rootfs iamge
  13. Kali Linux – Microsoft Store
  14. Oracle Linux – migrated from CentOS instance using CentOS migration tool from Oracle
  15. RedHat Enterprise Linux – created from virtual machine installation
  16. Rocky Linux – migrated from CentOS using RockyLinux migration tool
  17. Ubuntu – Microsoft Store
  18. Slackware – created from Docker image

As you can see, if you can find your favorite distribution in the Microsoft Store, great, if not or if its only available as a paid distribution, just bake your own. It’s not that difficult and really there is no reason why you should not have the Linux distribution of your choice available under WSL2.

A couple little Windows things

I recently had a discussion on a forum on the usefulness of virtual desktops and the implementation on Windows 10. Now everyone can have their opinions on virtual desktops and I do as well, but in that discussion some things came up that are apparently not so well known.

  1. Some people find it cumbersome to switch between virtual desktops to the point it becomes unusable for them. They need to go to Task View (WIN+TAB), then use the mouse to click the desktop they want.

    What people do not realize, is there is of course a keyboard shortcut for this too. Just hit CTRL+WIN+→ or CTRL+WIN+← to switch/scroll through all the virtual desktops you have created.

    Bonus tip: when in Task View, you can drag and drop any open window into any existing virtual desktop, or even on a new one to organize your windows.
  2. Some people find it a great feature that Windows launches programs in each virtual desktop in their own process. Others find that annoying. Few people seem to know, you can choose the behavior you want with the correct settings:

    Go to: Settings -> System -> Multitasking

    Then, under Virtual desktops, you will see the first line that says: “On the taskbar, show windows that are open on” with a drop down. This dropdown has two options. The first is “All desktops” and the second is “Only the desktop I’m using”.

    “All desktops” will have programs share process, so if you click an open program in the taskbar, it will switch to the desktop it is running on and make the window active. The other option will start a new instance of that program on the desktop you are on if it is not running there.
  3. Some people do not like the new timeline feature in the task view screen (WIN+TAB). This can be turned off if you want.

    Go to: Settings -> Privacy -> Activity history

    On that page, uncheck both items. Scroll down and toggle all accounts for which activities are shown, and done. No more timeline when doing WIN+TAB

New site

After building and setting up my new computer and subsequent move away from the Apple Mac ecosystem, I was left with an issue that the site I had built was setup using a Mac-only tool and could not be migrated to Windows. After mucking around with different tools on Windows and hacking things together, I have now decided to go platform independent and recreate my website using the ever popular WordPress CMS.

Setting it up on the Synology was a breeze and after finding a suitable theme for my needs I have now started to get things up and running again, adding the content, etc. So far, it looks pretty ok and this may indeed be the way to go.