A couple months ago, I switched over from Google Domains for DNS management to Cloudflare, largely for access to their CDN. I didn’t realize their free services were so expansive, and I’ve been happy to get access to all that they give me. However, the transition has not been without its pains. All of the sites on my server (along with this site) are served using an nginx reverse proxy, with SSL certificates managed through Let’s Encrypt’s certbot tool.
In my last post I wrote about gathering nginx VTS metrics with nginx-mod-vts, a dynamic nginx module I chose to build through an AUR package. This has come back to bite me a little bit, and I want to document the steps that went into my last nginx update so I have a reference going forward. After the latest update of nginx-mainline, I started receiving this error when I attempted to update my system: looking for conflicting packages.
Shortly after getting my basic alerting script marvin written, I got the feeling that my monitorix/marvin system was a bit too hacky for my liking and decided to upgrade to a more proper ecosystem. After some research, I settled on using Prometheus as my metrics aggregator, Grafana for visualizations, and integrating Prometheus’ Alertmanager with a webhook configuration to report alerts to my Matrix rooms. The work spanned about a month of on-and-off focus, so I wanted to get some documentation written on what was involved in this setup before I forget too much.
I recently setup a Matrix server to give my friends and I a place to collaborate on work. I quickly realized this is the perfect place to output reports and alerts regarding my services, so I wrote up a Python script to report on site outages, excessive server load and systemd services that are in a failed state. I like to personify bots as much as possible, so I named it Marvin after the paranoid android in The Hitchhiker’s Guide to the Galaxy series.
This weekend Nicolas Knoebber asked me if I would be willing to host the web server for his dotfile project. Always looking to play at being a sysadmin, I was ecstatic when he asked. The project is primarily a CLI application used to provide quick access and version control for your linux dotfiles such as your .bashrc, but the hope is to provide a file server with the codebase so that users can host their own remote file versions.
Last Tuesday I got an email from Let’s Encrypt that they needed to revoke a few million certificates that they had issued, and that the certificate I was using for this website happened to be one of them. I wasn’t particularly bothered; they seemed to have a handle on things and I generally prefer bugs that are found and fixed than bugs that are left to cause trouble. When I got to work I logged into my server figuring I could update the certificate quickly and be on my way.
This site is served using nginx, an extremely capable, open-source tool that makes serving static content remarkably simple. I’m using the nginx-mainline package, which is what I would call their bleeding-edge version. On Arch run pacman -S nginx-mainline to install it. After installing, you’ll need to get your configuration file set up. I just edit over my ssh session with vim, but you can do this however you see fit. With nginx-mainline, your configuration file will be located at /etc/nginx/nginx.
For a static website being hosted on a single server instance, your build and deploy pipeline don’t have to be particularly complex. I find the standard Hugo build tools and rsync to be more than enough for a personal site like this one. First, I make sure that I have an updated build of my site in my workspace. With Hugo, this isn’t too hard, I just go into the root of my project repository and use hugo --gc.
I decided to revitalize this website on a whim. My desktop computer had been sitting at my desk, largely unused except for occasional online games and managing my music library. With winter around the corner, I figured I could either dive deep into an RPG to pass the upcoming months (WoW Classic was very tempting), or disconnect the peripherals from my desktop, open up port 22, and use it as a hobby server for my side projects that never seem to get a proper deployment configured.
Since this website seems to be turning into a means of keeping my family updated on my adventures, I thought I would include a copy of an email I wrote up after riding in RAGBRAI in July of 2019. Hey everyone, For the last two weeks I’ve been saying I would get an email together to tell you all about how RAGBRAI went; this is that email. My ideas are pretty scattered in here, and I left out a lot.