This past week, I upgraded my primary PostgreSQL servers from version 13 to version 16. The process was relatively straightforward, but there were a few glitches that I wanted to document here. My intention is to do this upgrade annually and usually stay a year behind the current release (mostly …

After the recent death of one of my long-running Juniper EX-2200Cs, I needed to find a replacement. I decided to go with a Ruckus ICX 7150-C12P-2X10GR, which is a 12-port PoE switch with 2 10G SPF+ uplinks. It's basically a rebranded Brocade switch. I'd been happy with the Juniper, right …

Given a bit of downtime waiting for my AC repair, I decided to take a look at the login and startup items on my Mac. I've not been having issues, but it seems like good hygeine to know what these are. Unfortunately, although the Open at Login section of the …

I've been describing our Hashicorp Vault journey here at ClueTrust in a number of posts. Chief among the reasons to use Vault is its ability to generate and rotate credentials with specific systems and services. I've written before about PostgreSQL credential management using Vault, which has been quite successful. This …

After upgrading to Sonoma, I started occasionally (and then repeatedly) noticing warning messages and errors related to cloud files in my laptop and desktop machines in the area that is for iCloud. The specific files aren't important, although they seem to be related to applications (mostly on the phone) that …

The first time I did this I didn't document it very well, causing the next time to be more time consuming, so her'es the rundown. It's not a secret that we use some older Dell hardware as servers in our datacenter. We've been pretty happy with it since switching away …

Distroless images are all the rage in the container space these days due to the reduced attack surface. This is great and also results in much thinner images. But, when an image isn't behaving it can cause some additional trouble as you try to figure out what may be missing …

When using dynamic database credentials with Flask, we need to make sure that the flask instance picks up the right credentials, renews them when necessary, and uses the right roles. My flask code is pretty embedded with the database changes here, so pardon the dust, but I think it's relatively …

When I was confirming the configurations for my vault management of database credentials, I used a local postgresql and vault server. This may also be useful for development (especially testing code that may exercise the vault and database interactions). This can make it relatively easy to watch all of the …

This is part of a multi-part series on using postgres databases, vault, and a variety of other tools to effect short-lived database credentials for real use. As postgres uses user and role interchangably, so will I, although I'll generally try to use user to refer to a role with login …

When using dynamic database credentials with Django, we need to make sure that the django instance picks up the right credentials, renews them when necessary, and uses the right roles. This post includes the background and the necessary code. Migration and creation Migration and creation provide special problems because of …

This morning I had the need to change the IP address configuration for the load balancer in our k8s cluster. The basics of changing the main pool in metallb were straightforward enough, but when I applied my changes, I didn't get what I needed. So, what happened? Originally, I wasn't …

Another chapter in my learning kubernetes the hard way, this time Longhorn. Probably ill-advisedly, I'm using ephemeral volumes for my storage volumes in Longhorn and have a habit of leaving the nodes in the cluster as they're being rebuilt. Generally, this isn't a problem. This weekend, I was a bit …

Over the past year, I've been experimenting with Hashicorp Vault, using the open-source/community version for some internal experiments, including some with high availability. In a separate article, I'll go over a test configuration of Vault, but all of the notes here are agnostic to the use of HCP (Hashicorp's …

This post is mostly a warning to me for the future, but hopefully it'll prevent somebody else from going through the same problem. I've been running a small Kubernetes cluster for a couple of years now, mostly as an experiment and to keep my skills tuned for new tooling. Part …

Recently, I noticed a problem with my Index Lifecycle Management (ILM) not appropriately rotating indexes. The error was not super clear, but I did notice that the existing index had just reached 90 days without closing and that was the first move in the ILM. It was clear that the …

Background Generally speaking, I refresh most of my systems pretty regularly, spurred on by security concerns, general hygeine, a desire to make sure the automation doesn't age out, and certificate expiration. Although I don't need to refersh systems due to certificate expiration, it has historically been the easiest indicator of …

I've already written a bit about using renovate to keep dependencies current using Renovate On Prem in Renovating GitLab Repos. This has been working well. However, there are a couple of twists that I figured I'd document in the event that people run into them. For single-repositories with public dependencies …

Most of the system administration work that I do has been automated using Ansible, as I've mentioned in posts here, including Deploying with GitLab. Now that I've got Renovate in place (Renovating GitLab Repos), I am starting to look at how to expand beyond my existing automations in order to …

I note with some amusement the fact that I wrote on this exact day last year about this same subject (in much more detail). The reason for the new message on this subject is that I'm still cleaning up some of the decisions I made when first using Bacula. The …