After upgrading to Sonoma, I started occasionally (and then repeatedly) noticing warning messages and errors related to cloud files in my laptop and desktop machines in the area that is for iCloud. The specific files aren't important, although they seem to be related to applications (mostly on the phone) that …

The first time I did this I didn't document it very well, causing the next time to be more time consuming, so her'es the rundown. It's not a secret that we use some older Dell hardware as servers in our datacenter. We've been pretty happy with it since switching away …

Distroless images are all the rage in the container space these days due to the reduced attack surface. This is great and also results in much thinner images. But, when an image isn't behaving it can cause some additional trouble as you try to figure out what may be missing …

When using dynamic database credentials with Flask, we need to make sure that the flask instance picks up the right credentials, renews them when necessary, and uses the right roles. My flask code is pretty embedded with the database changes here, so pardon the dust, but I think it's relatively …

When I was confirming the configurations for my vault management of database credentials, I used a local postgresql and vault server. This may also be useful for development (especially testing code that may exercise the vault and database interactions). This can make it relatively easy to watch all of the …

This is part of a multi-part series on using postgres databases, vault, and a variety of other tools to effect short-lived database credentials for real use. As postgres uses user and role interchangably, so will I, although I'll generally try to use user to refer to a role with login …

When using dynamic database credentials with Django, we need to make sure that the django instance picks up the right credentials, renews them when necessary, and uses the right roles. This post includes the background and the necessary code. Migration and creation Migration and creation provide special problems because of …

This morning I had the need to change the IP address configuration for the load balancer in our k8s cluster. The basics of changing the main pool in metallb were straightforward enough, but when I applied my changes, I didn't get what I needed. So, what happened? Originally, I wasn't …

Another chapter in my learning kubernetes the hard way, this time Longhorn. Probably ill-advisedly, I'm using ephemeral volumes for my storage volumes in Longhorn and have a habit of leaving the nodes in the cluster as they're being rebuilt. Generally, this isn't a problem. This weekend, I was a bit …

Over the past year, I've been experimenting with Hashicorp Vault, using the open-source/community version for some internal experiments, including some with high availability. In a separate article, I'll go over a test configuration of Vault, but all of the notes here are agnostic to the use of HCP (Hashicorp's …

This post is mostly a warning to me for the future, but hopefully it'll prevent somebody else from going through the same problem. I've been running a small Kubernetes cluster for a couple of years now, mostly as an experiment and to keep my skills tuned for new tooling. Part …

Recently, I noticed a problem with my Index Lifecycle Management (ILM) not appropriately rotating indexes. The error was not super clear, but I did notice that the existing index had just reached 90 days without closing and that was the first move in the ILM. It was clear that the …

Background Generally speaking, I refresh most of my systems pretty regularly, spurred on by security concerns, general hygeine, a desire to make sure the automation doesn't age out, and certificate expiration. Although I don't need to refersh systems due to certificate expiration, it has historically been the easiest indicator of …

I've already written a bit about using renovate to keep dependencies current using Renovate On Prem in Renovating GitLab Repos. This has been working well. However, there are a couple of twists that I figured I'd document in the event that people run into them. For single-repositories with public dependencies …

Most of the system administration work that I do has been automated using Ansible, as I've mentioned in posts here, including Deploying with GitLab. Now that I've got Renovate in place (Renovating GitLab Repos), I am starting to look at how to expand beyond my existing automations in order to …

I note with some amusement the fact that I wrote on this exact day last year about this same subject (in much more detail). The reason for the new message on this subject is that I'm still cleaning up some of the decisions I made when first using Bacula. The …

I've been an ELK Stack (Elasticsearch, Logstash, Kibana, and Beats) user for quite some time, using exclusively the open source version fo the stack. Generally it's works well and, with some exceptions, supports our mostly-Solaris based environment (using LX zones to host most of the beefier components, and using custom-built …

After 18 months of using Bacula and sending copies of my data to the cloud (in this case, cloud I operate in another location) using an S3-compatible storage mechanism, I noticed I had a lot of data sitting around on my current server for backups. When I set out to …

Back in September of last year, I wrote in Bacula: 6 months on that cloud backups required part.0 in order to be recognized for automatic part retrieval. While this was mostly accurate, the critical file is actually part.1. As such, when referencing my own blog post when trimming …

In June, I mentioned in an article about Docker on SmartOS that we are doing some work with GitLab these days as a replacement for my venerable Gitolite server (and, to an increasing extent Jenkins). Deploying from Pelican I'm likely going to write more on GitLab in the near future …