[{"content":"I\u0026rsquo;ve recently been working with Pulumi1 in my Bazel monorepo, using Go as the runtime language and realized that the out of the box experience with the go_library, go_binary, etc, rules sort of sucks.\nSo I made it better.\nTLDR # Before: _deps = [ \u0026#34;//src/go/comptime\u0026#34;, \u0026#34;@com_github_pulumi_pulumi_sdk_v3//go/pulumi\u0026#34;, \u0026#34;@com_github_pulumi_pulumi_sdk_v3//go/pulumi/config\u0026#34;, \u0026#34;@com_github_pulumi_pulumi_vault_sdk_v7//go/vault\u0026#34;, \u0026#34;@com_github_pulumi_pulumi_vault_sdk_v7//go/vault/pkisecret\u0026#34;, ] go_binary( name = \u0026#34;openbao-pulumi-runtime\u0026#34;, srcs = [\u0026#34;main.go\u0026#34;], deps = _deps + [ \u0026#34;:config\u0026#34;, \u0026#34;:pki\u0026#34;, ], ) go_library( name = \u0026#34;config\u0026#34;, srcs = glob([\u0026#34;config/*.go\u0026#34;]), deps = _deps, importpath = \u0026#34;my.internal/pulumi/openbao/config\u0026#34;, ) go_library( name = \u0026#34;pki\u0026#34;, srcs = glob([\u0026#34;pki/*.go\u0026#34;]), deps = _deps, importpath = \u0026#34;my.internal/pulumi/openbao/pki\u0026#34;, ) After: _deps = [ \u0026#34;/comptime\u0026#34;, \u0026#34;github.com/pulumi/pulumi/sdk/v3:go/pulumi\u0026#34;, \u0026#34;github.com/pulumi/pulumi/sdk/v3:go/pulumi/config\u0026#34;, \u0026#34;github.com/pulumi/pulumi-vault/sdk/v7:go/vault\u0026#34;, \u0026#34;github.com/pulumi/pulumi-vault/sdk/v7:go/vault/pkisecret\u0026#34;, ] go.binary( name = \u0026#34;openbao-pulumi-runtime\u0026#34;, srcs = [\u0026#34;main.go\u0026#34;], deps = _deps + [ \u0026#34;:config\u0026#34;, \u0026#34;:pki\u0026#34;, ], ) go.library( name = \u0026#34;config\u0026#34;, srcs = glob([\u0026#34;config/*.go\u0026#34;]), deps = _deps, ) go.library( name = \u0026#34;pki\u0026#34;, srcs = glob([\u0026#34;pki/*.go\u0026#34;]), deps = _deps, ) Major points being:\nUsage of normal(ish) go module strings over trying to guess how exactly gazelle is naming your Bazel repo\n(And also some shorthands for in-monorepo go deps)\nNo need to specify importpaths for each go_library!\nHow it works # So this is basically a combination of three underlying functions:\nTranslating a given go module string into the correct Bazel label Adding a bit of structured string manipulation to deps resolution Calculating the importpath via Label resolution Translate # Starting with the first point, we\u0026rsquo;re basically reimplementing ImportPathToBazelRepoName from gazelle2, but in starlark:\ndef _go_pkg_to_repo_name(pkg): \u0026#34;\u0026#34;\u0026#34; Emulate what @gazelle does to map go pkg paths to bazel repository names \u0026#34;\u0026#34;\u0026#34;.strip() pkg = pkg.lower() parts = pkg.split(\u0026#34;/\u0026#34;) labels = parts[0].split(\u0026#34;.\u0026#34;) rlabels = [] for i in range(len(labels)): rlabels.append(labels[len(labels) - i - 1]) return _strip_nonwords(\u0026#34;.\u0026#34;.join(rlabels + parts[1:])) _replacements=[ \u0026#34;.\u0026#34;, \u0026#34;-\u0026#34;, \u0026#34;/\u0026#34;, \u0026#34;@\u0026#34;, \u0026#34;~\u0026#34;, \u0026#34;+\u0026#34;, ] def _strip_nonwords(s): out = s for r in _replacements: out = out.replace(r, \u0026#34;_\u0026#34;) for _ in range(10): if \u0026#34;__\u0026#34; not in out: break out = out.replace(\u0026#34;__\u0026#34;, \u0026#34;_\u0026#34;) return out Dep strings # Next, we do some string manipulation (a lot of this is omitted for brevity)\ndef _go_expand_dep_string(dep, aliases): # An absolute or package local repository label -- pass this through directly if dep.startswith(\u0026#34;@//\u0026#34;) or dep.startswith(\u0026#34;//\u0026#34;) or dep.startswith(\u0026#34;:\u0026#34;): return dep # An alias if dep.startswith(\u0026#34;@\u0026#34;): alias = _go_expand_dep_alias(dep, aliases) if alias: return _go_third_party_module_to_label(alias) fail(\u0026#34;unknown alias in Go dep: \u0026#39;%s\u0026#39;. Known aliases are: %s\u0026#34; % (dep, aliases.keys())) # A third_party vendored go dep if dep.startswith(\u0026#34;/third_party/\u0026#34;): return \u0026#34;//third_party/go/vendor\u0026#34; + dep.removeprefix(\u0026#34;/third_party\u0026#34;) # A first party go dep if dep.startswith(\u0026#34;/\u0026#34;): return \u0026#34;//src/go\u0026#34; + dep # Else we assume its a third_party module return _go_third_party_module_to_label(dep) Quoting the commit that introduced this function:\n@//\u0026lt;rest...\u0026gt;, //\u0026lt;rest...\u0026gt;, :\u0026lt;rest...\u0026gt; Anything that is a valid absolute label is passed through directly\n@\u0026lt;alias\u0026gt;:\u0026lt;rest...\u0026gt; Callers may reference any known alias via starting their dep spec with an \u0026lsquo;@\u0026rsquo;. These are primarily for shortening otherwise long module specs, but can be used for any Go module string.\nAliases can be created via the make_alias function, and passed into make_go_defs via the aliases dict kwarg.\n/third_party/\u0026lt;rest...\u0026gt; Reference a module in //third_party/go/vendor, for any third_party deps we have vendored ourselves.\n/\u0026lt;rest...\u0026gt; Reference a first party module from //src/go\nEverything else is interpreted as a Go module string and expanded as such\nThere is one key difference to normal Go module strings: You must separate the top level module (where the go.mod sits) from any subpackages by a \u0026lsquo;:\u0026rsquo;.\nCalculating importpaths # Lastly, we automatically calculate each go_* rule\u0026rsquo;s importpath, via a provided modbase, anchor and path. These roughly equal the module stanza in your go.mod (e.g my.internal), the current package (via native.package_relative_label) and the name of the go_* rule that is being created.\ndef _go_relative_import_path(parent, strip, anchor, path): p = anchor.package.removeprefix(strip).removeprefix(\u0026#34;/\u0026#34;).removesuffix(\u0026#34;/\u0026#34;) # Handle cases where the Bazel package name and the importpath right most # segment are equal # # Most commonly in cases where the go_library\u0026#39;s name equals the Bazel # package name # # Ex: //src/go/foo:foo if xtpath.basename(p) == path: return xtpath.join(parent, p) # If the importpath is a subpath under the current Bazel package, strip the # package name before concating so we don\u0026#39;t duplicate it # # Ex: //src/go/foo:foo/config if \u0026#34;/\u0026#34; in path and path.startswith(xtpath.basename(p)): return xtpath.join(parent, p, path.removeprefix(xtpath.basename(p)).removeprefix(\u0026#34;/\u0026#34;)).rstrip(\u0026#34;/\u0026#34;) # Otherwise we can just do things normally # # Ex: //src/go/foo:http return xtpath.join(parent, p, path).rstrip(\u0026#34;/\u0026#34;) Putting it all together # To tie it all together at the top level, we create a make_go_defs function which returns a struct with members that wrap each upstream rule_go rule:\ndef make_go_defs(modbase, strip, aliases): \u0026#34;\u0026#34;\u0026#34; Return a struct of rules_go rule wrappers, that utilize the given modbase, strip and aliases. ...snipped for brevity \u0026#34;\u0026#34;\u0026#34;.strip() return struct( binary = lambda **kwargs: make_go_binary_wrapper( modbase = modbase, strip = strip, aliases = aliases, **kwargs ), library = lambda **kwargs: make_go_library_wrapper( modbase = modbase, strip = strip, aliases = aliases, **kwargs ), test = lambda **kwargs: make_go_test_wrapper( modbase = modbase, strip = strip, aliases = aliases, **kwargs ), # ... rest of the rules_go rules as reexports ) Which is then used like so:\nload(\u0026#34;//lib/bazel/go:defs.bzl\u0026#34;, \u0026#34;make_go_defs\u0026#34;, mkalias = \u0026#34;make_alias\u0026#34;) _aliases = { # ...snipped for brevity } # This is what the go.binary, go.library stuff from the TLDR is referring to go = make_go_defs( modbase = \u0026#34;my.internal\u0026#34;, strip = \u0026#34;src/go\u0026#34;, aliases = _aliases, ) All in all, I think it makes it much nicer to use rules_go / Go in Bazel.\nClosing # Anyone else come across a similar issue when using rules_go? I figure at least the issue of writing importpaths everywhere is something that anyone using Go + Bazel encountered quickly and probably came up with a fix for.\nNone of this publicly available, but if you\u0026rsquo;re interested in it, I\u0026rsquo;m not opposed to making it into a Bazel module for you.\nMaybe my rules_pulumi rule set should be the subject of another post?\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://github.com/bazel-contrib/bazel-gazelle/blob/v0.49.0/label/label.go#L236\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"April 21, 2026","externalUrl":null,"permalink":"/p/a-better-rules-go/","section":"Posts","summary":"","title":"A Better rules_go","type":"p"},{"content":"","date":"April 21, 2026","externalUrl":null,"permalink":"/p/","section":"Posts","summary":"","title":"Posts","type":"p"},{"content":"","date":"April 21, 2026","externalUrl":null,"permalink":"/","section":"thirdbrain","summary":"","title":"thirdbrain","type":"page"},{"content":"I love remote work, and I think we will see a resurgence of remote positions in 2026. Here\u0026rsquo;s why.\nCaveat # This is for the European job market, not America (or elsewhere) as I work in Europe.\nJob market is recovering # Anecdotally I can tell you the market is recovering. I\u0026rsquo;ve had 6 recruiters reach out to me directly this week \u0026ndash; and two even bothered to proactively screen out the non remote positions when pinging me.\nI currently work in a fully remote team of contractors, and most of us are getting 3+ pings a week now. This experience is the norm, as of about mid February \u0026lsquo;26, at least for DevOps / Cloud specialists that have 8+ years under their belt.\nAlso, as a classically leading indicator, take a look at the recovery of HR and recruitment roles across Europe:\nGraph of EU ISCO 242 employment data 24Q4-25Q4, retrieved from https://ec.europa.eu/eurostat/databrowser/bookmark/1136e193-9940-4458-a9aa-475b5b7c0879?lang=en\u0026createdAt=2026-03-25T19:02:03Z This data doesn\u0026rsquo;t capture 26Q1 yet, but you can see the trend.\nGas gas gas # For better or worse, oil and gas are going to become much more expensive due to the\u0026hellip; war, er special military operation, that is\u0026hellip; I mean; the complete and ongoing victory in Iran and the surrounding nations. Yeah.\nI\u0026rsquo;m not going to get political in this article, but suffice to say that people smarter than me don\u0026rsquo;t see an easy road out of this for Europe and the world more generally1. Which means the price of oil and gas (and helium, and fertilizer, and \u0026hellip;) are going to be substantially higher for the foreseeable future.\nThis is impacting Asia right now, which had already started to prepare for the oil and gas crisis: mandating work from home for government employees (in Thailand), and moving to virtual meetings unless in-person is explicitly required or even testing out a \u0026ldquo;4 day work week\u0026rdquo; in the Philippines \u0026ndash; see the reg\u0026rsquo;s excellent article on the topic.\nUpdate The first EU country to introduce fuel rationing happened this week, too, in Slovenia. Moreover, Asian leaders are now considering a return to the pandemic playbook, enforcing work from home measures to conserve fuel2.\nWe\u0026rsquo;re not even 30 days into what will probably be a crisis for the rest of this year at least.\nEurope will fair slightly better, but we\u0026rsquo;re going to be in the same boat when the strategic reserves run dry. At least in Ireland, the prices are already going up quite badly3 \u0026ndash; and we haven\u0026rsquo;t even entered the crisis proper yet!\nErgo # A recovering jobs market and spiking energy costs combined are going to incentivize both companies and employees to find ways to avoid the increased energy costs\u0026hellip; which has a pretty simple and recently globally tested solution in the IT market: work from home.\n(as a side benefit you also help our ecology by burning less fossil fuels!)\nSo do your part to help our economy, keep the world habitable, and save some €€€:\nStart working (or hiring) remotely today! https://acoup.blog/2026/03/25/miscellanea-the-war-in-iran?ref=tb.luxolus.com\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.reuters.com/business/energy/asia-looks-covid-era-playbook-tackle-fuel-crisis-2026-03-25/?ref=tb.luxolus.com\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://www.irishtimes.com/politics/2026/03/24/explainer-what-is-the-government-doing-about-energy-prices-and-how-much-will-we-benefit/?ref=tb.luxolus.com\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"March 16, 2026","externalUrl":null,"permalink":"/p/return-to-remote/","section":"Posts","summary":"","title":"Return to Remote","type":"p"},{"content":"I have been a Ceph administrator for 5 years, and the Archlinux maintainer of Ceph for 3. With the release of Ceph 20 (tentacle) I made my first contributions to the upstream project, and I want to reflect on my journey so far.\nPrimer: what is Ceph? # Ceph is a storage solution that \u0026ldquo;provides a unified storage service with object, block, and file interfaces from a single cluster built from commodity hardware components.\u0026rdquo;\nIt provides distributed block (like your hard drive), file (like NFS), and object (like AWS S3) storage interfaces for Linux, OSX and Windows \u0026ndash; plus more exotic datacenter-centric platforms.\nIt is also the underlying technology of Red Hat Storage, which is sold by many providers for 7 and 8 figure deals, and stores \u0026gt;2 exabytes1 of data around the world.\nPrimer: why Ceph? # Bluntly put, because distributed storage is a super power.\nCeph has been the storage medium for all of my VMs, containers and PVCs for a little over 5 years now, operating on hard drives that are on average 6 years old, across multiple complete node refreshes and has never lost or corrupted a single byte of data in the entire period.\nEvery cloud provider uses network storage as their primary offering because it frees your state from the underlying physicalities.\nNo more hard drive failures2 No more IO limitations because of bad PCI buses3 Reliable, consistent storage of 1s and 0s (no more evil firmware lying to your OS)4 But those are just the table stakes. Here\u0026rsquo;s a couple of the crazier things I\u0026rsquo;ve done with Ceph over the years:\nMount a single block device as single-writer, multi-reader across ~20 nodes and 3 datacenters I\u0026rsquo;m not even sure a cloud provider has a product that is equivalent to this functionality Export a S3 bucket as an NFS filesystem without paying bills that would bankrupt a medium size org looking at you AWS FileGateway v13, the early years # If I recall correctly, it was mid 2019 that I first bootstrapped what would become my production cluster.\nIt was Ceph 13 back then, and I used the (at the time) only supported deployment tool for Archlinux: ceph-deploy. This, as it turns out, was a mistake in hindsight. But we\u0026rsquo;ll get there, it worked for the initial setup.\nAfter some testing and a nasty MTU bug which I only figured out after trawling the mailing lists and coming across a (now deleted?) post by grin, finally achieved a 3 healthy node cluster.\nretrieved from my documentation:\nThis is possibly some network issue, but I cannot see the indicator about what to see. mon0 usually stands in quorum alone, and other mons cannot join. They get the monmap, they intend to join, but it just never happens, mons get from synchronising to probing, forever. Raising log level doesn\u0026rsquo;t reveal anything to me.\nSpoiler alert: it was an MTU problem.\nA crappy switch (cisco nexus, by courtesy of Satan) actually wasn\u0026rsquo;t able to handle its designated MTU size so the cluster members pushed larger-than-switch-reality-of-mtu packets, some of which obviously have failed to deliver. After lowering the MTU on the nodes to the level of the crappy switch everything came alive.\nI just want to sidenote that I was absolutely unable to see that there were messages lost, and from where to where and what. I have seen no indicator on the initiator side that a message has been sent but never acknowledged(?) or acted upon; and I have no obvious sign that the destination have expected a message which was missing.\nThis maybe has been helped in the later versions, but if it wasn\u0026rsquo;t, maybe it ought to be.\nI initially added a couple OSDs and just let it sit for a while. Then two months later, I started migrating my existing Incus (then LXD) workloads to use the Ceph storage driver.\nThe v14 nautilus upgrade # At some point Thore (previous maintainer for Ceph on Archlinux) pushed the v14 packages to packages/community, and I promptly fragged one of the mon deployments by upgrading prematurely. I tried to fix it, but ceph-deploy ran into many and varied Python related troubles (narrator: this is foreshadowing) that left me worse off than before.\n\u0026hellip;However, Ceph continued to function normally throughout this entire kerfuffle. Even down a monitor and partially upgraded!\nI eventually ripped out all of the ceph-deploy machinery and replaced it with a couple pretty straight forward bash scripts that just directly configured the systemd services + config files.\nThis was one of the best decisions I made. What had previously been ~200 lines of Python became ~10 lines of bash that I could directly modify and understand.\nThis experience shaped the way I operate Ceph all the way to today.\nThe long sleep for v17 # The v15 upgrade came and went peacefully for my cluster, and in early 2021 I started preparing for v16 which was slated to release in Q1. It released, and I waited for the Arch package to drop, which had normally happened within a couple weeks. It did not come. Months went by and eventually in early 2022 Ceph started breaking again because of incompatibilities between Python 3.10 and the version Ceph was compiled against (v3.7 IIRC?).\nThen Archlinux pushed Python v3.10 which completely broke the Python parts of Ceph, making upgrades a bit awkward.\nI eventually opened a bug report with Archlinux, and in the background started to build Ceph myself so I could keep doing system upgrades.\nThe months dragged on, and Ceph v17 was released but still Archlinux remained on the now EOL v15.\nEventually, myself and a few others that use Ceph on Archlinux managed to contact the then maintainer of Ceph, and have him hand over the package to the AUR, where I adopted it.5\nMaintaining a \u0026gt;1 million LOC, mission critical C++ project # After about 3 eventful months I had managed to build, test and upgrade to Ceph versions 16 and 17, getting me back to a supported version and giving me strong opinions about C++, Python, CMake, Make and build systems generally \u0026ndash; but that is a story for another time.\nWhile this was not my first contribution to the open source community, it was definitely the largest and taught me a lot about how to deal with consumers of the software I write maintain, which has stuck with me to this day.\nv18, or: is the Python subinterpreter in the room with us now? # During the update to v18, my mgr services starting dying with a fateful error:\nImportError: PyO3 modules may only be initialized once per interpreter process\nNow, this is an interesting error for a couple reasons:\nThe mgr service is written in C++, so how are we getting Python ImportErrors? What is an \u0026ldquo;interpreter process\u0026rdquo;? What is a PyO3 module? Without getting to much into the weeds:\nCPython allows C API users to create \u0026gt;1 Python interpreters per single OS process What happens if you \u0026ldquo;share\u0026rdquo; Python objects between interpreters is \u0026ldquo;unspecified\u0026rdquo; and/or \u0026ldquo;undefined\u0026rdquo; Virtually no-one embeds \u0026gt;1 interpreters in a single process so this undefined-ness largely went unnoticed, minus a segfault here or there PyO3 is a library for building rust extension modules for Python and because of the Rust community\u0026rsquo;s emphasis on writing safe software added a check to ensure that PyO3 modules were never \u0026ldquo;initialized\u0026rdquo; more than once Hence the error message above This is not a small problem. It largely affects the entire C API extension ecosystem around Python, the Python 3.12 changes that cause it are now starting to make their way into \u0026ldquo;stable\u0026rdquo; distros like Rocky Linux and OpenSUSE which I\u0026rsquo;m very sure is going to be painful for everyone in involved \u0026ndash; speaking from personal experience.\nThis issue is the reason I\u0026rsquo;m working with PyO3 to implement the changes needed to support subinterpreters (and free threaded Python) which is going to become a pretty big issue for the ecosystem as more and more people start using Python \u0026gt;=3.12.\nEspecially given how much of the Python ecosystem now uses Rust as the extension language:\nGraph presented by David Hewitt at Rust Nation UK, retrieved from https://old.reddit.com/r/rust/comments/1p2tfi1/media_new_releases_on_pypi_rust_vs_cc at 2025-11-24T18:58:20Z v20 and my first upstream contributions # During my v19 upgrade, I came across a truly awful C++ / Boost metaprogramming issue during the build, and unlikely many of the previous issues, this one wasn\u0026rsquo;t google-able.\nEven now if you search for BOOST_PP_ITERATION_02 my GitHub issue comes up first.\nSo, naturally my first contribution to Ceph includes the ranty-sweary6 comments make in the heat of the moment 😅.\nI wouldn\u0026rsquo;t have it any other way.\nv21 and beyond # I expect to contribute more to Ceph in 2026, likely indirectly at first via enabling to PyO3 to support subinterpreters and then helping the important Python module consumers to upgrade to the relevant PyO3 version, and also perhaps directly via upstreaming some of the patches I maintain for Ceph on Archlinux.\nI may also start packaging Ceph for Debian, as I need it in Trixie and Ceph v19 is notably absent.\nAll in all, I\u0026rsquo;ve really enjoyed using and participating in the Ceph ecosystem. Here\u0026rsquo;s to another 5 years! 🥂\nhttps://telemetry-public.ceph.com\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nHardware failures still happen, but their impact on consumers mitigated entirely in a well managed system\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://ceph.io/en/news/blog/2024/ceph-a-journey-to-1tibps/\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nIt still lies to Ceph, but Ceph transparently handles this for you because of the \u0026ldquo;distributed\u0026rdquo; in distributed storage\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://bugs.archlinux.org/task/73335.html#comment210109\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://github.com/ceph/ceph/blob/v20.2.0/src/mgr/PyModule.cc#L42-L51\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"November 24, 2025","externalUrl":null,"permalink":"/p/five-years-in-ceph/","section":"Posts","summary":"","title":"5 Years Running Ceph","type":"p"},{"content":"Cloudflare is down, I\u0026rsquo;m not.\nThis blog runs off1 of a S3 website bucket, behind a self hosted edge router that handles the toxic wasteland of botnets, AI scrapers and vuln scanners the internet has become.\nYou don\u0026rsquo;t need to use Cloudflare for your website, and if you weren\u0026rsquo;t, your productive day, today, would be: 5h longer (11:48 - 17:44, UTC+0), times the number of staff for the managers in the back!.\nWhat is that worth to you?\nBlogging in the Wesland\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"November 18, 2025","externalUrl":null,"permalink":"/p/still-alive/","section":"Posts","summary":"","title":"Still Alive","type":"p"},{"content":"Starting from PyO3 v0.28, all Rust #[pymodule]s will automatically start using the modern multi-phase initialization machinery added in cpython 3.4, making Rust Python extensions similar to native py modules, with (overridable!) __new__ and __init__ stages.\nSee the pull-request here: github.com/PyO3/pyo3#5525.\nWhat does this mean to\u0026hellip; # Authors # Mostly this is a transparent change. Under the hood, your module is no longer a singleton, global python object, meaning that the Python interpreter now can manage your module(s) lifecycle \u0026ndash; for example, being unloaded from sys.modules will now actually deallocate your module, and if/when it is reimported, it will be reinitialized, the same as a normal .py module.\nThis makes your module a bit less surprising, and makes it easier to test the lifecycle functions the way python normally does.\nConsumers # Nothing really changes for module consumers \u0026ndash; CPython, Pypy and GraalPy should all just work as usual.\nFuture work # This is hopefully just the beginning, in the coming months I\u0026rsquo;m looking to expose per-module state to module authors, which should make state management on the Rust side much easier to deal with (less PyOnceLocks, statics and thread_local!s!), plus various improvements to the pyo3 machinery itself, so we can get to natively supported subinterpreters for all Rust pymodules.\nIf you\u0026rsquo;re interested in updates around this, subscribe to github.com/PyO3/pyo3#3451.\nPersonal thoughts # Special thanks to @Aequitosh, as him and @davidhewitt are responsible for the majority of this work, but lacked the time to push it over the finish line.\nThis is a pretty interesting area of work, as it requires me to roughly understand:\nRust C++ CPython APIs Python (the language) In order to implement features. This is a pretty wide set of technologies, that must be understood well enough to be confident about working between them and across FFI boundaries, which is always complicated to do right.\nI personally need this, because my self hosted Ceph cluster has suffered from the lack of subinterpreter support for nearly 3 years now1, something even Debian, Ubuntu, and even Proxmox are now suffering from.\nSo hopefully, my continued work here can finally resolve the root cause of the trouble. Time will tell.\ngithub.com/bazaah/aur-ceph#20\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"October 27, 2025","externalUrl":null,"permalink":"/p/pyo3-multi-phase-init/","section":"Posts","summary":"","title":"PyO3 Multiphase Initialization","type":"p"},{"content":" rant This is the output from a npm ci run from a project I maintain (not develop!):\nnpm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful. npm WARN deprecated inflight@1.0.6: This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful. npm WARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated npm WARN deprecated inside@1.0.0: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. npm WARN deprecated @babel/plugin-proposal-class-properties@7.18.6: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-class-properties instead. npm WARN deprecated request-promise-native@1.0.9: request-promise-native has been deprecated because it extends the now deprecated request package, see https://github.com/request/request/issues/3142 npm WARN deprecated @stylelint/postcss-markdown@0.36.2: Use the original unforked package instead: postcss-markdown npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated npm WARN deprecated har-validator@5.1.5: this library is no longer supported npm WARN deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported npm WARN deprecated abab@2.0.6: Use your platform\u0026#39;s native atob() and btoa() methods instead npm WARN deprecated rimraf@3.0.2: Rimraf versions prior to v4 are no longer supported npm WARN deprecated domexception@1.0.1: Use your platform\u0026#39;s native DOMException instead npm WARN deprecated debug@4.2.0: Debug versions \u0026gt;=3.2.0 \u0026lt;3.2.7 || \u0026gt;=4 \u0026lt;4.3.1 have a low-severity ReDos regression when used in a Node.js environment. It is recommended you upgrade to 3.2.7 or 4.3.1. (https://github.com/visionmedia/debug/issues/797) npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated glob@7.2.3: Glob versions prior to v9 are no longer supported npm WARN deprecated source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated npm WARN deprecated @babel/plugin-proposal-object-rest-spread@7.20.7: This proposal has been merged to the ECMAScript standard and thus this plugin is no longer maintained. Please use @babel/plugin-transform-object-rest-spread instead. npm WARN deprecated glob@7.1.6: Glob versions prior to v9 are no longer supported npm WARN deprecated w3c-hr-time@1.0.2: Use your platform\u0026#39;s native performance.now() and performance.timeOrigin. npm WARN deprecated domexception@4.0.0: Use your platform\u0026#39;s native DOMException instead npm WARN deprecated sourcemap-codec@1.4.8: Please use @jridgewell/sourcemap-codec instead npm WARN deprecated cucumber-tag-expressions@1.1.1: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details. npm WARN deprecated @stylelint/postcss-css-in-js@0.37.3: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. npm WARN deprecated critters@0.0.24: Ownership of Critters has moved to the Nuxt team, who will be maintaining the project going forward. If you\u0026#39;d like to keep using Critters, please switch to the actively-maintained fork at https://github.com/danielroe/beasties npm WARN deprecated dommatrix@1.0.3: dommatrix is no longer maintained. Please use @thednp/dommatrix. npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142 npm WARN deprecated gherkin@5.1.0: This package is now published under @cucumber/gherkin npm WARN deprecated ng-click-outside@9.0.1: ng-click-outside is no longer maintained. See https://github.com/arkon/ng-sidebar/issues/229 npm WARN deprecated cucumber-expressions@5.0.18: This package is now published under @cucumber/cucumber-expressions npm WARN deprecated cucumber-expressions@6.6.2: This package is now published under @cucumber/cucumber-expressions npm WARN deprecated cucumber-messages@8.0.0: This package is now published under @cucumber/messages npm WARN deprecated fork-awesome@1.1.7: Package no longer supported. Contact Support at https://www.npmjs.com/support for more info. npm WARN deprecated gherkin@9.0.0: This package is now published under @cucumber/gherkin npm WARN deprecated @applitools/eyes-sdk-core@12.23.12: Please see https://applitools.com/docs/api-ref/ to find the right package for your tests npm WARN deprecated cucumber@4.2.1: Cucumber is publishing new releases under @cucumber/cucumber npm WARN deprecated core-js@2.6.12: core-js@\u0026lt;3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js. npm WARN deprecated core-js@3.6.4: core-js@\u0026lt;3.23.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Some versions have web compatibility issues. Please, upgrade your dependencies to the actual version of core-js. This is bluntly, insane.\nAs a package maintainer it is technically my responsibility to ensure that I am providing functional, secure software to the people that consume my work. But that is fundamentally impossible for me, as an outsider (to the project\u0026rsquo;s development) to do here.\nIf I had to guess, its probably also impractical for the (paid) development team sit down and fully fix. And even if it wasn\u0026rsquo;t this code base is only ~5 years old\u0026hellip; so what expectation does the dev team have that if they invest the weeks (months) needed to unscrew this, it remains unscrewed for long enough that it becomes a good use of time?\nI don\u0026rsquo;t know.\nIt does seem to me however, that the NPM ecosystem is simply not ready for the challenges of writing stable software that remains usable (\u0026amp; secure) over decade long horizons.\nI wonder how much economic damage it has done in terms of time wasted on security evaluations, constant patch bumps (and the related CI resource wasted on testing such), cognitive overhead and \u0026ldquo;stop the world\u0026rdquo; refactors required to exist in it\u0026rsquo;s sphere of influence.\nBuilding on top of the NPM dependency tower sure looks a lot like trying to keep your head above the sewage line\u0026hellip; and I can smell it\u0026rsquo;s stench quite strongly right now.\n","date":"September 10, 2025","externalUrl":null,"permalink":"/p/npm-sewage/","section":"Posts","summary":"","title":"NPM Sewage","type":"p"},{"content":" Flow Overview # You arrived at this site via:\nsequenceDiagram actor User participant DNS participant Edge participant RadosGateway participant S3 User-\u003e\u003eEdge: Connect to https://tb.luxolus.com Edge-\u003e\u003eRadosGateway: Route to internal Ceph RGW replica-set RadosGateway--\u003e\u003eDNS: Lookup tb.luxolus.com CNAME for bucket routing RadosGateway-\u003e\u003eS3: Route to (librados) S3 website bucket content S3-\u003e\u003eUser: Navigate to index.html This blog post was published via the following:\n❯ just hugo deploy [blog@f230607|~]: Requesting IAM credentials from https://secret.i.st8l.com (you may be required to authenticate) Password (will be hidden): [blog@f230607|~]: Running `hugo deploy -e production --target production` Deploying to target \u0026#34;production\u0026#34; (s3://w3-luxolus-blog?endpoint=https://storage.st8l.com) Identified 40 file(s) to upload, totaling 942 kB, and 0 file(s) to delete. Success! And in just a few seconds, my new post was up and available.\nZero cloud technologies involved. Everything along this stack is opensource. You can do this too. Selfhosting Weslandia # Weslandia is a children\u0026rsquo;s book by Paul Fleischman, and was one of my favorite books when I was growing up. The brief, relevant synopsis:\nWhen Wesley, a somewhat eccentric boy with no friends, discovers a mysterious plant magically growing in his parents\u0026rsquo; backyard, he cultivates the plant over his summer vacation. The plant, which he names \u0026ldquo;swist\u0026rdquo;, provides him with a food source, and allows him to build shelter, tools, and even create his own entertainment and inspires Wesley to create his own writing system. Wesley\u0026rsquo;s resourcefulness and meticulous research eventually allow to him the basis of his own civilization which he names \u0026ldquo;Weslandia\u0026rdquo;, an eponymous micro-nation in his parents\u0026rsquo; backyard. His efforts are successful, and instead of being a social outcast, he gains a group of followers made up of his former grade-school tormentors.\nI have built my own digital Wesland. It has all of the managed services that are relevant:\nManaged block, filesystem \u0026amp; blob storage Managed VMs Managed workload scheduling \u0026amp; placement Managed databases Managed email MTAs Managed load balancers Managed queues Managed networks / DNS zones Managed logging + metrics Managed secrets engine Okay I lie, almost all: I don\u0026rsquo;t like Lambda/Function runtimes so I don\u0026rsquo;t use them.\nAll of the above is highly available excluding the MTA but that\u0026rsquo;s an intentional choice on my part to save money on public virtual IPs.\nI had better uptime in 2024 than every major cloud provider, and unless something horrible happens in 2025, I\u0026rsquo;m going to beat them all again, this year.\nOh yes, and the price: I run a ~500G mem, ~64 core, 150T disk cluster for slightly over $350/mo. The \u0026ldquo;equivalent\u0026rdquo; cost of just the cpu/mem in EC2s excluding storage, because AWS doesn\u0026rsquo;t really offer an equivalent of what I have is north of 5 grand a month. Add in my 26 databases and many terabytes of storage and we\u0026rsquo;re pretty easily over 20 grand a month.1\nInstead of $350. For worse uptime. For stagnant service offerings.\nUnrelated note but did you hear that AWS\u0026rsquo; gross margin as of the last earnings report was just short of 50%? I\u0026rsquo;m sure that has nothing to do with the prices above.\nOh yeah, last thing: I maintain this in my free time. After a 40-60 hour work week. This is not taking up multiple IT teams worth of budget to manage.\nThe point of my rant is this:\n\u0026lt;endRant /\u0026gt;\nRight what were we talking about? A blog?! # I invite you to join me in Weslandia. Over the next many posts I\u0026rsquo;m going to go into detail about parts of my setup, so you can build your own.\nGoing over the command line from above:\njust is a task runner I use it for approximately 99% of the usage Make gets outside of c/c++ build systems. If you want a build system use Make (actually, use Bazel but that\u0026rsquo;s whole \u0026rsquo;nother series of posts!), if you want a task runner use just.\nYou want a task runner by the way.\nWe then reach out to secrets.i.st8l.com My internal \u0026ldquo;secrets as a service\u0026rdquo;\u0026hellip; service. It\u0026rsquo;s just a highly available OpenBao cluster that has a plugin for managing Ceph Rados Gateway credentials.\nI login via OIDC, provided by a OpenBao\u0026lt;-\u0026gt;LDAP sync bridge plugin.\nWe then invoke hugo deploy with the following config targets: - name: production url: s3://w3-luxolus-blog?endpoint=https://storage.st8l.com matchers: - cacheControl: max-age=31536000, no-transform, public gzip: true pattern: ^.+\\.(js|css|svg|ttf)$ - cacheControl: max-age=31536000, no-transform, public gzip: false pattern: ^.+\\.(png|jpg)$ - contentType: application/xml gzip: true pattern: ^sitemap\\.xml$ Deploying my site to w3-luxolus-blog.storage.st8l.com Which is an S3 website bucket. With a twist.\nIf you dig this site\u0026rsquo;s DNS record you\u0026rsquo;ll see something impossible interesting:\n❯ drill tb.luxolus.com ;; ANSWER SECTION: tb.luxolus.com. 241 IN CNAME w3-luxolus-blog.w3storage.st8l.com. w3-luxolus-blog.w3storage.st8l.com. 241 IN CNAME w3storage.st8l.com. w3storage.st8l.com. 241 IN CNAME st8l.com. st8l.com. 241 IN A 159.69.232.211 Which, if you have worked with S3 website buckets before should be very surprising to you. This\u0026hellip; shouldn\u0026rsquo;t work. AWS\u0026rsquo; own documentation2 says this setup isn\u0026rsquo;t supported:\nAmazon S3 website endpoints do not support HTTPS or access points. If you want to use HTTPS, you can use Amazon CloudFront to serve a static website hosted on Amazon S3.\nSo rather than using 1 cheap service to accomplish what free, opensource software can do, you can instead use 3 services, one of them being expensive: S3, R53, and CloudFront.\nFinitione # I really like my setup. It\u0026rsquo;s easier to deal with than substack, and I have half a mind to add a git post-receive hook on master pushes that automatically deploys to prod when I push, so I don\u0026rsquo;t even need to remember to hugo deploy myself.\nIt reminds me of the wonder I felt when I first used Heroku, pushed to GitHub the first time and had a running site.\nAnd it\u0026rsquo;s all mine\u0026hellip;\nbut you could have it too\nFor those following along at home, I\u0026rsquo;m using 7x m7gd.4xlarge \u0026amp; 26x db.m2.2xlarge as my reference instances, but AWS doesn\u0026rsquo;t really provide storage in the same category I have, for any instance outside of metal ones\u0026#160;\u0026#x21a9;\u0026#xfe0e;\nhttps://docs.aws.amazon.com/AmazonS3/latest/userguide/website-hosting-custom-domain-walkthrough.html\u0026#160;\u0026#x21a9;\u0026#xfe0e;\n","date":"August 29, 2025","externalUrl":null,"permalink":"/p/wesland-blogging/","section":"Posts","summary":"","title":"Blogging in the Wesland","type":"p"},{"content":" Welcome to my thirdbrain # This space will be a selection of short snippets, articles about interesting tech, and technical deep dives into topics, so that I have a place to refer to when I\u0026rsquo;m talking about them with colleagues.\nIt will have a very heavy slant towards the operational side of things, and be focused mostly on the tech, tools and services I use to run my own private cloud.\nI may also discuss the odd $dayJob topic too.\n\u0026hellip; the first of which, will be on the hosting of this blog using nothing more than AWS S3 + S3 website hosting; but you know, without AWS.\n","date":"August 4, 2025","externalUrl":null,"permalink":"/p/welcome/","section":"Posts","summary":"","title":"Welcome","type":"p"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"},{"content":"","externalUrl":null,"permalink":"/t/","section":"Tags","summary":"","title":"Tags","type":"t"},{"content":"","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"}]