Remove upstream tests, docs, examples, dev env and release tooling
Replace upstream's README with a short one for this fork, and add docs/fork-maintenance.md describing what was removed and how to maintain the rest. Keeps helper/update-fixed-output-derivation.sh for pkgs/mempool/generate.sh. There is no test suite. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,248 @@
|
||||
# Maintaining this fork
|
||||
|
||||
Upstream (fort-nix/nix-bitcoin) is archived. This fork is now the only
|
||||
source of fixes, so every module and package left in it is something we
|
||||
port across each NixOS release. This note lists what squirtle actually
|
||||
uses, what can be removed, and what still needs maintenance after that.
|
||||
|
||||
Assessed 2026-11-15, against fork `master` at `da27f27` (upstream's tree
|
||||
minus its last two SECURITY.md commits; no fork-only commits yet) and
|
||||
nix-pleb's squirtle config on NixOS 26.05 and 26.11pre.
|
||||
|
||||
**Status (2026-11-15):** steps 1–5 of *Suggested order* are done on
|
||||
branch `slim`. Step 3's nix-pleb half (squirtle's `services.nix`,
|
||||
`configuration.nix`, `nixosConfigurations.nix`) is uncommitted in
|
||||
nix-pleb. Step 6 is still to do. Upstream's tests were dropped, not
|
||||
replaced (see *Testing*).
|
||||
|
||||
## Who consumes the fork
|
||||
|
||||
Only one host does: squirtle, in nix-pleb
|
||||
(`systems/brenise.dev/nixosConfigurations.nix`). It imports
|
||||
`nixosModules.default` and `modules/presets/secure-node.nix`. No other
|
||||
flake in `~/ops` uses the fork.
|
||||
|
||||
These are squirtle's evaluated settings, not just what the config file
|
||||
declares:
|
||||
|
||||
| Feature | State on squirtle |
|
||||
|---|---|
|
||||
| `services.bitcoind` | on, `bitcoind-32.1` from system nixpkgs, `i2p = false` |
|
||||
| `services.lnd` | on, package overridden to nix-pleb's `unstablePkgs.lnd` |
|
||||
| `services.electrs` | on, from system nixpkgs |
|
||||
| `services.mempool` | on, fork-packaged `mempool-*-3.2.1`, `electrumServer = "electrs"` |
|
||||
| `nix-bitcoin.operator` | on (`pleb`) |
|
||||
| `nix-bitcoin.nodeinfo` | on (from secure-node preset) |
|
||||
| `nix-bitcoin.generateSecrets` | on, `secretsSetupMethod = "setup-secrets"` |
|
||||
| onion services | bitcoind (public), electrs, sshd |
|
||||
| `nix-bitcoin.security.dbusHideProcessInformation` | on (preset) |
|
||||
| `services.tor` | on (preset) |
|
||||
| `nix-bitcoin.netns-isolation` | **off** |
|
||||
| `services.i2pd` | **off** |
|
||||
| `services.backups` | **off** |
|
||||
| `nix-bitcoin.configVersion` | unset |
|
||||
| clightning, rtl, fulcrum, btcpayserver, nbxplorer, joinmarket, liquidd, loop, pool, charge-lnd, clightning-rest, hardware-wallets, lndconnect | all **off** |
|
||||
|
||||
Squirtle runs four services: **bitcoind, lnd, electrs and mempool**. The
|
||||
rest of the fork is unused.
|
||||
|
||||
## What can go
|
||||
|
||||
### Unused services: remove
|
||||
|
||||
These modules and their packages have no consumer. Each one breaks
|
||||
independently whenever nixpkgs renames an option or drops a dependency.
|
||||
|
||||
| Module | Package(s) in `pkgs/` | Lines (module + pkg) |
|
||||
|---|---|---|
|
||||
| `clightning.nix`, `clightning-plugins/`, `clightning-rest.nix`, `clightning-replication.nix` | `clightning-plugins/`, `clightning-rest/`, `clnrest/`, `trustedcoin/`, most of `python-packages/` (pyln-*, pylightning) | ~1,500 |
|
||||
| `joinmarket.nix`, `joinmarket-ob-watcher.nix` | `joinmarket/`, `python-packages/` (joinmarket, bencoderpyx, chromalog, txzmq, python-bitcointx…) | ~1,000 |
|
||||
| `btcpayserver.nix` | `nbxplorer/` | ~410 |
|
||||
| `rtl.nix` | `rtl/` | ~345 |
|
||||
| `liquid.nix` | `liquid-swap/` | ~320 |
|
||||
| `lndconnect.nix` | — | ~280 |
|
||||
| `lightning-loop.nix`, `lightning-pool.nix`, `charge-lnd.nix` | — | ~400 |
|
||||
| `fulcrum.nix` | — | ~140 |
|
||||
| `hardware-wallets.nix` | `hwi` (python-packages) | ~85 + |
|
||||
| — | `bitcoind_29/` (old bitcoind pin) | ~215 |
|
||||
|
||||
`python-packages/` (~520 lines) disappears completely once clightning,
|
||||
joinmarket and hwi are gone. Its own comments note that it pins older
|
||||
cython, ecdsa and similar, and that kind of pin breaks on each nixpkgs
|
||||
bump.
|
||||
|
||||
### Unused support features: remove
|
||||
|
||||
- **`netns-isolation.nix`** (366 lines). It's off on squirtle, but it is
|
||||
the module that reaches into every other service (btcpayserver,
|
||||
clightning, joinmarket, rtl, i2pd, …). The 26.11 i2pd breakage came from
|
||||
here. Removing it also drops `pkgs/netns-exec/` and the
|
||||
netns-only helpers in `pkgs/lib.nix`.
|
||||
- **i2p support in `bitcoind.nix`.** `i2p` is false on squirtle. Dropping
|
||||
the `i2p` option, `i2pSAM` and the `services.i2pd` block makes the
|
||||
26.11 compatibility patch on branch `nixos-26.11` unnecessary. **Prefer
|
||||
this over that patch.**
|
||||
- **`backups.nix`.** It's off on squirtle; the secure-node preset only
|
||||
sets its `frequency`. Squirtle isn't in raichu's restic jobs either, so
|
||||
if you want its lnd channel backup or config backed up, add it there
|
||||
rather than reviving this module.
|
||||
- **`versioning.nix`** (292 lines) and **`obsolete-options.nix`** (145
|
||||
lines). Both exist for migrating *other people's* configs across
|
||||
upstream releases. Squirtle has no `configVersion` set and has never
|
||||
needed either. Most of their entries reference removed services
|
||||
(joinmarket, liquidd, clightning-rest…).
|
||||
- **`modules/deployment/`** (krops, nixops), **`pkgs/krops/`,
|
||||
`pkgs/nixops/`, `pkgs/generate-secrets-deprecated.nix`.** Squirtle
|
||||
deploys with `nixos-rebuild` from nix-pleb.
|
||||
|
||||
### Presets: fold into squirtle, then remove
|
||||
|
||||
`presets/secure-node.nix` and `presets/enable-tor.nix` mostly configure
|
||||
services squirtle doesn't run. What squirtle actually gets from them:
|
||||
|
||||
- `services.tor.enable`, `tor.client.enable`
|
||||
- `tor.proxy`/`tor.enforce` defaults for bitcoind, lnd, electrs, mempool
|
||||
(squirtle then overrides `enforce = false` on bitcoind, lnd, electrs)
|
||||
- onion services for bitcoind, electrs and sshd, plus `onionAddresses.access.pleb`
|
||||
- `nodeinfo.enable`, `operator.enable`, `dbusHideProcessInformation`
|
||||
- `bitcoind.listen = true`, `bitcoind.dbCache = 1000`
|
||||
- doas instead of sudo (`squirtle/configuration.nix` already sets this),
|
||||
`networking.firewall.enable = true`
|
||||
|
||||
Those lines belong in `squirtle/services.nix`. That also fixes the
|
||||
existing annoyance noted there: `dbCache` "cannot be changed here",
|
||||
because the preset sets it without `mkDefault`. `hardened.nix`,
|
||||
`hardened-extended.nix`, `wireguard.nix` and `bitcoind-remote.nix` have no
|
||||
users.
|
||||
|
||||
### Repo scaffolding: remove or shrink
|
||||
|
||||
- `test/` (2,700 lines). The VM scenarios exercise every service. Removed
|
||||
entirely; see *Testing*.
|
||||
- `examples/` (1,250), `dev/` (900), most of `docs/` (1,700). These are
|
||||
upstream's user-facing docs and dev environment. Replace them with a
|
||||
short README covering what's left.
|
||||
- `helper/`: `fetch-release`, `push-release.sh` and `key-jonasnick.bin`
|
||||
are upstream's release signing. `update-flake.{nix,sh}` regenerates
|
||||
`pkgs/pinned.nix` and becomes unnecessary once pins are gone (below).
|
||||
Keep `update-fixed-output-derivation.sh` only if it's still handy for
|
||||
mempool hashes.
|
||||
|
||||
### Extra nixpkgs pins: remove
|
||||
|
||||
The fork's flake has three nixpkgs inputs plus `extra-container` and
|
||||
`flake-utils`. nix-pleb overrides only `nixpkgs` (`inputs.nixpkgs.follows`).
|
||||
`nixpkgs-unstable` and `nixpkgs-25_05` stay pinned at whatever the fork's
|
||||
`flake.lock` says, and they feed `pinned.nix`:
|
||||
|
||||
- `pkgsUnstable` supplies `lnd` (squirtle overrides it), plus
|
||||
`bitcoind-knots`, `clboss`, `fulcrum` and `lightning-loop` (all unused).
|
||||
- `pkgs-25_05` is only plumbed through to python-packages.
|
||||
|
||||
Once the unused packages are gone, take everything from the consumer's
|
||||
`pkgs`. Make `lnd` default to `pkgs.lnd`; squirtle sets the package
|
||||
explicitly anyway. Then delete `pinned.nix`, `nixpkgs-pinned.nix`, the
|
||||
two extra inputs, `extra-container` (used only for container tests and
|
||||
examples), and the `useVersionLockedPkgs` option. The fork then has one
|
||||
input, `nixpkgs`, which nix-pleb already follows.
|
||||
|
||||
## What's left, and what it costs to maintain
|
||||
|
||||
After the removals the fork is roughly these modules: `nix-bitcoin.nix`,
|
||||
`secrets/`, `operator.nix`, `security.nix`, `bitcoind.nix` (with
|
||||
`bitcoind-rpc-public-whitelist.nix`, which it imports), `lnd.nix`,
|
||||
`electrs.nix`, `mempool.nix`, `onion-services.nix`, `onion-addresses.nix`,
|
||||
`nodeinfo.nix`. Plus `pkgs/lib.nix`, `pkgs/mempool/` and
|
||||
`pkgs/build-support/fetch-node-modules.nix`. That's about 2,300 lines,
|
||||
down from about 16,500.
|
||||
|
||||
Ordered from most to least work:
|
||||
|
||||
1. **mempool package (`pkgs/mempool/`)**: the only real package we
|
||||
maintain. It has five fixed-output hashes (source, frontend and backend
|
||||
node modules, the frontend-assets fetch, and the rust-gbt cargo
|
||||
vendor), a local patch
|
||||
(`0001-allow-disabling-mining-pool-fetching.patch`), and a pinned
|
||||
`nodejs_22`. It's at 3.2.1 against upstream's 3.3.2. Expect every
|
||||
version bump to need the hashes redone. Node 22 also has an end-of-life
|
||||
date, after which nixpkgs will drop `nodejs_22` and force a Node bump
|
||||
here. nixpkgs doesn't package mempool, so there's no upstream to lean
|
||||
on. If you ever stop wanting the mempool UI, removing it is the
|
||||
biggest maintenance cut available.
|
||||
2. **`mempool.nix` module**: configures mariadb, nginx (includes
|
||||
`mempool-nginx-conf` from the package) and the backend's JSON config.
|
||||
It follows NixOS changes to the mysql and nginx modules and mempool's
|
||||
config schema. Drop its `fulcrum` option along with fulcrum.
|
||||
3. **`bitcoind.nix`**: tracks Bitcoin Core config changes across major
|
||||
versions (nixpkgs moves bitcoind on its own schedule; 32.1 today). It
|
||||
disables NixOS's own `services/networking/bitcoind.nix`, so watch for
|
||||
that module being renamed or split. That `disabledModules` path is
|
||||
in `modules.nix`.
|
||||
4. **`lnd.nix`**: tracks lnd config changes. It uses `lndinit` from
|
||||
`pkgs/lndinit/`, which is **0.1.3** while nixpkgs and upstream are at
|
||||
**0.1.38**. Switch to `pkgs.lndinit` and delete `pkgs/lndinit/`; check
|
||||
that `gen-seed` and `init-wallet` flags still match. Also drop its
|
||||
clightning port-conflict assertion.
|
||||
5. **Tor plumbing** (`onion-services.nix`, `onion-addresses.nix`,
|
||||
`nbLib.tor`): built on `services.tor.relay.onionServices` and
|
||||
`tor.client.socksListenAddress`, which NixOS has kept stable for years.
|
||||
Low churn.
|
||||
6. **Secrets (`secrets/`)**: generates RPC passwords, the lnd cert and
|
||||
similar on the host via `setup-secrets`. It's mostly shell and openssl,
|
||||
so low churn. Note that nix-pleb uses sops for everything else; moving
|
||||
these to sops would remove this too, at the cost of a migration on a
|
||||
live node.
|
||||
7. **`electrs.nix`, `nodeinfo.nix`, `operator.nix`, `security.nix`,
|
||||
`nix-bitcoin.nix`**: small. The package comes from nixpkgs. Remove the
|
||||
dead service entries from `nodeinfo.nix` (clightning, liquidd,
|
||||
joinmarket-ob-watcher…) along with those services.
|
||||
8. **systemd hardening in `pkgs/lib.nix`** (`defaultHardening`,
|
||||
`allowedIPAddresses`, `allowNetlink`): occasionally a systemd or NixOS
|
||||
change makes a service fail at runtime, not at evaluation. That only
|
||||
shows up after a deploy or in a VM test.
|
||||
|
||||
### Per-release checklist
|
||||
|
||||
On each NixOS release, or each nixpkgs bump in nix-pleb:
|
||||
|
||||
1. `nix build .#nixosConfigurations.squirtle.config.system.build.toplevel`
|
||||
in nix-pleb. Evaluation errors land in this fork's modules.
|
||||
2. If mempool or Node changed, build `services.mempool.package` and
|
||||
`services.mempool.frontend.package` first, and update the hashes.
|
||||
3. Switch squirtle and check that bitcoind, lnd, electrs, mempool and
|
||||
nginx are running. Hardening regressions don't show at build time.
|
||||
If one fails, `nixos-rebuild switch --rollback`.
|
||||
|
||||
## Testing
|
||||
|
||||
There is no test suite. The fork isn't distributed and only squirtle
|
||||
consumes it, so problems are handled as they show up on squirtle, with
|
||||
rollback as the fallback. A regtest VM test mirroring squirtle was
|
||||
written and then dropped as not worth maintaining; if runtime breakage
|
||||
becomes frequent, upstream's `test/tests.py` (in git history before
|
||||
branch `slim`) has per-service checks for all four services to start
|
||||
from.
|
||||
|
||||
## Suggested order
|
||||
|
||||
Each step should leave squirtle's evaluated system unchanged on 26.05.
|
||||
Compare `toplevel.drvPath` before and after, as was done for the i2pd
|
||||
patch.
|
||||
|
||||
1. Remove unused services, their packages, python-packages and
|
||||
`bitcoind_29`. Edit `modules.nix`, `nodeinfo.nix`, `onion-services.nix`,
|
||||
`enable-tor.nix`, `backups.nix`, `versioning.nix`, `obsolete-options.nix`
|
||||
and `netns-isolation.nix` only enough to drop references. Expect the
|
||||
drvPath to change slightly here, because nodeinfo's script and the
|
||||
module set shrink.
|
||||
2. Remove netns-isolation, i2p support, backups, versioning,
|
||||
obsolete-options and deployment, and abandon branch `nixos-26.11`'s
|
||||
compatibility patch.
|
||||
3. Move what squirtle uses from the presets into nix-pleb's
|
||||
`squirtle/services.nix`, stop importing `secure-node.nix`, and delete
|
||||
`presets/`.
|
||||
4. Drop `pinned.nix` and the extra flake inputs; switch to nixpkgs `lndinit`.
|
||||
5. Remove `test/`, `examples/`, `dev/`, `docs/` and `helper/` (except
|
||||
`update-fixed-output-derivation.sh`, used by `pkgs/mempool/generate.sh`)
|
||||
and replace upstream's README with a short one (keep this file).
|
||||
6. Rebuild squirtle on 26.11 in nix-pleb.
|
||||
Reference in New Issue
Block a user