Commit Graph
100 Commits
Author SHA1 Message Date
Erik Arvstedt c041079ae1 configuration.nix: reorder sections
Move backups and netns-isolation to the end.
2021-08-15 11:29:33 +02:00
Erik Arvstedt 3734ab38a6 configuration.nix: improve wording and formatting 2021-08-15 11:29:33 +02:00
Erik Arvstedt a2454975a5 doas: fix recursive calls to doas
Doas was broken for recursive calls like `doas -u operator lncli`
where `lncli` internally calls doas.
2021-08-14 10:46:42 +02:00
Erik Arvstedt 7c876664b1 netns test: update matching of 'capsh' output
The output now contains multiple lines.
2021-08-14 10:46:42 +02:00
Erik Arvstedt 308a11f22b tests: avoid postgresql timeout failures on CI nodes 2021-08-14 10:46:42 +02:00
Erik Arvstedt 01804e6dfb tests: improve test script formatting
Remove annyoing spacing constraints enforced by the previous 'black' linter.
2021-08-14 10:46:42 +02:00
Erik Arvstedt 1be924529d tests: adapt to new linter
The Python test driver now uses 'pyflakes'.
Remove hacks that were needed for the 'black' linter.
2021-08-14 10:46:42 +02:00
Erik Arvstedt c1c663d0a9 tests: fix formatting 2021-08-14 10:46:42 +02:00
Erik Arvstedt c4c2b03e19 extra-container: 0.6 -> 0.7
Version 0.7 adds support for NixOS 21.05.
2021-08-14 10:46:41 +02:00
Erik Arvstedt 161baa7e68 joinmarket-ob-watcher: allow required 'mbind' system call 2021-08-14 10:46:41 +02:00
Erik Arvstedt ca64a4a64f clightning-plugins.prometheus: use current nixpkgs version of prometheus-client 2021-08-14 10:46:41 +02:00
Erik Arvstedt 3aab1fc267 spark-wallet: update to new node-env 2021-08-14 10:46:41 +02:00
Erik Arvstedt a0e5894f1f backups: remove illegal option definition 2021-08-14 10:46:41 +02:00
Erik Arvstedt 35fe939cf8 security: update /proc restriction mechanism
NixOS option `security.hideProcessInformation` for globally restricting
access to /proc has been removed.
Use per-service restrictions via 'ProtectProc' instead.

Rename
`nix-bitcoin.security.hideProcessInformation` to
`nix-bitcoin.security.dbusHideProcessInformation`
because this option now only implements the dbus restriction.
2021-08-14 10:46:41 +02:00
Erik Arvstedt 178a0dcf8f services: use new 'tor' options 2021-08-14 10:46:41 +02:00
Erik Arvstedt e44f78ebb8 services: set isSystemUser for service users
'isSystemUser' has to be explicitly set in NixOS 21.05.
Previously, it was the implicit default.
2021-08-14 10:46:40 +02:00
Erik Arvstedt 0ef66c920b treewide: use services.getty option
services.mingetty is equivalent but deprecated.
2021-08-14 10:46:40 +02:00
Erik Arvstedt a25ceecca5 update to NixOS 21.05 2021-08-12 11:18:26 +02:00
Erik Arvstedt b758150c9e pinned: expose nixpkgsStable, nixpkgsUnstable
This allows accessing the pinned nixpkgs.
E.g., this is useful for comparing package versions between stable
and unstable.
2021-08-12 11:18:08 +02:00
Erik Arvstedt 881a3aa004 joinmarket: use preStart, postStart 2021-08-10 10:12:42 +00:00
Erik Arvstedt ed364f5932 tests: speed up stopping of clightning with clboss
This speeds up the 'backups' test by 90 s.
2021-07-29 20:27:34 +02:00
Erik Arvstedt 637a58d826 lnd: improve waiting for active RPC server
- Simplify
- Add comment
- Avoid the unneeded default call to sleep
2021-07-29 20:27:34 +02:00
Erik Arvstedt adeccce06e lnd: simplify use of curl 2021-07-29 20:27:33 +02:00
Erik Arvstedt 7ae0a38701 electrs: replace local version with nixpkgs 2021-05-04 11:31:35 +00:00
Erik Arvstedt c5f67629e6 joinmarket: add copy of cryptography 3.3.2 2021-04-07 19:05:40 +02:00
Erik Arvstedt 3cce822c64 run-tests: add 'vm' command 2021-03-26 23:23:22 +01:00
Erik Arvstedt 4cddf284e9 treewide: remove use of deprecated stdenv.lib 2021-03-22 14:39:32 +01:00
Erik Arvstedt 6ba5880b2c test.py: improve composability of test 'banlist-and-restart'
The test now works if not all services previously used in the
`systemctl restart` command are available.
2021-03-22 14:39:31 +01:00
Erik Arvstedt 44439e2a81 tests: optimize building multiple tests at once
The result of `import tests.nix {}` is now an attrset of tests.
This makes it easier and more efficient to evaluate or build multiple
tests in one call to `nix build`.

Simplify tests.nix by removing the large module args scope in favor of
self-contained scenario module definitions.

Add CPU core and memory size defaults to the test configuration to
enable building tests without `run-tests.sh`.

Add the following top-level args to tests.nix:
- `extraScenarios` to provide a nix-level way to define extra scenarios.
- `pkgs` to allow building tests with custom pkgs or systems.
2021-03-22 14:35:29 +01:00
Erik Arvstedt 9ca52af523 tests: improve make-test-vm.nix
1. fixedTest: Remove some unneeded layers of function calls.

2. Don't add a modified version of `black` to the global pkgs set.
   Tests should not affect the pkgs of the tested system modules.
   Fix the driver build script instead by adding an extra arg to the
   call to `black`.
2021-03-22 14:35:29 +01:00
Erik Arvstedt 08fe9ba84a services: add finer-grained address family restrictions
Due to a possible NixOS bug, this commit has no effect on NixOS 20.09
where `RestrictAddressFamilies` is a no-op.
It's only relevant for NixOS unstable with cgroups v2.

bitcoind+zmq: instead of allowing all address families, only add the required
AF_NETLINK family.

lnd: lnd only runs a zmq client, not a server, therefore it requires
no additional address families.

lightning-pool, clightning-plugin-zmq: add AF_NETLINK.
2021-03-22 14:35:29 +01:00
Erik Arvstedt 020433cec6 services: add helper fn setAllowedIPAddresses
Also use 'allowLocalIPAddresses' instead of 'allowTor' in bitcoind-import-banlist
which doesn't use Tor.
2021-03-22 13:20:45 +01:00
Erik Arvstedt cdf27d9d0c bitcoind: improve service timeouts
- Improve readability by using minutes
- set `TimeoutStopSec` like in bitcoin/contrib/init/bitcoind.service.
  Stopping bitcoind can exceed the default timeout during IBD.
2021-03-16 12:46:19 +01:00
Erik Arvstedt 09cd3ce5e4 lnd: show curl error messages 2021-03-16 12:46:19 +01:00
Erik Arvstedt d214605b32 spark-wallet: add flakes compatibility
Pure flakes can't use NIX_PATH.
2021-03-16 12:46:19 +01:00
Erik Arvstedt 81db927f66 spark-wallet/generate: remove supplement.json
This file is empty and has no effect.
2021-03-16 12:46:19 +01:00
Erik Arvstedt 84b3217c3d fetch-release: minor improvements
This script is potentially fetched from an untrusted source and should
be in good shape to be easily auditable.

- Create just one TMPDIR
- Improve comments
- Use `cut` to extract sha256
- Use camelCase var names like in other scripts
2021-03-16 12:46:18 +01:00
Erik Arvstedt 45d0964e27 examples/shell.nix: minor improvements
- Use idiomatic var name `pkgs` for the imported nixpkgs.
- Don't add `figlet` to PATH because it's only used internally.
- Only print figlet in interactive shells to avoid interfering with
  stdout when running `nix-shell --run <cmd>`.
- Define `fetch-release` as a function to enable running it via
  `nix-shell --run fetch-release`
2021-03-16 12:46:18 +01:00
Erik Arvstedt cc7149eb78 examples: improve robustness of deploy scripts
- The scripts now work within arbitrary nix-shells.
  Previously, they failed when run from nix shells other than
  `examples/shell.nix`.

- The scripts now work from arbitrary working dirs.
2021-03-16 12:46:18 +01:00
Erik Arvstedt e8b47f099c examples: add deploy-krops.sh 2021-03-15 19:02:58 +01:00
Erik Arvstedt 03db1a61b1 lnd, joinmarket: don't write to secrets dir
Keeping the secrets dir read-only is more simple and robust.

- lnd seed mnemonic creation and joinmarket wallet creation can be
  run as the regular service user instead of root.

- It is easier to switch to a third-party secrets deployment
  method in the future.

Don't create a seed mnemonic for lnd when a wallet exists.
This avoids creating unused mnemonics and helps simplifying
the migration command in `versioning.nix`.
2021-03-15 18:50:15 +01:00
Erik Arvstedt 55d87490ec secrets: make configuration more robust
- Fail at evaluation when secrets setup is not configured.
  Previously, bitcoind failed at runtime due to the missing secrets target.

- Fail at evaluation when conflicting secrets setup methods are used.
  This happens when `secretsSetupMethod` has more than one definition.
2021-03-15 12:42:52 +00:00
Erik Arvstedt b0c223e716 secrets: don't stop services when the secrets target stops
With krops or nixops the secrets target is always restarted during
the deployment process.
This previously caused unnecessary restarts of all nix-bitcoin services.
2021-03-15 12:42:52 +00:00
Erik Arvstedt b701cb5603 secrets: add option 'generateSecrets'
Move this feature from a module preset to a regular option, so that it's
easily discoverable and accessible.

Simplify the implementation of `generateSecrets` by adding it to the
existing `setup-secrets` service script.

Also rename option setup-secrets -> setupSecrets.
2021-03-15 12:42:52 +00:00
Erik Arvstedt 03515a8da6 examples/qemu: disable hardened preset
- Hardening is irrelevant in examples
- The decreased performance is especially noticeable in VMs
2021-03-15 12:42:52 +00:00
Erik Arvstedt 908af3bfb8 examples/deploy-qemu-vm: show progress when waiting
This improves the user experience on VM startup, which can take a few
seconds.
2021-03-15 12:42:51 +00:00
Erik Arvstedt ccba86a0f0 examples: extract common features for running QEMU VMs
Needed by commit: "examples: add deploy-krops.sh"
2021-03-15 12:42:51 +00:00
Erik Arvstedt cb3d5446c8 examples: use relative paths to example files
This ensures that the examples files from the current example dir are used.
Previously, the default examples from the nix-bitcoin source were used.
2021-03-15 12:42:51 +00:00
Erik Arvstedt 8e3feece67 run-tests: fix interrupt handling for --copy-src
Previously, `run-tests.sh --copy-src ...` exited with status 0 (success) when interrupted (SIGINT).
It now exits with an error status.
2021-02-12 21:39:46 +01:00
Erik Arvstedt ce2b445777 treewide: use runuser for dropping privileges
When running as root, use runuser instead of sudo.
As opposed to sudo or doas, runuser is a standalone
binary that needs no external configuration.
Also, it's a bit faster.
2021-02-09 12:44:01 +00:00
Erik Arvstedt 0a2c8e4864 run-tests: add option --copy-src 2021-02-08 12:20:20 +01:00
Erik Arvstedt 803584a288 backups: don't use hardcoded secrets dir 2021-02-07 22:45:38 +01:00
Erik Arvstedt c29d44b49a ci: use 'cachix watch-exec'
Simplifies the build script.
This feature appeared in a recent cachix update.
2021-02-07 22:45:37 +01:00
Erik Arvstedt 6a32812412 services: add names for systemd helper scripts
The systemd journal now shows a specific script name instead of
the generic name "script" before script output.
2021-02-07 22:45:36 +01:00
Erik Arvstedt 6982699613 services: use consistent layout
Use the following order of definitions for all services:
- assertions
- configuration of other services
- environment.systemPackages
- tmpfiles
- own service
- users
- secrets
2021-02-07 22:42:23 +01:00
Erik Arvstedt a43534dda0 services: improve config file setup
- btcpayserver, nbxplorer: Add quotes to the
  dataDir arg. (dataDir can contain spaces.)

- clightning, liquidd: use 'install'
2021-02-07 22:42:22 +01:00
Erik Arvstedt 18f2002cf0 joinmarket-yieldgenerator: improve systemd journal output
Journal entries now look like
`joinmarket-yieldgenerator[9795]: User data location: /var/lib/joinmarket`
instead of
`bash[9795]: User data location: /var/lib/joinmarket`
2021-02-07 22:41:46 +01:00
Erik Arvstedt 9d0b8c8f6f joinmarket-ob-watcher: use DynamicUser
DynamicUser simplifies services that don't need a persistent uid/gid,
like joinmarket-ob-watcher.

For existing installations the data dir migration to dynamic users
is automatically handled by systemd.
2021-02-07 22:41:44 +01:00
Erik Arvstedt e9c98f415c joinmarket: explain need for tor control socket 2021-02-07 22:41:31 +01:00
Erik Arvstedt d9c87b6a8f joinmarket: fix wallet creation
- Fix jm-wallet-seed being globally readable.

- Handle seed extraction failures.
  If seed extraction fails, remove the newly created wallet.
  This guarantees that wallets always have an accompanying seed.
2021-02-07 22:41:31 +01:00
Erik Arvstedt 7458350108 treewide: remove deprecated types.loaOf 2021-02-07 22:41:31 +01:00
Erik Arvstedt 9cf038939c treewide: use mkEnableOption 2021-02-07 22:41:31 +01:00
Erik Arvstedt 7a97304f13 treewide: remove unit descriptions
Systemd's `Description` option is a misnomer (as confessed by `man systemd.unit`):
Its value is used by user-facing tools in place of the unit file name, so this option
could have been more aptly named `label` or `name`.
`Description` should only be set if the unit file name is not sufficient for naming a unit.
This is not the case for our services, except for `systemd.services.nb-netns-bridge`
whose description has been kept.

As an example how this affects users, weird journal lines like
```
nb-test systemd[1]: Starting Run clightningd...
```
are now replaced by
```
nb-test systemd[1]: Starting clightning.service...
```
2021-02-07 22:41:31 +01:00
Erik Arvstedt a942177ecf treewide: remove user descriptions
User descriptions are stored in the `comment` field in /etc/passwd.
In our case, these are completely redundant and don't add any useful information.
2021-02-07 22:41:30 +01:00
Erik Arvstedt 4f6ff408ef treewide: remove unneeded string literals 2021-02-07 22:41:29 +01:00
Erik Arvstedt e6a6c721c1 treewide: streamline 'extraConfig' descriptions 2021-02-07 22:40:11 +01:00
Erik Arvstedt e774c045de treewide: fix formatting 2021-02-07 22:40:10 +01:00
Erik Arvstedt 0b5b29a2a3 netns-isolation: simplify permission definition for netns-exec
The new definition is equivalent to the old one.
2021-02-07 22:39:06 +01:00
Erik Arvstedt a587a2b02a defaultHardening: explain where @system-service is defined 2021-02-07 22:39:06 +01:00
Erik Arvstedt bb3a69797e README: minor improvements
- Simplify examples link text.
  That the examples README is located in a subdirectory is not relevant here.

- The backup frequency is freely configurable.
  It's set to 'daily' only by secure-node.nix.
2021-02-07 22:39:06 +01:00
Erik Arvstedt 13fc9dfabf examples: improve introductory comments 2021-02-07 22:39:05 +01:00
Erik Arvstedt af2040f4c4 netns-isolation: use 'true' for systemd option 2021-02-07 22:39:05 +01:00
Erik Arvstedt c246bbb36e bitcoind, clightning, lnd: improve descriptions
bitcoind: The previous description of 'prune' didn't match the int-only
values supported by our option.
2021-02-07 22:39:05 +01:00
Erik Arvstedt 7533f12ef1 bitcoind, clightning, run-tests: minor refactoring
bitcoind: use builtins.toFile
clightning: use boolToString
run-tests: remove leftover var
2021-02-07 22:39:05 +01:00
Erik Arvstedt 41fe9b0c1d elementsd: minor refactoring
- Use pname
- urls -> url
2021-02-07 22:39:05 +01:00
Erik Arvstedt f0850d3f23 btcpayserver: reorder config settings
Move 'bind' and 'port' next to each other and to the top.
2021-02-07 22:39:05 +01:00
Erik Arvstedt d1c0ea9f85 btcpayserver: add missing systemd postgresql dependency
btcpayserver fails if it starts before postgresql.
2021-02-07 22:39:05 +01:00
Erik Arvstedt a2f265cd35 secp256k1: move to top-level packages
Reason: secp256k1 is not a Python package.
2021-02-06 11:43:36 +01:00
Erik Arvstedt d41a843167 jmbitcoin: remove secp256k1 from propagatedBuildInputs
Adding this input has no effect. jmbitcoin accesses secp256k1 via bitcointx.
2021-02-06 11:43:36 +01:00
Erik Arvstedt c22adb03af extra-container: 0.5 -> 0.6 2021-02-06 11:43:36 +01:00
Erik Arvstedt a344ae95c9 move mkHiddenService to lib 2021-02-04 12:39:54 +00:00
Erik Arvstedt a26ed03d77 rename nix-bitcoin-services.nix -> lib.nix 2021-02-04 12:39:48 +00:00
Erik Arvstedt b114d0c3b1 bitcoind: use systemd startup notification 2021-01-31 22:26:49 +01:00
Erik Arvstedt 332d0e70c8 bitcoind: support onion address announcing 2021-01-31 22:26:49 +01:00
Erik Arvstedt 9662c19ab1 onionServices: use actual user name of services
Previously, onionAddresses definitions in onionServices were of the form
onionAddresses.access.<service> = [<service>];

This caused failures for configurations where a service user name was
overridden or for bitcoind whose default user is 'bitcoin' instead of 'bitcoind'.

Now set the equivalent of:
onionAddresses.access.<actualServiceUser> = [<service>];

Implement this via a new option `onionAddresses.services` to make things more
readable and to work around an infinite recursion error in onionServices.
2021-01-31 22:26:49 +01:00
Erik Arvstedt 5c09845e6f bitcoind: tag incoming connections as onion on enforceTor 2021-01-31 22:26:49 +01:00
Erik Arvstedt 8f9ea61d6e update nixpkgs-unstable
- bitcoind 0.20.1 -> 0.21.0
  Manually create a wallet in the backup test because bitcoind
  does not create a default wallet anymore

- disable the failing elementsd build on unstable
2021-01-31 22:26:30 +01:00
Erik Arvstedt 05e5ec99ec modules packages: build electrs, lightning-loop with nixpkgs stable
Building with nixBitcoinPkgsUnstable was only a temporary measure to
fix build errors on stable.
2021-01-30 11:38:47 +01:00
Erik Arvstedt 44546561fc run-tests: allow defining scenarios via cmdline args
This simplifies running self-contained scenarios for testing and debugging.
2021-01-30 11:38:47 +01:00
Erik Arvstedt fc40776689 improve backup test
Only check enabled services. This allows running the backup test with a custom subset of other
tests.

Also, show a meaningful error on test failure.
Previously, just an AssertionError without a message was shown.
Because the test code is evaluated from a string and not a file, there
was also no backtrace to the tests file.
2021-01-30 11:38:47 +01:00
Erik Arvstedt 9a67a32779 fix build-to-cachix
- Don't fail on `kill $cachixPid` when cachix has already exited.
  This fixes some failing CI builds.

- Stop the cachix background worker before the final `cachix push`.
  This can avoid unneeded reuploads.
  Use the coreutils version of tail on cirrus which supports
  argument --pid.
2021-01-30 11:38:47 +01:00
Erik Arvstedt 915df059f4 joinmarket: 0.8.0-bcfa7eb -> 0.8.0-a5e8879 2021-01-17 17:40:01 +00:00
Erik Arvstedt 254246cf39 joinmarket: use installPhase
This simplifies the build.
2021-01-17 14:17:14 +01:00
Erik Arvstedt e2922eb4ce move rpc thread count setting to lightning modules 2021-01-14 13:25:12 +01:00
Erik Arvstedt 352fc4e8fe liquid: remove insecure and redundant option 'rpcpassword' 2021-01-14 13:25:11 +01:00
Erik Arvstedt 757a66b9bd liquid: move rpcuser definition to module 2021-01-14 13:25:11 +01:00
Erik Arvstedt 0e00c39d47 secure-node: improve layout 2021-01-14 13:25:11 +01:00
Erik Arvstedt 5f7a7962f7 backups: remove redundant option 'program'
Not needed until we support other backup backends.
2021-01-14 13:25:11 +01:00
Erik Arvstedt 04d8560f86 secure-node: remove qrencode, tor from systemPackages
Keep jq which is useful for analyzing service cli output.
2021-01-14 13:25:10 +01:00
Erik Arvstedt 323a431aba improve nodeinfo
- enable usage outside of secure-node.nix
- use json as the output format
- show ports
- also show local addresses, which is particularly useful when
  netns-isolation is enabled
- only show enabled services
2021-01-14 13:25:10 +01:00