Commit Graph
100 Commits
Author SHA1 Message Date
Erik Arvstedt cb3d5446c8 examples: use relative paths to example files
This ensures that the examples files from the current example dir are used.
Previously, the default examples from the nix-bitcoin source were used.
2021-03-15 12:42:51 +00:00
Erik Arvstedt 8e3feece67 run-tests: fix interrupt handling for --copy-src
Previously, `run-tests.sh --copy-src ...` exited with status 0 (success) when interrupted (SIGINT).
It now exits with an error status.
2021-02-12 21:39:46 +01:00
Erik Arvstedt ce2b445777 treewide: use runuser for dropping privileges
When running as root, use runuser instead of sudo.
As opposed to sudo or doas, runuser is a standalone
binary that needs no external configuration.
Also, it's a bit faster.
2021-02-09 12:44:01 +00:00
Erik Arvstedt 0a2c8e4864 run-tests: add option --copy-src 2021-02-08 12:20:20 +01:00
Erik Arvstedt 803584a288 backups: don't use hardcoded secrets dir 2021-02-07 22:45:38 +01:00
Erik Arvstedt c29d44b49a ci: use 'cachix watch-exec'
Simplifies the build script.
This feature appeared in a recent cachix update.
2021-02-07 22:45:37 +01:00
Erik Arvstedt 6a32812412 services: add names for systemd helper scripts
The systemd journal now shows a specific script name instead of
the generic name "script" before script output.
2021-02-07 22:45:36 +01:00
Erik Arvstedt 6982699613 services: use consistent layout
Use the following order of definitions for all services:
- assertions
- configuration of other services
- environment.systemPackages
- tmpfiles
- own service
- users
- secrets
2021-02-07 22:42:23 +01:00
Erik Arvstedt a43534dda0 services: improve config file setup
- btcpayserver, nbxplorer: Add quotes to the
  dataDir arg. (dataDir can contain spaces.)

- clightning, liquidd: use 'install'
2021-02-07 22:42:22 +01:00
Erik Arvstedt 18f2002cf0 joinmarket-yieldgenerator: improve systemd journal output
Journal entries now look like
`joinmarket-yieldgenerator[9795]: User data location: /var/lib/joinmarket`
instead of
`bash[9795]: User data location: /var/lib/joinmarket`
2021-02-07 22:41:46 +01:00
Erik Arvstedt 9d0b8c8f6f joinmarket-ob-watcher: use DynamicUser
DynamicUser simplifies services that don't need a persistent uid/gid,
like joinmarket-ob-watcher.

For existing installations the data dir migration to dynamic users
is automatically handled by systemd.
2021-02-07 22:41:44 +01:00
Erik Arvstedt e9c98f415c joinmarket: explain need for tor control socket 2021-02-07 22:41:31 +01:00
Erik Arvstedt d9c87b6a8f joinmarket: fix wallet creation
- Fix jm-wallet-seed being globally readable.

- Handle seed extraction failures.
  If seed extraction fails, remove the newly created wallet.
  This guarantees that wallets always have an accompanying seed.
2021-02-07 22:41:31 +01:00
Erik Arvstedt 7458350108 treewide: remove deprecated types.loaOf 2021-02-07 22:41:31 +01:00
Erik Arvstedt 9cf038939c treewide: use mkEnableOption 2021-02-07 22:41:31 +01:00
Erik Arvstedt 7a97304f13 treewide: remove unit descriptions
Systemd's `Description` option is a misnomer (as confessed by `man systemd.unit`):
Its value is used by user-facing tools in place of the unit file name, so this option
could have been more aptly named `label` or `name`.
`Description` should only be set if the unit file name is not sufficient for naming a unit.
This is not the case for our services, except for `systemd.services.nb-netns-bridge`
whose description has been kept.

As an example how this affects users, weird journal lines like
```
nb-test systemd[1]: Starting Run clightningd...
```
are now replaced by
```
nb-test systemd[1]: Starting clightning.service...
```
2021-02-07 22:41:31 +01:00
Erik Arvstedt a942177ecf treewide: remove user descriptions
User descriptions are stored in the `comment` field in /etc/passwd.
In our case, these are completely redundant and don't add any useful information.
2021-02-07 22:41:30 +01:00
Erik Arvstedt 4f6ff408ef treewide: remove unneeded string literals 2021-02-07 22:41:29 +01:00
Erik Arvstedt e6a6c721c1 treewide: streamline 'extraConfig' descriptions 2021-02-07 22:40:11 +01:00
Erik Arvstedt e774c045de treewide: fix formatting 2021-02-07 22:40:10 +01:00
Erik Arvstedt 0b5b29a2a3 netns-isolation: simplify permission definition for netns-exec
The new definition is equivalent to the old one.
2021-02-07 22:39:06 +01:00
Erik Arvstedt a587a2b02a defaultHardening: explain where @system-service is defined 2021-02-07 22:39:06 +01:00
Erik Arvstedt bb3a69797e README: minor improvements
- Simplify examples link text.
  That the examples README is located in a subdirectory is not relevant here.

- The backup frequency is freely configurable.
  It's set to 'daily' only by secure-node.nix.
2021-02-07 22:39:06 +01:00
Erik Arvstedt 13fc9dfabf examples: improve introductory comments 2021-02-07 22:39:05 +01:00
Erik Arvstedt af2040f4c4 netns-isolation: use 'true' for systemd option 2021-02-07 22:39:05 +01:00
Erik Arvstedt c246bbb36e bitcoind, clightning, lnd: improve descriptions
bitcoind: The previous description of 'prune' didn't match the int-only
values supported by our option.
2021-02-07 22:39:05 +01:00
Erik Arvstedt 7533f12ef1 bitcoind, clightning, run-tests: minor refactoring
bitcoind: use builtins.toFile
clightning: use boolToString
run-tests: remove leftover var
2021-02-07 22:39:05 +01:00
Erik Arvstedt 41fe9b0c1d elementsd: minor refactoring
- Use pname
- urls -> url
2021-02-07 22:39:05 +01:00
Erik Arvstedt f0850d3f23 btcpayserver: reorder config settings
Move 'bind' and 'port' next to each other and to the top.
2021-02-07 22:39:05 +01:00
Erik Arvstedt d1c0ea9f85 btcpayserver: add missing systemd postgresql dependency
btcpayserver fails if it starts before postgresql.
2021-02-07 22:39:05 +01:00
Erik Arvstedt a2f265cd35 secp256k1: move to top-level packages
Reason: secp256k1 is not a Python package.
2021-02-06 11:43:36 +01:00
Erik Arvstedt d41a843167 jmbitcoin: remove secp256k1 from propagatedBuildInputs
Adding this input has no effect. jmbitcoin accesses secp256k1 via bitcointx.
2021-02-06 11:43:36 +01:00
Erik Arvstedt c22adb03af extra-container: 0.5 -> 0.6 2021-02-06 11:43:36 +01:00
Erik Arvstedt a344ae95c9 move mkHiddenService to lib 2021-02-04 12:39:54 +00:00
Erik Arvstedt a26ed03d77 rename nix-bitcoin-services.nix -> lib.nix 2021-02-04 12:39:48 +00:00
Erik Arvstedt b114d0c3b1 bitcoind: use systemd startup notification 2021-01-31 22:26:49 +01:00
Erik Arvstedt 332d0e70c8 bitcoind: support onion address announcing 2021-01-31 22:26:49 +01:00
Erik Arvstedt 9662c19ab1 onionServices: use actual user name of services
Previously, onionAddresses definitions in onionServices were of the form
onionAddresses.access.<service> = [<service>];

This caused failures for configurations where a service user name was
overridden or for bitcoind whose default user is 'bitcoin' instead of 'bitcoind'.

Now set the equivalent of:
onionAddresses.access.<actualServiceUser> = [<service>];

Implement this via a new option `onionAddresses.services` to make things more
readable and to work around an infinite recursion error in onionServices.
2021-01-31 22:26:49 +01:00
Erik Arvstedt 5c09845e6f bitcoind: tag incoming connections as onion on enforceTor 2021-01-31 22:26:49 +01:00
Erik Arvstedt 8f9ea61d6e update nixpkgs-unstable
- bitcoind 0.20.1 -> 0.21.0
  Manually create a wallet in the backup test because bitcoind
  does not create a default wallet anymore

- disable the failing elementsd build on unstable
2021-01-31 22:26:30 +01:00
Erik Arvstedt 05e5ec99ec modules packages: build electrs, lightning-loop with nixpkgs stable
Building with nixBitcoinPkgsUnstable was only a temporary measure to
fix build errors on stable.
2021-01-30 11:38:47 +01:00
Erik Arvstedt 44546561fc run-tests: allow defining scenarios via cmdline args
This simplifies running self-contained scenarios for testing and debugging.
2021-01-30 11:38:47 +01:00
Erik Arvstedt fc40776689 improve backup test
Only check enabled services. This allows running the backup test with a custom subset of other
tests.

Also, show a meaningful error on test failure.
Previously, just an AssertionError without a message was shown.
Because the test code is evaluated from a string and not a file, there
was also no backtrace to the tests file.
2021-01-30 11:38:47 +01:00
Erik Arvstedt 9a67a32779 fix build-to-cachix
- Don't fail on `kill $cachixPid` when cachix has already exited.
  This fixes some failing CI builds.

- Stop the cachix background worker before the final `cachix push`.
  This can avoid unneeded reuploads.
  Use the coreutils version of tail on cirrus which supports
  argument --pid.
2021-01-30 11:38:47 +01:00
Erik Arvstedt 915df059f4 joinmarket: 0.8.0-bcfa7eb -> 0.8.0-a5e8879 2021-01-17 17:40:01 +00:00
Erik Arvstedt 254246cf39 joinmarket: use installPhase
This simplifies the build.
2021-01-17 14:17:14 +01:00
Erik Arvstedt e2922eb4ce move rpc thread count setting to lightning modules 2021-01-14 13:25:12 +01:00
Erik Arvstedt 352fc4e8fe liquid: remove insecure and redundant option 'rpcpassword' 2021-01-14 13:25:11 +01:00
Erik Arvstedt 757a66b9bd liquid: move rpcuser definition to module 2021-01-14 13:25:11 +01:00
Erik Arvstedt 0e00c39d47 secure-node: improve layout 2021-01-14 13:25:11 +01:00
Erik Arvstedt 5f7a7962f7 backups: remove redundant option 'program'
Not needed until we support other backup backends.
2021-01-14 13:25:11 +01:00
Erik Arvstedt 04d8560f86 secure-node: remove qrencode, tor from systemPackages
Keep jq which is useful for analyzing service cli output.
2021-01-14 13:25:10 +01:00
Erik Arvstedt 323a431aba improve nodeinfo
- enable usage outside of secure-node.nix
- use json as the output format
- show ports
- also show local addresses, which is particularly useful when
  netns-isolation is enabled
- only show enabled services
2021-01-14 13:25:10 +01:00
Erik Arvstedt f6b883a9ac remove webindex
This module is outdated and incomplete. We can readd an improved version in
the future.

Move nanopos nginx proxy tests to the nanopos test.
2021-01-14 13:25:10 +01:00
Erik Arvstedt 2a240d6f4a enable-tor: disable default onion services for clightning, lnd, btcpayserver
In case of btcpayserver the default onion service is a security risk
because any visitor can register an admin account on a freshly setup node.
2021-01-14 13:25:09 +01:00
Erik Arvstedt 18c7842e1a modules: show warnings for obsolete options 2021-01-14 13:25:09 +01:00
Erik Arvstedt 45c40c4eb9 versioning: simplify assertion evaluation 2021-01-14 13:25:09 +01:00
Erik Arvstedt bed00fe937 lnd: use onionServices for address announcing 2021-01-14 13:25:09 +01:00
Erik Arvstedt 3980cd5a41 clightning: use onionServices for address announcing 2021-01-14 13:25:08 +01:00
Erik Arvstedt bd2a46cb73 spark-wallet: use onionServices
Also remove the unneeded definition of ReadWritePaths because the
service doesn't need write access to onion files.
2021-01-14 13:25:08 +01:00
Erik Arvstedt 87fb9f246b add 'enable-tor' preset
Move 'enforceTor' and onion-service definitions from secure-node.nix.
Use the onionServices module to define onion services.

Onion services now automatically work for services that bind to an INADDR_ANY (`0.0.0.0`) address.
2021-01-14 13:25:08 +01:00
Erik Arvstedt 05b5402bb1 add nix-bitcoin.onionServices 2021-01-14 13:25:07 +01:00
Erik Arvstedt fffe988248 onionAddresses: add readonly option 'dataDir'
Used by 'onionServices' in a later commit for services that announce
their onion address.
2021-01-14 13:25:07 +01:00
Erik Arvstedt 5f34b094d3 onionAddresses: improve script
- use -e to check for existence of /var/lib/tor/state, use shorter
  polling interval
- clear existing dataDir contents to avoid accumulating obsolete data
- use concatMapStrings instead of foldl'
2021-01-14 13:25:07 +01:00
Erik Arvstedt b266f23251 onionAddresses: use service 'script' option
This also makes the script stop on errors.
2021-01-14 13:25:07 +01:00
Erik Arvstedt 6d13b26d0a onionAddresses: add more precise type for option 'access' 2021-01-14 13:25:06 +01:00
Erik Arvstedt 93562f76dd onionAddresses: remove redundant option 'enable'
The service can be disabled via `onion-addresses.access = mkForce {};`

Also remove redundant description.
2021-01-14 13:25:06 +01:00
Erik Arvstedt 43c247e3fe onionAddresses: use StateDirectory instead of tmpfiles
Simplifies the dataDir setup.
2021-01-14 13:25:06 +01:00
Erik Arvstedt 5c6977b006 rename onion-chef -> nix-bitcoin.onionAddresses
This clarifies its function.
2021-01-14 13:25:05 +01:00
Erik Arvstedt 55073eee70 remove nix-bitcoin.pkgs.lib
Type ipv4Address is not needed anymore because all services have
separate 'port' and 'address' options.
2021-01-14 13:25:05 +01:00
Erik Arvstedt 09e0042aa8 spark-wallet: add consistent address options 2021-01-14 13:25:05 +01:00
Erik Arvstedt 39f16c0b4a liquidd: add consistent address options 2021-01-14 13:25:05 +01:00
Erik Arvstedt b5d76ba1b3 electrs: add consistent address options 2021-01-14 13:25:04 +01:00
Erik Arvstedt 8fa32b7f91 btcpayserver: add consistent address options 2021-01-14 13:25:04 +01:00
Erik Arvstedt e78a609687 clightning: add consistent address options
Also remove option 'autolisten'. This option has no effect because
option 'bind-addr' is always set.
2021-01-14 13:25:04 +01:00
Erik Arvstedt b41a720c28 lnd: add consistent address options
Also fix btcpayserver by connecting to the lnd restAddress instead of the p2p address.
2021-01-14 13:25:03 +01:00
Erik Arvstedt dd4a0238f9 bitcoind: group rpc options under parent option 'rpc' 2021-01-14 13:25:03 +01:00
Erik Arvstedt 5b7e0d09b2 bitcoind: add consistent address options 2021-01-14 13:25:03 +01:00
Erik Arvstedt 2bfb4efbd8 make-container: fix usage comment 2020-12-19 13:18:50 +01:00
Erik Arvstedt 3403795c86 tests: add example scripts 2020-12-18 19:56:56 +01:00
Erik Arvstedt ff94985b8b tests: add test 'hardened' 2020-12-18 19:56:56 +01:00
Erik Arvstedt c8e73c959e fix 'hardened' profile for NixOS 20.09
The 'scudo' memory allocator set by the 'hardened' profile breaks some
services on 20.09.
The fix for NixOS unstable (https://github.com/NixOS/nixpkgs/pull/104052)
is ineffective on 20.09.

As a workaround, add a custom 'hardened' preset that uses the default allocator.
2020-12-18 19:56:56 +01:00
Erik Arvstedt 44b06aea5a extra-container: 0.5-pre -> 0.5 2020-12-18 19:56:56 +01:00
Erik Arvstedt a359cdfb66 generate-secrets: use pwgen
Password length and alphabet is unchanged, but the restriction to
include at least one numeric and one capital char has been removed.
This restriction is not needed by client applications,
adds code complexity, and even (insignificantly) reduces entropy.

Reason for switching to pwgen:
apg uses /dev/random instead of /dev/urandom which brings no security
benefits but can stall the generate-secrets script on low-entropy
devices due to blocking.

Since `security.rngd` has been disabled in NixOS 20.09, blocking
in generate-secrets can also appear on regular NixOS desktop systems.
2020-12-18 19:56:56 +01:00
Erik Arvstedt a5a2fc7274 make-container: fix renamed variable
The variable was only renamed in run-tests.sh, which broke containers.
2020-12-16 01:29:12 +01:00
Erik Arvstedt 9977fa69af ci: use run-tests.sh 2020-12-11 13:27:06 +01:00
Erik Arvstedt a82f0f5f48 add test 'pkgsUnstable'
Included in 'basic' tests.
Function 'doBuild' is needed by the following commit.
2020-12-11 13:27:05 +01:00
Erik Arvstedt 95bc1237e2 run-tests: rename testDir -> scriptDir 2020-12-11 13:27:05 +01:00
Erik Arvstedt 466d23deaa ci: extract build-to-cachix.sh
Needed by the following commits.
Also, don't use the cachix cache as a substituter for local, non-CI
builds. This obviates the need for the 'untrusted' warning in build.sh.
2020-12-11 13:27:05 +01:00
Erik Arvstedt a70c3bf210 make-test-vm: remove unneeded leftover arg attrs 2020-12-11 13:27:05 +01:00
Erik Arvstedt ed65e78a2b make-test: expose test config
This is useful for programmatically exploring a test config or for building a test system
on a custom platform.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 7265742655 run-tests: add 'instantiate' command
Useful for diffing test derivations.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 8cbdef8bf6 run-tests: fix CLI
Restore the original behavior that was accidentally changed:
When no args are given, run the basic test suite.
Otherwise, run the given command with default scenario 'default'.
Previously, `run-tests.sh build` ran the basic test suite instead of
building the default scenario.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 1c0233c0a8 use Cirrus CI
- Make more economic use of the free CI resources by removing redundant build tasks:
  - Build unstable pkgs in a single separate task ("pkgs_unstable").
  - All stable pkgs are implicitly built by the modules tests.
- The build script (ci/build.sh) can now be executed locally for easier
  debugging.
- Use an explicit 'cachix push' command instead of helper/wait-for-network-idle.rb.
  This is simpler and more reliable.
2020-12-06 19:07:54 +01:00
Erik Arvstedt a6346c2561 electrs: 0.8.5 -> 0.8.6 2020-12-01 12:51:36 +01:00
Erik Arvstedt 4640821f96 make-test.nix: use writeText
Needed for the following commit which adds derivation outputs to `dataFile`.
2020-11-18 20:21:34 -06:00
Erik Arvstedt 5399f73b20 add txzmq python pkg 2020-11-18 20:21:34 -06:00
Erik Arvstedt e62e163177 add clightning python pkgs 2020-11-18 20:21:34 -06:00
Erik Arvstedt 1a16e55237 move python packages to pkgs/python-packages
Remove obsolete passthru from joinmarket because joinmarket packages are
now accessible via pkgs/python-packages.
2020-11-18 20:21:34 -06:00
Erik Arvstedt 7e3d2965a5 testing framework: re-add features removed in nixpkgs 20.09
Since nixpks 20.09, the test output is just an empty directory.
Restore saving the log output and linking to the driver.

Without linking to the driver, the driver is eligible for
garbage collection after running a test via `run-tests.sh --out-link-prefix`,
which implies lengthy driver rebuilds.
2020-11-11 22:27:46 +01:00