Commit Graph
100 Commits
Author SHA1 Message Date
Erik Arvstedt 44b06aea5a extra-container: 0.5-pre -> 0.5 2020-12-18 19:56:56 +01:00
Erik Arvstedt a359cdfb66 generate-secrets: use pwgen
Password length and alphabet is unchanged, but the restriction to
include at least one numeric and one capital char has been removed.
This restriction is not needed by client applications,
adds code complexity, and even (insignificantly) reduces entropy.

Reason for switching to pwgen:
apg uses /dev/random instead of /dev/urandom which brings no security
benefits but can stall the generate-secrets script on low-entropy
devices due to blocking.

Since `security.rngd` has been disabled in NixOS 20.09, blocking
in generate-secrets can also appear on regular NixOS desktop systems.
2020-12-18 19:56:56 +01:00
Erik Arvstedt a5a2fc7274 make-container: fix renamed variable
The variable was only renamed in run-tests.sh, which broke containers.
2020-12-16 01:29:12 +01:00
Erik Arvstedt 9977fa69af ci: use run-tests.sh 2020-12-11 13:27:06 +01:00
Erik Arvstedt a82f0f5f48 add test 'pkgsUnstable'
Included in 'basic' tests.
Function 'doBuild' is needed by the following commit.
2020-12-11 13:27:05 +01:00
Erik Arvstedt 95bc1237e2 run-tests: rename testDir -> scriptDir 2020-12-11 13:27:05 +01:00
Erik Arvstedt 466d23deaa ci: extract build-to-cachix.sh
Needed by the following commits.
Also, don't use the cachix cache as a substituter for local, non-CI
builds. This obviates the need for the 'untrusted' warning in build.sh.
2020-12-11 13:27:05 +01:00
Erik Arvstedt a70c3bf210 make-test-vm: remove unneeded leftover arg attrs 2020-12-11 13:27:05 +01:00
Erik Arvstedt ed65e78a2b make-test: expose test config
This is useful for programmatically exploring a test config or for building a test system
on a custom platform.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 7265742655 run-tests: add 'instantiate' command
Useful for diffing test derivations.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 8cbdef8bf6 run-tests: fix CLI
Restore the original behavior that was accidentally changed:
When no args are given, run the basic test suite.
Otherwise, run the given command with default scenario 'default'.
Previously, `run-tests.sh build` ran the basic test suite instead of
building the default scenario.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 1c0233c0a8 use Cirrus CI
- Make more economic use of the free CI resources by removing redundant build tasks:
  - Build unstable pkgs in a single separate task ("pkgs_unstable").
  - All stable pkgs are implicitly built by the modules tests.
- The build script (ci/build.sh) can now be executed locally for easier
  debugging.
- Use an explicit 'cachix push' command instead of helper/wait-for-network-idle.rb.
  This is simpler and more reliable.
2020-12-06 19:07:54 +01:00
Erik Arvstedt a6346c2561 electrs: 0.8.5 -> 0.8.6 2020-12-01 12:51:36 +01:00
Erik Arvstedt 4640821f96 make-test.nix: use writeText
Needed for the following commit which adds derivation outputs to `dataFile`.
2020-11-18 20:21:34 -06:00
Erik Arvstedt 5399f73b20 add txzmq python pkg 2020-11-18 20:21:34 -06:00
Erik Arvstedt e62e163177 add clightning python pkgs 2020-11-18 20:21:34 -06:00
Erik Arvstedt 1a16e55237 move python packages to pkgs/python-packages
Remove obsolete passthru from joinmarket because joinmarket packages are
now accessible via pkgs/python-packages.
2020-11-18 20:21:34 -06:00
Erik Arvstedt 7e3d2965a5 testing framework: re-add features removed in nixpkgs 20.09
Since nixpks 20.09, the test output is just an empty directory.
Restore saving the log output and linking to the driver.

Without linking to the driver, the driver is eligible for
garbage collection after running a test via `run-tests.sh --out-link-prefix`,
which implies lengthy driver rebuilds.
2020-11-11 22:27:46 +01:00
Erik Arvstedt f1681f5b45 add option nix-bitcoin.pkgs, remove overlay
This works around a nixpkgs bug where overlays are ignored in containers.
2020-11-09 22:10:07 +01:00
Erik Arvstedt b1a5f5e7b9 don't add service-only pkgs to systemPackages
These packages are not useful in shell environemnts.
2020-11-09 22:10:07 +01:00
Erik Arvstedt e39ad358a4 lnd: remove unused bitcoind from PATH 2020-11-09 22:10:07 +01:00
Erik Arvstedt 0972af55f1 netns: fix setup service restarts 2020-11-03 21:55:13 +01:00
Erik Arvstedt 63a464431b netns: fail when netns already exists
Under normal circumstances, service-specific netns should never exist
before the netns setup service starts.
An existing netns is a genuine error that should not be silently ignored.
2020-11-03 21:55:13 +01:00
Erik Arvstedt b4b607dfa5 netns: simplify firewall setup 2020-10-29 22:36:20 +01:00
Erik Arvstedt 25639cec42 netns: fix error msg when starting netns
Previously, the failing initial `netns delete` resulted in a
"Cannot remove namespace file ..." error visible in the journal
and status output.
2020-10-29 21:21:30 +01:00
Erik Arvstedt 67068afd6b netns: fix error when stopping netns
A short time after `netns delete` finishes, the peer link in the main
netns is automatically removed.
When `link del` is run before that, it fails with
`Cannot find device "nb-veth-br-*"` and the netns service enters a failed state.
2020-10-29 21:21:30 +01:00
Erik Arvstedt 4ff88efc50 netns: add address binding test
Proposed by Jonas Nick.
2020-10-29 21:21:30 +01:00
Erik Arvstedt 8da01fe8a6 lightning-loop: allow RPC access from main netns
Note that this also exposes the REST server, which is secured by
macaroon auth like the RPC server.
2020-10-29 21:21:29 +01:00
Erik Arvstedt d76b080b74 lightning-loop: add RPC and REST server options 2020-10-29 21:21:29 +01:00
Erik Arvstedt 9ddf7864a4 lightning-loop regtest: fix incorrectly succeeding test
When 'loop getparams' fails, jq gets no stdin and exits with code 0.
Because -o pipefail is not enabled in the testing shell, the whole test
command succeeds, although it should fail.

Just test "loop getparams" instead and ignore its output.
2020-10-29 21:21:29 +01:00
Erik Arvstedt e66636ef0e liquidd: use type str for rpcbind 2020-10-29 21:21:29 +01:00
Erik Arvstedt de23fdd377 lnd: use type str for rpclisten, restlisten 2020-10-29 21:21:28 +01:00
Erik Arvstedt 8b053326cc bitcoind: use type str for rpcbind
Extra RPC bind addresses can still be added via extraConfig.
2020-10-29 21:21:28 +01:00
Erik Arvstedt 6903e8afcc netns-liquidd: allow RPC access from main netns 2020-10-29 21:21:28 +01:00
Erik Arvstedt 82f4901880 netns-lnd: allow RPC access from main netns 2020-10-29 21:21:27 +01:00
Erik Arvstedt 58d24e735d netns-bitcoind: allow RPC access from main netns 2020-10-29 21:21:27 +01:00
Erik Arvstedt 0e2ff948d3 test: add scenario 'netnsRegtest'
The 'basic' test command now cover regtest mode and using nix-bitcoin without
the secure-node preset.
2020-10-29 21:21:27 +01:00
Erik Arvstedt e0675cb256 move enforceTor logic to service modules
This enables tor support for services without using secure-node.nix
2020-10-29 21:21:27 +01:00
Erik Arvstedt 0cc8caa737 lnd: only set tor.active on enforceTor
This also enables the test scenario 'netnsRegtest' introduced in a
later commit by fixing the following bug:
For unknown reasons, when tor.active=true and tor is not running, lnd
fails with a tor connection error on netns-isolation, but runs fine
without netns-isolation.
2020-10-29 21:21:26 +01:00
Erik Arvstedt 9a931483b9 netns test: remove strict dependency on clightning, electrs
This allows the netns test to be run with a reduced service set for debugging.
2020-10-29 21:21:26 +01:00
Erik Arvstedt bae1b7f413 netns test: improve ping test
- Use fping for pinging multiple hosts in parallel.
  Significantly improves test runtime:
  >13 s -> ~200 ms for the negative ping tests.
- Only test network namespaces that are enabled.
  This allows running the netns test with a reduced service set for debugging.
- Remove deprecated services, instead add btcpayserver, spark-wallet
2020-10-29 21:21:26 +01:00
Erik Arvstedt 2a9b918f72 generate-secrets: always run with Bash, stop on errors 2020-10-23 10:54:15 +02:00
Erik Arvstedt 5e0e16529c netns: fix default addressblock value type
Also remove redundant definition in secure-node.nix
2020-10-20 18:21:37 +02:00
Erik Arvstedt b574cb097f examples: add deploy-container-minimal.sh 2020-10-20 12:35:23 +02:00
Erik Arvstedt 67e49fe415 services: auto-enable dependencies 2020-10-19 14:55:59 +02:00
Erik Arvstedt d2dbad256f shell.nix: fix failing generate-secrets
generate-secrets failed due to the `PYTHONPATH` env var that was set
through the nixops19_09 buildInput.

Fixes:
- Don't pull in the build environments of binaries that should be
  available in PATH. Only extend PATH instead.
- Run generate-secrets in an empty environment
2020-10-19 11:35:32 +02:00
Erik Arvstedt e61d7b1d46 test: improve lightning-loop regtest 2020-10-19 08:59:26 +00:00
Erik Arvstedt e6340426c1 deploy-container.sh: fix sudo
The 'echo sudo' approach used previously failed when PATH or
NIX_PATH contains spaces. Exec the script with sudo instead.
2020-10-18 13:42:58 +02:00
Erik Arvstedt c19f7ebb01 examples: add option --interactive|-i 2020-10-18 13:42:58 +02:00
Erik Arvstedt 33ff8d82be examples: fix running outside of examples dir 2020-10-18 13:42:54 +02:00
Erik Arvstedt 9951f10e74 test: add scenario 'regtest' 2020-10-16 23:55:13 +02:00
Erik Arvstedt d3ece59919 add module 'versioning' 2020-10-16 23:23:00 +02:00
Erik Arvstedt 1f96ca67c5 electrs test: make service shutdown optional
Needed for regtest scenario.
2020-10-16 18:01:52 +02:00
Erik Arvstedt eb42fc8e06 test: extract test 'joinmarket-yieldgenerator'
Needed for regtest scenario.
2020-10-16 18:01:52 +02:00
Erik Arvstedt 06b2ec5b02 joinmarket: add regtest support 2020-10-16 18:01:52 +02:00
Erik Arvstedt 975b30c90e joinmarket: don't hardcode bitcoind rpc port 2020-10-16 18:01:52 +02:00
Erik Arvstedt 031df4231f joinmarket: move comment out of config file 2020-10-16 18:01:52 +02:00
Erik Arvstedt 848c4c6eda joinmarket: add variable 'bitcoind' 2020-10-16 18:01:52 +02:00
Erik Arvstedt 96b08f5d60 btcpayserver: add regtest support 2020-10-16 18:01:52 +02:00
Erik Arvstedt bd2145dc77 btcpayserver: add 'port' option 2020-10-16 18:01:51 +02:00
Erik Arvstedt 001f8fe8d3 btcpayserver: use option bitcoind.rpc.port 2020-10-16 18:01:51 +02:00
Erik Arvstedt 6f4715ac2a electrs: add regtest support 2020-10-16 18:01:51 +02:00
Erik Arvstedt 46efd141a1 lightning-loop: add regtest support 2020-10-16 18:01:51 +02:00
Erik Arvstedt 75ec85bea2 lnd: add regtest support 2020-10-16 18:01:51 +02:00
Erik Arvstedt 1935c252ec lnd: remove redundant option 'bitcoind-host'
Also set bitcoind rpc port.
2020-10-16 18:01:51 +02:00
Erik Arvstedt b1a8629223 lnd: add variable 'bitcoind' 2020-10-16 18:01:51 +02:00
Erik Arvstedt 937aee0062 spark-wallet: add regtest support 2020-10-16 18:01:50 +02:00
Erik Arvstedt 47d611b5ef spark-wallet: use tor rate provider only when enforceTor 2020-10-16 18:01:50 +02:00
Erik Arvstedt 127b186c3c spark-wallet: simplify start script
Also:
- quote paths
- use long form args
2020-10-16 18:01:50 +02:00
Erik Arvstedt 0f32f3c99e clightning: add regtest support 2020-10-16 18:01:50 +02:00
Erik Arvstedt c24ac5d363 clightning: remove redundant option 'bitcoin-rpcconnect' 2020-10-16 18:01:50 +02:00
Erik Arvstedt abd32cde30 clightning: enable config file read access for group
Enables lightning-cli group access when nonstandard config options are set.
2020-10-16 18:01:50 +02:00
Erik Arvstedt ddadaed3da clightning: always use bind-addr in config
bind-addr can't be null.
2020-10-16 18:01:50 +02:00
Erik Arvstedt 9e928e2097 bitcoind: add regtest support
Remove unsupported option 'testnet'.
2020-10-16 18:01:49 +02:00
Erik Arvstedt bfed10b2fa run-tests: add command 'all'
'all' includes test 'full', which now succeeds.
2020-10-16 16:46:56 +02:00
Erik Arvstedt 0a6b9beda5 run-tests: simplify setting default scenario 2020-10-16 16:46:56 +02:00
Erik Arvstedt 1a32292e07 test: speed up clightning startup when offline 2020-10-16 16:46:56 +02:00
Erik Arvstedt 7d1797cec7 clightning: add option 'extraConfig' 2020-10-16 16:46:56 +02:00
Erik Arvstedt e0117d56d1 spark-wallet: fix always-on onion-chef setting
Previously, the service failed when onion-service was disabled.
2020-10-16 16:46:55 +02:00
Erik Arvstedt 480d0d3959 liquid: fix bitcoin rpc settings
- Remove redundant option mainchainrpchost.
  This option is already provided by bitcoind.
- Set a working default for rpcport and rpcuser.
  Enables use without secure-node.
2020-10-16 16:46:55 +02:00
Erik Arvstedt c07e767889 test: add python test requirements
This allows running the Python tests without importing secure-node.
2020-10-16 16:46:55 +02:00
Erik Arvstedt 9aa19c3fdd extract operator module 2020-10-16 16:46:55 +02:00
Erik Arvstedt 2dd1a741f7 modules: group imports 2020-10-16 16:46:55 +02:00
Erik Arvstedt 1cc432a136 examples/deploy-container: use new extra-container features 2020-10-16 15:53:34 +02:00
Erik Arvstedt 16b2783ae7 examples: enable running outside of working dir 2020-10-16 15:53:34 +02:00
Erik Arvstedt 04075b108c test: use QEMU from stable nixpkgs 2020-10-16 15:53:34 +02:00
Erik Arvstedt 03f8dbba47 test: add non-secure-node eval test
This tests that the modules work without the secure-node template.

The test currently fails at runtime, but evaluating already helps
catching module-related errors.
2020-10-16 15:53:34 +02:00
Erik Arvstedt bb763d6a26 run-tests: add 'eval' command 2020-10-16 15:53:34 +02:00
Erik Arvstedt 36358066e4 spark-wallet: don't disable tor when onion-service is disabled
This fixes modules-only usage.

We can leave enabling tor and tor.client to secure-node.nix, on which
spark-wallet has a strict dependency.
2020-10-16 15:53:33 +02:00
Erik Arvstedt c9251e72a1 README: add run-tests.sh to examples 2020-10-16 15:53:33 +02:00
Erik Arvstedt 5a565dff66 netns test: use netns ips from config 2020-10-16 15:53:33 +02:00
Erik Arvstedt ac95fe7c82 netns test: don't test recurring-donations
This service is not enabled and its netns doesn't exist.
2020-10-16 15:53:33 +02:00
Erik Arvstedt 84744f38d7 netns test: disable backup test 2020-10-16 15:53:33 +02:00
Erik Arvstedt fcc67da9f4 test: add container support 2020-10-16 15:53:33 +02:00
Erik Arvstedt e99b7edb8e run-tests: pass script args verbatim to command without word splitting
Needed for the upcoming 'container' command
2020-10-16 15:53:33 +02:00
Erik Arvstedt e7c397a485 run-tests: rename scriptDir -> testDir
Needed for container support.
2020-10-16 15:53:32 +02:00
Erik Arvstedt b552d17d55 run-tests: fix arg error messages
$1 was not substituted due to single quotes.
2020-10-16 15:53:32 +02:00
Erik Arvstedt 572967d3ad extra-container: pre-release -> 0.5-pre 2020-10-16 15:53:32 +02:00
Erik Arvstedt ac6cee5c12 pkgs: add extra-container 2020-10-11 19:40:26 +02:00
Erik Arvstedt be2127ae5b test: fix noConnections configs
- bitcoind: remove mkForce because otherwise the whole extraConfig is replaced
  by the value of mkForce.

- liquidd: don't disable 'listen' because it is entirely benign in offline
  mode, we also allow it for bitcoind.
2020-10-11 19:40:08 +02:00