Commit Graph
100 Commits
Author SHA1 Message Date
Erik Arvstedt 54a21874ae nodeinfo/lnd: add rest_address 2023-03-11 20:10:55 +01:00
Erik Arvstedt a4bfefd562 add presets/wireguard.nix
This allows using `lndconnect` via a direct WireGuard connection.
2023-03-11 20:10:54 +01:00
Erik Arvstedt 477e1709fb lndconnect: update to Zeus 0.7.1
- Generate lndconnect URLs with protocol `c-lightning-rest` for clightning.
  (Zeus now auto-detects the lightning implementation by the URL protocol.)
- Use improved QR code format (via qrencode)  .
2023-02-27 14:39:29 +01:00
Erik Arvstedt f996ef37d9 lnd, clightning-rest: remove lndconnectOnion, add generic option lndconnect
For both lnd and clightning-rest, `lndconnectOnion` is replaced by
options `lndconnect.enable` and `lndconnect.onion`.

This allows using lndconnect without Tor.
2023-02-27 14:39:29 +01:00
Erik Arvstedt b4bc621b8c rename lndconnect-onion.nix -> lndconnect.nix 2023-02-27 14:39:29 +01:00
Erik Arvstedt 907cfe4f4c docs/services: improve title, fix numbering 2023-02-27 14:39:29 +01:00
Erik Arvstedt 3a2730d287 btcpayserver: support restarting from the web interface
This is required since version 1.7.4.
See: https://github.com/btcpayserver/btcpayserver/releases/tag/v1.7.4
2023-02-27 12:44:08 +00:00
Erik Arvstedt bf8de6744e README: add mempool extension module 2023-02-19 22:02:23 +01:00
Erik Arvstedt c12ff6251d README: add some module descriptions 2023-02-04 13:22:41 +01:00
Erik Arvstedt 2c3fa63baa dev/features: improve enter_service
Read uid/gid directly from the service pid.

This makes this fn work with arbitrary services, and with `bitcoind`,
where, for historical reasons, the service user name (`bitcoin`) doesn't
equal the service name.
2023-02-04 13:22:41 +01:00
Erik Arvstedt addfa8ec6b test: support run, debug commands in basic NixOS tests
Currently, this only affects the basic NixOS test `clightning-replication`.
2023-02-02 10:51:41 +01:00
Erik Arvstedt ae733d887e tests/clightning-replication: reuse pkgs instance
This reduces eval time by 30%.
2023-02-02 10:51:41 +01:00
Erik Arvstedt 6cbd0d93ae tests: rename clightningReplication -> clightning-replication
The test name now matches the file name.
2023-02-02 10:51:41 +01:00
Erik Arvstedt 85310b533a secrets: use type lines for generateSecretsCmds
This allows users to amend secrets cmds.
2023-02-02 10:51:41 +01:00
Erik Arvstedt bc2f66d4f1 bitcoind, liquid: increase start/stop timeouts 2023-02-02 10:51:41 +01:00
Erik Arvstedt 519ae31202 netns-isolation: improve formatting 2023-02-02 10:51:41 +01:00
Erik Arvstedt a1023696e6 netns-isolation: reserve netns id for mempool
This allows using the old id in the extension flake, so that
existing configs are not changed.
2023-02-02 10:51:41 +01:00
Erik Arvstedt 34fe8675bd add option nix-bitcoin.pkgOverlays
This simplifies extending `nix-bitcoin.pkgs` and is required for
extension flakes.
For now, mark this as `internal`.
2023-02-02 10:51:40 +01:00
Erik Arvstedt a3bdecb10b helper: add start-bash-session.sh 2023-02-02 10:51:40 +01:00
Erik Arvstedt 6291d4fbea rtl: 0.13.2 -> 0.13.4 2023-01-26 23:08:05 +01:00
Erik Arvstedt 690a8f6256 nodeinfo: extract fn mkInfoLong
This is required by the mempool extension flake.
2023-01-26 11:17:03 +01:00
Erik Arvstedt 2af642f56a improve comments
The comment in python-packackges was obsolete.
2023-01-26 11:17:02 +01:00
Erik Arvstedt 56c2abd91a update nixpkgs
btcpayserver: 1.7.2 -> 1.7.3
electrs: 0.9.10 -> 0.9.11
hwi: 2.1.1 -> 2.2.0
2023-01-25 23:57:29 +01:00
Erik Arvstedt 86dc7e2669 fulcrum: allow access to /proc/meminfo
This still hides the proc subdirectories for other processes.

Without this setting, fulcrum fails when the config value of
`fast-sync` is greater than 2^31 bytes.
2023-01-21 13:28:32 +01:00
Erik Arvstedt c948af2e18 dev/dev-features: add enter_service helper 2023-01-21 13:20:49 +01:00
Erik Arvstedt 5634f08873 rtl: make extraConfig recursively mergeable
Previously, when merging different definitions of `extraConfig`,
only the top-level attrset was merged.

Example:
The two separate settings
  nodes.lnd.extraConfig.Settings.userPersona = "MERCHANT";
  nodes.lnd.extraConfig.Settings.logLevel = "DEBUG";
were previously merged into
  nodes.lnd.extraConfig.Settings = { logLevel = "DEBUG" };
(The last definition has precedence.)
2023-01-20 13:46:08 +01:00
Erik Arvstedt b76728a1ec treewide: use bool literals for systemd
Run this from the repo root to check that there are no more remaining
bool strings:
grep -P '"true"|"false"' -r --exclude-dir=.git
2023-01-20 13:46:08 +01:00
Erik Arvstedt b4d7e1aa8f add dev helper and docs 2023-01-15 20:28:49 +01:00
Erik Arvstedt b35d08d3f2 docs: move test docs from examples/README to test/README 2023-01-15 20:28:48 +01:00
Erik Arvstedt 4d76eb9183 docs/configuration: fix typo 2023-01-15 20:28:48 +01:00
Erik Arvstedt dc0710f3f4 tests: add example scenario customTest 2023-01-15 20:28:48 +01:00
Erik Arvstedt 9019a17bfc versioning: add fulcrum db change info 2023-01-07 19:32:54 +00:00
Erik Arvstedt 9e30d2728b tests: formatting
Move line next to `services.lnd` config for clarity.
2023-01-06 23:46:43 +01:00
Erik Arvstedt c6d85c6fe3 tests: fix broken unit file when clightning is disabled
Previously, an incomplete clightning unit was always created because
attr `clightning` was always defined in option attrset `systemd.services`.
2023-01-06 23:46:43 +01:00
Erik Arvstedt a51f7b419e run-tests: use arg instead of env var for scenario overrides
This removes a source of implicit state and guarantees that regular
calls to `run-tests.sh` always run the builtin tests.
2023-01-06 23:46:43 +01:00
Erik Arvstedt 4b5b4eac58 examples/deploy-container: fix sudo env propagation
Env vars can't be reliably passed through `sudo`, so always
call nix-shell to setup the env after running sudo.
2023-01-06 23:23:54 +01:00
Erik Arvstedt 8d476cfeaf nix-bitcoin/runAsUserCmd: remove workaround 2023-01-03 16:18:27 +01:00
Erik Arvstedt 314020b246 rtl: 0.13.1 -> 0.13.2 2022-12-20 23:24:24 +01:00
Erik Arvstedt 00cceca861 joinmarket: fix Python packages 2022-12-18 20:01:53 +01:00
Erik Arvstedt e4b8e14d3a clightning: fix Python packages
Patching `pyln-proto` to use cryptography 38 lets
us avoid adding many older Python pkg versions.

The backwards incompatible changes from cryptography 36 to 38
only include the removal of deprecated fns that pyln-proto
doesn't use.
See string "BACKWARDS INCOMPATIBLE" in
https://cryptography.io/en/latest/changelog/
2022-12-18 20:01:53 +01:00
Erik Arvstedt d1ef2a6e1e pythonPackages: improve layout
- Move the creation of the joinmarket Python pkgs from
  `joinmarket/default.nix` to `pkgs/python-packages/default.nix`.

- Move definitions of old pkg versions from the main Python pkgs
  to the joinmarket Python pkgs.
  These old versions are only required by joinmarket.
2022-12-18 20:01:52 +01:00
Erik Arvstedt 74c8593407 pythonPackages: add indentation
This makes the following commit more readable.
2022-12-18 20:01:52 +01:00
Erik Arvstedt 109dccca27 treewide: use mdDoc for descriptions
Enable markdown syntax (instead of docbook) for descriptions.
This only affects external doc tooling that renders the descriptions.
2022-12-18 20:01:52 +01:00
Erik Arvstedt a9c1995ed9 treewide: rename maintainer earvstedt -> erikarvstedt 2022-12-18 20:01:52 +01:00
Erik Arvstedt 9e456ea3a9 shellcheck-services.nix: update to NixOS 22.11 2022-12-18 20:01:52 +01:00
Erik Arvstedt 77d58162e7 test: update to NixOS 22.11 2022-12-18 20:01:52 +01:00
Erik Arvstedt 142cbcfb37 flake: remove 32-bit systems 2022-12-18 20:01:52 +01:00
Erik Arvstedt c9b1e59f20 update to NixOS 22.11
This includes no pkg version updates.
2022-12-18 20:01:52 +01:00
Erik Arvstedt 62515a5696 helper/update-flake: support updating NixOS versions 2022-12-18 20:01:48 +01:00
Erik Arvstedt c5493717b7 lnd: support INADDR_ANY addresses for bitcoind.zmqpubraw*
Also use `mkDefault` when defining `bitcoind.zmqpubraw*` to simplify
overriding for users.
2022-11-11 12:10:00 +01:00
Erik Arvstedt 0447c5bacb treewide: set shebang for bash scripts
These scripts previously failed when called with syscalls like
`execve` (used by, e.g., Python's `subprocess.run`) that use no default
interpreter for scripts without a shebang.
2022-11-08 23:04:56 +01:00
Erik Arvstedt 0de16095e1 clightning-replication: switch system before waiting for server sshd
This is primarily a cosmetic change.
- Increases code clarity because all system test blocks now start with `switch_to_system`
- Optimizes dependency ordering because `switch_to_system` has no
  dependency on the server sshd
2022-11-04 11:51:44 +01:00
Erik Arvstedt d332177d3e clightning: extract var bitcoind
Follow the default module formatting style.
2022-11-04 11:07:36 +01:00
Erik Arvstedt 1b5e51b7fe examples/vm-config: fix syntax error 2022-11-04 00:33:53 +01:00
Erik Arvstedt 565deb770a examples/minimal-vm: add lightning-cli demo command 2022-11-04 00:33:31 +01:00
Erik Arvstedt edbaeb9813 tests: define tests via flake
Advantages:
- Pure test evaluations
- The test framework can now be used by flakes that extend nix-bitcoin
- Most features of `run-tests.sh` are now accessible via `nix build`/`nix run`.
  We keep `run-tests.sh` for advanced features like `scenarioOverridesFile` and adhoc scenarios.

Other changes:
- `run-tests.sh` now builds aggregate VM tests like `basic` or
  `buildable` by creating all VMs in a single evaluation.
  This speeds up the tests and eases debugging by separating the eval and build steps.
- Use the new `nix` CLI which has improved build output logging
  by prefixing output lines with the origin drv name.
2022-11-03 23:08:06 +01:00
Erik Arvstedt 90e942e5ae nodeinfo: rename nodeinfoLib -> lib 2022-11-03 23:08:06 +01:00
Erik Arvstedt 8eaa4cce30 tests: move mkIfTest to nix-bitcoin.lib 2022-11-03 23:08:06 +01:00
Erik Arvstedt 47a09ec214 flake: expose supportedSystems 2022-11-03 23:08:05 +01:00
Erik Arvstedt b0dfa69e84 nixos-search/flake: formatting 2022-11-03 23:08:05 +01:00
Erik Arvstedt d428755399 flake: rename input nixpkgsUnstable -> nixpkgs-unstable
This follows common flake naming conventions.
2022-11-03 23:08:05 +01:00
Erik Arvstedt a12b701e75 tests/container: don't require services.clightning to be defined 2022-11-03 23:08:05 +01:00
Erik Arvstedt 450de19803 tests/run-tests.sh: print examples before running
This eases debugging example failures.
2022-11-03 23:08:05 +01:00
Erik Arvstedt 5f1bb2a8fc tests/copy-src: always copy .git dir
This is required by a later commit that introduces flakes-based test
evaluation. Evaluating local flakes needs a repo dir.
2022-11-03 23:08:05 +01:00
Erik Arvstedt a87a59a86b make-container.sh: improve root handling
Don't auto-switch to root when executing make-container.sh, because
auto root switching is also implemented in extra-container.

Besides simplifying the code, this is useful for a later commit that
introduces flakes-based container building.
With this change, the container is built under the regular user
instead of root, thereby utilizing the user's regular fetcher and
evaluation caches.
2022-11-03 23:08:05 +01:00
Erik Arvstedt b616d7ac1b profiles/hardened: support pure eval mode 2022-11-03 23:08:05 +01:00
Erik Arvstedt 73d2fbb448 add compatibility with Nix PR #6530 (Source tree abstraction)
Avoid adding flake resource paths to the store (via string
interpolation).
This reduces performance and can lead to modules getting imported
twice, once through a local path and once through a store path.

This might not be needed in a future Nix release, in which case we can
revert this.
2022-11-03 23:08:05 +01:00
Erik Arvstedt 3c816b862c tests/vmWithoutTests: poweroff on shell exit
This allows quitting the VM with Ctrl-D like in the minimal example VM.
2022-11-03 23:08:04 +01:00
Erik Arvstedt 1d3f49f8da tests, example: avoid lengthy documentation build
This options manual rebuild takes 30-60s and is triggered by the extra
NixOS options defined by nix-bitcoin.
2022-11-03 23:08:04 +01:00
Erik Arvstedt b840548d40 test/shellcheck-services: add configurable source prefix
This allows using this module for services defined outside of nix-bitcoin.
2022-11-03 23:08:04 +01:00
Erik Arvstedt b412de3ad7 bitcoind: fix rare startup error
Previously, dhcpcd and bitcoind starting up in parallel could lead to
the following error in bitcoind:
```
bitcoind: libevent: getaddrinfo: address family for nodename not supported
bitcoind: Binding RPC on address 127.0.0.1 port 8332 failed.
bitcoind: Unable to bind any endpoint for
```
After the initial failure, the bitcoind service would always restart successfully.

This race condition, where both applications were simultaneously
manipulating network resources, was only triggered under specific
hardware conditions.

Fix it by running bitcoind after dhcp has started (by running after
`network-online.target`).
This bug and the fix only affect the default NixOS scripted
networking backend.
2022-11-02 12:02:03 +01:00
Erik Arvstedt 57b76d4461 pkgs: add lnd 0.15.4 (hotfix)
Includes an emergency hotfix:
https://github.com/lightningnetwork/lnd/releases/tag/v0.15.4-beta
2022-11-01 14:12:56 +01:00
Erik Arvstedt 67949a002a lnd: fix missing RPC permissions when bitcoind is pruned 2022-10-25 22:56:51 +02:00
Erik Arvstedt 49303be2e0 test/shellcheck-services: fix error by excluding unavailable services 2022-10-25 22:36:30 +02:00
Erik Arvstedt 46f17fe313 test/shellcheck-services: simplify accessing service definitions
This also improves performance by removing the extra module evaluation.
2022-10-25 22:36:30 +02:00
Erik Arvstedt 277510c7ee tests: run flake-info in sandbox
Don't use sandboxing in Cirrus CI where namespace support is missing.
2022-10-25 22:04:17 +02:00
Erik Arvstedt c88acbb1bb btcpayserver: use new option certfilepath for lnd 2022-10-24 12:47:01 +02:00
Erik Arvstedt 13a835e88f Revert "pkgs: add lnd 0.15.2"
This reverts commit cf836b5d3b.
2022-10-24 11:50:36 +02:00
Erik Arvstedt 3549725b51 update nixpkgs
btcpayserver: 1.6.10 -> 1.6.12
clightning: 0.12.0 -> 0.12.1
fulcrum: 1.8.1 -> 1.8.2
nbxplorer: 2.3.33 -> 2.3.41
2022-10-24 11:49:03 +02:00
Erik Arvstedt 61c539d5b6 defaultHardening: allow syscall set_mempolicy
This syscall is safe to allow.
It's required by the dotnet runtime (btcpayserver, nbxplorer) update
introduced in the following commit.
2022-10-22 23:54:08 +02:00
Erik Arvstedt cf836b5d3b pkgs: add lnd 0.15.2
Includes an emergency hotfix:
https://github.com/lightningnetwork/lnd/releases/tag/v0.15.2-beta
2022-10-10 13:27:49 +02:00
Erik Arvstedt d3b7e8c432 revert "tests: disable nixosSearch" 2022-09-23 09:04:57 +02:00
Erik Arvstedt c3b97e6728 tests: add shellcheckServices 2022-09-12 21:00:00 +02:00
Erik Arvstedt 01fa900633 shellcheck: fix setup-secrets.sh, spark-wallet 2022-09-12 21:00:00 +02:00
Erik Arvstedt 49e30f7172 test/lib/create-git-repo: fix empty var expansion
When `amend` is empty, expand it to nothing instead of an empty
string (which causes a git error).
2022-09-07 11:02:15 +02:00
Erik Arvstedt 7985e075e5 README: remove nixos-search 2022-09-07 10:24:16 +02:00
Erik Arvstedt c63da4734e tests: disable nixosSearch 2022-09-05 21:11:15 +02:00
Erik Arvstedt 67a573dae8 create-git-repo.sh: fix amend error if there's no branch checked out
In this case, `git commit --amend` fails.

The state of a .git repo existing without a branch being checked out
can be reached through the following sequence of events (which I just
seem to have triggered):
1. The script calls `git init`
2. The user interrupts the script before `git commit` is finished
2022-08-29 00:12:19 +02:00
Erik Arvstedt d9eecb7d4c nixos-search/flake: remove unused output 2022-08-29 00:12:19 +02:00
Erik Arvstedt 4e70db14bc README: fix capitalization 2022-08-29 00:12:18 +02:00
Erik Arvstedt 030be842c8 secrets: use fetchurl derivation
This moves fetching from eval time to build time, which is a better
fit for fetching build time resources.
2022-08-29 00:12:17 +02:00
Erik Arvstedt 1713e016a0 clightning-replication: clarify comment about FUSE mounts 2022-08-28 23:49:12 +02:00
Erik Arvstedt cd3878e357 delete modules/default.nix
This file has been missing some modules for quite some time.
Remove it to simplify maintenance.
2022-08-28 23:49:12 +02:00
Erik Arvstedt f2a4d1fe65 docs/security: fix typo 2022-08-28 23:49:12 +02:00
Erik Arvstedt 3a65dc0dc9 docs/configuration: improve section Remote bitcoind 2022-08-28 23:49:12 +02:00
Erik Arvstedt af115d746b internal scripts: use pinned, cached pkgs
Instead of setting up the script PATH via nix-shell, use
`nix shell` with inputs from the nix-bitcoin flake.
Advantages:
- Uses the nixpkgs version from the nix-bitcoin flake instead of
  `<nixpkgs>` from the user env (NIX_PATH), so the script runtime
  env is reproducible.
- The pkg derivations for the runtime env are cached, which greatly
  increases script startup speed.

This commit was generated by running the following script inside the
repo root dir:

def transform(path, src)
  if src =~ /#! *nix-shell +-i +bash +-p +(.*)/
    pkgs = $1
    if src =~ /^.*?(set -e.*?pipefail)\n/
      set_statement = $1
      src.sub!($&, '')
    end
    src.sub!(/\A.*?#! *nix-shell.*?\n/m, '')

    parents = ([ '..' ] * (path.split('/').count - 1)).join('/')

    [
      '#!/usr/bin/env bash',
      *set_statement,
      %(. "${BASH_SOURCE[0]%/*}/#{parents}/helper/run-in-nix-env" "#{pkgs}" "$@"),
      nil,
      src
    ].join("\n")
  end
end

Dir['**/*.sh'].each do |f|
  src = File.read(f)
  if new_src = transform(f, src)
    puts "Changed file #{f}"
    File.write(f, new_src)
  end
end
2022-08-28 23:49:12 +02:00
Erik Arvstedt b625325487 ci/modules_test: use flake-enabled nix 2022-08-28 18:25:37 +02:00
Erik Arvstedt 02f1599503 clightning-rest: 0.8.0 -> 0.9.0 2022-08-27 14:09:48 +02:00
Erik Arvstedt c3769014b1 rtl: 0.13.0 -> 0.13.1 2022-08-27 14:09:48 +02:00
Erik Arvstedt db3c0f0519 img/logo: add LICENSE 2022-08-26 10:22:29 +02:00