Commit Graph
100 Commits
Author SHA1 Message Date
Erik Arvstedt 721ba1aeba python-packages: separate specific-versions pkgs
This simplifies maintenance.
2021-10-24 21:18:56 +02:00
Erik Arvstedt ec4a4dbe41 btcpayserver: fix whitelist security issue
Whitelisting localhost implicitly whitelists all inbound onion
connections. This prevents banning misbehaving inbound onion peers
and enables message `mempool` which can cause privacy leaks.

Instead, grant `download` as the single bitcoind whitelist permission, which
should be safe for onion peers.
Remove liquidd whitelisting because it doesn't support fine-grained permissions.

After a cursory glance at the nbxplorer code I think that nbxplorer
requires none of the other default whitelist permissions (noban, mempool,
relay).
Details: https://github.com/dgarage/NBXplorer/issues/344
2021-10-21 11:40:40 +02:00
Erik Arvstedt df2070b44a bitcoind: add separate p2p socket for tor connections
This re-enables onion tagging while still supporting untagged connections.

Onion sockets are not yet supported in the latest liquidd/elements
version 0.18.1.12 available on nixpkgs.
2021-10-21 11:40:40 +02:00
Erik Arvstedt ced1637d07 joinmarket: share IRC server definitions between jm and ob-watcher
Also add server name comments.
2021-10-13 14:44:36 +02:00
Erik Arvstedt 8938eadf0c bitcoind: don't tag all incoming connections as 'Tor'
We're also accepting local, non-Tor connections.
2021-10-06 16:55:41 +02:00
Erik Arvstedt b9301ce0d9 emergency fix: lnd: 0.13.1-beta -> 0.13.3-beta
Fixes CVE-2021-41593.
Temporarily switch to nixpkgs/master.
2021-10-06 15:34:24 +02:00
Erik Arvstedt 265fc1911d extra-container: pin to nixpkgs-unstable
extra-container is now part of nixpkgs.
2021-10-06 15:34:24 +02:00
Erik Arvstedt 75b89f3957 electrs: adapt to version 0.9.0
- `waitfornewblock` was previously not included in the public RPC
  whitelist because it's reserved for testing and marked as hidden
  in bitcoind.

- electrs changed its verbosity settings. `-vv` is now the best choice
  for normal usage.

- bitcoind option `dataDirReadableByGroup` is now unused.
  Because it can be valuable for other use cases and implementing
  it is intricate, we're keeping it for now.

- test: keep `nc` connection open because otherwise the electrs
  RPC server would now close the connection before sending a response.
2021-10-06 15:34:24 +02:00
Erik Arvstedt 6f42fa8181 update nixpkgs (electrs: 0.8.11 -> 0.9.0) 2021-10-06 15:34:24 +02:00
Erik Arvstedt 6d694a6269 backups: allow extraFiles to override default settings
By moving them to the top they take precedence over the remaining
filelist entries.
2021-10-06 11:27:52 +02:00
Erik Arvstedt 0c45415c86 backups: exclude bitcoind, liquidd txindex data 2021-10-06 11:27:52 +02:00
Erik Arvstedt 0853dedc43 tests/regtest: don't fail when restarting bitcoind 2021-10-06 11:27:52 +02:00
Erik Arvstedt b73c093d3d joinmarket-ob-watcher: require nix-bitcoin.service
This caused failures in the tests which were ignored because
ob-watcher was expected to fail for other reasons.
2021-10-06 11:27:47 +02:00
Erik Arvstedt 27905e2c3a tests: disable restarting joinmarket-ob-watcher
This removes the repeated failure messages from the test log.
2021-10-05 16:45:00 +02:00
Erik Arvstedt c8251cdad7 onion-services: don't always enable Tor
Previously, Tor was always enabled because `cfg` was always nonempty
(via definitions at `Set sensible defaults for some services`).
Now only enable Tor if there are active onion services.

Also rename var `services` -> `onionServices` to improve readability in
section `Set getPublicAddressCmd ...` where the same name is also used for
option `config.services`.
2021-10-05 15:11:41 +02:00
Erik Arvstedt 3c6a664b7b examples/configuration: show how to enable sudo/doas for operator
This was brought up by a user on IRC.
2021-10-04 00:33:27 +02:00
Erik Arvstedt 4d5bc810eb secrets: fix setup-secrets in case of no secrets
Previously, the glob (*) returned '*' when no files existed in the
secrets dir, leading to error `chown: cannot access '*'`.

Now `unprocessedFiles` is empty when there are no secrets.

Also remove the unneeded sorting of `unprocessedFiles` and
remove redundant leading zero in the default mode.
2021-10-04 00:33:27 +02:00
Erik Arvstedt e61c743644 test: add option extraTestScript
This allows extending tests.py.
2021-10-04 00:33:27 +02:00
Erik Arvstedt 2cf12d8765 README: minor fixes
- `packages and dependencies are pinned`
  Reworded and moved to `Integrity`
- `most packages are built from the NixOS stable channel...`
  Removed because this is false
- `builds happen in a sandboxed environment`
  Removed because this depends on the user's local nix configuration
  and cannot be enforced by nix-bitcoin
2021-10-04 00:33:27 +02:00
Erik Arvstedt e57ab83a51 docs/hardware: update
- Update pruning info
- Add Raspberry Pi 4
  Mentioned here: https://github.com/fort-nix/nix-bitcoin/issues/39#issuecomment-917418830
2021-10-04 00:33:27 +02:00
Erik Arvstedt 1b597f92a6 docs/hardware: add line breaks 2021-10-04 00:33:27 +02:00
Erik Arvstedt a92d6a8e80 netns: expose bridgeIp as an option
Previously, this variable was not accessible to other modules.
2021-10-04 00:33:26 +02:00
Erik Arvstedt f36df8f563 secure-node: remove redundant bitcoind settings
- `discover` is automatically disabled by bitcoind because we're
   setting `externalip` via the `nix-bitcoin.onionServices` mechanism
- `bech32` is bitcoind's default addresstype
2021-10-04 00:33:26 +02:00
Erik Arvstedt 09169365d8 liquid: remove unused features
- `hexStr` is unused
- Simplify ExecStart options
- Quote `dataDir`
- Remove unneeded `pidFile` setting
2021-10-04 00:33:26 +02:00
Erik Arvstedt 82d910e937 nbxplorer: fix bitcoind, liquidd settings
- Add nbxplorer to whitelists.
  This is recommended by the nbxplorer docs and guarantees that nbxplorer
  can always p2p-connect to bitcoind/liquidd.

- Enable bitcoind/liquidd p2p servers via `listen`.
2021-10-04 00:33:26 +02:00
Erik Arvstedt f61e928139 services: support 0.0.0.0/:: in address options
Previously, client services didn't decode these special INADDR_ANY
addresses and failed to connect.
2021-10-04 00:33:26 +02:00
Erik Arvstedt 1848c3dd98 btcpayserver: minor improvements
- Quote datadir
- Extract liquidd service variable
- Move btcpayserver below liquid in modules list because it depends
  on liquid
2021-10-01 11:52:57 +02:00
Erik Arvstedt e561637600 minor fixes
- bitcoind: Remove obsolete defaultText
- clightning: Fix description
  Option `address` can't be used to specify a socket path because it's
  used explicitly as an IP address in many places.
- lnd: Break up overlong line
  This is required by commit `services: support 0.0.0.0/:: in `address` options`
- nix-bitcoin.nix: Formatting
- secrets: Improve descriptions
2021-10-01 11:52:56 +02:00
Erik Arvstedt 46af0b1a8d examples: fix shell on non-Linux systems
extra-container is not supported on these systems.

Also mention that VM examples require Linux.
2021-09-26 23:58:46 +02:00
Erik Arvstedt 2a16db6919 readme: add 'Get started' section
Also, shorten the overview a bit by removing some redundancy:
- Remove sentence 'A Bitcoin node verifies ... network.' because
  its content is implied by the use cases (wallet, public infrastructure) mentioned in the
  following sentence.

- 'second layer public infrastructure' -> 'public infrastructure',
  because we're also serving the first layer.
2021-09-26 22:34:39 +02:00
Erik Arvstedt d713e7b15c examples: add importable-configuration.nix
This replaces minimal-configuration.nix.
importable-configuration.nix can be directly added to an existing
NixOS configuration. This makes it easy for users to get started quickly.
2021-09-26 22:34:39 +02:00
Erik Arvstedt 9114ec669a lnd: improve options formatting 2021-09-16 12:51:00 +02:00
Erik Arvstedt 1f8f2b2139 examples/shell.nix: Add shell version variable
This simplifies future shell upgrades.
2021-09-15 12:22:10 +02:00
Erik Arvstedt 0186b2a764 examples/shell.nix: Add upgrade note for NixOps users 2021-09-15 12:01:37 +02:00
Erik Arvstedt b49c74545f fetch-release: make GPG key searchable
The non-spaced version gives no matches in major search engines.
This is useful for auditing this script.
2021-09-14 19:56:35 +02:00
Erik Arvstedt 7356a34d88 docs/install.md: update 2021-09-14 19:56:35 +02:00
Erik Arvstedt 77af2e4538 makeShell: improve update-nix-bitcoin
- Don't overwrite `nix-bitcoin-release.nix` on errors
- Show a message to indicate whether `nix-bitcoin-release.nix` was
  updated
- Don't start a shell when called noninteractively

Also, update `usage.md` and reformat `shell.nix`.
2021-09-14 19:56:35 +02:00
Erik Arvstedt 52aaa8388e fetch-release: write error messages to stderr
Previously, when used to update `nix-bitcoin-release`, the error
wasn't displayed but instead written to `nix-bitcoin-release`.

Also, show curl error messages.
2021-09-14 19:56:23 +02:00
Erik Arvstedt c8774375d3 modules: use consistent service variables
Benefits of adding top-level variables for used services:
- Makes it obvious which other services are referenced by a service
- Less code

We already do this in many other places.
2021-09-13 13:41:47 +02:00
Erik Arvstedt ad97c268c6 modules: move user/group options to bottom
These are insignificant, generic options; place them above readonly options.
We already do this in other services.

Also move user/group config to bottom in spark-wallet.
2021-09-13 13:41:47 +02:00
Erik Arvstedt 27c45b82cc modules: move options to the top
This greatly improves readability and makes it easier to discover options.

This commit was genereated by running the following script inside the
repo root dir:

#!/usr/bin/env ruby

def transform(src)
  return false if src.include?('inherit options;')

  success = false

  options = nil
  src.sub!(/^  options.*?^  }.*?;/m) do |match|
    options = match
    "  inherit options;"
  end
  return false if !options

  src.sub!(/^with lib;\s*let\n+/m) do |match|
    success = true
    <<~EOF
      with lib;
      let
      #{options}

    EOF
  end

  success
end

Dir['modules/**/*.nix'].each do |f|
  src = File.read(f)
  if transform(src)
    puts "Changed file #{f}"
    File.write(f, src)
  end
end
2021-09-13 13:41:47 +02:00
Erik Arvstedt 731cf647ff modules: remove unneeded use of options module arg
Needed by the following commit.
2021-09-13 13:41:47 +02:00
Erik Arvstedt fdcb68e96e examples/shell.nix: add new commands 2021-09-12 11:29:54 +02:00
Erik Arvstedt a2466b1127 secrets: allow extending generate-secrets
`generate-secrets` is no longer a monolithic script. Instead, it's
composed of the values of option `nix-bitcoin.generateSecretsCmds`.

This has the following advantages:
- generate-secrets is now extensible by users
- Only secrets of enabled services are generated
- RPC IPs in the `lnd` and `loop` certs are no longer hardcoded.

Secrets are no longer automatically generated when entering nix-shell.
Instead, they are generated before deployment (via `krops-deploy`)
because secrets generation is now dependant on the node configuration.
2021-09-12 11:29:54 +02:00
Erik Arvstedt 24fd1e9bdc improve examples/shell.nix
The user's local node configuration directory usually contains a copy of
examples/shell.nix.

1. Move the shell implementation from shell.nix to nix-bitcoin/helper/makeShell.nix
   Because the shell is no longer defined locally in the user's config
   directory, we can now ship new shell features via nix-bitcoin updates.

2. Simplify examples/nix-bitcoin-release.nix
   nix-bitcoin-release.nix, as generated via `fetch-release`, now
   contains a simple fetchTarball statement which can be directly imported.
   This allows us to get rid of the extra `nix-bitcoin-unpacked` derivation
   which adds a dependency on the user's local nixpkgs.

   To keep `fetch-release` as simple as possible for easy auditing, we just
   fetch and verify a `nar-hash.txt` file that is now uploaded
   via `push-release.sh`.

A migration guide for updating the user's local `shell.nix` is
automatically printed when the user starts a new shell after updating
nix-bitcoin.
This is achieved by throwing an error in `generate-secrets`, which is called
on shell startup.

This commit is required to deploy the new extensible `generate-secrets`
mechanism introduced in the next commit.
2021-09-12 11:29:54 +02:00
Erik Arvstedt 8a757e0486 push-release.sh: improve --dry-run mode
- Don't require OAUTH_TOKEN
- Skip interactive query
2021-09-11 15:07:24 +02:00
Erik Arvstedt 82a2b148d8 secrets: minor fixes
- Improve comment.
- `secretsSetupMethod` is not internal because it can be set to "manual"
  by the user.
2021-09-11 15:07:24 +02:00
Erik Arvstedt e1e3d8a92b secrets: simplify cert generation
- Remove openssl.cnf which includes many unused settings.
- Generate the key and cert files with a single call to openssl.
  - Option `-nodes` ("no DES") disables encryption of the key file.
  - Option `-addext` is used to specify `subjectAltName` settings
    that were previously defined by openssl.cnf.

The key type is unchanged.
Certificate changes:
- Certificate duration is now 10 years
- Organization (subj 'O') is now 'loop' instead of 'loopd' for
  lightning-loop to simplify the code.
  For reference, the org. name in auto-generated loop certs is
  "loop autogenerated cert".
- The certificate now includes all default x509v3 extensions.
  These were previously restricted to just `subjectAltName` by openssl.cnf.
  We now use the openssl defaults for simplicity.
2021-09-11 15:07:24 +02:00
Erik Arvstedt 2c8e29b35b lnd: extract option certPath
Improves service encapsulation.
2021-09-11 15:07:24 +02:00
Erik Arvstedt be12a49933 lightning-pool/loop: extract lnd variable 2021-09-11 15:07:24 +02:00
Erik Arvstedt 955b44404c delete helper/fetch-channel
This script was obsoleted by switching to flakes.
2021-09-11 15:07:23 +02:00
Erik Arvstedt 5087ce245f minor cleanups
- btcpayserver: remove unneeded trailing semicolons

- krops/get-sha256:
  `tail` is unneeded because `nix-prefetch-url` just outputs a single
  line containing the hash.
2021-09-11 15:07:23 +02:00
Erik Arvstedt 0d2db4e79f backups: add option postgresqlDatabases
This simplifies defining postgresql backups.
This change is covered by tests.py.
2021-09-11 15:07:23 +02:00
Erik Arvstedt c35e96a553 joinmarket: update patch hash
The patch hash has changed due to an update of the PR branch.
The PR has now been merged.
2021-09-05 22:33:17 +02:00
Erik Arvstedt 32ce9d0ff4 run-tests: fix 'eval' command for newer versions of nix
There's no common `nix` command argument syntax for eval'ing a nix
expression that supports both older and newer (flake support) versions of nix.
So fall back to nix-instantiate.
2021-09-04 08:17:38 +02:00
Erik Arvstedt 926f1febb7 make-container: update extra-container version
Keep this file in sync with the latest extra-container update.
2021-09-04 08:17:38 +02:00
Erik Arvstedt 9730be9282 joinmarket-yieldgenerator: simplify start script 2021-08-30 13:37:05 +02:00
Erik Arvstedt 179b86d19c joinmarket: allow recreating wallet from seed
This allows users to easily upgrade their wallets to use Fidelity Bonds.
2021-08-30 13:37:05 +02:00
Erik Arvstedt 7c5ef32b50 versioning: move list of changes to the top
Improves readability.
2021-08-30 13:37:05 +02:00
Erik Arvstedt b15d71605e joinmarket: fix leaking passwords
Previously, `bitcoin-rpcpassword-privileged` and `jm-wallet-password` were
passed as world readable arguments to sed and jm-genwallet subprocesses.
2021-08-30 13:37:04 +02:00
Erik Arvstedt d7f9e33e1c joinmarket-ob-watcher: move resource files to extra dir
Don't clutter joinmarket/bin with ob-watcher resource files.
2021-08-30 13:37:04 +02:00
Erik Arvstedt dde04f8cbe update nixpkgs-unstable
Includes:
btcpayserver: 1.1.2 -> 1.2.0
lightning-loop: 0.14.2-beta -> 0.15.0-beta
nbxplorer: 2.1.52 -> 2.1.58
2021-08-26 12:45:10 +02:00
Erik Arvstedt 87df809a88 add helper/update-flake.sh
This greatly simplifies updating nixpkgs.
See the comment at the top of update-flake.sh for a description.
2021-08-26 12:45:10 +02:00
Erik Arvstedt f7c2133250 add flake support
This change is fully backwards compatible.

We continue to use the standard non-flake evaluation mode in our
examples and internal tooling until the flakes design has stabilized.

'clightning-plugins = pkgs.recurseIntoAttrs' in pkgs/default.nix is
needed by flake-utils.lib.flattenTree in flake.nix.
It transforms the packages in `clightning-plugins` to top-level packages
named like `clightning-plugins/summary`. (The flake attr `packages`
must be a non-nested attrset of derivations.)
2021-08-26 12:45:10 +02:00
Erik Arvstedt de77281cba pkgs: import pinned nixpkgs in default.nix
pkgs/default.nix now explicitly specifies all its dependencies as arguments.
This is required for flake support.

Also simplify pinned.nix and python-packages by removing unused attrs.
2021-08-16 10:43:07 +02:00
Erik Arvstedt b0c66c41e1 tests: add container-minimal example 2021-08-15 22:42:03 +02:00
Erik Arvstedt a8a8b9ce4d backups: backup NixOS uid, gid mappings
Now that service uid, gid mappings are included in the backups, along
with the service data dirs, we can remove 'chown -R' for
clightning and liquidd data dirs.

Note that we used 'chown -R' only for these two services, while this
approach would have been relevant for all services with data dirs.
2021-08-15 22:40:35 +02:00
Erik Arvstedt ee8b83681b modules: document module dependencies 2021-08-15 22:40:35 +02:00
Erik Arvstedt 9f7d048769 modules: move assertion to lnd.nix
nix-bitcoin.nix is now no longer dependent on clightning.nix and lnd.nix.
Due to condition '!(config.services ? clightning)' lnd.nix still
doesn't depend on clightning.nix.

Also fix the assertion message by renaming clightning.bindPort to clightning.port.
2021-08-15 22:40:35 +02:00
Erik Arvstedt cce9a3f6b2 modules: move nix-bitcoin options to file 'nix-bitcoin.nix'
This allows modules.nix to consist only of a list of modules.
2021-08-15 22:40:35 +02:00
Erik Arvstedt fdc278a0b8 lib: fix comment 2021-08-15 11:29:36 +02:00
Erik Arvstedt 13b4650e84 versioning: add usage comment 2021-08-15 11:29:36 +02:00
Erik Arvstedt ca3c7a281b secrets: mark option 'secretsSetupMethod' as internal 2021-08-15 11:29:36 +02:00
Erik Arvstedt f9a0fd7a17 nodeinfo: fix indentation 2021-08-15 11:29:36 +02:00
Erik Arvstedt 4ece606e8b examples/minimal-configuration: improve comment 2021-08-15 11:29:35 +02:00
Erik Arvstedt 6de9aba854 run-tests: quote scriptDir
scriptDir may contain spaces.
2021-08-15 11:29:35 +02:00
Erik Arvstedt 1ef8cbb384 joinmarket: fix allowRunAsUsers setting
This option requires user names instead of groups.
2021-08-15 11:29:35 +02:00
Erik Arvstedt fb36f2abe5 joinmarket-ob-watcher: use consistent mode formatting
Remove redundant leading zero.
2021-08-15 11:29:34 +02:00
Erik Arvstedt f14af1fc48 treewide: use consistent echo message formatting
Quote the echo message.
2021-08-15 11:29:34 +02:00
Erik Arvstedt b8043d3db5 treewide: use consistent bash script indentation
Always use two spaces.
2021-08-15 11:29:34 +02:00
Erik Arvstedt c758d68ea4 lib: rename privileged -> rootScript
The naming is now analogous the related function `script`.
2021-08-15 11:29:34 +02:00
Erik Arvstedt 1c3735b600 examples/README: add nixbitcoin.org server repo 2021-08-15 11:29:33 +02:00
Erik Arvstedt c041079ae1 configuration.nix: reorder sections
Move backups and netns-isolation to the end.
2021-08-15 11:29:33 +02:00
Erik Arvstedt 3734ab38a6 configuration.nix: improve wording and formatting 2021-08-15 11:29:33 +02:00
Erik Arvstedt a2454975a5 doas: fix recursive calls to doas
Doas was broken for recursive calls like `doas -u operator lncli`
where `lncli` internally calls doas.
2021-08-14 10:46:42 +02:00
Erik Arvstedt 7c876664b1 netns test: update matching of 'capsh' output
The output now contains multiple lines.
2021-08-14 10:46:42 +02:00
Erik Arvstedt 308a11f22b tests: avoid postgresql timeout failures on CI nodes 2021-08-14 10:46:42 +02:00
Erik Arvstedt 01804e6dfb tests: improve test script formatting
Remove annyoing spacing constraints enforced by the previous 'black' linter.
2021-08-14 10:46:42 +02:00
Erik Arvstedt 1be924529d tests: adapt to new linter
The Python test driver now uses 'pyflakes'.
Remove hacks that were needed for the 'black' linter.
2021-08-14 10:46:42 +02:00
Erik Arvstedt c1c663d0a9 tests: fix formatting 2021-08-14 10:46:42 +02:00
Erik Arvstedt c4c2b03e19 extra-container: 0.6 -> 0.7
Version 0.7 adds support for NixOS 21.05.
2021-08-14 10:46:41 +02:00
Erik Arvstedt 161baa7e68 joinmarket-ob-watcher: allow required 'mbind' system call 2021-08-14 10:46:41 +02:00
Erik Arvstedt ca64a4a64f clightning-plugins.prometheus: use current nixpkgs version of prometheus-client 2021-08-14 10:46:41 +02:00
Erik Arvstedt 3aab1fc267 spark-wallet: update to new node-env 2021-08-14 10:46:41 +02:00
Erik Arvstedt a0e5894f1f backups: remove illegal option definition 2021-08-14 10:46:41 +02:00
Erik Arvstedt 35fe939cf8 security: update /proc restriction mechanism
NixOS option `security.hideProcessInformation` for globally restricting
access to /proc has been removed.
Use per-service restrictions via 'ProtectProc' instead.

Rename
`nix-bitcoin.security.hideProcessInformation` to
`nix-bitcoin.security.dbusHideProcessInformation`
because this option now only implements the dbus restriction.
2021-08-14 10:46:41 +02:00
Erik Arvstedt 178a0dcf8f services: use new 'tor' options 2021-08-14 10:46:41 +02:00
Erik Arvstedt e44f78ebb8 services: set isSystemUser for service users
'isSystemUser' has to be explicitly set in NixOS 21.05.
Previously, it was the implicit default.
2021-08-14 10:46:40 +02:00
Erik Arvstedt 0ef66c920b treewide: use services.getty option
services.mingetty is equivalent but deprecated.
2021-08-14 10:46:40 +02:00
Erik Arvstedt a25ceecca5 update to NixOS 21.05 2021-08-12 11:18:26 +02:00