This enables generating module option documentation.
This commit was genereated by running the following script inside the
repo root dir:
def add_default_text(file)
src = File.read(file)
src2 = src.gsub(/( = mkOption\s+\{[^{]*?)(\n\s+default = )(.*?);$(.*?\})/m) do |str|
pre, defaultVar, default, post = Regexp.last_match.captures
replacement =
if !post.include?('defaultText =')
if default =~ /\bpkgs\b/
defaultText = default.lines.length == 1 ? default : "(See source)"
"#{pre}#{defaultVar}#{default};#{defaultVar.sub('default', 'defaultText')}#{defaultText.inspect};#{post}"
end
end
replacement or str
end
File.write(file, src2) if src2 != src
end
Dir["modules/**/*.nix"].each do |f|
next if File.basename(f) == "nix-bitcoin.nix"
add_default_text f
end
This enables generating module option documentation.
Add `defaultText` to options where the default value can't be
evaluated during metadata generation.
This simplifies integrating a remote bitcoind instance and
makes `bitcoin-cli` work with the remote node.
Add note regarding `whitelistedPort` to docs.
- Fix firewall.allowedTCPPorts settings
- Section `Allow bitcoind RPC connections`:
Set catch-all listen for `rpc.address` instead of `address`.
- rpc.allowip: Set subnet zero to fix allowing all addresses
The `nix` command is an experimental feature and should be avoided in
user-facing code.
I recently saw `nix eval` failing for a user who was asking for help
via Matrix.
- Move section `updating` from `usage.md` to `configuration.md` and
rename `usage.md` -> `services.md`.
`services.md` documents how enable and interact with node services.
- README: Move `docs` below `Get started`. The `docs` section is short and
should be easily accessible.
This is simpler and more memory-efficient.
We've also changed other services to use this appraoch.
Also remove unneded `wait_for_unit` in the electrs regtest test.
Split `enforceTor` into `tor.proxy` and `tor.enforce`.
By enabling `tor.proxy` without `tor.enforce`, a service can accept
incoming clearnet connections.
E.g., this allows setting up a Tor-proxied bitcoind node that accepts
RPC connections from LAN.
When NixOS is already running and Tor is restarted due to config
changes, `/var/lib/tor/state` may be present even when Tor has not
yet finished setting up onion services.
This caused the previous version of `onion-addresses` to not wait for
Tor and to skip not yet present onion service files.
`onion-addresses` now waits until each required onion service file
has appeared.
- Improves readability
- `netns exec ...` (called via `netnsIptables`) incurs a large
overhead: In addition to netns switching, a mount namespace
is setup and populated with the contents of /etc/netns/<ns>/.
Instead, simply run `nsenter`.
Previously, restarting a service implied restarting its netns.
For unknown reasons, this sometimes caused the netns-local address
to not be routable from the root netns for up to 20 s.
I.e., the service was sometimes unreachable after restarting.
Now the netns is no longer stopped when the service is stopped.
- import pkgs without the global config to avoid pulling in external state
- rename `path` -> `setPath`
- export `nixpkgsUnstable`
This avoids garbage collection of nixpkgs-unstable for gcrooted
shell environments (like those created by lorri)
- The message is now a nix string, which simplifies formatting.
- The message can be now be modified via overrideAttrs in client shells.
This is more effective than changing the message in Bash.
Otherwise liquidd startup fails. This bug was not covered by our tests,
because we're not combining `regtest` with `secure-node`.
But nixbitcoin.org does, which should suffice for now.
This simplifies the host's address configuration.
This also removes unused addresses that are returned when resolving
container hostnames via nss-mymachines:
`getent ahosts nb-test`
- README: add matrix room
- examples/configuration.nix: explain why bitcoind is enabled by default
- btcpayserver: group lnd service settings
- clightning:
Use public onion port only when the onion service is public
This allows users to enable the onion service while announcing a
non-onion public address.
- netns-isolation: move `readOnly` attr to the top
- tests: use mkDefault to allow for easier overriding
- tests/btcpayserver: test web server response
This allows whitelisting local services without implicitly
whitelisting all inbound onion connections, which would happen when
setting bitcoind/liquidd option `whitelist=localhost`.
Used by electrs and nbxplorer, which requires the unsafe `mempool`
permission.
This removes the module-level dependency from onion-services to
bitcoind.
Due to the `or false` fallback, there's no dependency added in
the reverse direction.
In particular, this allows us to not add a dependency on liquidd in
the following commit.
Whitelisting localhost implicitly whitelists all inbound onion
connections. This prevents banning misbehaving inbound onion peers
and enables message `mempool` which can cause privacy leaks.
Instead, grant `download` as the single bitcoind whitelist permission, which
should be safe for onion peers.
Remove liquidd whitelisting because it doesn't support fine-grained permissions.
After a cursory glance at the nbxplorer code I think that nbxplorer
requires none of the other default whitelist permissions (noban, mempool,
relay).
Details: https://github.com/dgarage/NBXplorer/issues/344
This re-enables onion tagging while still supporting untagged connections.
Onion sockets are not yet supported in the latest liquidd/elements
version 0.18.1.12 available on nixpkgs.
- `waitfornewblock` was previously not included in the public RPC
whitelist because it's reserved for testing and marked as hidden
in bitcoind.
- electrs changed its verbosity settings. `-vv` is now the best choice
for normal usage.
- bitcoind option `dataDirReadableByGroup` is now unused.
Because it can be valuable for other use cases and implementing
it is intricate, we're keeping it for now.
- test: keep `nc` connection open because otherwise the electrs
RPC server would now close the connection before sending a response.
Previously, Tor was always enabled because `cfg` was always nonempty
(via definitions at `Set sensible defaults for some services`).
Now only enable Tor if there are active onion services.
Also rename var `services` -> `onionServices` to improve readability in
section `Set getPublicAddressCmd ...` where the same name is also used for
option `config.services`.
Previously, the glob (*) returned '*' when no files existed in the
secrets dir, leading to error `chown: cannot access '*'`.
Now `unprocessedFiles` is empty when there are no secrets.
Also remove the unneeded sorting of `unprocessedFiles` and
remove redundant leading zero in the default mode.
- `packages and dependencies are pinned`
Reworded and moved to `Integrity`
- `most packages are built from the NixOS stable channel...`
Removed because this is false
- `builds happen in a sandboxed environment`
Removed because this depends on the user's local nix configuration
and cannot be enforced by nix-bitcoin
- `discover` is automatically disabled by bitcoind because we're
setting `externalip` via the `nix-bitcoin.onionServices` mechanism
- `bech32` is bitcoind's default addresstype
- Add nbxplorer to whitelists.
This is recommended by the nbxplorer docs and guarantees that nbxplorer
can always p2p-connect to bitcoind/liquidd.
- Enable bitcoind/liquidd p2p servers via `listen`.
- bitcoind: Remove obsolete defaultText
- clightning: Fix description
Option `address` can't be used to specify a socket path because it's
used explicitly as an IP address in many places.
- lnd: Break up overlong line
This is required by commit `services: support 0.0.0.0/:: in `address` options`
- nix-bitcoin.nix: Formatting
- secrets: Improve descriptions
Also, shorten the overview a bit by removing some redundancy:
- Remove sentence 'A Bitcoin node verifies ... network.' because
its content is implied by the use cases (wallet, public infrastructure) mentioned in the
following sentence.
- 'second layer public infrastructure' -> 'public infrastructure',
because we're also serving the first layer.
This replaces minimal-configuration.nix.
importable-configuration.nix can be directly added to an existing
NixOS configuration. This makes it easy for users to get started quickly.
- Don't overwrite `nix-bitcoin-release.nix` on errors
- Show a message to indicate whether `nix-bitcoin-release.nix` was
updated
- Don't start a shell when called noninteractively
Also, update `usage.md` and reformat `shell.nix`.
Previously, when used to update `nix-bitcoin-release`, the error
wasn't displayed but instead written to `nix-bitcoin-release`.
Also, show curl error messages.
Benefits of adding top-level variables for used services:
- Makes it obvious which other services are referenced by a service
- Less code
We already do this in many other places.
These are insignificant, generic options; place them above readonly options.
We already do this in other services.
Also move user/group config to bottom in spark-wallet.
This greatly improves readability and makes it easier to discover options.
This commit was genereated by running the following script inside the
repo root dir:
#!/usr/bin/env ruby
def transform(src)
return false if src.include?('inherit options;')
success = false
options = nil
src.sub!(/^ options.*?^ }.*?;/m) do |match|
options = match
" inherit options;"
end
return false if !options
src.sub!(/^with lib;\s*let\n+/m) do |match|
success = true
<<~EOF
with lib;
let
#{options}
EOF
end
success
end
Dir['modules/**/*.nix'].each do |f|
src = File.read(f)
if transform(src)
puts "Changed file #{f}"
File.write(f, src)
end
end
`generate-secrets` is no longer a monolithic script. Instead, it's
composed of the values of option `nix-bitcoin.generateSecretsCmds`.
This has the following advantages:
- generate-secrets is now extensible by users
- Only secrets of enabled services are generated
- RPC IPs in the `lnd` and `loop` certs are no longer hardcoded.
Secrets are no longer automatically generated when entering nix-shell.
Instead, they are generated before deployment (via `krops-deploy`)
because secrets generation is now dependant on the node configuration.
The user's local node configuration directory usually contains a copy of
examples/shell.nix.
1. Move the shell implementation from shell.nix to nix-bitcoin/helper/makeShell.nix
Because the shell is no longer defined locally in the user's config
directory, we can now ship new shell features via nix-bitcoin updates.
2. Simplify examples/nix-bitcoin-release.nix
nix-bitcoin-release.nix, as generated via `fetch-release`, now
contains a simple fetchTarball statement which can be directly imported.
This allows us to get rid of the extra `nix-bitcoin-unpacked` derivation
which adds a dependency on the user's local nixpkgs.
To keep `fetch-release` as simple as possible for easy auditing, we just
fetch and verify a `nar-hash.txt` file that is now uploaded
via `push-release.sh`.
A migration guide for updating the user's local `shell.nix` is
automatically printed when the user starts a new shell after updating
nix-bitcoin.
This is achieved by throwing an error in `generate-secrets`, which is called
on shell startup.
This commit is required to deploy the new extensible `generate-secrets`
mechanism introduced in the next commit.
- Remove openssl.cnf which includes many unused settings.
- Generate the key and cert files with a single call to openssl.
- Option `-nodes` ("no DES") disables encryption of the key file.
- Option `-addext` is used to specify `subjectAltName` settings
that were previously defined by openssl.cnf.
The key type is unchanged.
Certificate changes:
- Certificate duration is now 10 years
- Organization (subj 'O') is now 'loop' instead of 'loopd' for
lightning-loop to simplify the code.
For reference, the org. name in auto-generated loop certs is
"loop autogenerated cert".
- The certificate now includes all default x509v3 extensions.
These were previously restricted to just `subjectAltName` by openssl.cnf.
We now use the openssl defaults for simplicity.
Erik Arvstedt