Instead of forking this repo, it is now recommended that users simply import the
nix-bitcoin module. This commit adds an example directory that contains the
network/ examples and a shell.nix for deployment with nixops.
28cf7ebe74 Add nixops19_09 to default pkgs. (Jonas Nick)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: e10f7d8d94df506cc5848477956da6cd3cc1c7ee87950df8c09da27e2fcac87b97c7dff1facafde5b114a9d7f6076f492956c2b684a7776b2566e86ba78a9d1d
323b2a7f17 Allow adding multiple nodes to bitcoind with the addnodes option and improve bitcoin module option descriptions (Jonas Nick)
ed6511c96e Document how to override attributes in configuration.nix (Jonas Nick)
9d3588e1de Convert nix-bitcoin extraConfig options to regular options (Jonas Nick)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: 02d7a38e41742f76979e2e12ae2195304a11a86c6547f5e1f3ff82b6031ad36b80d006a78cb4ec03fdfc4227ffdd60c5cc15bf898c32a3f213acaf2598da8eaf
31ad56950f CI build: print number of VM CPUs (Erik Arvstedt)
Pull request description:
ACKs for top commit:
jonasnick:
ACK 31ad56950f
Tree-SHA512: ff6c43d24f4121d0f1a89e82bb58e182ba044662cb00e7fc879168e7f697d171bc6fbed5d22d983fc6ded11fadbbb4e433133709bf4512882412ac8908e15156
187ff884db add modules test (Erik Arvstedt)
826245484e make secrets dir location configurable (Erik Arvstedt)
b1e13e9415 simplify secrets file format (Erik Arvstedt)
314272a228 lnd, nanopos: move user and group definitions to the bottom (Erik Arvstedt)
766fa4f300 travis: cache all build outputs with cachix (Erik Arvstedt)
b0e759160d travis: set NIX_PATH as early as possible (Erik Arvstedt)
c51bbcf104 travis: move comment (Erik Arvstedt)
7092dce0c7 travis: remove use of deprecated statements (Erik Arvstedt)
190a92507c travis: split up scripts into statements (Erik Arvstedt)
10d6b04ac8 support enabling clightning and lnd simultaneously (Erik Arvstedt)
ad7a519284 bitcoind: wait until RPC port is open (Erik Arvstedt)
5536b64fb3 lnd: wait until wallet is created (Erik Arvstedt)
6f2a55d63c lnd: wait until RPC port is open (Erik Arvstedt)
1868bef462 lnd: add option 'rpcPort' (Erik Arvstedt)
120e3e8cfe lnd postStart: suppress curl response output (Erik Arvstedt)
3e86637327 lnd postStart: poll for REST service availability (Erik Arvstedt)
795c51dc01 lnd postStart: make more idiomatic (Erik Arvstedt)
6e58beae8a lnd: use postStart option for script (Erik Arvstedt)
86167c6e6d clightning: wait until the RPC socket appears (Erik Arvstedt)
60c732a6a1 onion-chef: set RemainAfterExit, fix tor dependency (Erik Arvstedt)
2b9b3ba1c5 systemPackages: improve readability with shorter service references (Erik Arvstedt)
14ecb5511a liquid: add cli option (Erik Arvstedt)
cd5ed39b9c lnd: add cli option (Erik Arvstedt)
1833b15888 clightning: add cli option (Erik Arvstedt)
b90bf6691b add generate-secrets.service (Erik Arvstedt)
6447694214 add generate-secrets pkg (Erik Arvstedt)
e34093a8ac generate_secrets.sh: add opensslConf option (Erik Arvstedt)
9d14d5ba64 generate_secrets.sh: write secrets to working directory (Erik Arvstedt)
51fb054001 generate_secrets.sh: extract makepw command (Erik Arvstedt)
e3b47ce18a add setup-secrets.service (Erik Arvstedt)
437b268433 extract make-secrets.nix (Erik Arvstedt)
f9c29b9318 simplify secret definitions (Erik Arvstedt)
cd0fd6926b don't copy secret files to store during nixops deployment (Erik Arvstedt)
f0a36fe0c7 add 'nix-bitcoin-services' option (Erik Arvstedt)
7aaf30501c nix-bitcoin-services: simplify formatting (Erik Arvstedt)
760da232e0 add nix-bitcoin pkgs namespace (Erik Arvstedt)
6def181dbc add modules.nix (Erik Arvstedt)
3b842e5fe7 add nix-bitcoin-secrets.target (Erik Arvstedt)
bbf2bbc04a network.nix: simplify import of main config (Erik Arvstedt)
7e021a2629 simplify overlay.nix (Erik Arvstedt)
07dc3e04ac move bitcoinrpc group definition to bitcoind (Erik Arvstedt)
d61b185c3a simplify user and group definitions (Erik Arvstedt)
Pull request description:
The nix-bitcoin modules consist of three fundamental components:
1. a set of bitcoin-related modules for general use.
2. an opinionated configuration of these modules (`nix-bitcoin.nix`), to be deployed on a
dedicated machine.
3. machinery for nixops deployment.
This PR removes dependencies that reach from top to bottom in the list.
This means that 1. is now usable on its own and that 2. can be used without 3.
Besides improving nix-bitcoin's general usefulness, this
- simplifies testing. This PR includes a Travis-enabled modules test using the NixOS testing framework.
- paves the way for krops deployment.
- unlocks direct deployment in NixOS containers which allows for super fast experimentation.
### Details
Here are the unnecessary inter-component dependencies and how they're resolved by the commits. I'm using the numbering from the list above.
- `1. -> 3.` The modules (1.) use the nixops-specific (3.) `keys` group.
Resolved by `add nix-bitcoin-secrets.target`.
- `1. -> 3.` 1. requires nixops-specific key services.
Resolved by `add nix-bitcoin-secrets.target`.
- `1. -> 2.` bitcoind needs the bitcoinrpc group which is defined in `nix-bitcoin.nix` (2.).
Resolved by `move bitcoinrpc group definition to bitcoind`.
Further obstacles for standalone usage of 1.:
- We can't easily import 1. as a standalone module set.
Resolved by `add modules.nix`.
- Users of 1. shouldn't be forced to import nix-bitcoin's packages as top-level items in the pkgs namespace.
Resolved by `add nix-bitcoin pkgs namespace`.
### Non-nixops deployments
Commit `add setup-secrets.service` simplifies non-nixops deployment methods like containers, NixOS VMs or krops.
Secrets can now deployed as follows:
1. create local secrets.
2. transfer secrets to machine.
3. on the machine, `setup-secrets.service` creates extra secrets from `secrets.nix` and sets owner and
permissions for all secrets.
As krops integrates step 2. we now have all ingredients for automatic krops deployment.
The service is complicated by the creation of secrets like `bitcoin-rpcpassword` that are composed of attrs from `secrets.nix` instead of being simply backed by a file like `lnd_key`. We could simplify this by creating all secret files locally.
Running nix-bitcoin in NixOS containers gives you faster rebuild cycles when developing. [Here's](https://gist.github.com/5db4fa7dd3f1137920b58e39647116f6) an example.
### Test
The last commits starting with `clightning: add cli option` are testing-related and mostly fix non-critical bugs that were exposed by the test.
All `STABLE=1` builds from the Travis build matrix are implicit in the modules test.
Should we remove these individual builds?
Regarding commit `travis: cache all build outputs with cachix`:
To replace my cache with a cache that's owned by you (maybe named `nix-bitcoin-ci`), run
```
nix-shell -p travis --run 'travis encrypt CACHIX_SIGNING_KEY=... -r fort-nix/nix-bitcoin'
```
where `...` is the value of `secretKey` in `~/.config/cachix/cachix.dhall`. Let me know the travis secret and I'll fixup the commit.
### Docs
If you like the proposed changes, I'll add another PR with updates to the docs regarding the project layout, non-nixops deployment, and how to use nix-bitcoin within a larger NixOS config.
ACKs for top commit:
jonasnick:
ACK 187ff884db
Tree-SHA512: f4be65215c592a4f41bb7fa991a6d8d7c463cf631b88bf53051ca57ba280e7a60b8b09d0d1521345d5b656f844daa2166fff5d00a3105077c9e263465eacfb0a
43507a7ce5 Update assumevalid to block 605181 (Jonas Nick)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: 717b4885aad72b5ac86e1d7c9644732e6559055cef138f1a8adad0414760886ef58593fe24717cd070ce86728df9e738613f26c7f08a7c711088226f04f394f3
c36c496507 banlist: fail on unexpected errors (Erik Arvstedt)
e0276503ed fixup! ignore banlist errors (like in master) (Erik Arvstedt)
d64156e485 banlist: don't wait in preStart until bitcoind is ready (Erik Arvstedt)
d87c50a305 banlist: simplify unit, bind to bitcoind, fix wantedBy (Erik Arvstedt)
39885d37c1 banlist: simplify script, remove package (Erik Arvstedt)
55e73f32e3 bitcoind: add cli option (Erik Arvstedt)
8807b9f6b2 bitcoind: remove 'StateDirectory' (Erik Arvstedt)
Pull request description:
Please see the individual commit messages for more infos.
This time I've used commit msg titles starting with lower-case letters.
I think this style is the simplest and cleanest, especially with
`topic:` prefixes. Let me know if I should reformat this.
Regarding commit `banlist: simplify script, remove package`:
I've added the current version of Greg's banlist, but it has 5600 entries instead of the previous 739. Is this ok?
Rationale for commit `bitcoind: add cli option`:
For some cases it can be useful when bitcoin-cli is an executable instead of a shell alias.
- non-interactive ssh commands like `ssh mynode 'bitcoin-cli getnetworkinfo'`.
This can also be achieved with shell aliases in `environment.shellInit`, but it's good
practice to avoid aliases in noninteractive shells.
- Scripts that call `bitcoin-cli`. The banlist importer which uses this is a good example.
ACKs for top commit:
jonasnick:
ACK c36c496507
Tree-SHA512: 31dacf5e5b051ada24151c9884a97cfd83883c2b48e3d25e16f228fb575cb915e1093d12d171a44d25064fda7d7b98a27c897cc4bdace527c52fdf652033a160
46edb39d2f Add content hashes for pinned channels (Erik Arvstedt)
961e8217a9 Rename contrib/ to helper/ (Erik Arvstedt)
Pull request description:
Unhashed external content is bad for security and performance (due to
re-fetches when the cache times out).
Use this simple fix until https://github.com/fort-nix/nix-bitcoin/pull/78 is fleshed out.
For testing, run this in the repo root dir:
```bash
nix eval '(import ./pkgs/nixpkgs-pinned.nix)'
```
ACKs for top commit:
jonasnick:
ACK 46edb39d2f
Tree-SHA512: cb098a4714aecf00e8d0f9fe6d388b6322416c1d2f8d55b54dc16328145331a87a71fbf68e2faa85105727cbd6370542799f1c2d84ac2bee90a6710b96eba9bd
6ba4a43193 Fix release.nix.patch for our nixops (Jonas Nick)
Pull request description:
Otherwise nix-shell doesn't start up:
```
building '/nix/store/c03b0sb0b822m7n1j0rhg8bfbmqv2vv2-src.drv'...
patching file release.nix
Hunk #3 FAILED at 83.
1 out of 3 hunks FAILED -- saving rejects to file release.nix.rej
builder for '/nix/store/c03b0sb0b822m7n1j0rhg8bfbmqv2vv2-src.drv' failed with exit code 1
error: build of '/nix/store/c03b0sb0b822m7n1j0rhg8bfbmqv2vv2-src.drv' failed
(use '--show-trace' to show detailed location information)
```
CC @erikarvstedt
Top commit has no ACKs.
Tree-SHA512: fd92325f9cf6f44592a0733131efc147669140402fb87a9efb6e407520048b8e84a58ccbf12c9c3a637a24f3d381cfaa54a34e2341e6542eb1e3769447de0f8f
cfafcb5d32 Fix virtualbox deployment for 19.09 (Erik Arvstedt)
Pull request description:
Without this, starting the virtualbox guest service fails during machine activation.
This is due to an incompatible NixOS machine base image.
Fix this by using an updated version of nixops.
I've already reported [this issue](https://github.com/NixOS/nixops/issues/1207) upstream.
@jonasnick, could you check if this new nixops version works
correctly for your non-virtualbox deployments?
Top commit has no ACKs.
Tree-SHA512: ac8a86ec11739c4bd527c45438bbbd92cb8a5b94509d45a56a22c3294419a78e01cdfa9a9cac17be84b9a1dd79eee35150eb182904265c3fbf387be59d8d85d3
6e26453689 Set default size of the virtualbox machine disk (Erik Arvstedt)
Pull request description:
This eliminates the manual resize step in vbox deployment
ACKs for top commit:
jonasnick:
ACK 6e26453689
Tree-SHA512: 31521de732ad891af94c007bf6a9947131b476c904c84c8cad35786e95b08a26b29eacc58041877f4ef80ebebd1bf1c2e863694022c72e56130c02d7417332b4
0c22af03b7 Allow AnyProtocol for bitcoin if zmq options are set (and not if lnd is enabled) (Jonas Nick)
cf39d88c63 Move zmq options from nix-bitcoin.nix to bitcoind module (Jonas Nick)
Pull request description:
... which is a better place for this. CC @cypherpunk2140
Top commit has no ACKs.
Tree-SHA512: 47d1b95fef78ee31711b5ad5a59000adfb0fcd3bbfe82c7321d87f5a6d7c998646d3428a1c86ff9b0103b167501c8cf3b16e00d4e2b5c09425ab09f732f75a57
664c5c6762 Switch from python 3.5 to python 3.x for trezor (Jonas Nick)
8dd27b6334 Use types.str instead of types.string to avoid warning (Jonas Nick)
09d2df1a81 Use stable tor module instead of unstable which we had to use because stable didn't support v3 onion services (Jonas Nick)
b2fb83c910 Use our own bitcoind module instead of nixpkgs' (Jonas Nick)
26d2e11a6b Update to NixOS 19.09 (Jonas Nick)
1d022f2017 Remove PyQt dependency from liquid-swap tool because it doesn't work with NixOs 19.09 and we don't need it anyway (Jonas Nick)
Pull request description:
ACKs for top commit:
erikarvstedt:
Tested ACK 664c5c6762
Tree-SHA512: eecbb15011ce8849b8c54788e296109a7f0529286b5a38cd2dd90268cbd03ca396555296ac2018e7e82d88cace18d97ab69201a252b30b3b87db0ae136c410c5
a152c19945 spark-wallet: 0.2.8 -> 0.2.9 (William Casarin)
Pull request description:
needed to fix issues with deprecated apis such as `listpayments`
ACKs for top commit:
jonasnick:
ACK a152c19945
Tree-SHA512: 875a07aa2e3ddd944d41bb01471cdaa837dfa513de88b990d78a2986fad913c13268847de1f0eca82fb87f413ca6f4c56b6adc710c1bad0ef81324c8679631d0
c5024d0f15 Add liquid-swap tool to CI (Jonas Nick)
29e612d3bd Remove spark-wallet with unstable from nixpkgs because it doesn't work (Jonas Nick)
c1d67c4cee Update nixpkgs (Jonas Nick)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: 64de79713b656a7535c0a89f8cead5be0168b067d2e79d9b9dfa7152635d09cea677494ad04f8a0b5f9c5278860ff8f75813561ddafb5ca8024f1f66b4fd4f34
8e69e42c12 Update elementsd to 0.18.1.1 (Jonas Nick)
Pull request description:
Because `LC_ALL` must be set to make the tests work (https://github.com/bitcoin/bitcoin/issues/14948#issuecomment-447049325) I also copied the rest from the current nixpkgs bitcoind package.
Top commit has no ACKs.
Tree-SHA512: 6380c0bbb5b35cf49e7553d561d46546701c260c25771b35e3d4ce5c5c5078eed2062f85fa505ae99daa9975beff8ec5d1f33483c0d335a11d275679a968f799
ae15205689 Mention another way to get nixops to pick up the correct IP address of a virtualbox (Jonas Nick)
4df0c9fcfd Fix nodeinfo for clightning (Jonas Nick)
Pull request description:
ACKs for top commit:
cypherpunk2140:
ACK ae15205689
Tree-SHA512: 56fff8c687e0070c0dcd1d7c44cd1b82f6d86103e8634a06fc823dea4bf9d1d986bcfb19caa6c72836c4cbcb636cd5360b3326ae71ee05ecf0942c02566b61b9
aabf4a4730 travis: remove cachix push (William Casarin)
8e751c5f3c travis: disable unstable electrs, it is broken (William Casarin)
1e3afa54c5 travis: add travis yml (William Casarin)
d7d35b875a nixpkgs-pinned: fetch a tarball instead (William Casarin)
Pull request description:
This builds everything to make sure everything works on either the latest release or unstable branch. In the future we could do more interesting nixops builds to test modules, but for now we just nix-build everything and make sure it compiles.
We test the build with two different versions of nixpkgs:
- nixpkgs-channels/nixos-19.03
- nixpkgs-channels/nixos-unstable
ACKs for top commit:
jonasnick:
ACK aabf4a4730
Tree-SHA512: 7241552e65fe2e5325f23e271eeb1f87bdf59c85e5dbd8263326d14340bbbee17163d5cb9df3f562c86182b0585ab766fefddea627ee45f0cf8258b85ece6404
9d029fd1af Remove lnd explicit tor onion service config (Ștefan D. Mihăilă)
1f407ef22c Remove lnd user from onion-chef (Ștefan D. Mihăilă)
5880023158 Increase xxd column size (Ștefan D. Mihăilă)
101ae3c370 Instruct user to backup channel.backup (Ștefan D. Mihăilă)
fccd91972a Fix "value is a list [...]" error when lnd is not enabled (Ștefan D. Mihăilă)
700fdf6feb Add logdir and tor.privatekeypath to lnd.conf (Ștefan D. Mihăilă)
5a2517b926 Check for existing secrets and create them more granularly (Ștefan D. Mihăilă)
d6f961db89 Reuse lnd seed (Ștefan D. Mihăilă)
9b0753135c Add LND support (Ștefan D. Mihăilă)
4acf5cd32c Remove unused nginx.csr file (Ștefan D. Mihăilă)
19b971f21f Rename nginx certificate files (Ștefan D. Mihăilă)
Pull request description:
ACKs for top commit:
jonasnick:
ACK 9d029fd1af
Tree-SHA512: 58ee80bcab6c3a1c4642a5d40b94e10d28311557ae7c69539fee90d6f252a6afc70b8066cc7d7ddc0a45e2675978718a369b0341c518f8ce7590cbde1403eaeb
820abe213d Update nixpkgs (Jonas Nick)
Pull request description:
Includes lnd 0.7.1
Top commit has no ACKs.
Tree-SHA512: 842716ac246882d74392ea942d81a1425bd5f5cfd9d63876535dc271c6979c571198d2da7a910e33d0ce95c0a4f67a9ff2bb67085a20ffc44052df1fe8b6830d
67a464d097 Mention problems with hardened kernel and NUCs in README (Jonas Nick)
7771a4c931 Refer to systemd man pages for hardening options (Jonas Nick)
a5e10a82d8 Simplify clightning preStart (Jonas Nick)
Pull request description:
CC @cypherpunk2140
Top commit has no ACKs.
Tree-SHA512: aa726f29e499cc268b21cac8cd07617be591cfdaa89dd0495cb979ebd3e49cc01164af25924c554429a1d35d14167dea276f7d61877452b69f027143cc3eee97
1c8dadd876 Add allowAnyProtocol option to nix-bitcoin-services (Jonas Nick)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: 9769b04f815dc72bd0b9e6fca8cde6b9098fd678021929d9541f81f433c6cf715dbfada1a58b9bf11f35d1a7681bdcae091032f884cd157f3f02900ec8b75560
161ee02550 style: remove extra space (Ștefan D. Mihăilă)
4e6e05a4a8 Improve electrs ports descriptions (Ștefan D. Mihăilă)
cd722cac1a Fix identation (Ștefan D. Mihăilă)
df784b341e Expose electrs high-memory option in configuration.nix (Ștefan D. Mihăilă)
Pull request description:
ACKs for top commit:
jonasnick:
ACK 161ee02550
Tree-SHA512: c730b6979f28f8a26a1a10d47aeb0eaf4a41343b058c271a2710c27c171bf521f64358be1fedafeefb30bb2cdafb37a8665c0292e8cd673eeb575f3afbc8919e
Jonas Nick