How does Core Lightning communicate with the Bitcoin network?

• builtin bcli,

• https://github.com/nbd-wtf/trustedcoin: A lightningd plugin that replaces bitcoind with trusted public explorers.

• Sauron in https://github.com/lightningd/plugins: A Bitcoin backend plugin relying on Esplora.

• https://github.com/clightning4j/btcli4j: It is a core lightning plugin to override Bitcoin backend plugin with esplora by Blockstream and give the possibility to make the running process with bitcoind in pruning mode more solid.

• https://github.com/coffee-tools/folgore: Universal Bitcoin backend for core lightning with BIP 157 support full based on the btcli4j Kotlin plugin! Ah, yes it is written in Rust

Note that as estimatefees changed in CLN v23.05 and the deprecation cycle (as far as I know) is about 2 releases, you can use those plugins with CLN v23.05 (and the next CLN v23.08) by enabling deprecated APIs with allow-deprecated-apis flag.

I opened issues regarding estimatefees new version in the corresponding repositories:

• https://github.com/nbd-wtf/trustedcoin/issues/20,

• https://github.com/clightning4j/btcli4j/issues/79,

• https://github.com/coffee-tools/folgore/issues/26,

• https://github.com/lightningd/plugins/issues/458.

Pull requests are welcome if you want to give it a try.

Note: during the live, I wrongly used ~/.lightning/conf as lightningd config file. The default config file is ~/.lightning/config. This had no consequences on what we did (as we disable bcli plugin at the command line). In this writeup, I use the correct default config file.

Let's install trustedcoin bitcoin backend plugin and try to connect to William Casarin (https://jb55.com) the author of Damus Nostr client.

First, we ssh into lnroom server and check lightning-cli version:

◉ tony@tony:~/clnlive:
$ ssh lnroom
◉ tony@lnroom:~:
$ lightning-cli --version
v23.02.2

We move into .lightning node directory and check for its content:

◉ tony@lnroom:~:
$ cd .lightning/
◉ tony@lnroom:~/.lightning:
$ ls
plugins/  config
◉ tony@lnroom:~/.lightning:
$ cat config
network=bitcoin
log-level=debug
allow-deprecated-apis=false

We move into ~/.lightning/plugins/ directory (the default directory for installing plugins), we download trustedcoin binary for linux amd64 (I'm running Ubuntu 22.04 on lnroom server):

◉ tony@lnroom:~/.lightning:
$ cd plugins
◉ tony@lnroom:~/.lightning:
$ wget https://github.com/nbd-wtf/trustedcoin/releases/download/v0.6.1/trustedcoin-v0.6.1-linux-amd64.tar.gz
Resolving github.com (github.com)... 140.82.121.3
Connecting to github.com (github.com)|140.82.121.3|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://objects.githubusercontent.com/github-production-release-asset-2e65be/286887571/7eb71b3c-9597-42ff-be07-6c2d036b0417?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230622%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230622T141122Z&X-Amz-Expires=300&X-Amz-Signature=000450beeff7067fd4a587fbeaeacf6186e0aa7867586a29c54541fb0f72c2c0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=286887571&response-content-disposition=attachment%3B%20filename%3Dtrustedcoin-v0.6.1-linux-amd64.tar.gz&response-content-type=application%2Foctet-stream [following]
--2023-06-22 14:11:23--  https://objects.githubusercontent.com/github-production-release-asset-2e65be/286887571/7eb71b3c-9597-42ff-be07-6c2d036b0417?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20230622%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20230622T141122Z&X-Amz-Expires=300&X-Amz-Signature=000450beeff7067fd4a587fbeaeacf6186e0aa7867586a29c54541fb0f72c2c0&X-Amz-SignedHeaders=host&actor_id=0&key_id=0&repo_id=286887571&response-content-disposition=attachment%3B%20filename%3Dtrustedcoin-v0.6.1-linux-amd64.tar.gz&response-content-type=application%2Foctet-stream
Resolving objects.githubusercontent.com (objects.githubusercontent.com)... 185.199.110.133, 185.199.109.133, 185.199.108.133, ...
Connecting to objects.githubusercontent.com (objects.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5539005 (5.3M) [application/octet-stream]
Saving to: ‘trustedcoin-v0.6.1-linux-amd64.tar.gz’

trustedcoin-v0.6.1-linu 100%[==============================>]   5.28M  --.-KB/s    in 0.1s

2023-06-22 14:11:23 (37.3 MB/s) - ‘trustedcoin-v0.6.1-linux-amd64.tar.gz’ saved [5539005/5539005]

◉ tony@lnroom:~/.lightning:
$ tar -xvf trustedcoin-v0.6.1-linux-amd64.tar.gz
trustedcoin

To start lightningd with trustedcoin as Bitcoin backend we need to disable the builtin bcli Bitcoin backend plugin, this is why we use --disable-plugin=bcli in the following command:

◉ tony@lnroom:~/.lightning:
$ lightningd --daemon --disable-plugin=bcli --log-file=/home/tony/.lightning/log

We can check that lightningd is actually running:

◉ tony@lnroom:~/.lightning:
$ cd
◉ tony@lnroom:~:
$ ps -ax | rg light
  56996 ?        Ss     0:02 lightningd --daemon --disable-plugin=bcli --log-file=/home/tony/.lightning/log
  56997 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/autoclean
  56998 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/chanbackup
  56999 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/commando
  57001 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/funder
  57002 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/topology
  57003 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/keysend
  57004 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/offers
  57005 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/pay
  57006 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/txprepare
  57007 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/spenderp
  57008 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/sql
  57009 ?        S      0:00 /usr/local/libexec/c-lightning/plugins/bookkeeper
  57010 ?        Sl     0:02 /home/tony/.lightning/plugins/trustedcoin
  57021 ?        SL     0:00 /usr/local/libexec/c-lightning/lightning_hsmd
  57022 ?        S      0:00 /usr/local/libexec/c-lightning/lightning_connectd
  57025 ?        S      0:00 /usr/local/libexec/c-lightning/lightning_gossipd
  57060 pts/1    S+     0:00 rg light
◉ tony@lnroom:~:
$ lightning-cli getinfo
{
   "id": "03f990e2dfe3e4ab3c6f007a08d40b5720bc665eb07d9c332421c84ad417241747",
   "alias": "YELLOWSPORK",
   "color": "03f990",
   "num_peers": 0,
   "num_pending_channels": 0,
   "num_active_channels": 0,
   "num_inactive_channels": 0,
   "address": [
      {
         "type": "ipv4",
         "address": "157.230.114.178",
         "port": 9735
      }
   ],
   "binding": [
      {
         "type": "ipv6",
         "address": "::",
         "port": 9735
      },
      {
         "type": "ipv4",
         "address": "0.0.0.0",
         "port": 9735
      }
   ],
   "version": "v23.02.2",
   "blockheight": 795440,
   "network": "bitcoin",
   "msatoshi_fees_collected": 0,
   "fees_collected_msat": "0msat",
   "lightning-dir": "/home/tony/.lightning/bitcoin",
   "warning_lightningd_sync": "Still loading latest blocks from bitcoind.",
   "our_features": {
      "init": "08a000080269a2",
      "node": "88a000080269a2",
      "channel": "",
      "invoice": "02000000024100"
   }
}

Now we connect our node to William's node and check that it appears listed in our peers:

◉ tony@lnroom:~:
$ lightning-cli connect 03f3c108ccd536b8526841f0a5c58212bb9e6584a1eb493080e7c1cc34f82dad71@24.84.152.187:9735
{
   "id": "03f3c108ccd536b8526841f0a5c58212bb9e6584a1eb493080e7c1cc34f82dad71",
   "features": "08a0800a0a69a2",
   "direction": "out",
   "address": {
      "type": "ipv4",
      "address": "24.84.152.187",
      "port": 9735
   }
}
◉ tony@lnroom:~:
$ lightning-cli listpeers
{
   "peers": [
      {
         "id": "03f3c108ccd536b8526841f0a5c58212bb9e6584a1eb493080e7c1cc34f82dad71",
         "connected": true,
         "num_channels": 0,
         "netaddr": [
            "24.84.152.187:9735"
         ],
         "remote_addr": "157.230.114.178:44452",
         "features": "08a0800a0a69a2",
         "channels": []
      }
   ]
}

Now we can list some channels:

◉ tony@lnroom:~:
$ lightning-cli listchannels
{
   "channels": [
      {
         "source": "03f3c108ccd536b8526841f0a5c58212bb9e6584a1eb493080e7c1cc34f82dad71",
         "destination": "02bed77fa014c429169ae6c6e10883e3762efd2346004c8e3aba7af26922fa2d49",
         "short_channel_id": "776280x130x1",
         "direction": 1,
         "public": true,
         "satoshis": 100000,
         "amount_msat": "100000000msat",
         "message_flags": 1,
         "channel_flags": 1,
         "active": true,
         "last_update": 1686546009,
         "base_fee_millisatoshi": 0,
         "fee_per_millionth": 2500,
         "delay": 34,
         "htlc_minimum_msat": "1msat",
         "htlc_maximum_msat": "99000000msat",
         "features": ""
      }
   ]
}

During the live I share lnroom's node information in the chat so that people could try to connect to it. It didn't work, people tried to connect to lnroom's node but got errors.

I checked after the live why people couldn't connect to my node: I had the port 9735 closed. If it happens to you and you are using ufw as firewall, you can run the following command to enable the port 9735:

$ sudo ufw allow 9735

Anyway, to get the needed information we ran the following command:

◉ tony@lnroom:~:
$ lightning-cli getinfo -F
id=03f990e2dfe3e4ab3c6f007a08d40b5720bc665eb07d9c332421c84ad417241747
alias=YELLOWSPORK
color=03f990
num_peers=1
num_pending_channels=0
num_active_channels=0
num_inactive_channels=0
address[0].type=ipv4
address[0].address=157.230.114.178
address[0].port=9735
binding[0].type=ipv6
binding[0].address=::
binding[0].port=9735
binding[1].type=ipv4
binding[1].address=0.0.0.0
binding[1].port=9735
version=v23.02.2
blockheight=795445
network=bitcoin
msatoshi_fees_collected=0
fees_collected_msat=0msat
lightning-dir=/home/tony/.lightning/bitcoin
our_features.init=08a000080269a2
our_features.node=88a000080269a2
our_features.channel=
our_features.invoice=02000000024100

If you're reading this, don't try to connect to lnroom's node. I shut it down after the live.

The top commentary of lightningd/bitcoind.c contains the following:

/* Code for talking to bitcoind.  We use a plugin as the Bitcoin backend.
 * The default one shipped with C-lightning is a plugin which talks to bitcoind
 * by using bitcoin-cli, but the interface we use to gather Bitcoin data is
 * standardized and you can use another plugin as the Bitcoin backend, or
 * even make your own! */

What does "the interface we use to gather Bitcoin data is standardized and you can use another plugin as the Bitcoin backend" mean?

That means that when some part of CLN needs to gather Bitcoin data, that part doesn't do it directly but uses functions declared in lightningd/bitcoind.h and defined in lightningd/bitcoind.c. And those functions retrieve Bitcoin data via JSON-RPC calls to the following 5 methods defined by any Bitcoin backends plugins:

static const char *methods[] = {"getchaininfo", "getrawblockbyheight",
                                "sendrawtransaction", "getutxout",
                                "estimatefees"};

Let's take an example.

When lightningd is started the block topology is initialized using setup_topology. The function setup_topology gets Bitcoin data by calling the macro bitcoind_getchaininfo defined in lightningd/bitcoind.h like this:

#define bitcoind_getchaininfo(bitcoind_, first_call_, cb, arg)             \
        bitcoind_getchaininfo_((bitcoind_), (first_call_),                 \
                              typesafe_cb_preargs(void, void *,            \
                                                  (cb), (arg),             \
                                                  struct bitcoind *,       \
                                                  const char *, u32, u32,  \
                                                  const bool, const bool), \
                              (arg))

So the Bitcoin data is retrieved by the function bitcoind_getrawblockbyheight_ which gets it by doing a call to the JSON-RPC method getrawblockbyheight using the macro jsonrpc_request_start:

void bitcoind_getrawblockbyheight_(struct bitcoind *bitcoind,
                                   u32 height,
                                   void (*cb)(struct bitcoind *bitcoind,
                                              struct bitcoin_blkid *blkid,
                                              struct bitcoin_block *blk,
                                              void *arg),
                                   void *cb_arg)
{
        struct jsonrpc_request *req;
        struct getrawblockbyheight_call *call = tal(NULL,
                                                    struct getrawblockbyheight_call);

        call->bitcoind = bitcoind;
        call->cb = cb;
        call->cb_arg = cb_arg;

        req = jsonrpc_request_start(bitcoind, "getrawblockbyheight", NULL, true,
                                    bitcoind->log,
                                    NULL,  getrawblockbyheight_callback,
                                    call);
        json_add_num(req->stream, "height", height);
        jsonrpc_request_end(req);
        bitcoin_plugin_send(bitcoind, req);
}

What a nice interface!

To communicate with the Bitcoin network, trustedcoin plugin uses several Esplora block explorers. trustedcoin queries those Esplora instances via GET and POST HTTP calls.

You can find the Blockstream Esplora implementation at https://github.com/Blockstream/esplora and its HTTP API description at esplora:API.md.

To understand how a Bitcoin backend plugin can be implemented using Esplora instances, let's take getrawblockbyheight JSON-RPC method as an example and see how trustedcoin implements it (part of the implementation).

As we can read in CLN plugin documentation, getrawblockbyheight is a method that must be called with one parameter (height) which determines the block height of the block to fetch.

The plugin must set all fields to null if no block was found at the specified height. And the plugin must respond to getrawblockbyheight with the following fields:

• blockhash (string), the block hash as a hexadecimal string,

• block (string), the block content as a hexadecimal string.

As we have trustedcoin already running, let see what getrawblockbyheight call returns if we pass it the genesis block height:

◉ tony@lnroom:~:
$ lightning-cli getrawblockbyheight 0
{
   "blockhash": "000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f",
   "block": "0100000000000000000000000000000000000000000000000000000000000000000000003ba3edfd7a7b12b27ac72c3e67768f617fc81bc3888a51323a9fb8aa4b1e5e4a29ab5f49ffff001d1dac2b7c0101000000010000000000000000000000000000000000000000000000000000000000000000ffffffff4d04ffff001d0104455468652054696d65732030332f4a616e2f32303039204368616e63656c6c6f72206f6e206272696e6b206f66207365636f6e64206261696c6f757420666f722062616e6b73ffffffff0100f2052a01000000434104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac00000000"
}

Let's decode the block and read the famous sentence:

◉ tony@lnroom:~:
$ lightning-cli getrawblockbyheight 0 | jq -r .block | xxd -r -p
...The Times 03/Jan/2009 Chancellor on brink of second bailout for banks...

In its implementation trustedcoin needs at some point to retrieve the blockhash of the first block. If we use https://blockstream.info Esplora instances, this can be done like this:

◉ tony@lnroom:~:
$ curl -s https://blockstream.info/api/block-height/0
000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f

(Note that code from ndb-wtf/trustedcoin is licensed under the following MIT license.)

And this is how trustedcoin does it but using the Go language.

First, it defined in the esplora variable the Esplora instances that might be used:

var (
  ...
  esplora = map[string][]string{
    "bitcoin": {
      "https://mempool.space/api",
      "https://blockstream.info/api",
      "https://mempool.emzy.de/api",
    },
    "testnet": {
      "https://mempool.space/testnet/api",
      "https://blockstream.info/testnet/api",
    },
    "signet": {
      "https://mempool.space/signet/api",
    },
    "liquid": {
      "https://liquid.network/api",
      "https://blockstream.info/liquid/api",
    },
  }
  ...
)

Then it register getrawblockbyheight JSON-RPC method to lightningd and defined the function that handles it:

func main() {
  p := plugin.Plugin{
    Name:    "trustedcoin",
    Version: "v0.6.1",
    Options: []plugin.Option{...},
    RPCMethods: []plugin.RPCMethod{
      {
        "getrawblockbyheight",
        "height",
        "Get the bitcoin block at a given height",
        "",
        func(p *plugin.Plugin, params plugin.Params) (resp interface{}, errCode int, err error) {
          height := params.Get("height").Int()

          blockUnavailable := map[string]interface{}{
            "blockhash": nil,
            "block":     nil,
          }

          block, hash, err := getBlock(height)
          if err != nil {
            p.Logf("getblock error: %s", err.Error())
            return blockUnavailable, 0, nil
          }
          if block == "" {
            return blockUnavailable, 0, nil
          }

          p.Logf("returning block %d, %s…, %d bytes",
            height, string(hash[:26]), len(block)/2)

          return struct {
            BlockHash string `json:"blockhash"`
            Block     string `json:"block"`
          }{hash, string(block)}, 0, nil
        },
      }
    ...
    },
    ...
  }
  p.Run()
}

So when trustedcoin receives a getrawblockbyheight request, it uses the function defined above, in that function it calls the function getBlock which calls the function getHash which does a GET request to one of the Esplora instances in esplora variable to retrieve the block hash given its height:

func getHash(height int64) (hash string, err error) {
  ...
  for _, endpoint := range esploras(network) {
    w, errW := http.Get(fmt.Sprintf(endpoint+"/block-height/%d", height))
    ...
    }
  ...
}

That's it!

As estimatefees changed in CLN v23.05 (the last release) we can take this as an opportunity to talk about CLN deprecation period.

As far as I know the deprecation period is 2 releases (6 months). During that time, the deprecated APIs can still be used if we set the flag allow-deprecated-apis to true in lightningd config file.

For instance we could have ran the plugin trustedcoin with CLN v23.05 by using the following config file:

network=bitcoin
log-level=debug
allow-deprecated-apis=true

But it seems that deprecated apis are for developer to catch breaking bugs so I preferred to run trustedcoin with CLN v23.02.2 during this live.

Before estimatefees was changed, this is what lightningd was expected as response to a JSON-RPC call to the method estimatefees (during the live we used feerates command to demonstrate this):

◉ tony@lnroom:~:
$ lightning-cli estimatefees
{
   "opening": 9128,
   "mutual_close": 25708,
   "unilateral_close": 39756,
   "delayed_to_us": 9128,
   "htlc_resolution": 25708,
   "penalty": 32752,
   "min_acceptable": 4564,
   "max_acceptable": 3975800
}

From CLN v23.05, calling the JSON-RPC estimatefees method must return a response that look like this:

{
  "feerate_floor": <sat per kVB>,
  "feerates": {
    { "blocks": 2, "feerate": <sat per kVB> },
    { "blocks": 6, "feerate": <sat per kVB> },
    { "blocks": 12, "feerate": <sat per kVB> }
    { "blocks": 100, "feerate": <sat per kVB> }
  }
}

Why is this like this? I you are interested in the answer you can look at https://github.com/ElementsProject/lightning/pull/6120.

Now let's look at the code during the deprecation period in the case of estimatefees JSON-RPC method.

Let's say we try to run CLN 23.05.1 with trustedcoin but without allowing deprecated APIs. What will happen?

When lightningd will try to start, the function setup_topology will try to set up the block topology. That function will call start_fee_estimate which will call bitcoind_estimate_fees which will do a call to the JSON-RPC method estimatefees:

void bitcoind_estimate_fees(struct bitcoind *bitcoind,
                            void (*cb)(struct lightningd *ld,
                                       u32 feerate_floor,
                                       const struct feerate_est *feerates))
{
        struct jsonrpc_request *req;
        struct estimatefee_call *call = tal(bitcoind, struct estimatefee_call);

        call->bitcoind = bitcoind;
        call->cb = cb;

        req = jsonrpc_request_start(bitcoind, "estimatefees", NULL, true,
                                    bitcoind->log,
                                    NULL, estimatefees_callback, call);
        jsonrpc_request_end(req);
        plugin_request_send(strmap_get(&bitcoind->pluginsmap,
                                       "estimatefees"), req);
}

Then trustedcoin plugin will answer lightningd with the old specification. lightningd will then call estimatefees_callback with that wrong JSON response. Specifically, that JSON response won't have the field feerate_floor set which will force lightningd to raise an error (not that in our example deprecated_apis is false):

static void estimatefees_callback(const char *buf, const jsmntok_t *toks,
                                  const jsmntok_t *idtok,
                                  struct estimatefee_call *call)
{
        ...
        resulttok = json_get_member(buf, toks, "result");
        ...
        /* Modern style has floor. */
        floortok = json_get_member(buf, resulttok, "feerate_floor");
        if (floortok) {
                ...
        } else {
                if (!deprecated_apis)
                        bitcoin_plugin_error(call->bitcoind, buf, resulttok,
                                             "estimatefees",
                                             "missing fee_floor field");
                ...
        }
        ...
}

Interesting isn't it?

The following is not in the video.

To show what happens if we do it for real, when writing this writeup, I tried to run trustedcoin as Bitcoin backend plugin with CLN v23.05.1 without allowing deprecated APIs.

I used the following ~/.lightning/config file

network=bitcoin
log-level=debug
log-file=/home/tony/.lightning/log
disable-plugin=bcli
allow-deprecated-apis=false

and tried to start lightningd which lead to the following error:

◉ tony@lnroom:~/.lightning:
$ lightning-cli --version
v23.05.1
◉ tony@lnroom:~/.lightning:
$ lightningd --daemon
/home/tony/.lightning/plugins/trustedcoin error: bad response to estimatefees (missing fee_floor field), response was {"opening":10100,"mutual_close":22098,"unilateral_close":36202,"delayed_to_us":10100,"htlc_resolution":22098,"penalty":31105,"min_acceptable":5050,"max_acceptable":3620200}

If we relpace allow-deprecated-apis=false by allow-deprecated-apis=true in the config file, everything works fine.

• Thank you for this informative session!

• I tried to connect and I got an error (two people).

• We can try in futur sessions.