Understand CLN Plugin mechanism with a Bash example

LNROOM #14・May 10, 2023

In this live, we add the method mymethod to Core Lightning by writing a dynamic Bash plugin called myplugin.bash. Doing this, we try to understand: (1) how startup options are passed to the plugin, (2) how cli parameters are passed to the plugin and (3) how to communicate to the lightning node via JSON-RPC over unix sockets.

Transcript with corrections and improvements

In this episode, we write a plugin in Bash that registers a JSON-RPC method to lightningd called mymethod.

Assuming l1-cli is an alias for lightning-cli --lightning-dir=/tmp/l1-regtest, when we start our plugin myplugin.bash with the startup option foo_opt set to BAR like this

◉ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR

we want mymethod method to return

the node id of the node running the plugin in the id field,
the startup options in the options field and
the cli parameters in the cli_params field.

Specifically, we want this:

◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "0204663461c108e470b83a111c241b34cb79e6661890a78a211bbfdb909c934b59",
   "options": {
      "foo_opt": "BAR"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Core Lightning plugin system

Before implementing that plugin, let's describe Core Lightning plugin system.

Once lightningd has started the plugin as a subprocess, the plugin and lightningd communicate using pipes. Specifically,

lightningd writes to the stdin stream of the plugin to send it messages and
the plugin writes to its stdout stream to send messages to lightningd.

To understand each other they use the JSON-RPC protocol.

That communcation can be represented like this:

          ->  stdin
         |        |
lightningd        plugin
         |        |
          <- stdout

Now that we know how lightningd and plugins communicate with each other, let's take a look at the life cycle of a plugin like myplugin.bash which registers a JSON-RPC method called mymethod:

When lightningd starts myplugin.bash, it sends the getmanifest request to myplugin.bash. myplugin.bash replies to lightningd with a response containing the declaration of the startup option foo_opt and the information to register mymethod JSON-RPC method.
Then lightningd sends the init request to myplugin.bash. This tells myplugin.bash that lightningd is ready to communicate. That init request contains informations that the plugin might needs to work correctly. For instance, myplugin.bash needs to know the value of the startup option foo_opt and to know the Unix socket file to use to connect to the node and to send JSON-RPC requests in order to retrieve the node id. Those informations are available in the init request.
Then myplugin.bash starts an IO loop and waits in its stdin stream incoming requests from lightningd:
- (a). A client sends a mymethod request to lightningd. As myplugin.bash has registered mymethod JSON-RPC method to lightningd, lightningd forwards that request to myplugin.bash,
- (b). myplugin.bash receives that request, builds a response and sends it back to lightningd,
- (c). Finally, lightningd forwards that response to the client,
- (d). We repeat (a) to (c).

Setup

Here is my setup

◉ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
lightningd v23.02.2
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)

Start 2 Lightning nodes running on regtest

Let's start two Lightning nodes running on the Bitcoin regtest chain by sourcing the script lightning/contrib/startup_regtest.sh provided in CLN repository and by running the command start_ln:

◉ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is hashed (/usr/local/bin/lightning-cli)
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
◉ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 1139027
[2] 1139063
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes

We can check that l1-cli is just an alias for lightning-cli with the base directory being /tmp/l1-regtest:

◉ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'

Write in the /tmp/myplugin file

When we start myplugin.bash script as a plugin with lightningd

#!/usr/bin/bash

set -e

we get the following error because the script stops before replying to the getmanifest request:

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

Note that myplugin.bash is an executable file.

As plugins use their stdout stream to send messages to lightningd, plugins can't write to their stdout to give us (the programmers) information about their behavior.

Indeed, if we modify myplugin.bash to be

#!/usr/bin/bash

set -e

echo foo

we get the same error as above and foo didn't get printed in our terminal:

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

Not a problem, in this video to get informations about myplugin.bash we'll write stuff in /tmp/myplugin file like this:

#!/usr/bin/bash

set -e

[[ -e /tmp/myplugin ]] && rm /tmp/myplugin

echo foo >> /tmp/myplugin

Let's try to start our plugin again:

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

We get the same error but this time we wrote foo in /tmp/myplugin:

foo

This is our tool to try things out and to get informations about the "state" of myplugin.bash plugin.

getmanifest request

Let's treat the getmanifest request.

First we receive the getmanifest request and store it in the variable JSON that we wrote in /tmp/myplugin:

...
# getmanifest

read -r JSON
read -r _
echo "$JSON" >> /tmp/myplugin

Note that the second read is to read the empty line as lightningd request ends with two consecutive new lines (\n\n).

We get the same error as before trying to start myplugin.bash plugin

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

but this time we wrote the getmanifest request in /tmp/myplugin that we can prettify like this:

{
  "jsonrpc": "2.0",
  "id": 52,
  "method": "getmanifest",
  "params": {
    "allow-deprecated-apis": false
  }
}

Let's see if we can extract the id of that request.

To do so we use jq utility and modify myplugin.bash like this:

...
# getmanifest

read -r JSON
read -r _

id=$(echo "$JSON" | jq .id)
echo "$JSON" >> /tmp/myplugin
echo "$id" >> /tmp/myplugin

We get the same error as before trying to start myplugin.bash plugin

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}

but now we can check in /tmp/myplugin that we correctly extracted the request id:

{"jsonrpc":"2.0","id":59,"method":"getmanifest","params":{"allow-deprecated-apis":false}}
59

The moment to reply to the getmanifest request has come.

In the getmanifest response, we tells lightningd that:

we want the plugin to be dynamic,
we want to declare foo_opt startup option with bar as default value and
we want to register mymethod JSON-RPC method.

To do so, we use the fields dynamic, options and rpcmethods like this (where "id" will be changed in the Bash script to the value we received from lightningd):

{
  "jsonrpc": "2.0",
  "id": "id",
  "result": {
    "dynamic": true,
    "options": [
      {
        "name": "foo_opt",
        "type": "string",
        "default": "bar",
        "description": "description"
      }
    ],
    "rpcmethods": [
      {
        "name": "mymethod",
        "usage": "",
        "description": "description"
      }
    ]
  }
}

So we modify myplugin.bash like this:

...
# getmanifest

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"dynamic": true, "options": [{"name": "foo_opt", "type": "string", "default": "bar", "description": "description"}], "rpcmethods": [{"name": "mymethod", "usage": "", "description": "description"}]}}'

Back to our terminal, we try to start our plugin and we get a new error because the plugin doesn't reply to the init request:

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

init request

Let's modify myplugin.bash such that we can write the init request and its id to /tmp/myplugin file:

...
# init

read -r JSON
read -r _

id=$(echo "$JSON" | jq .id)
echo "$JSON" >> /tmp/myplugin
echo "$id" >> /tmp/myplugin

When we try to start our plugin still get the same error

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

but now we wrote the init request and its id to /tmp/myplugin which is now

{"jsonrpc":"2.0","id":"cln:init#79","method":"init","params":{"options":{"foo_opt":"bar"},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}
"cln:init#79"

and we can prettify the init request like this:

{
  "jsonrpc": "2.0",
  "id": "cln:init#79",
  "method": "init",
  "params": {
    "options": {
      "foo_opt": "bar"
    },
    "configuration": {
      "lightning-dir": "/tmp/l1-regtest/regtest",
      "rpc-file": "lightning-rpc",
      "startup": false,
      "network": "regtest",
      "feature_set": {
        "init": "08a000080269a2",
        "node": "88a000080269a2",
        "channel": "",
        "invoice": "02000000024100"
      }
    }
  }
}

In the field options we have the value of foo_opt startup option. And with the value of the fields lightningd-dir and rpc-file we can form the Unix socket file we need to connect to our node in order to get the node id of the node running the plugin.

Using jq and the following jq filter

.params.options.foo_opt

we can set the variable foo_opt to the startup option value.

Let's modify myplugin.bash accordingly and also write that value to /tmp/myplugin file:

...
# init

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
echo "$foo_opt" >> /tmp/myplugin

We try to start myplugin.bash, we get the same error

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

and /tmp/myplugin file is now:

"foo"

Using jq and the following jq filter

.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"

we can set the variable socket_path corresponding to the node's Unix socket file.

Let's modify myplugin.bash accordingly and also write that value to /tmp/myplugin file:

...
# init

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
socket_path=$(echo "$JSON" | jq '.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"' -r)
echo "$socket_path" >> /tmp/myplugin

We try to start myplugin.bash, we get the same error

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}

and /tmp/myplugin file is now:

/tmp/l1-regtest/regtest/lightning-rpc

So far, we extracted the information we need from the init request, but we didn't reply to that request.

As we do nothing special at that stage, we just return the following response (where "id" will be changed in the Bash script to the value we received from lightningd):

{
  "jsonrpc": "2.0",
  "id": "id",
  "result": {}
}

We modify myplugin.bash consequently:

...
# init

read -r JSON
read -r _
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
socket_path=$(echo "$JSON" | jq '.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"' -r)

echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {}}'

And now for the first time we start our plugin successfully:

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bcli",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/commando",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/funder",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/topology",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/keysend",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/offers",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/pay",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/txprepare",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/spenderp",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/sql",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/home/tony/lnroom/myplugin.bash",
         "active": true,
         "dynamic": true
      }
   ]
}

In the previous output, we see /home/tony/lnroom/myplugin.bash plugin listed, but if we look for a process associated with that file we get nothing:

◉ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1139988 pts/1    S+     0:00 rg myplugin

This is totally normal, because after replying to the init request myplugin.bash script does nothing more and the process stops.

IO loop

Get the plugin working

Last part to take on in this video is the IO loop.

Let's start by writing to /tmp/myplugin file the request lightningd forwards us when a client calls the mymethod method. This is done like this:

...
# i/o loop

while read -r JSON; do
    read -r _
    id=$(echo "$JSON" | jq .id)
    echo "$JSON" >> /tmp/myplugin
done

Let's start myplugin.bash plugin and check that we have a process associated to our pluging:

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1140059 pts/1    S      0:00 /usr/bin/bash /home/tony/lnroom/myplugin.bash
1140078 pts/1    S+     0:00 rg myplugin

Now when we call mymethod, the terminal hangs (we can stop it with C-c key binding) because we don't answer to mymethod requests in myplugin.bash plugin:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod
^C

Before fixing this we can take a look at mymethod which has been written in /tmp/myplugin file:

{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140094/cln:mymethod#130", "params" :[ ] }

Let's call mymethod with key/value arguments

◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
^C

and observe the changes in the params field of mymethod request that are accessible in /tmp/myplugin file:

{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140094/cln:mymethod#130", "params" :[ ] }
{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140195/cln:mymethod#139", "params" :{ "foo1" : "bar1", "foo2" : "bar2"} }

Let's call mymethod with positional arguments

◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod bar1 bar2
^C

and observe the changes in the params field of mymethod request that are accessible in /tmp/myplugin file:

{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140094/cln:mymethod#130", "params" :[ ] }
{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140195/cln:mymethod#139", "params" :{ "foo1" : "bar1", "foo2" : "bar2"} }
{ "jsonrpc" : "2.0", "method" : "mymethod", "id" : "cli:mymethod#1140214/cln:mymethod#142", "params" :[ "bar1", "bar2"] }

For the first time, we make our plugin working correctly by replying to mymethod requests with the payload being {"foo":"bar"} like this

...
# i/o loop

while read -r JSON; do
    read -r _
    id=$(echo "$JSON" | jq .id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":"bar"}}'
done

We restart the plugin

◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}

and we call mymethod method which returns the following:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "bar"
}

cli params

Now let's extract the cli parameters out of the request. To do that we modify myplugin.bash like this:

...
# i/o loop

while read -r JSON; do
    read -r _
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":'"$cli_params"'}}'
done

We restart the plugin

◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...

and we call mymethod with no argument, with key/value arguments and with positional arguments which gives us the following:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": []
}
◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "foo": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod bar1 bar2
{
   "foo": [
      "bar1",
      "bar2"
   ]
}

node id and JSON-RPC over Unix socket

Now let's retrieve the node id of the node running the plugin.

In our terminal using lightning-cli client, we can get the node by calling getinfo command like this:

◉ tony@tony:~/lnroom:
$ l1-cli getinfo | jq .id -r
024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830

We could also use nc utility to open a Unix socket connection with the node l1 like this

◉ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc

then inserting the a getinfo request in the terminal

◉ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}

and by pressing return we would receive getinfo response from l1 node like this

◉ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}
{"jsonrpc":"2.0","id":"1","result":{"id":"024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"}}

^C

This is something that we can do in our Bash script.

So, in myplugin.bash we use nc with the variable socket_path instead of the file /tmp/l1-regtest/regtest/lightning-rpc, we do a process substition (<(...)) that we redirect (<) to the read command in order to store the getinfo response in the variable JSON_1:

...
# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":'"$JSON_1"'}}'
done

We restart the plugin

◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...

and we call mymethod to check that this returns the getinfo request filtered in the field foo:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": {
      "jsonrpc": "2.0",
      "id": "1",
      "result": {
         "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
      }
   }
}

Now we store the node id in the variable node_id and return it the payload of the response:

...
# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"foo":'"$node_id"'}}'
done

We restart the plugin and call mymethod to check that everything is working as expected:

◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
}

Finally, we modify myplugin.bash to returns the node id, the startup options and the cli parameters in the payload of the repsonses to mymethod requests:

...
# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"id": '"$node_id"', "options": {"foo_opt": '"$foo_opt"'}, "cli_params": '"$cli_params"'}}'
done

We check that this works by restarting the plugin and calling mymethod method with no arguments and key/value arguments:

◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
...
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": []
}
◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Last thing! Does foo_opt startup option get set correctly? Let's find out by restarting the plugin with foo_opt set to BAR

◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
...
◉ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR
...

and calling mymethod method:

◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "BAR"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Everything is OK.

We are done!

Terminal session

We ran the following commands in this order:

$ ./setup.sh
$ source lightning/contrib/startup_regtest.sh
$ start_ln
$ alias l1-cli
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ ps -ax | rg myplugin
$ l1-cli plugin start $(pwd)/myplugin.bash
$ ps -ax | rg myplugin
$ l1-cli mymethod
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
$ l1-cli mymethod bar1 bar2
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
$ l1-cli mymethod bar1 bar2
$ l1-cli getinfo | jq .id -r
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli plugin start $(pwd)/myplugin.bash
$ l1-cli mymethod
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
$ l1-cli plugin stop $(pwd)/myplugin.bash
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR
$ l1-cli -k mymethod foo1=bar1 foo2=bar2

And below you can read the terminal session (command lines and outputs):

◉ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
lightningd v23.02.2
GNU bash, version 5.1.16(1)-release (x86_64-pc-linux-gnu)
◉ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is hashed (/usr/local/bin/lightning-cli)
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
◉ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 1139027
[2] 1139063
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes
◉ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to getmanifest"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "code": -3,
   "message": "/home/tony/lnroom/myplugin.bash: exited before replying to init"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bcli",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/commando",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/funder",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/topology",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/keysend",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/offers",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/pay",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/txprepare",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/spenderp",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/sql",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/home/tony/lnroom/myplugin.bash",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1139988 pts/1    S+     0:00 rg myplugin
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ ps -ax | rg myplugin
1140059 pts/1    S      0:00 /usr/bin/bash /home/tony/lnroom/myplugin.bash
1140078 pts/1    S+     0:00 rg myplugin
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
^C
◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
^C
◉ tony@tony:~/lnroom:
$ l1-cli mymethod bar1 bar2
^C
◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "bar"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": []
}
◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "foo": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod bar1 bar2
{
   "foo": [
      "bar1",
      "bar2"
   ]
}
◉ tony@tony:~/lnroom:
$ l1-cli getinfo | jq .id -r
024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830
◉ tony@tony:~/lnroom:
$ nc -U /tmp/l1-regtest/regtest/lightning-rpc
{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}
{"jsonrpc":"2.0","id":"1","result":{"id":"024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"}}

^C
◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": {
      "jsonrpc": "2.0",
      "id": "1",
      "result": {
         "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
      }
   }
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830"
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.bash
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": []
}
◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.bash
{
   "command": "stop",
   "result": "Successfully stopped myplugin.bash."
}
◉ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.bash foo_opt=BAR
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "id": "024da50dc4c50f06c2c00c5a2363fc0ccdd28b0d30323d290de43415d63445a830",
   "options": {
      "foo_opt": "BAR"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

Source code

myplugin.bash

#!/usr/bin/bash

set -e

[[ -e /tmp/myplugin ]] && rm /tmp/myplugin

# getmanifest

read -r JSON
read -r _
# echo "$JSON" >> /tmp/myplugin
id=$(echo "$JSON" | jq .id)
echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"dynamic": true, "options": [{"name": "foo_opt", "type": "string", "default": "bar", "description": "description"}], "rpcmethods": [{"name": "mymethod", "usage": "", "description": "description"}]}}'

# init

read -r JSON
read -r _
# echo "$JSON" >> /tmp/myplugin
id=$(echo "$JSON" | jq .id)
foo_opt=$(echo "$JSON" | jq .params.options.foo_opt)
socket_path=$(echo "$JSON" | jq '.params.configuration."lightning-dir" + "/" + .params.configuration."rpc-file"' -r)

echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {}}'

# i/o loop

while read -r JSON; do
    read -r _
    # echo "$JSON" >> /tmp/myplugin
    id=$(echo "$JSON" | jq .id)
    cli_params=$(echo "$JSON" | jq .params)
    read -r JSON_1 < <(echo '{"jsonrpc": "2.0", "id": "1", "method": "getinfo", "params": [], "filter": { "id": true }}' | nc -U $socket_path)
    node_id=$(echo "$JSON_1" | jq .result.id)
    echo '{"jsonrpc": "2.0", "id": '"$id"', "result": {"id": '"$node_id"', "options": {"foo_opt": '"$foo_opt"'}, "cli_params": '"$cli_params"'}}'
done

setup.sh

#!/usr/bin/env bash

ubuntu=$(lsb_release -ds)
lightningd=$(lightningd --version | xargs printf "lightningd %s\n")
bash_version=$(bash --version | head -n1)

printf "%s\n%s\n%s\n%s" "$ubuntu" "$lightningd" "$bash_version"

Resources

Understand CLN Plugin mechanism with a Bash example

Transcript with corrections and improvements

Core Lightning plugin system

Setup

Start 2 Lightning nodes running on regtest

Write in the /tmp/myplugin file

getmanifest request

init request

IO loop

Get the plugin working

cli params

node id and JSON-RPC over Unix socket

Terminal session

Source code

myplugin.bash

setup.sh

Resources

Create and pay an invoice with CLN

Start a lightning node on regtest with CLN

Connect Lightning nodes on regtest with CLN

Fund the wallet of a CLN Lightning node running on regtest

Open a channel between two nodes running on regtest using CLN

Close payment channels of lightning nodes running on regtest with CLN

List payments of a CLN lightning node greater than 20000sat

A penalty transaction managed by Core Lightning

Another penalty transaction on regtest with Core Lightning

Understand CLN Plugin mechanism with a Python example

Register a JSON-RPC method to Core Lightning using pyln-client Python package

Start writing Core Lightning plugins with pyln-client TODAY!

Overview of pyln-client implementation - Plugin.run() - Part 1

How to write tests for CLN plugins

Overview of pyln-client implementation - @plugin.method() - Part 2

Overview of pyln-client implementation - LightningRpc - Part 3

Understand CLN Plugin mechanism with a Bash example

Learn how to subscribe to lightningd event notifications with CLN plugins

Subscribe to lightningd notification topics with a Python plugin

Subscribe to connect notifications with pyln-client

How Core Lightning plugins can communicate with each other?

Core lightning rpc_command hook, pay command and BOLT11 invoice

How does Core Lightning communicate with the Bitcoin network?

Core Lightning plugins in 2023

Write a Core Lightning plugin in Javascript

Write a Core Lightning plugin in Go

Write a Core Lightning plugin in Rust

Introduction to commando and commando-rune

Simple CLN bookkeeper web app powered by lnsocket & Golang - part 1

Simple CLN bookkeeper web app powered by lnsocket & Golang - part 2

Overview of lnmessage implementation

Create invoices with a Node.JS cli using lnmessage and commando

Core Lightning release v23.08 live exploration

Get started with Lightning Network Splicing on Core Lightning v23.08

Getting started with CLNRest plugin

Learn how to test libraries/apps that interact with CLN nodes

Get started with cln-grpc plugin

Remote control of Core Lightning Nodes in 2023

Core Lightning implementation of BOLT #11 invoices - part 1

Core Lightning implementation of BOLT #11 invoices - part 2

Core Lightning implementation of BOLT #11 invoices - part 3

Core Lightning implementation of BOLT #11 invoices - part 4

Core Lightning implementation of BOLT #11 invoices - part 5

Core Lightning implementation of BOLT #11 invoices - part 6

Core Lightning implementation of BOLT #11 invoices - part 7

Core Lightning implementation of BOLT #11 invoices - part 8