Write a Core Lightning plugin in Go

Before implementing that plugin, let's describe Core Lightning plugin system.

Once lightningd has started the plugin as a subprocess, the plugin and lightningd communicate using pipes. Specifically,

1. lightningd writes to the stdin stream of the plugin to send it messages and

2. the plugin writes to its stdout stream to send messages to lightningd.

To understand each other they use the JSON-RPC protocol.

That communcation can be represented like this:

          ->  stdin
         |        |
lightningd        plugin
         |        |
          <- stdout

Now that we know how lightningd and plugins communicate with each other, let's take a look at the life cycle of a plugin like myplugin which registers a JSON-RPC method called mymethod:

1. When lightningd starts myplugin, it sends the getmanifest request to myplugin. myplugin replies to lightningd with a response containing the declaration of the startup option foo_opt and the information to register mymethod JSON-RPC method.

2. Then lightningd sends the init request to myplugin. This tells myplugin that lightningd is ready to communicate. That init request contains informations that the plugin might needs to work correctly. For instance, myplugin needs to know the value of the startup option foo_opt and to know the Unix socket file to use to connect to the node and to send JSON-RPC requests in order to retrieve the node id. Those informations are available in the init request.

3. Then myplugin starts an IO loop and waits in its stdin stream incoming requests from lightningd:

   • (a). A client sends a mymethod request to lightningd. As myplugin has registered mymethod JSON-RPC method to lightningd, lightningd forwards that request to myplugin,

   • (b). myplugin receives that request, builds a response and sends it back to lightningd,

   • (c). Finally, lightningd forwards that response to the client,

   • (d). We repeat (a) to (c).

Here is my setup:

lightningd v23.05.2
go 1.9
lightningd-gjson-rpc v1.6.2
Ubuntu 22.04.2 LTS

Let's start two Lightning nodes running on the Bitcoin regtest chain by sourcing the script lightning/contrib/startup_regtest.sh provided in CLN repository and by running the command start_ln:

◉ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
...
◉ tony@tony:~/lnroom:
$ start_ln
...

We can check that l1-cli is just an alias for lightning-cli with the base directory being /tmp/l1-regtest:

◉ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'

In the file main.go, first we import the module lightningd-gjson-rpc/plugin. Then in the main function we define p a dynamic plugin of type plugin.Plugin. Finally, at the end of the main function we start the I/O loop with p.Run():

package main

import "github.com/fiatjaf/lightningd-gjson-rpc/plugin"

func main() {
  p := plugin.Plugin{
    Dynamic: true,
}

  p.Run()
}

This defines a CLN plugin (thought it does nothing).

Let's create go.mod and go.sum files (which also downloads the dependencies - they are already present on my machine) like this:

◉ tony@tony:~/lnroom:
$ go mod init myplugin
go: creating new go.mod: module myplugin
go: to add module requirements and sums:
        go mod tidy
◉ tony@tony:~/lnroom:
$ go mod tidy
go: finding module for package github.com/fiatjaf/lightningd-gjson-rpc/plugin
go: found github.com/fiatjaf/lightningd-gjson-rpc/plugin in github.com/fiatjaf/lightningd-gjson-rpc v1.6.2

Now we compile main.go into myplugin binary:

◉ tony@tony:~/lnroom:
$ go build -o myplugin

We can now start the myplugin plugin:

◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin
2023-07-13T17:51:33.905Z plugin- initialized plugin
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bcli",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/commando",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/funder",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/topology",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/keysend",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/offers",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/pay",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/txprepare",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/spenderp",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/sql",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/home/tony/lnroom/myplugin",
         "active": true,
         "dynamic": true
      }
   ]
}

Although our plugin doesn't do anything, we can check that it's running.

◉ tony@tony:~/lnroom:
$ ps -ax | rg mypl
 110238 pts/1    Sl     0:00 /home/tony/lnroom/myplugin
 110258 pts/1    S+     0:00 rg mypl

Now let's declare mymethod method which always returns the json object {"foo": "bar"}:

package main

import "github.com/fiatjaf/lightningd-gjson-rpc/plugin"

func main() {
  p := plugin.Plugin{
    Dynamic: true,
    RPCMethods: []plugin.RPCMethod{
      {
        Name: "mymethod",
        Usage: "",
        Description: "Description",
        LongDescription: "LongDescription",
        Handler: func(p *plugin.Plugin, params plugin.Params) (interface{}, int, error){
          return map[string]interface{}{
            "foo": "bar",
          }, 0, nil
        },
      },
    },
  }

  p.Run()
}

Note that in the video, I used (resp interface{}, errCode int, err error) as returned values of the handler function instead of (interface{}, int, error).

Let's compile and restart the plugin like this

◉ tony@tony:~/lnroom:
$ go build -o myplugin
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin
2023-07-13T17:57:55.718Z plugin- initialized plugin
{
   "command": "start",
   "plugins": [...]
}

and call mymethod method by running the following command:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo": "bar"
}

Not so bad. We know how to register JSON-RPC commands.

Let's continue.

We can declare startup options to lightningd using Options field of the struct plugin.Plugin.

After we receive the init request from lightningd, lightningd-gjson-rpc takes care to put the startup options we've declared to lightningd in p.Args.

For instance, we can declare the startup option foo_opt and access its value in mymethod's handler this way:

package main

import "github.com/fiatjaf/lightningd-gjson-rpc/plugin"

func main() {
  p := plugin.Plugin{
    Dynamic: true,
    Options: []plugin.Option{
      {
        Name: "foo_opt",
        Type: "string",
        Default: "bar",
        Description: "Description",
      },
    },
    RPCMethods: []plugin.RPCMethod{
      {
        Name: "mymethod",
        Usage: "",
        Description: "Description",
        LongDescription: "LongDescription",
        Handler: func(p *plugin.Plugin, params plugin.Params) (interface{}, int, error){
          fooOpt := p.Args.Get("foo_opt").String()
          return map[string]interface{}{
              "foo_opt": fooOpt,
          }, 0, nil
        },
      },
    },
  }

  p.Run()
}

Back to our terminale, we compile the plugin, we restart it and call mymethod method:

◉ tony@tony:~/lnroom:
$ go build -o myplugin
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin
2023-07-13T18:02:34.424Z plugin- initialized plugin
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo_opt": "bar"
}

Fine, we got the default value of foo_opt option. Let's see if we can set foo_opt to BAR. To do this, first we stop the plugin:

◉ tony@tony:~/lnroom:
$ l1-cli plugin stop myplugin
{
   "command": "stop",
   "result": "Successfully stopped myplugin."
}

Then we start the plugin with foo_opt option set to BAR like this:

◉ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin foo_opt=BAR
2023-07-13T18:03:38.548Z plugin- initialized plugin
{
   "command": "start",
   "plugins": [...]
}

Finally, calling mymethod method we check that foo_opt has been set correctly:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "foo_opt": "BAR"
}

Great! We know how to declare startup options and use them.

In this part we focus on cli parameters handling.

In the field Usage of a RPCMethod struct we can define:

• no parameters (using empty string),

• optional parameters (enclosing the name of a parameter with brackets) or

• mandatory parameters (put the name with no brackets).

For instance using Usage: "foo1 [foo2]" defined foo1 as mandatory parameter and foo2 as an optional parameter.

When parameters are passed to the command, they are available in the handler function via its params argument.

Here the new implemention of our plugin which takes two parameters and returns those parameters:

package main

import "github.com/fiatjaf/lightningd-gjson-rpc/plugin"

func main() {
  p := plugin.Plugin{
    Dynamic: true,
    Options: []plugin.Option{
      {
        Name: "foo_opt",
        Type: "string",
        Default: "bar",
        Description: "Description",
      },
    },
    RPCMethods: []plugin.RPCMethod{
      {
        Name: "mymethod",
        Usage: "foo1 [foo2]",
        Description: "Description",
        LongDescription: "LongDescription",
        Handler: func(p *plugin.Plugin, params plugin.Params) (interface{}, int, error){
          foo1 := params.Get("foo1").String()
          foo2 := params.Get("foo2").String()
          return map[string]interface{}{
            "cli_params": map[string]interface{}{
              "foo1": foo1,
              "foo2": foo2,
            },
          }, 0, nil
        },
      },
    },
  }

  p.Run()
}

Let's compile and restart the plugin:

◉ tony@tony:~/lnroom:
$ go build -o myplugin
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin
2023-07-13T18:07:53.802Z plugin- initialized plugin
{
   "command": "start",
   "plugins": [...]
}

Calling mymethod with no parameters raises an error as we can see below:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod
2023-07-13T18:08:11.923Z plugin- Error decoding params 'foo1 [foo2]': required parameter foo1 missing.
{
   "code": 400,
   "message": "Error decoding params: foo1 [foo2]",
   "data": null
}

Let's call mymethod method with one positional parameter, 2 positional parameters and 2 key/value parameters:

◉ tony@tony:~/lnroom:
$ l1-cli mymethod bar1
{
   "cli_params": {
      "foo1": "bar1",
      "foo2": ""
   }
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod bar1 bar2
{
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

We know how to deal with cli parameters.

Let's continue.

Let's see how we can get the node id of l1 node, the node running the plugin.

We can get that node id by doing a getinfo RPC call to the node.

Indeed, in the I/O loop of our plugin (started with p.Run()) the plugin first answers to the getmanifest request and then answers to the init request. Just before sending the init response, lightningd-gjson-rpc set p.Client field with the struct lightning.Client. This allows us to do JSON-RPC call to the node running the plugin like this:

p.Client.Call(...)

So to retrieve the node id and return it in the json response, we change our plugin to be this:

package main

import "github.com/fiatjaf/lightningd-gjson-rpc/plugin"

func main() {
  p := plugin.Plugin{
    Dynamic: true,
    Options: []plugin.Option{
      {
        Name: "foo_opt",
        Type: "string",
        Default: "bar",
        Description: "Description",
      },
    },
    RPCMethods: []plugin.RPCMethod{
      {
        Name: "mymethod",
        Usage: "",
        Description: "Description",
        LongDescription: "LongDescription",
        Handler: func(p *plugin.Plugin, params plugin.Params) (interface{}, int, error){
          getinfo, _ := p.Client.Call("getinfo")
          return map[string]interface{}{
            "node_id": getinfo.Get("id").String(),
          }, 0, nil
        },
      },
    },
  }

  p.Run()
}

Back to our terminal, we compile and restart our plugin, and call mymethod method which returns l1's node id:

$ go build -o myplugin
◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin
2023-07-13T18:13:49.722Z plugin- initialized plugin
{
   "command": "start",
   "plugins": [...]
}
◉ tony@tony:~/lnroom:
$ l1-cli mymethod
{
   "node_id": "03e815a542572d3427cac4270e20f34e47fe4dcde9c214fdf64a44fb2393033841"
}

We can check that we got the correct id by calling directly getinfo method:

◉ tony@tony:~/lnroom:
$ l1-cli getinfo | jq -r .id
03e815a542572d3427cac4270e20f34e47fe4dcde9c214fdf64a44fb2393033841

Now we just have to put all the pieces together like this:

package main

import "github.com/fiatjaf/lightningd-gjson-rpc/plugin"

func main() {
  p := plugin.Plugin{
    Dynamic: true,
    Options: []plugin.Option{
      {
        Name: "foo_opt",
        Type: "string",
        Default: "bar",
        Description: "Description",
      },
    },
    RPCMethods: []plugin.RPCMethod{
      {
        Name: "mymethod",
        Usage: "foo1 [foo2]",
        Description: "Description",
        LongDescription: "LongDescription",
        Handler: func(p *plugin.Plugin, params plugin.Params) (interface{}, int, error){
          fooOpt := p.Args.Get("foo_opt").String()
          foo1 := params.Get("foo1").String()
          foo2 := params.Get("foo2").String()
          getinfo, _ := p.Client.Call("getinfo")
          return map[string]interface{}{
            "node_id": getinfo.Get("id").String(),
            "options": map[string]interface{}{
              "foo_opt": fooOpt,
            },
            "cli_params": map[string]interface{}{
              "foo1": foo1,
              "foo2": foo2,
            },
          }, 0, nil
        },
      },
    },
  }

  p.Run()
}

Therefore, after compiling and restarting our plugin with the startup option foo_opt set to BAR like this

◉ tony@tony:~/lnroom:
$ go build -o myplugin
◉ tony@tony:~/lnroom:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin foo_opt=BAR
2023-07-13T18:16:53.479Z plugin- initialized plugin
{
   "command": "start",
   "plugins": [...]
}

and calling mymethod method with key/value pairs foo1=bar1 and foo2=bar2 we get the following:

◉ tony@tony:~/lnroom:
$ l1-cli -k mymethod foo1=bar1 foo2=bar2
{
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   },
   "node_id": "03e815a542572d3427cac4270e20f34e47fe4dcde9c214fdf64a44fb2393033841",
   "options": {
      "foo_opt": "BAR"
   }
}

We are done!

(Note that code from fiatjaf/lightningd-gjson-rpc is licensed under the following MIT license.)

During the demo, we've looked at:

• Plugin type defined in lightningd-gjson-rpc:plugin/plugin.go:

  type Plugin struct {
    Client  *lightning.Client            `json:"-"`
    Log     func(...interface{})         `json:"-"`
    Logf    func(string, ...interface{}) `json:"-"`
    Name    string                       `json:"-"`
    Version string                       `json:"-"`
    Network string                       `json:"-"`
  
    Options       []Option            `json:"options"`
    RPCMethods    []RPCMethod         `json:"rpcmethods"`
    Subscriptions []Subscription      `json:"subscriptions"`
    Hooks         []Hook              `json:"hooks"`
    Features      Features            `json:"featurebits"`
    Dynamic       bool                `json:"dynamic"`
    Notifications []NotificationTopic `json:"notifications"`
  
    Configuration  Params            `json:"-"`
    Args   Params        `json:"-"`
    OnInit func(*Plugin) `json:"-"`
  }

• RPCMethod type defined in lightningd-gjson-rpc:plugin/plugin.go:

  type RPCMethod struct {
    Name            string     `json:"name"`
    Usage           string     `json:"usage"`
    Description     string     `json:"description"`
    LongDescription string     `json:"long_description"`
    Handler         RPCHandler `json:"-"`
  }

• RPCHandler function type defined in lightningd-gjson-rpc:plugin/plugin.go:

  type RPCHandler func(p *Plugin, params Params) (resp interface{}, errCode int, err error)

• Params type defined in lightningd-gjson-rpc:plugin/params.go:

  type Params map[string]interface{}

• Option type defined in lightningd-gjson-rpc:plugin/plugin.go:

  type Option struct {
    Name        string      `json:"name"`
    Type        string      `json:"type"`
    Default     interface{} `json:"default"`
    Description string      `json:"description"`
  }

package main

import "github.com/fiatjaf/lightningd-gjson-rpc/plugin"

func main() {
  p := plugin.Plugin{
    Dynamic: true,
    Options: []plugin.Option{
      {
        Name: "foo_opt",
        Type: "string",
        Default: "bar",
        Description: "Description",
      },
    },
    RPCMethods: []plugin.RPCMethod{
      {
        Name: "mymethod",
        Usage: "foo1 [foo2]",
        Description: "Description",
        LongDescription: "LongDescription",
        Handler: func(p *plugin.Plugin, params plugin.Params) (interface{}, int, error){
          fooOpt := p.Args.Get("foo_opt").String()
          foo1 := params.Get("foo1").String()
          foo2 := params.Get("foo2").String()
          getinfo, _ := p.Client.Call("getinfo")
          return map[string]interface{}{
            "node_id": getinfo.Get("id").String(),
            "options": map[string]interface{}{
              "foo_opt": fooOpt,
            },
            "cli_params": map[string]interface{}{
              "foo1": foo1,
              "foo2": foo2,
            },
          }, 0, nil
        },
      },
    },
  }

  p.Run()
}

module myplugin

go 1.19

require github.com/fiatjaf/lightningd-gjson-rpc v1.6.2

require (
        github.com/aead/siphash v1.0.1 // indirect
        github.com/btcsuite/btcd v0.23.2 // indirect
        github.com/btcsuite/btcd/btcec/v2 v2.2.0 // indirect
        github.com/btcsuite/btcd/btcutil v1.1.1 // indirect
        github.com/btcsuite/btcd/btcutil/psbt v1.1.4 // indirect
        github.com/btcsuite/btcd/chaincfg/chainhash v1.0.1 // indirect
        github.com/btcsuite/btclog v0.0.0-20170628155309-84c8d2346e9f // indirect
        github.com/btcsuite/btcwallet v0.15.1 // indirect
        github.com/btcsuite/btcwallet/wallet/txauthor v1.2.3 // indirect
        github.com/btcsuite/btcwallet/wallet/txrules v1.2.0 // indirect
        github.com/btcsuite/btcwallet/wallet/txsizes v1.1.0 // indirect
        github.com/btcsuite/btcwallet/walletdb v1.4.0 // indirect
        github.com/btcsuite/btcwallet/wtxmgr v1.5.0 // indirect
        github.com/btcsuite/go-socks v0.0.0-20170105172521-4720035b7bfd // indirect
        github.com/btcsuite/websocket v0.0.0-20150119174127-31079b680792 // indirect
        github.com/davecgh/go-spew v1.1.1 // indirect
        github.com/decred/dcrd/crypto/blake256 v1.0.0 // indirect
        github.com/decred/dcrd/dcrec/secp256k1/v4 v4.0.1 // indirect
        github.com/decred/dcrd/lru v1.0.0 // indirect
        github.com/go-errors/errors v1.0.1 // indirect
        github.com/kkdai/bstream v1.0.0 // indirect
        github.com/lightninglabs/gozmq v0.0.0-20191113021534-d20a764486bf // indirect
        github.com/lightninglabs/neutrino v0.14.2 // indirect
        github.com/lightningnetwork/lnd v0.15.0-beta // indirect
        github.com/lightningnetwork/lnd/clock v1.1.0 // indirect
        github.com/lightningnetwork/lnd/queue v1.1.0 // indirect
        github.com/lightningnetwork/lnd/ticker v1.1.0 // indirect
        github.com/lightningnetwork/lnd/tlv v1.0.3 // indirect
        github.com/lightningnetwork/lnd/tor v1.0.1 // indirect
        github.com/miekg/dns v1.1.43 // indirect
        github.com/tidwall/gjson v1.6.1 // indirect
        github.com/tidwall/match v1.0.1 // indirect
        github.com/tidwall/pretty v1.0.2 // indirect
        golang.org/x/crypto v0.0.0-20210921155107-089bfa567519 // indirect
        golang.org/x/net v0.0.0-20211015210444-4f30a5c0130f // indirect
        golang.org/x/sys v0.0.0-20220520151302-bc2c85ada10a // indirect
        golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1 // indirect
)