Register a JSON-RPC method to Core Lightning using pyln-client Python package

LIVE #2April 13, 2023

In this live, we register a JSON-RPC method in Core Lightning using pyln-client Python package. As pyln-client uses Python decorators to declare JSON-RPC methods, in the second part we try to understand how they work by playing with 2 examples, the second one being very close to the way pyln-client uses them.

Transcript with corrections and improvements

Introduction

Hi everybody, I'm really happy to be with you for another live.

To make this session more interactive, we will divide it into three parts of about 20 minutes of coding and 10 minutes of chat.

In the first live we tried to understand CLN Plugin mechanism by building a plugin in Python from scratch. In some way, we used Python as a "proxy" language to demonstrate how plugins work but we could have done it the same way with any other general purpose language.

Today we'll start by registering the same JSON-RPC method in Core Lightning as in the first live but using pyln-client Python package.

This is faster to write as it takes only 25% of the LOC that we wrote last time to get the plugin working the same way.

As pyln-client uses Python decorators to declare JSON-RPC methods, in the second part we'll try to understand how they work by playing with 2 examples, the second one being very close to the way pyln-client uses them.

Finally we'll look at Plugin.method method implementation (We haven't seen this part, but I'll talk about it in a video that will be posted soon).

Let's go.

Register myplugin method to CLN using pyln-client

Let's add the method myplugin to CLN by writing a dynamic Python plugin called myplugin.py using pyln-client.

When we start the plugin myplugin.py with the option foo_opt set to BAR like this

◉ tony@tony:~/clnlive:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.py foo_opt=BAR

where l1-cli is an alias for lightning-cli --lightning-dir=/tmp/l1-regtest, we expect myplugin method called with the parameters foo1=bar1 and foo2=bar2 to gives us the following

◉ tony@tony:~/clnlive:
$ l1-cli -k myplugin foo1=bar1 foo2=bar2
{
   "node_id": "029999ce6a1b74cf077f503b81bb4d95af34114363bdac5922a9b353ce0449bac4",
   "options": {
      "foo_opt": "BAR"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

where the value of node_id is the ID of the node l1.

Setup

Here is my setup:

◉ tony@tony:~/clnlive:
$ lightningd --version | xargs printf "lightningd %s\n" && python --version && lsb_release -ds
lightningd v23.02.2
Python 3.10.6
Ubuntu 22.04.2 LTS

Start 2 Lightning nodes running on regtest

Let's start two Lightning nodes running on the Bitcoin regtest chain by sourcing the script lightning/contrib/startup_regtest.sh provided in CLN repository and by running the command start_ln:

◉ tony@tony:~/clnlive:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
◉ tony@tony:~/clnlive:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 108917
[2] 108952
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes

We can check that l1-cli is just an alias for lightning-cli with the base directory being /tmp/l1-regtest:

◉ tony@tony:~/clnlive:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'

Installing and using pyln-client

Note that during the live we got the Python error ModuleNotFoundError: No module named 'pyln' when we tried to use pyln-client installed in a Python virtual environment.

I described here what we did, why we got that error (not in the video) and how not do to that mistake again (not in the video).

In the executable file myplugin.py, we import the class Plugin from the module pyln.client, then we instantiate the object plugin with the class Plugin and finally we start the I/O loop with plugin.run():

#!/usr/bin/env python

from pyln.client import Plugin

plugin = Plugin()

plugin.run()

This is the minimum we need to have the plugin working (thought it does nothing).

Now, as we haven't installed pyln-client yet (note that it is not installed globlally in my computer), if we try to start myplugin.py plugin we get the following error:

◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
Traceback (most recent call last):
  File "/home/tony/clnlive/myplugin.py", line 3, in <module>
    from pyln.client import Plugin
ModuleNotFoundError: No module named 'pyln'
{
   "code": -3,
   "message": "/home/tony/clnlive/myplugin.py: exited before replying to getmanifest"
}

Let's install pyln-client in the virtual environment .venv using pip like this:

◉ tony@tony:~/clnlive:
$ python -m venv .venv
◉ tony@tony:~/clnlive:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/clnlive:
$ which python
/home/tony/clnlive/.venv/bin/python
(.venv) ◉ tony@tony:~/clnlive:
$ pip install pyln-client
...
Successfully installed PySocks-1.7.1 asn1crypto-1.5.1 base58-2.1.1 bitstring-3.1.9 cffi-1.15.1 coincurve-17.0.0 cryptography-36.0.2 pycparser-2.21 pyln-bolt7-1.0.246 pyln-client-23.2 pyln-proto-23.2

We can list the newly installed packages in our virtual environment that we have activated:

(.venv) ◉ tony@tony:~/clnlive:
$ pip list
Package      Version
------------ -------
asn1crypto   1.5.1
base58       2.1.1
bitstring    3.1.9
cffi         1.15.1
coincurve    17.0.0
cryptography 36.0.2
pip          22.0.2
pycparser    2.21
pyln-bolt7   1.0.246
pyln-client  23.2
pyln-proto   23.2
PySocks      1.7.1
setuptools   59.6.0

Now let's try to start myplugin.py again:

(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
Traceback (most recent call last):
  File "/home/tony/clnlive/myplugin.py", line 3, in <module>
    from pyln.client import Plugin
ModuleNotFoundError: No module named 'pyln'
{
   "code": -3,
   "message": "/home/tony/clnlive/myplugin.py: exited before replying to getmanifest"
}

During the live I didn't understand why we got that error thought .venv virtual environment was activated with pyln-client installed and the Python binary used in that environment was /home/tony/clnlive/.venv/bin/python. In order not to waste too much time at this stage of the live, I decided to change the shebang of myplugin.py file from

#!/usr/bin/env python

to

#!/home/tony/clnlive/.venv/bin/python

which tells the Python script to use the modules installed in .venv virtual environment. We deactivated .venv virtual environment and succeeded to start myplugin.py plugin. It worked perfectly and we moved on.

Now let's take look at the mistake I made.

When we tell lightningd to start a plugin, it spawned it as a child process. Child processes inherit their environment variables from their parent. Now, remember that we started lightningd (with start_ln) before activating .venv environment. So /home/tony/clnlive/.venv/bin directory didn't appear at the beginning of PATH environment variable of lightningd process, and so neither appeared in PATH environment variable of the child process and finally /home/tony/clnlive/.venv/bin/python couldn't be found.

What could we have done differently to develop our plugin without changing the shebang #!/usr/bin/env python?

  1. We could have push /home/tony/clnlive/.venv/bin at the beginning of PATH environment variable before starting lightningd (for instance by activating .venv virtual environment before) or,

  2. we could have install pyln-client globally (without using Python virtual environments).

Declare myplugin JSON-RPC method

The file myplugin.py being

#!/home/tony/clnlive/.venv/bin/python

from pyln.client import Plugin

plugin = Plugin()

plugin.run()

we can start the plugin like this

◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bcli",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/commando",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/funder",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/topology",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/keysend",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/offers",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/pay",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/txprepare",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/spenderp",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/sql",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}

and verify that it is indeed running by checking for its process:

◉ tony@tony:~/clnlive:
$ ps -ax | rg myplugin
 109548 pts/1    S      0:00 /home/tony/clnlive/.venv/bin/python /home/tony/clnlive/myplugin.py
 109555 pts/1    D+     0:00 rg myplugin

When we started the plugin, the expression plugin.run() did two things:

  1. first it answered to the getmanifest request sent by lightningd, saying something like this: "I'm declaring no options, no methods, no notifications, nothing" and then

  2. it answered to the init request sent by lightningd saying something like this: "I have no specific initialization to do, we can talk together now"

and after this the plugin started waiting for incoming data in its stdin.

So with no surprise, as myplugin has not been registered in that communication, we see that lightningd doesn't know about myplugin method:

◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "code": -32601,
   "message": "Unknown command 'myplugin'"
}

Now let's declare myplugin method that takes no arguments and returns always the json object {"foo": "bar"}:

#!/home/tony/clnlive/.venv/bin/python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("myplugin")
def myplugin_func(plugin):
    return {"foo": "bar"}

plugin.run()

Let's restart the plugin

◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}

and call myplugin method by running the following command:

◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "foo": "bar"
}

Not so bad. We know how to register JSON-RPC commands.

Let's continue.

Add foo_opt startup option to plugin.py

The class Plugin defined the method get_option that let us get the options we pass when we start the plugin.

In the function myplugin_func we assigned the variable foo_opt to the startup option that we get using get_option method and we return its value in a dictionary like this:

#!/home/tony/clnlive/.venv/bin/python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("myplugin")
def myplugin_func(plugin):
    foo_opt = plugin.get_option("foo_opt")
    return {"foo_opt": foo_opt}

plugin.run()

We restart the plugin

◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}

and we call myplugin method which produces the following error:

◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "code": -32600,
   "message": "Error while processing myplugin: No option with name foo_opt registered",
   "traceback": "Traceback (most recent call last):\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 639, in _dispatch_request\n    result = self._exec_func(method.func, request)\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 616, in _exec_func\n    ret = func(*ba.args, **ba.kwargs)\n  File \"/home/tony/clnlive/myplugin.py\", line 9, in myplugin_func\n    foo_opt = plugin.get_option(\"foo_opt\")\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 435, in get_option\n    raise ValueError(\"No option with name {} registered\".format(name))\nValueError: No option with name foo_opt registered\n"
}

This is totally normal. We tried to use a startup option that we didn't declared in the getmanifest response we sent to lightningd. If we had more complicated error that we need to fix, we could print in a more readable way the error traceback like this:

◉ tony@tony:~/clnlive:
$ python
Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Traceback (most recent call last):\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 639, in _dispatch_request\n    result = self._exec_func(method.func, request)\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 616, in _exec_func\n    ret = func(*ba.args, **ba.kwargs)\n  File \"/home/tony/clnlive/myplugin.py\", line 9, in myplugin_func\n    foo_opt = plugin.get_option(\"foo_opt\")\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 435, in get_option\n    raise ValueError(\"No option with name {} registered\".format(name))\nValueError: No option with name foo_opt registered\n")
Traceback (most recent call last):
  File "/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py", line 639, in _dispatch_request
    result = self._exec_func(method.func, request)
  File "/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py", line 616, in _exec_func
    ret = func(*ba.args, **ba.kwargs)
  File "/home/tony/clnlive/myplugin.py", line 9, in myplugin_func
    foo_opt = plugin.get_option("foo_opt")
  File "/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py", line 435, in get_option
    raise ValueError("No option with name {} registered".format(name))
ValueError: No option with name foo_opt registered

>>>
◉ tony@tony:~/clnlive:

Let's fix that error by declaring the startup option foo_opt with bar as default value using add_option method:

#!/home/tony/clnlive/.venv/bin/python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("myplugin")
def myplugin_func(plugin):
    foo_opt = plugin.get_option("foo_opt")
    return {"foo_opt": foo_opt}

plugin.add_option(name="foo_opt",
                  default="bar",
                  description="'foo_opt description")

plugin.run()

We restart the plugin

◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
...
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
...

and we call myplugin method that should return bar, the default value of foo_opt startup option:

◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "foo_opt": "bar"
}

To be sure that this is working correctly, let's restart myplugin.py plugin with the startup option foo_opt set to BAR

◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
...
◉ tony@tony:~/clnlive:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.py foo_opt=BAR
...

and verify that calling myplugin method returns BAR as value for foo_opt option:

◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "foo_opt": "BAR"
}

Great! We know how to declare startup options and use them.

Pass cli parameters

To take into account cli parameters we have to modify myplugin_func function signature.

To add the two optional cli parameters foo1 and foo2 with their default value being respectively foo1 and foo2, we modify myplugin_func as follow (note that we've also modify the returned value):

#!/home/tony/clnlive/.venv/bin/python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("myplugin")
def myplugin_func(plugin,foo1="foo1", foo2="foo2"):
    foo_opt = plugin.get_option("foo_opt")
    return {
        "node_id": "029999ce6a1b74cf077f503b81bb4d95af34114363bdac5922a9b353ce0449bac4",
        "options": {
            "foo_opt": foo_opt
        },
        "cli_params": {
            "foo1": foo1,
            "foo2": foo2
        }
    }

plugin.add_option(name="foo_opt",
                  default="bar",
                  description="'foo_opt description")

plugin.run()

We restart the plugin

◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
...
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
...

and call myplugin method passing the key/value parameters foo1=bar1 and foo2=bar2 which returned the expected json object with the cli parameters set correctly:

◉ tony@tony:~/clnlive:
$ l1-cli -k myplugin foo1=bar1 foo2=bar2
{
   "node_id": "029999ce6a1b74cf077f503b81bb4d95af34114363bdac5922a9b353ce0449bac4",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}

To be sure we did it the right way, we can call it again with the key/value parameters foo1=bar_1 and foo2=bar_2:

◉ tony@tony:~/clnlive:
$ l1-cli -k myplugin foo1=bar_1 foo2=bar_2
{
   "node_id": "029999ce6a1b74cf077f503b81bb4d95af34114363bdac5922a9b353ce0449bac4",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar_1",
      "foo2": "bar_2"
   }
}

Cool! We know how to pass cli parameters.

Get the node id of the node l1 running plugin.py

Finally we complete myplugin method so that it also returns the node id of the node running the plugin.

We can get the node id via a call to getinfo method like this

◉ tony@tony:~/clnlive:
$ l1-cli getinfo | jq .id
"0341075d01cd6bd67d2410ffaa5607649fcebce23efc0fe7faef882742b0ef3f3f"

and this call can also be done using pyln-client.

Indeed, in the I/O loop of our plugin (started with plugin.run()) the plugin first answers to the getmanifest request and then answers to the init request. Just before sending the init response, pyln-client instantiate plugin.rpc property with the class LightningRpc. This allows us to do JSON-RPC call to the node running the plugin.

For instance, the following expression returns the node id:

plugin.rpc.getinfo()["id"]

So we can modify our plugin like this

#!/home/tony/clnlive/.venv/bin/python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("myplugin")
def myplugin_func(plugin,foo1="foo1", foo2="foo2"):
    node_id = plugin.rpc.getinfo()["id"]
    foo_opt = plugin.get_option("foo_opt")
    return {
        "node_id": node_id,
        "options": {
            "foo_opt": foo_opt
        },
        "cli_params": {
            "foo1": foo1,
            "foo2": foo2
        }
    }

plugin.add_option(name="foo_opt",
                  default="bar",
                  description="'foo_opt description")

plugin.run()

and after restarting the plugin

◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
...
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
...

we can call myplugin method like this

◉ tony@tony:~/clnlive:
$ l1-cli -k myplugin foo1=bar_1 foo2=bar_2
{
   "node_id": "0341075d01cd6bd67d2410ffaa5607649fcebce23efc0fe7faef882742b0ef3f3f",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar_1",
      "foo2": "bar_2"
   }
}

and check that the node id is the same as in:

◉ tony@tony:~/clnlive:
$ l1-cli getinfo | jq .id
"0341075d01cd6bd67d2410ffaa5607649fcebce23efc0fe7faef882742b0ef3f3f"

We are done with this part.

Chat

Are there clients for other languages?
I read in the docs that plugin lightningd communication is done via stdin/stdout. Does it give the best performances we can get? Is there other ways to do the communication?

I don't know.

Python decorators

Motivation

Python decorators are used in pyln-client and in CLN tests which are written in Python and uses pytest.

So if we want either to understand how pyln-client is implemented or we want to read/write tests for CLN, knowing about Python decorators can be useful.

For instance if we search for the occurences of @pytest in lightning repository we get 448 hits:

◉ tony@tony:~/work/repos/lightning:[git»(HEAD detached at v23.02.2)]
$ rg '^@pytest'
contrib/pyln-testing/pyln/testing/fixtures.py
25:@pytest.fixture(scope="session")
46:@pytest.fixture(autouse=True)
69:@pytest.fixture
110:@pytest.fixture
121:@pytest.fixture
126:@pytest.fixture
189:@pytest.fixture
209:@pytest.fixture
426:@pytest.fixture(autouse=True)
453:@pytest.fixture
614:@pytest.fixture
622:@pytest.fixture
629:@pytest.fixture

external/lnprototest/tests/conftest.py
27:@pytest.fixture()  # type: ignore
35:@pytest.fixture()
49:@pytest.fixture()

...

And to be specific here we have test_reconnect_sender_add test which is written composing decorators:

@pytest.mark.developer
@pytest.mark.openchannel('v1')
@pytest.mark.openchannel('v2')
def test_reconnect_sender_add(node_factory):
    disconnects = ['-WIRE_COMMITMENT_SIGNED',
                   '+WIRE_COMMITMENT_SIGNED',
                   '-WIRE_REVOKE_AND_ACK',
                   '+WIRE_REVOKE_AND_ACK']
    if EXPERIMENTAL_DUAL_FUND:
        disconnects = ['=WIRE_COMMITMENT_SIGNED'] + disconnects

    # Feerates identical so we don't get gratuitous commit to update them
    l1 = node_factory.get_node(disconnect=disconnects,
                               may_reconnect=True,
                               feerates=(7500, 7500, 7500, 7500))
    l2 = node_factory.get_node(may_reconnect=True)
    l1.rpc.connect(l2.info['id'], 'localhost', l2.port)

    l1.fundchannel(l2, 10**6)

    amt = 200000000
    inv = l2.rpc.invoice(amt, 'testpayment', 'desc')
    rhash = inv['payment_hash']
    assert only_one(l2.rpc.listinvoices('testpayment')['invoices'])['status'] == 'unpaid'

    route = [{'amount_msat': amt, 'id': l2.info['id'], 'delay': 5, 'channel': first_scid(l1, l2)}]

    # This will send commit, so will reconnect as required.
    l1.rpc.sendpay(route, rhash, payment_secret=inv['payment_secret'])
    # Should have printed this for every reconnect.
    for i in range(0, len(disconnects)):
        l1.daemon.wait_for_log('Already have funding locked in')

A simple decorator example

Decorators are a Python mechanism that allows to modify a function definition by applying it a function (called a decorator) that takes a function as argument and returns another function that becomes the new function definition.

Do do that Python uses the @ sign at the beginning of line followed by a decorator function.

For instance assuming bar is a function that take a function of one argument as argument and return a function with the same signature, we can use it as decorator function to modify the definition of foo function like this:

@bar
def foo(x):
    print(f"'{x}' in 'foo'")

If we send the previous snippet to the Python interpreter, the function foo will be set to something like this:

# this is done by Python, we don't write this assignment
foo = bar(foo)

Now, let's do it for real and define the decorator bar as the identy function which take a function f and return f like this:

def bar(f):
    return f

@bar
def foo(x):
    print(f"'{x}' in 'foo'")

When we send the previous snippet to the Python interpreter, the function foo is set to bar(foo) which is exactly foo and we get the following:

>>> foo("cln")
'cln' in 'foo'

Nothing fancy here but it's a good start.

Let's modify bar by defining in its body a function bar_inner with the same signature as foo, that first prints something and then calls f passed as argument of bar and finally bar return that new function bar_inner:

def bar(f):
    def bar_inner(x):
        print(f"'{x}' in 'bar_inner'")
        f(x)
    return bar_inner

@bar
def foo(x):
    print(f"'{x}' in 'foo'")

When we send the previous snippet to the Python interpreter, the function foo is set to bar(foo) which is bar_inner and we get the following:

>>> foo("cln")
'cln' in 'bar_inner'
'cln' in 'foo'

Using decorators to build a dictionary

In the previous example, the decorator bar takes a function as argument and returns a function and is used with the syntax @bar which is not exactly the same as

@plugin.method("myplugin")
def myplugin_func(...):
    ...
    return {...}

that we used previously to register the JSON-RPC method myplugin using the class Plugin defined in pyln-client package.

Let's have a look to an example closer to what we did to define myplugin JSON-RPC method.

Python decorators mechanism allows also the expression after the @ sign to be a function name (for instance make_decorator) followed by the arguments to be passed to that function enclosed by parentheses (...) like this:

@make_decorator("myplugin")
def method_func(x):
    return {"x_field": x}

In that case, make_decorator must be a function that takes one string as argument and return a function which is a decorator (that means that takes a function as argument and return a function with the same signature).

Hence, assuming make_decorator is well defined, if we send the previous snippet to the Python interpreter, the function method_func will be set to something like this:

# this is done by Python, we don't write this assignment
method_func = make_decorator("myplugin")(method_func)

Now, let's define make_decorator such that the function that it returns (decorator) is in charge to add to the gobal dictionary methods a key/value pair where the key is rpc_method_name (argument of make_decorator function) and the value is f (its argument which will be method_func in our example):

methods = {}

def make_decorator(rpc_method_name):
    def decorator(f):
        methods[rpc_method_name] = f
        return f
    return decorator

@make_decorator("myplugin")
def method_func(x):
    return {"x_field": x}

When we send the previous snippet to the Python interpreter, an entry for myplugin is added to methods dictionary its value being method_func function and method_func is left unmodified. So we have the following:

>>> method_func
<function method_func at 0x7f26d69bdcf0>
>>> methods
{'myplugin': <function method_func at 0x7f26d69bdcf0>}
>>> methods['myplugin']("XXXXXXXXX")
{'x_field': 'XXXXXXXXX'}

Now we can make a parallel between the function make_decorator of the previous example and the method Plugin.method of Plugin class which uses Plugin.add_method method to add method_name entry to self.methods property its value being a Method object instantiated using the argument passed to Plugin.add_method:

class Plugin(object):
    ...
    def add_method(self, name: str, func: Callable[..., Any],
               background: bool = False,
               category: Optional[str] = None,
               desc: Optional[str] = None,
               long_desc: Optional[str] = None,
               deprecated: bool = False) -> None:
        """..."""
        if name in self.methods:
            raise ValueError(
                "Name {} is already bound to a method.".format(name)
            )

        # Register the function with the name
        method = Method(
            name, func, MethodType.RPCMETHOD, category, desc, long_desc,
            deprecated
        )

        method.background = background
        self.methods[name] = method
    ...
    def method(self, method_name: str, category: Optional[str] = None,
               desc: Optional[str] = None,
               long_desc: Optional[str] = None,
               deprecated: bool = False) -> JsonDecoratorType:
        """..."""
        def decorator(f: Callable[..., JSONType]) -> Callable[..., JSONType]:
            self.add_method(method_name,
                            f,
                            background=False,
                            category=category,
                            desc=desc,
                            long_desc=long_desc,
                            deprecated=deprecated)
            return f
        return decorator

We are done!

Chat

I'm not very clear on the kind of things we use lightning plugin for.

As far as I understand, CLN plugin systems is part of the design of the software and many builtin features offer by CLN are implemented as plugins. For instance pay command is implemented in the pay plugin.

You can find more ideas in https://github.com/lightningd/plugins.

Terminal session

We ran the following commands in this order:

$ lightningd --version | xargs printf "lightningd %s\n" && python --version && lsb_release -ds
$ source lightning/contrib/startup_regtest.sh
$ start_ln
$ alias l1-cli
$ l1-cli plugin start $(pwd)/myplugin.py
$ python -m venv .venv
$ source .venv/bin/activate
$ which python
$ pip install pyln-client
$ pip list
$ l1-cli plugin start $(pwd)/myplugin.py
$ deactivate
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ ps -ax | rg myplugin
$ l1-cli myplugin
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli myplugin
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli myplugin
$ python
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli myplugin
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.py foo_opt=BAR
$ l1-cli myplugin
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ ps -ax | rg myplugin
$ l1-cli myplugin
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli -k myplugin foo1=bar1 foo2=bar2
$ l1-cli -k myplugin foo1=bar_1 foo2=bar_2
$ l1-cli getinfo
$ l1-cli getinfo | jq .id
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli getinfo | jq .id
$ l1-cli -k myplugin foo1=bar_1 foo2=bar_2

And below you can read the terminal session (command lines and outputs):

◉ tony@tony:~/clnlive:
$ lightningd --version | xargs printf "lightningd %s\n" && python --version && lsb_release -ds
lightningd v23.02.2
Python 3.10.6
Ubuntu 22.04.2 LTS
◉ tony@tony:~/clnlive:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
◉ tony@tony:~/clnlive:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 108917
[2] 108952
WARNING: eatmydata not found: instal it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes
◉ tony@tony:~/clnlive:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
Traceback (most recent call last):
  File "/home/tony/clnlive/myplugin.py", line 3, in <module>
    from pyln.client import Plugin
ModuleNotFoundError: No module named 'pyln'
{
   "code": -3,
   "message": "/home/tony/clnlive/myplugin.py: exited before replying to getmanifest"
}
◉ tony@tony:~/clnlive:
$ python -m venv .venv
◉ tony@tony:~/clnlive:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/clnlive:
$ which python
/home/tony/clnlive/.venv/bin/python
(.venv) ◉ tony@tony:~/clnlive:
$ pip install pyln-client
Collecting pyln-client
  Using cached pyln_client-23.2-py3-none-any.whl (29 kB)
Collecting pyln-bolt7>=1.0
  Using cached pyln_bolt7-1.0.246-py3-none-any.whl (18 kB)
Collecting pyln-proto>=0.12
  Using cached pyln_proto-23.2-py3-none-any.whl (31 kB)
Collecting PySocks<2.0.0,>=1.7.1
  Using cached PySocks-1.7.1-py3-none-any.whl (16 kB)
Collecting base58<3.0.0,>=2.1.1
  Using cached base58-2.1.1-py3-none-any.whl (5.6 kB)
Collecting coincurve<18.0.0,>=17.0.0
  Using cached coincurve-17.0.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (1.3 MB)
Collecting cryptography<37.0.0,>=36.0.1
  Using cached cryptography-36.0.2-cp36-abi3-manylinux_2_24_x86_64.whl (3.6 MB)
Collecting bitstring<4.0.0,>=3.1.9
  Using cached bitstring-3.1.9-py3-none-any.whl (38 kB)
Collecting asn1crypto
  Using cached asn1crypto-1.5.1-py2.py3-none-any.whl (105 kB)
Collecting cffi>=1.3.0
  Using cached cffi-1.15.1-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (441 kB)
Collecting pycparser
  Using cached pycparser-2.21-py2.py3-none-any.whl (118 kB)
Installing collected packages: bitstring, asn1crypto, PySocks, pyln-bolt7, pycparser, base58, cffi, cryptography, coincurve, pyln-proto, pyln-client
Successfully installed PySocks-1.7.1 asn1crypto-1.5.1 base58-2.1.1 bitstring-3.1.9 cffi-1.15.1 coincurve-17.0.0 cryptography-36.0.2 pycparser-2.21 pyln-bolt7-1.0.246 pyln-client-23.2 pyln-proto-23.2
(.venv) ◉ tony@tony:~/clnlive:
$ pip list
Package      Version
------------ -------
asn1crypto   1.5.1
base58       2.1.1
bitstring    3.1.9
cffi         1.15.1
coincurve    17.0.0
cryptography 36.0.2
pip          22.0.2
pycparser    2.21
pyln-bolt7   1.0.246
pyln-client  23.2
pyln-proto   23.2
PySocks      1.7.1
setuptools   59.6.0
(.venv) ◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
Traceback (most recent call last):
  File "/home/tony/clnlive/myplugin.py", line 3, in <module>
    from pyln.client import Plugin
ModuleNotFoundError: No module named 'pyln'
{
   "code": -3,
   "message": "/home/tony/clnlive/myplugin.py: exited before replying to getmanifest"
}
(.venv) ◉ tony@tony:~/clnlive:
$ deactivate
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bcli",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/commando",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/funder",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/topology",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/keysend",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/offers",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/pay",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/txprepare",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/spenderp",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/sql",
         "active": true,
         "dynamic": true
      },
      {
         "name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
         "active": true,
         "dynamic": false
      },
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ ps -ax | rg myplugin
 109548 pts/1    S      0:00 /home/tony/clnlive/.venv/bin/python /home/tony/clnlive/myplugin.py
 109555 pts/1    D+     0:00 rg myplugin
◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "code": -32601,
   "message": "Unknown command 'myplugin'"
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "foo": "bar"
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "code": -32600,
   "message": "Error while processing myplugin: No option with name foo_opt registered",
   "traceback": "Traceback (most recent call last):\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 639, in _dispatch_request\n    result = self._exec_func(method.func, request)\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 616, in _exec_func\n    ret = func(*ba.args, **ba.kwargs)\n  File \"/home/tony/clnlive/myplugin.py\", line 9, in myplugin_func\n    foo_opt = plugin.get_option(\"foo_opt\")\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 435, in get_option\n    raise ValueError(\"No option with name {} registered\".format(name))\nValueError: No option with name foo_opt registered\n"
}
◉ tony@tony:~/clnlive:
$ python
Python 3.10.6 (main, Mar 10 2023, 10:55:28) [GCC 11.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> print("Traceback (most recent call last):\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 639, in _dispatch_request\n    result = self._exec_func(method.func, request)\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 616, in _exec_func\n    ret = func(*ba.args, **ba.kwargs)\n  File \"/home/tony/clnlive/myplugin.py\", line 9, in myplugin_func\n    foo_opt = plugin.get_option(\"foo_opt\")\n  File \"/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py\", line 435, in get_option\n    raise ValueError(\"No option with name {} registered\".format(name))\nValueError: No option with name foo_opt registered\n")
Traceback (most recent call last):
  File "/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py", line 639, in _dispatch_request
    result = self._exec_func(method.func, request)
  File "/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py", line 616, in _exec_func
    ret = func(*ba.args, **ba.kwargs)
  File "/home/tony/clnlive/myplugin.py", line 9, in myplugin_func
    foo_opt = plugin.get_option("foo_opt")
  File "/home/tony/clnlive/.venv/lib/python3.10/site-packages/pyln/client/plugin.py", line 435, in get_option
    raise ValueError("No option with name {} registered".format(name))
ValueError: No option with name foo_opt registered

>>>
◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "foo_opt": "bar"
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli -k plugin subcommand=start plugin=$(pwd)/myplugin.py foo_opt=BAR
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "foo_opt": "BAR"
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ ps -ax | rg myplugin
 110334 pts/1    S      0:00 /home/tony/clnlive/.venv/bin/python /home/tony/clnlive/myplugin.py
 110357 pts/1    S+     0:00 rg myplugin
◉ tony@tony:~/clnlive:
$ l1-cli myplugin
{
   "node_id": "029999ce6a1b74cf077f503b81bb4d95af34114363bdac5922a9b353ce0449bac4",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ l1-cli -k myplugin foo1=bar1 foo2=bar2
{
   "node_id": "029999ce6a1b74cf077f503b81bb4d95af34114363bdac5922a9b353ce0449bac4",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar1",
      "foo2": "bar2"
   }
}
◉ tony@tony:~/clnlive:
$ l1-cli -k myplugin foo1=bar_1 foo2=bar_2
{
   "node_id": "029999ce6a1b74cf077f503b81bb4d95af34114363bdac5922a9b353ce0449bac4",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar_1",
      "foo2": "bar_2"
   }
}
◉ tony@tony:~/clnlive:
$ l1-cli getinfo
{
   "id": "0341075d01cd6bd67d2410ffaa5607649fcebce23efc0fe7faef882742b0ef3f3f",
   "alias": "VIOLETWATER",
   "color": "034107",
   "num_peers": 0,
   "num_pending_channels": 0,
   "num_active_channels": 0,
   "num_inactive_channels": 0,
   "address": [],
   "binding": [
      {
         "type": "ipv4",
         "address": "127.0.0.1",
         "port": 7171
      }
   ],
   "version": "v23.02.2",
   "blockheight": 1,
   "network": "regtest",
   "fees_collected_msat": 0,
   "lightning-dir": "/tmp/l1-regtest/regtest",
   "our_features": {
      "init": "08a000080269a2",
      "node": "88a000080269a2",
      "channel": "",
      "invoice": "02000000024100"
   }
}
◉ tony@tony:~/clnlive:
$ l1-cli getinfo | jq .id
"0341075d01cd6bd67d2410ffaa5607649fcebce23efc0fe7faef882742b0ef3f3f"
◉ tony@tony:~/clnlive:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
   "command": "stop",
   "result": "Successfully stopped myplugin.py."
}
◉ tony@tony:~/clnlive:
$ l1-cli plugin start $(pwd)/myplugin.py
{
   "command": "start",
   "plugins": [
      {
         "name": "/usr/local/libexec/c-lightning/plugins/autoclean",
         "active": true,
         "dynamic": false
      },
      ...
      {
         "name": "/home/tony/clnlive/myplugin.py",
         "active": true,
         "dynamic": true
      }
   ]
}
◉ tony@tony:~/clnlive:
$ l1-cli getinfo | jq .id
"0341075d01cd6bd67d2410ffaa5607649fcebce23efc0fe7faef882742b0ef3f3f"
◉ tony@tony:~/clnlive:
$ l1-cli -k myplugin foo1=bar_1 foo2=bar_2
{
   "node_id": "0341075d01cd6bd67d2410ffaa5607649fcebce23efc0fe7faef882742b0ef3f3f",
   "options": {
      "foo_opt": "bar"
   },
   "cli_params": {
      "foo1": "bar_1",
      "foo2": "bar_2"
   }
}

pyln-client source code

As we briefly looked at Plugin class implementation, here is Plugin.run method source code

class Plugin(object):

    ...
    def run(self) -> None:
        # If we are not running inside lightningd we'll print usage
        # and some information about the plugin.
        if os.environ.get('LIGHTNINGD_PLUGIN', None) != '1':
            return self.print_usage()

        partial = b""
        for l in self.stdin.buffer:
            partial += l

            msgs = partial.split(b'\n\n')
            if len(msgs) < 2:
                continue

            partial = self._multi_dispatch(msgs)

and Plugin.method method source code:

class Plugin(object):
    ...
    def method(self, method_name: str, category: Optional[str] = None,
               desc: Optional[str] = None,
               long_desc: Optional[str] = None,
               deprecated: bool = False) -> JsonDecoratorType:
        """Decorator to add a plugin method to the dispatch table.

        Internally uses add_method.
        """
        def decorator(f: Callable[..., JSONType]) -> Callable[..., JSONType]:
            self.add_method(method_name,
                            f,
                            background=False,
                            category=category,
                            desc=desc,
                            long_desc=long_desc,
                            deprecated=deprecated)
            return f
        return decorator

Even if we didn't look at Plugin._init method (the method that does the instantiation that allows to do RPC calls with the node running the plugin) during the live, let's reproduce its source code, :

class Plugin(object):
    ...
    def _init(self, options: Dict[str, JSONType],
              configuration: Dict[str, JSONType],
              request: Request) -> JSONType:

        def verify_str(d: Dict[str, JSONType], key: str) -> str:
            v = d.get(key)
            if not isinstance(v, str):
                raise ValueError("Wrong argument to init: expected {key} to be"
                                 " a string, got {v}".format(key=key, v=v))
            return v

        def verify_bool(d: Dict[str, JSONType], key: str) -> bool:
            v = d.get(key)
            if not isinstance(v, bool):
                raise ValueError("Wrong argument to init: expected {key} to be"
                                 " a bool, got {v}".format(key=key, v=v))
            return v

        self.rpc_filename = verify_str(configuration, 'rpc-file')
        self.lightning_dir = verify_str(configuration, 'lightning-dir')

        path = os.path.join(self.lightning_dir, self.rpc_filename)
        self.rpc = LightningRpc(path)
        self.startup = verify_bool(configuration, 'startup')
        for name, value in options.items():
            self.options[name]['value'] = value

        # Dispatch the plugin's init handler if any
        if self.child_init:
            return self._exec_func(self.child_init, request)
        return None

Source code

myplugin.py

See Intalling and using pyln-client for a discussion about Python virtual environments.

#!/usr/bin/env python

from pyln.client import Plugin

plugin = Plugin()

@plugin.method("myplugin")
def myplugin_func(plugin,foo1="foo1", foo2="foo2"):
    node_id = plugin.rpc.getinfo()["id"]
    foo_opt = plugin.get_option("foo_opt")
    return {
        "node_id": node_id,
        "options": {
            "foo_opt": foo_opt
        },
        "cli_params": {
            "foo1": foo1,
            "foo2": foo2
        }
    }

plugin.add_option(name="foo_opt",
                  default="bar",
                  description="'foo_opt description")

plugin.run()

decorators.py

def bar(f):
    def bar_inner(x):
        print(f"'{x}' in 'bar_inner'")
        f(x)
    return bar_inner

@bar
def foo(x):
    print(f"'{x}' in 'foo'")

foo("cln")

# foo = bar(foo)

methods = {}

def make_decorator(rpc_method_name):
    def decorator(f):
        methods[rpc_method_name] = f
        return f
    return decorator

@make_decorator("myplugin")
def method_func(x):
    return {"x_field": x}

# method_func = make_decorator("myplugin")(method_func)

Resources