Overview of pyln-client implementation - Plugin.run() - Part 1
In this episode, we look a pyln-client
Python package implementation focusing specifically on the method run
of the class Plugin
.
Transcript with corrections and improvements
Install pyln-client
Let's install pyln-client
in .venv
Python virtual
environment:
◉ tony@tony:~/lnroom:
$ python -m venv .venv
◉ tony@tony:~/lnroom:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/lnroom:
$ pip install pyln-client
...
Setup
Here is my setup
(.venv) ◉ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
Python 3.10.6
lightningd v23.02.2
pyln-client 23.2
Start 2 Lightning nodes running on regtest
Let's start two Lightning nodes running on the Bitcoin regtest
chain
by sourcing the script lightning/contrib/startup_regtest.sh
provided
in CLN repository and by running the command start_ln
:
(.venv) ◉ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
start_ln 3: start three nodes, l1, l2, l3
connect 1 2: connect l1 and l2
fund_nodes: connect all nodes with channels, in a row
stop_ln: shutdown
destroy_ln: remove ln directories
(.venv) ◉ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 697735
[2] 697769
WARNING: eatmydata not found: instal it for faster testing
Commands:
l1-cli, l1-log,
l2-cli, l2-log,
bt-cli, stop_ln, fund_nodes
(.venv) ◉ tony@tony:~/lnroom:
We can check that l1-cli
is just an alias for lightning-cli
with the
base directory being /tmp/l1-regtest
:
(.venv) ◉ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
myplugin.py
To demonstrate how Plugin.run
method works, we'll use myplugin.py
plugin. This plugin registers the JSON-RPC method node-id
to
lightningd
. That method returns the node id of the node running the
plugin:
#!/usr/bin/env python
from pyln.client import Plugin
plugin = Plugin()
@plugin.method("node-id")
def node_id_func(plugin):
node_id = plugin.rpc.getinfo()["id"]
return {"node_id":node_id}
plugin.run()
If you don't know how to write Core Lightning plugins with
pyln-client
you can check Start writing Core Lightning plugins with
pyln-client TODAY!.
Let's jump in our terminal, start myplugin.py
plugin
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
and call node-id
method:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
Part of Plugin.run() "call stack"
The method Plugin.run()
is an IO loop waiting for incoming JSON-RPC
requests from lightningd
.
Those incoming requests can be "normal" requests or notifications.
When an incoming request is received, it is passed to
Plugin._multi_dispatch
method.
And in the case of "normal" requests (what we are studying today), the
method Plugin._multi_dispatch
passes the request to
Plugin._dispatch_request
method.
In Plugin._dispatch_request
:
Plugin._exec_func
method constructs the payload (what goes into theresult
field of the JSON-RPC response) corresponding to the method of the request andRequest.set_result
produces the JSON-RPC response to the request and passes it toRequest._write_result
method.
Then, Request._write_result
replies to lightningd
by writing to the
plugin's stdout stream.
Finally, we are back in the IO loop waiting for incoming JSON-RPC
request from lightningd
.
What we've just described can be represented with the following schema
where the methods in the boxes are the methods we are going to modify
in that video to better understand how Plugin.run
method works:
┌────────────┐
│Plugin.run()│
└┬───────────┘
└── Plugin._multi_dispatch
│ ┌────────────────────────┐
└──│Plugin._dispatch_request│
└┬───────────────────────┘
├── Plugin._exec_func
│ ┌──────────────────┐
└──│Request.set_result│
└┬─────────────────┘
└── Request._write_result
└── Plugin._write_locked
└── (write to Plugin.stdout)
Plugin.run
The method Plugin.run is defined in lightning:contrib/pyln-client/pyln/client/plugin.py like this:
class Plugin(object):
...
def run(self) -> None:
# If we are not running inside lightningd we'll print usage
# and some information about the plugin.
if os.environ.get('LIGHTNINGD_PLUGIN', None) != '1':
return self.print_usage()
partial = b""
for l in self.stdin.buffer:
partial += l
msgs = partial.split(b'\n\n')
if len(msgs) < 2:
continue
partial = self._multi_dispatch(msgs)
The requests received by run
method are stored in the first field of
msgs
array which is then passed to Plugin._multi_dispatch.
Let's adds the following Python snippet
with open("/tmp/myplugin_out", "a") as output:
output.write(f"\n--------\n---> in 'Plugin.run'\n{repr(msgs)}\n")
to Plugin.run (in the file plugin.py
in .venv
virtual environment)
like this
class Plugin(object):
...
def run(self) -> None:
# If we are not running inside lightningd we'll print usage
# and some information about the plugin.
if os.environ.get('LIGHTNINGD_PLUGIN', None) != '1':
return self.print_usage()
partial = b""
for l in self.stdin.buffer:
partial += l
msgs = partial.split(b'\n\n')
if len(msgs) < 2:
continue
with open("/tmp/myplugin_out", "a") as output:
output.write(f"\n--------\n---> in 'Plugin.run'\n{repr(msgs)}\n")
partial = self._multi_dispatch(msgs)
in order to see how does the requests look like by writing each
incoming request to the file /tmp/myplugin_out
.
Back to our terminal we restart our plugin in order to take into
account the changes we made in pyln-client
package:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
Now we can check that the file /tmp/myplugin_out
contains the
getmanifest
and init
requests sent by lightningd
to myplugin.py
plugin
after we started it:
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":96,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#97","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
Then we can call node-id
method like this
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
and observe that node-id
request has been written in the file
/tmp/myplugin_out
which is now:
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":96,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#97","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#698621/cln:node-id#108", "params" :[ ] }', b'']
Plugin._muti_dispatch
The method Plugin._multi_dispatch is defined in lightning:contrib/pyln-client/pyln/client/plugin.py like this:
class Plugin(object):
...
def _multi_dispatch(self, msgs: List[bytes]) -> bytes:
"""We received a couple of messages, now try to dispatch them all.
Returns the last partial message that was not complete yet.
"""
for payload in msgs[:-1]:
# Note that we use function annotations to do Millisatoshi
# conversions in _exec_func, so we don't use LightningJSONDecoder
# here.
request = self._parse_request(json.loads(payload.decode('utf8')))
# If this has an 'id'-field, it's a request and returns a
# result. Otherwise it's a notification and it doesn't
# return anything.
if request.id is not None:
self._dispatch_request(request)
else:
self._dispatch_notification(request)
return msgs[-1]
From msgs
, Plugin._multi_dispatch instantiates request
object of type
Request
, check that the request
has an id (which is the case of node-id
request for instance) and passes it to Plugin._dispatch_request (we
don't look at notifications in this video).
As it might be useful to see how Request class is defined we reproduce below its Request.__init__ method:
class Request(dict):
"""A request object that wraps params and allows async return
"""
def __init__(self, plugin: 'Plugin', req_id: Optional[str], method: str,
params: Any, background: bool = False):
self.method = method
self.params = params
self.background = background
self.plugin = plugin
self.state = RequestState.PENDING
self.id = req_id
self.termination_tb: Optional[str] = None
Plugin._dispatch_request
In Plugin._dispatch_request method, we look for the name of the
request
in Plugin.methods
dictionary. If we find it, we set method
variable as the object of the class Method
associated with that name
in Plugin.methods
. Then we try to execute the function method.func
using Plugin._exec_func and set result
variable to the value returned
by Plugin._exec_func. If there is no error, we finally reply to
lightningd
passing result
to Request.set_result method:
class Plugin(object):
...
def _dispatch_request(self, request: Request) -> None:
name = request.method
if name not in self.methods:
raise ValueError("No method {} found.".format(name))
method = self.methods[name]
request.background = method.background
try:
result = self._exec_func(method.func, request)
if not method.background:
# Only if this is a synchronous (background=False) call do we need to
# return the result. Otherwise the callee (method) will eventually need
# to call request.set_result or request.set_exception to
# return a result or raise an exception.
request.set_result(result)
except Exception as e:
if name in hook_fallbacks:
response = hook_fallbacks[name]
self.log((
"Hook handler for {name} failed with an exception. "
"Returning safe fallback response {response} to avoid "
"crashing the main daemon. Please contact the plugin "
"author!"
).format(name=name, response=response), level="error")
request.set_result(response)
else:
request.set_exception(e)
self.log(traceback.format_exc())
To have a better understanding of the composition of Plugin.methods
dictionary, we can send the following expressions
from pyln.client import Plugin
plugin = Plugin()
to a Python interpreter and then looks at plugin.methods
value:
>>> plugin.methods
{'init': <pyln.client.plugin.Method object at 0x7f03545d6080>,
'getmanifest': <pyln.client.plugin.Method object at 0x7f03545d5ed0>}
>>> plugin.methods["init"]
<pyln.client.plugin.Method object at 0x7f03545d6080>
>>> plugin.methods["init"].name
'init'
>>> plugin.methods["init"].func
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7f03545d7d60>>
It means that when myplugin.py
receives the init
request from
lightningd
, the function used in Plugin._dispatch_request to build the
result
field of the JSON-RPC reponse is Plugin._init method.
Now if we send the following Python decorator snippet
@plugin.method("node-id")
def node_id_func(plugin):
node_id = plugin.rpc.getinfo()["id"]
return {"node_id":node_id}
to our Python interpreter and look at plugin.methods
value
>>> plugin.methods
{'init': <pyln.client.plugin.Method object at 0x7f03545d6080>,
'getmanifest': <pyln.client.plugin.Method object at 0x7f03545d5ed0>,
'node-id': <pyln.client.plugin.Method object at 0x7f0353423cd0>}
we can see that an entry for node-id
method has been added. And if we
look at
>>> plugin.methods["node-id"].func
<function node_id_func at 0x7f03534315a0>
we can see that when myplugin.py
receives a node-id
request from
lightningd
, the function used in Plugin._dispatch_request to build the
result
field of the JSON-RPC reponse is node_id_func
function we
defined just below the line @plugin.method("node-id")
.
How does this work? Well, if you are intereted you can check Overview of pyln-client implementation - @plugin.method() - Part 2.
Back to lightning:contrib/pyln-client/pyln/client/plugin.py file and Plugin._dispatch_request method.
Let's add the following Python snippet
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Plugin._dispatch_request\n{name}\n"+
f"{repr(method.func)}\n"+
f"{json.dumps(result)}\n")
to Plugin._dispatch_request (in the file plugin.py
in .venv
virtual
environment) like this
class Plugin(object):
...
def _dispatch_request(self, request: Request) -> None:
name = request.method
if name not in self.methods:
raise ValueError("No method {} found.".format(name))
method = self.methods[name]
request.background = method.background
try:
result = self._exec_func(method.func, request)
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Plugin._dispatch_request\n{name}\n"+
f"{repr(method.func)}\n"+
f"{json.dumps(result)}\n")
if not method.background:
# Only if this is a synchronous (background=False) call do we need to
# return the result. Otherwise the callee (method) will eventually need
# to call request.set_result or request.set_exception to
# return a result or raise an exception.
request.set_result(result)
except Exception as e:
if name in hook_fallbacks:
response = hook_fallbacks[name]
self.log((
"Hook handler for {name} failed with an exception. "
"Returning safe fallback response {response} to avoid "
"crashing the main daemon. Please contact the plugin "
"author!"
).format(name=name, response=response), level="error")
request.set_result(response)
else:
request.set_exception(e)
self.log(traceback.format_exc())
in order to see in the file /tmp/myplugin_out
for each request from
lightningd
which function is used to produce the result field of the JSON-RPC response and
what does the result field of the JSON-RPC response contains.
Back to our terminal we restart our plugin in order to take into
account the changes we made in pyln-client
package:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
Now we can check that the following
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":187,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#188","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
null
has been appended to /tmp/myplugin_out
file.
Let's call node-id
method like this
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
and check that the following informations about node-id
request
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#699655/cln:node-id#205", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7f8b44c59fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
have been appended to /tmp/myplugin_out
file.
Request.set_result
In Plugin._dispatch_request we built a result
that we passed to
Request.set_result method which is defined in
lightning:contrib/pyln-client/pyln/client/plugin.py like this:
class Request(dict):
...
def set_result(self, result: Any) -> None:
if self.state != RequestState.PENDING:
assert(self.termination_tb is not None)
raise ValueError(
"Cannot set the result of a request that is not pending, "
"current state is {state}. Request previously terminated at\n"
"{tb}".format(state=self.state, tb=self.termination_tb))
self.result = result
self._write_result({
'jsonrpc': '2.0',
'id': self.id,
'result': self.result
})
self.state = RequestState.FINISHED
self.termination_tb = "".join(traceback.extract_stack().format()[:-1])
In that method we build the JSON-RPC response corresponding the request that Plugin.run received in the stdin stream of the plugin process. And that response is then passed to Request._write_result method
class Request(dict):
...
def _write_result(self, result: dict) -> None:
self.plugin._write_locked(result)
which delegates the work to Plugin._write_locked that finally writes
to the plugin stdout stream. This leads to reply to lightningd
request:
class Plugin(object):
...
def _write_locked(self, obj: JSONType) -> None:
# ensure_ascii turns UTF-8 into \uXXXX so we need to suppress that,
# then utf8 ourselves.
s = bytes(json.dumps(
obj,
cls=LightningRpc.LightningJSONEncoder,
ensure_ascii=False
) + "\n\n", encoding='utf-8')
with self.write_lock:
self.stdout.buffer.write(s)
self.stdout.flush()
And we are back at the beginning of Plugin.run IO loop waiting for
incoming requests from lightningd
.
To see exactly what is the JSON-RPC response that we send back to
lightningd
, we modify Request.set_result (in the file plugin.py
in
.venv
virtual environment) like this:
class Request(dict):
...
def set_result(self, result: Any) -> None:
if self.state != RequestState.PENDING:
assert(self.termination_tb is not None)
raise ValueError(
"Cannot set the result of a request that is not pending, "
"current state is {state}. Request previously terminated at\n"
"{tb}".format(state=self.state, tb=self.termination_tb))
self.result = result
response = {
'jsonrpc': '2.0',
'id': self.id,
'result': self.result
}
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Request.set_result'\n{json.dumps(response)}\n")
self._write_result(response)
self.state = RequestState.FINISHED
self.termination_tb = "".join(traceback.extract_stack().format()[:-1])
Now each time our plugin receives a request from lightningd
we'll
append to /tmp/myplugin_out
file the JSON-RPC response we send back to
lightningd
.
Back to our terminal we restart our plugin in order to take into
account the changes we made in pyln-client
package:
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
Now we can check that the following
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":232,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": 232, "result": {"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#233","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
null
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cln:init#233", "result": null}
has been appended to /tmp/myplugin_out
file.
Let's call node-id
method like this
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
and check that the following informations about node-id
request
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#700027/cln:node-id#242", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7fbe22e41fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cli:node-id#700027/cln:node-id#242", "result": {"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}}
have been appended to /tmp/myplugin_out
file.
We are done!
Terminal session
We ran the following commands in this order:
$ python -m venv .venv
$ source .venv/bin/activate
$ pip install pyln-client
$ ./setup.sh
$ source lightning/contrib/startup_regtest.sh
$ start_ln
$ alias l1-cli
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
$ l1-cli plugin stop $(pwd)/myplugin.py
$ l1-cli plugin start $(pwd)/myplugin.py
$ l1-cli node-id
And below you can read the terminal session (command lines and outputs):
◉ tony@tony:~/lnroom:
$ python -m venv .venv
◉ tony@tony:~/lnroom:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/lnroom:
$ pip install pyln-client
...
(.venv) ◉ tony@tony:~/lnroom:
$ ./setup.sh
Ubuntu 22.04.2 LTS
Python 3.10.6
lightningd v23.02.2
pyln-client 23.2
(.venv) ◉ tony@tony:~/lnroom:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
start_ln 3: start three nodes, l1, l2, l3
connect 1 2: connect l1 and l2
fund_nodes: connect all nodes with channels, in a row
stop_ln: shutdown
destroy_ln: remove ln directories
(.venv) ◉ tony@tony:~/lnroom:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
[1] 697735
[2] 697769
WARNING: eatmydata not found: instal it for faster testing
Commands:
l1-cli, l1-log,
l2-cli, l2-log,
bt-cli, stop_ln, fund_nodes
(.venv) ◉ tony@tony:~/lnroom:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [
{
"name": "/usr/local/libexec/c-lightning/plugins/autoclean",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/chanbackup",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/bcli",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/commando",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/funder",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/topology",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/keysend",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/offers",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/pay",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/txprepare",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/spenderp",
"active": true,
"dynamic": false
},
{
"name": "/usr/local/libexec/c-lightning/plugins/sql",
"active": true,
"dynamic": true
},
{
"name": "/usr/local/libexec/c-lightning/plugins/bookkeeper",
"active": true,
"dynamic": false
},
{
"name": "/home/tony/lnroom/myplugin.py",
"active": true,
"dynamic": true
}
]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin stop $(pwd)/myplugin.py
{
"command": "stop",
"result": "Successfully stopped myplugin.py."
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli plugin start $(pwd)/myplugin.py
{
"command": "start",
"plugins": [...]
}
(.venv) ◉ tony@tony:~/lnroom:
$ l1-cli node-id
{
"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"
}
pyln-client source code
During the demo, we've modified Request.set_result method of
pyln-client
package like this:
class Request(dict):
...
def set_result(self, result: Any) -> None:
if self.state != RequestState.PENDING:
assert(self.termination_tb is not None)
raise ValueError(
"Cannot set the result of a request that is not pending, "
"current state is {state}. Request previously terminated at\n"
"{tb}".format(state=self.state, tb=self.termination_tb))
self.result = result
response = {
'jsonrpc': '2.0',
'id': self.id,
'result': self.result
}
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Request.set_result'\n{json.dumps(response)}\n")
self._write_result(response)
self.state = RequestState.FINISHED
self.termination_tb = "".join(traceback.extract_stack().format()[:-1])
We also have modified Plugin._dispatch_request and Plugin.run
methods of pyln-client
package like this:
class Plugin(object):
...
def _dispatch_request(self, request: Request) -> None:
name = request.method
if name not in self.methods:
raise ValueError("No method {} found.".format(name))
method = self.methods[name]
request.background = method.background
try:
result = self._exec_func(method.func, request)
with open("/tmp/myplugin_out", "a") as output:
output.write(f"---> in 'Plugin._dispatch_request\n{name}\n"+
f"{repr(method.func)}\n"+
f"{json.dumps(result)}\n")
if not method.background:
# Only if this is a synchronous (background=False) call do we need to
# return the result. Otherwise the callee (method) will eventually need
# to call request.set_result or request.set_exception to
# return a result or raise an exception.
request.set_result(result)
except Exception as e:
if name in hook_fallbacks:
response = hook_fallbacks[name]
self.log((
"Hook handler for {name} failed with an exception. "
"Returning safe fallback response {response} to avoid "
"crashing the main daemon. Please contact the plugin "
"author!"
).format(name=name, response=response), level="error")
request.set_result(response)
else:
request.set_exception(e)
self.log(traceback.format_exc())
...
def run(self) -> None:
# If we are not running inside lightningd we'll print usage
# and some information about the plugin.
if os.environ.get('LIGHTNINGD_PLUGIN', None) != '1':
return self.print_usage()
partial = b""
for l in self.stdin.buffer:
partial += l
msgs = partial.split(b'\n\n')
if len(msgs) < 2:
continue
with open("/tmp/myplugin_out", "a") as output:
output.write(f"\n--------\n---> in 'Plugin.run'\n{repr(msgs)}\n")
partial = self._multi_dispatch(msgs)
Source code
myplugin.py
#!/usr/bin/env python
from pyln.client import Plugin
plugin = Plugin()
@plugin.method("node-id")
def node_id_func(plugin):
node_id = plugin.rpc.getinfo()["id"]
return {"node_id":node_id}
plugin.run()
myplugin_out
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":96,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#97","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#698621/cln:node-id#108", "params" :[ ] }', b'']
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":187,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#188","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7f8b44d3bc10>>
null
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#699655/cln:node-id#205", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7f8b44c59fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":232,"method":"getmanifest","params":{"allow-deprecated-apis":false}}', b'']
---> in 'Plugin._dispatch_request
getmanifest
<bound method Plugin._getmanifest of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
{"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": 232, "result": {"options": [], "rpcmethods": [{"name": "node-id", "category": "plugin", "usage": "", "description": "Undocumented RPC method from a plugin."}], "subscriptions": [], "hooks": [], "dynamic": true, "nonnumericids": true, "notifications": [], "featurebits": {}}}
--------
---> in 'Plugin.run'
[b'{"jsonrpc":"2.0","id":"cln:init#233","method":"init","params":{"options":{},"configuration":{"lightning-dir":"/tmp/l1-regtest/regtest","rpc-file":"lightning-rpc","startup":false,"network":"regtest","feature_set":{"init":"08a000080269a2","node":"88a000080269a2","channel":"","invoice":"02000000024100"}}}}', b'']
---> in 'Plugin._dispatch_request
init
<bound method Plugin._init of <pyln.client.plugin.Plugin object at 0x7fbe22f23c10>>
null
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cln:init#233", "result": null}
--------
---> in 'Plugin.run'
[b'{ "jsonrpc" : "2.0", "method" : "node-id", "id" : "cli:node-id#700027/cln:node-id#242", "params" :[ ] }', b'']
---> in 'Plugin._dispatch_request
node-id
<function node_id_func at 0x7fbe22e41fc0>
{"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}
---> in 'Request.set_result'
{"jsonrpc": "2.0", "id": "cli:node-id#700027/cln:node-id#242", "result": {"node_id": "03fd248f93b8352d6cfc6a168f0fd10b1462f72d84b310a56f5c18076c5cf04fe4"}}
setup.sh
#!/usr/bin/env bash
ubuntu=$(lsb_release -ds)
lightningd=$(lightningd --version | xargs printf "lightningd %s\n")
python=$(python --version)
pyln_client=$(pip list | rg pyln-client)
printf "%s\n%s\n%s\n%s\n" "$ubuntu" "$python" "$lightningd" "$pyln_client"