Get started with cln-grpc plugin

Let's start two Lightning nodes running on the Bitcoin regtest chain by sourcing the script lightning/contrib/startup_regtest.sh provided by CLN repository and running the command start_ln:

◉ tony@tony:~/clnlive:
$ source lightning/contrib/startup_regtest.sh
lightning-cli is /usr/local/bin/lightning-cli
lightningd is /usr/local/bin/lightningd
Useful commands:
  start_ln 3: start three nodes, l1, l2, l3
  connect 1 2: connect l1 and l2
  fund_nodes: connect all nodes with channels, in a row
  stop_ln: shutdown
  destroy_ln: remove ln directories
◉ tony@tony:~/clnlive:
$ start_ln
Bitcoin Core starting
awaiting bitcoind...
Making "default" bitcoind wallet.
error code: -35
error message:
Wallet "default" is already loaded.
[1] 968150
[2] 968193
WARNING: eatmydata not found: install it for faster testing
Commands:
        l1-cli, l1-log,
        l2-cli, l2-log,
        bt-cli, stop_ln, fund_nodes

We can check that l1-cli is just an alias for lightning-cli with the base directory being /tmp/l1-regtest:

◉ tony@tony:~/clnlive:
$ alias l1-cli
alias l1-cli='lightning-cli --lightning-dir=/tmp/l1-regtest'

To be sure that we have at least a lightning node running on regtest, we can call the subcommand getinfo like this:

◉ tony@tony:~/clnlive:
$ l1-cli getinfo
{
   "id": "030da9d745d82472909034ab7caf11c101978f7d38e46487e69ae4e011ddfa5b78",
   "alias": "ANGRYAUTO",
   "color": "030da9",
   "num_peers": 0,
   "num_pending_channels": 0,
   "num_active_channels": 0,
   "num_inactive_channels": 0,
   "address": [],
   "binding": [
      {
         "type": "ipv4",
         "address": "127.0.0.1",
         "port": 7171
      }
   ],
   "version": "v23.08.1",
   "blockheight": 1,
   "network": "regtest",
   "fees_collected_msat": 0,
   "lightning-dir": "/tmp/l1-regtest/regtest",
   "our_features": {
      "init": "08a0000a0269a2",
      "node": "88a0000a0269a2",
      "channel": "",
      "invoice": "02000002024100"
   }
}

Note that if we don't specify a port for cln-grpc plugin, the plugin is disable by default.

And in the previous script that we used, no port is specified, so cln-grpc plugin is not running on the node l1 nor in l2. We can check this by listing the running plugins like this

◉ tony@tony:~/clnlive:
$ l1-cli -F plugin list
command=list
plugins[0].name=/usr/local/libexec/c-lightning/plugins/autoclean
plugins[0].active=true
plugins[0].dynamic=false
plugins[1].name=/usr/local/libexec/c-lightning/plugins/chanbackup
plugins[1].active=true
plugins[1].dynamic=false
plugins[2].name=/usr/local/libexec/c-lightning/plugins/bcli
plugins[2].active=true
plugins[2].dynamic=false
plugins[3].name=/usr/local/libexec/c-lightning/plugins/commando
plugins[3].active=true
plugins[3].dynamic=false
plugins[4].name=/usr/local/libexec/c-lightning/plugins/funder
plugins[4].active=true
plugins[4].dynamic=true
plugins[5].name=/usr/local/libexec/c-lightning/plugins/topology
plugins[5].active=true
plugins[5].dynamic=false
plugins[6].name=/usr/local/libexec/c-lightning/plugins/keysend
plugins[6].active=true
plugins[6].dynamic=false
plugins[7].name=/usr/local/libexec/c-lightning/plugins/offers
plugins[7].active=true
plugins[7].dynamic=true
plugins[8].name=/usr/local/libexec/c-lightning/plugins/pay
plugins[8].active=true
plugins[8].dynamic=true
plugins[9].name=/usr/local/libexec/c-lightning/plugins/txprepare
plugins[9].active=true
plugins[9].dynamic=true
plugins[10].name=/usr/local/libexec/c-lightning/plugins/cln-renepay
plugins[10].active=true
plugins[10].dynamic=true
plugins[11].name=/usr/local/libexec/c-lightning/plugins/spenderp
plugins[11].active=true
plugins[11].dynamic=false
plugins[12].name=/usr/local/libexec/c-lightning/plugins/sql
plugins[12].active=true
plugins[12].dynamic=true
plugins[13].name=/usr/local/libexec/c-lightning/plugins/bookkeeper
plugins[13].active=true
plugins[13].dynamic=false

and filtering by grpc using rg utility like this

◉ tony@tony:~/clnlive:
$ l1-cli -F plugin list | rg grpc

which outputs nothing. So we know that cln-grpc is not running on l1 node.

Let's stop l1 node

◉ tony@tony:~/clnlive:
$ l1-cli stop
"Shutdown complete"

and start it again specifying a port in the option --grpc-port in order to have cln-grpc plugin running

◉ tony@tony:~/clnlive:
$ lightningd --lightning-dir=/tmp/l1-regtest --grpc-port=3030 --daemon

Now we verify that cln-grpc plugin is running like this:

◉ tony@tony:~/clnlive:
$ l1-cli -F plugin list | rg grpc
plugins[13].name=/usr/local/libexec/c-lightning/plugins/cln-grpc

The cln-grpc plugin is listening on port 3030 and we can check that using nc utility like this:

◉ tony@tony:~/clnlive:
$ nc -z -v 127.0.0.1 3030
Connection to 127.0.0.1 3030 port [tcp/*] succeeded!

Now, what we want to do is to connect to cln-grpc with a gRPC channel and send it.

1. getinfo requests and then

2. invoice requests.

When we started l1 node with cln-grpc we also generated certificate files to authenticate and encrypt gRPC channels with our node:

◉ tony@tony:~/clnlive:
$ ls /tmp/l1-regtest/regtest/*.pem
/tmp/l1-regtest/regtest/ca-key.pem      /tmp/l1-regtest/regtest/client.pem
/tmp/l1-regtest/regtest/ca.pem          /tmp/l1-regtest/regtest/server-key.pem
/tmp/l1-regtest/regtest/client-key.pem  /tmp/l1-regtest/regtest/server.pem

To use those certificates in our Python script, we copy them in the current directory:

◉ tony@tony:~/clnlive:
$ cp /tmp/l1-regtest/regtest/client*.pem /tmp/l1-regtest/regtest/ca.pem .
◉ tony@tony:~/clnlive:
$ ls
clnlive-scratch/  lightning/  ca.pem  client-key.pem  client.pem  invoice.py  notes.org

As far as I understand gRPC protocol, services are defined once in proto files and we use those proto files to derive library bindings for different languages.

Let's take a look to the proto files used by cln-grpc plugins which defined the CLN methods that can be used with that plugin:

◉ tony@tony:~/clnlive:
$ ls lightning/cln-grpc/proto/
node.proto  primitives.proto

cln-grpc/proto/node.proto

...
service Node {
        rpc Getinfo(GetinfoRequest) returns (GetinfoResponse) {}
        rpc ListPeers(ListpeersRequest) returns (ListpeersResponse) {}
        rpc ListFunds(ListfundsRequest) returns (ListfundsResponse) {}
        rpc SendPay(SendpayRequest) returns (SendpayResponse) {}
        rpc ListChannels(ListchannelsRequest) returns (ListchannelsResponse) {}
        rpc AddGossip(AddgossipRequest) returns (AddgossipResponse) {}
        rpc AutoCleanInvoice(AutocleaninvoiceRequest) returns (AutocleaninvoiceResponse) {}
        rpc CheckMessage(CheckmessageRequest) returns (CheckmessageResponse) {}
        rpc Close(CloseRequest) returns (CloseResponse) {}
        rpc ConnectPeer(ConnectRequest) returns (ConnectResponse) {}
        rpc CreateInvoice(CreateinvoiceRequest) returns (CreateinvoiceResponse) {}
        rpc Datastore(DatastoreRequest) returns (DatastoreResponse) {}
        rpc CreateOnion(CreateonionRequest) returns (CreateonionResponse) {}
        rpc DelDatastore(DeldatastoreRequest) returns (DeldatastoreResponse) {}
        rpc DelExpiredInvoice(DelexpiredinvoiceRequest) returns (DelexpiredinvoiceResponse) {}
        rpc DelInvoice(DelinvoiceRequest) returns (DelinvoiceResponse) {}
        rpc Invoice(InvoiceRequest) returns (InvoiceResponse) {}
        rpc ListDatastore(ListdatastoreRequest) returns (ListdatastoreResponse) {}
        rpc ListInvoices(ListinvoicesRequest) returns (ListinvoicesResponse) {}
        rpc SendOnion(SendonionRequest) returns (SendonionResponse) {}
        rpc ListSendPays(ListsendpaysRequest) returns (ListsendpaysResponse) {}
        rpc ListTransactions(ListtransactionsRequest) returns (ListtransactionsResponse) {}
        rpc Pay(PayRequest) returns (PayResponse) {}
        rpc ListNodes(ListnodesRequest) returns (ListnodesResponse) {}
        rpc WaitAnyInvoice(WaitanyinvoiceRequest) returns (WaitanyinvoiceResponse) {}
        rpc WaitInvoice(WaitinvoiceRequest) returns (WaitinvoiceResponse) {}
        rpc WaitSendPay(WaitsendpayRequest) returns (WaitsendpayResponse) {}
        rpc NewAddr(NewaddrRequest) returns (NewaddrResponse) {}
        rpc Withdraw(WithdrawRequest) returns (WithdrawResponse) {}
        rpc KeySend(KeysendRequest) returns (KeysendResponse) {}
        rpc FundPsbt(FundpsbtRequest) returns (FundpsbtResponse) {}
        rpc SendPsbt(SendpsbtRequest) returns (SendpsbtResponse) {}
        rpc SignPsbt(SignpsbtRequest) returns (SignpsbtResponse) {}
        rpc UtxoPsbt(UtxopsbtRequest) returns (UtxopsbtResponse) {}
        rpc TxDiscard(TxdiscardRequest) returns (TxdiscardResponse) {}
        rpc TxPrepare(TxprepareRequest) returns (TxprepareResponse) {}
        rpc TxSend(TxsendRequest) returns (TxsendResponse) {}
        rpc ListPeerChannels(ListpeerchannelsRequest) returns (ListpeerchannelsResponse) {}
        rpc ListClosedChannels(ListclosedchannelsRequest) returns (ListclosedchannelsResponse) {}
        rpc DecodePay(DecodepayRequest) returns (DecodepayResponse) {}
        rpc Decode(DecodeRequest) returns (DecodeResponse) {}
        rpc Disconnect(DisconnectRequest) returns (DisconnectResponse) {}
        rpc Feerates(FeeratesRequest) returns (FeeratesResponse) {}
        rpc FundChannel(FundchannelRequest) returns (FundchannelResponse) {}
        rpc GetRoute(GetrouteRequest) returns (GetrouteResponse) {}
        rpc ListForwards(ListforwardsRequest) returns (ListforwardsResponse) {}
        rpc ListPays(ListpaysRequest) returns (ListpaysResponse) {}
        rpc ListHtlcs(ListhtlcsRequest) returns (ListhtlcsResponse) {}
        rpc Ping(PingRequest) returns (PingResponse) {}
        rpc SendCustomMsg(SendcustommsgRequest) returns (SendcustommsgResponse) {}
        rpc SetChannel(SetchannelRequest) returns (SetchannelResponse) {}
        rpc SignInvoice(SigninvoiceRequest) returns (SigninvoiceResponse) {}
        rpc SignMessage(SignmessageRequest) returns (SignmessageResponse) {}
        rpc Stop(StopRequest) returns (StopResponse) {}
        rpc PreApproveKeysend(PreapprovekeysendRequest) returns (PreapprovekeysendResponse) {}
        rpc PreApproveInvoice(PreapproveinvoiceRequest) returns (PreapproveinvoiceResponse) {}
        rpc StaticBackup(StaticbackupRequest) returns (StaticbackupResponse) {}
}

message GetinfoRequest {
}

message GetinfoResponse {
        bytes id = 1;
        optional string alias = 2;
        bytes color = 3;
        uint32 num_peers = 4;
        uint32 num_pending_channels = 5;
        uint32 num_active_channels = 6;
        uint32 num_inactive_channels = 7;
        string version = 8;
        string lightning_dir = 9;
        optional GetinfoOur_features our_features = 10;
        uint32 blockheight = 11;
        string network = 12;
        Amount fees_collected_msat = 13;
        repeated GetinfoAddress address = 14;
        repeated GetinfoBinding binding = 15;
        optional string warning_bitcoind_sync = 16;
        optional string warning_lightningd_sync = 17;
}
...
message InvoiceRequest {
        AmountOrAny amount_msat = 10;
        string description = 2;
        string label = 3;
        optional uint64 expiry = 7;
        repeated string fallbacks = 4;
        optional bytes preimage = 5;
        optional uint32 cltv = 6;
        optional bool deschashonly = 9;
}

message InvoiceResponse {
        string bolt11 = 1;
        bytes payment_hash = 2;
        bytes payment_secret = 3;
        uint64 expires_at = 4;
        optional uint64 created_index = 10;
        optional string warning_capacity = 5;
        optional string warning_offline = 6;
        optional string warning_deadends = 7;
        optional string warning_private_unused = 8;
        optional string warning_mpp = 9;
}
...

cln-grpc/proto/primitives.proto

...
message Amount {
        uint64 msat = 1;
}
...
message AmountOrAny {
        oneof value {
                Amount amount = 1;
                bool any = 2;
        }
}
...

In that section with generate gRPC bindings for Python using protoc to which we pass the previous cln-grpc proto files as argument.

First we install grpcio-tools in a Python virtual environment:

◉ tony@tony:~/clnlive:
$ python -m venv .venv
◉ tony@tony:~/clnlive:
$ source .venv/bin/activate
(.venv) ◉ tony@tony:~/clnlive:
$ pip install grpcio-tools
...

Then, from node.proto file we generate the Python files node_pb2_grpc.py and node_pb2.py using grpc_tools.protoc module:

(.venv) ◉ tony@tony:~/clnlive:
$ python -m grpc_tools.protoc \
►   -I lightning/cln-grpc/proto \
►   lightning/cln-grpc/proto/node.proto \
►   --python_out=. \
►   --grpc_python_out=. \
►   --experimental_allow_proto3_optional
(.venv) ◉ tony@tony:~/clnlive:
$ ls
clnlive-scratch/  ca.pem          client.pem  node_pb2_grpc.py  notes.org
lightning/        client-key.pem  invoice.py  node_pb2.py

And similarly from primitives.proto file we generate the Python file primitives_pb2.py:

(.venv) ◉ tony@tony:~/clnlive:
$ python -m grpc_tools.protoc \
►   -I lightning/cln-grpc/proto \
►   lightning/cln-grpc/proto/primitives.proto \
►   --python_out=. \
►   --experimental_allow_proto3_optional
(.venv) ◉ tony@tony:~/clnlive:
$ ls
clnlive-scratch/  ca.pem          client.pem  node_pb2_grpc.py  notes.org
lightning/        client-key.pem  invoice.py  node_pb2.py       primitives_pb2.py

Here is our invoice.py script which connects to l1 node via a gRPC channel importing node_pb2 module and NodeStub from node_pb2_grpc, defined by the Python bindings generated in the previous section and using the self-signed certificates generated by cln-grpc plugin.

#!/usr/bin/env python

from pathlib import Path
from node_pb2_grpc import NodeStub
import node_pb2
import grpc

p = Path(".")
cert_path = p / "client.pem"
key_path = p / "client-key.pem"
ca_cert_path = p / "ca.pem"

creds = grpc.ssl_channel_credentials(
    root_certificates=ca_cert_path.open("rb").read(),
    private_key=key_path.open("rb").read(),
    certificate_chain=cert_path.open("rb").read()
)

channel = grpc.secure_channel(
        "localhost:3030",
        creds,
        options=(("grpc.ssl_target_name_override", "cln"),)
)
stub = NodeStub(channel)

print(stub.Getinfo(node_pb2.GetinfoRequest()))

Now we can do a getinfo request on our node via a gRPC channel like this:

(.venv) ◉ tony@tony:~/clnlive:
$ ./invoice.py
id: "\003\r\251\327E\330$r\220\2204\253|\257\021\301\001\227\217}8\344d\207\346\232\344\340\021\335\372[x"
alias: "ANGRYAUTO"
color: "\003\r\251"
version: "v23.08.1"
lightning_dir: "/tmp/l1-regtest/regtest"
our_features {
  init: "\010\240\000\n\002i\242"
  node: "\210\240\000\n\002i\242"
  invoice: "\002\000\000\002\002A\000"
}
blockheight: 1
network: "regtest"
fees_collected_msat {
}
binding {
  item_type: IPV6
  address: "127.0.0.1"
  port: 7171
}

If we want we can print out only the alias of l1 node by modifying invoice.py script like this

...
print(stub.Getinfo(node_pb2.GetinfoRequest()).alias)

and running it this way:

(.venv) ◉ tony@tony:~/clnlive:
$ ./invoice.py
ANGRYAUTO

Let's continue and modify invoice.py script such that we do an invoice request on l1 node via a gRPC channel and print out the response we get:

...
inv = stub.Invoice(node_pb2.InvoiceRequest(
    amount_msat=node_pb2.primitives__pb2.AmountOrAny(
        amount=node_pb2.primitives__pb2.Amount(
            msat=10000
        )),
    label="label",
    description="description"
))

print(inv)

Let's run that script:

(.venv) ◉ tony@tony:~/clnlive:
$ ./invoice.py
bolt11: "lnbcrt100n1pjjszzmsp5aqy5k8ka2fcqfkmavfk2v6z367jkt48ftmhg6hvt9hprdzt6je3spp59uqphe4e6zduvg4khfghyucgzt2f7fmpra73dx4c3a9kf5qn7psqdqjv3jhxcmjd9c8g6t0dcxqyjw5qcqp29qxpqysgqtv68pqzfy4htxdz9quhv2jzmwm8eujxsfaxql0n47td274ymjhpycyjn60hj4p688lh0t05jk6vhn4rukt4xttwpf5yx9d6duy09a0gqvyv9f5"
payment_hash: "/\000\033\346\271\320\233\306\"\266\272Qrs\010\022\324\237\'a\037}\026\232\270\217Kd\320\023\360`"
payment_secret: "\350\tK\036\335Rp\004\333}bl\246hQ\327\245e\324\351^\356\215]\213-\3026\211z\226c"
expires_at: 1697727195
created_index: 1
warning_capacity: "Insufficient incoming channel capacity to pay invoice"

If we just want to print the BOLT #11 invoice string, we can do this by printing the bolt11 property of the inv object like this:

...
inv = stub.Invoice(node_pb2.InvoiceRequest(
    amount_msat=node_pb2.primitives__pb2.AmountOrAny(
        amount=node_pb2.primitives__pb2.Amount(
            msat=10000
        )),
    label="label-1",
    description="description"
))

print(inv.bolt11)

Note that we also modified the label parameter to not conflict with the previous one already registered by l1 node.

Let's generate a BOLT #11 on l1 node like this:

(.venv) ◉ tony@tony:~/clnlive:
$ ./invoice.py
lnbcrt100n1pjjsz97sp58q5vkxahpxyrhxd560a4akc82xxmmf50rukdc9lpmtlyeq300f0qpp5f625gpfjn84tqhxmfxnz7v53rnu8kseyq3vzdnzpqjllsy40ng9qdqjv3jhxcmjd9c8g6t0dcxqyjw5qcqp29qxpqysgq6nglnq8wcr5vc7dszv2arhm42tar64zcrduf694u0jdtwy5vshx38r5fe6gpzcnljudp8vz2tw8tpvdpp9ynzek9csuxk08k24cjh4sqhmyz2f

Finally, let's modify our script such that it takes two arguments, the first one being the amount in msat and the second one being the invoice's description and it generates a random label for the invoice request:

...
inv = stub.Invoice(node_pb2.InvoiceRequest(
    amount_msat=node_pb2.primitives__pb2.AmountOrAny(
        amount=node_pb2.primitives__pb2.Amount(
            msat=int(sys.argv[1])
        )),
    label=f"label-{random.random()}",
    description=sys.argv[2]
))

print(inv.bolt11)

Here we are, we can generate a BOLT #11 invoice for a pizza which costs 10000msat connecting to l1 node via Grcp like this:

(.venv) ◉ tony@tony:~/clnlive:
$ ./invoice.py 10000 pizza
lnbcrt100n1pjjszfzsp5ztx99xmaxy8ggzlfkufa00pglefrrlqaxfmpzaeejg34ke07nxcspp5wn6pseckunnlhsymwap4zfvnw4y8v83syp95qycl46xkjq8y5ykqdqgwp5h57npxqyjw5qcqp29qxpqysgqxulngjvd3hrq3yt0ddw7436syff74zyaxvsd3xdwesx59gg0uvuppmcle3l2dxf79q2qpghv5z35kqj0fdx578745w60a8wken7ws6spyl69th

We are done!

#!/usr/bin/env python

from pathlib import Path
from node_pb2_grpc import NodeStub
import node_pb2
import grpc
import random
import sys

p = Path(".")
cert_path = p / "client.pem"
key_path = p / "client-key.pem"
ca_cert_path = p / "ca.pem"

creds = grpc.ssl_channel_credentials(
    root_certificates=ca_cert_path.open("rb").read(),
    private_key=key_path.open("rb").read(),
    certificate_chain=cert_path.open("rb").read()
)

channel = grpc.secure_channel(
        "localhost:3030",
        creds,
        options=(("grpc.ssl_target_name_override", "cln"),)
)
stub = NodeStub(channel)

# print(stub.Getinfo(node_pb2.GetinfoRequest()).alias)

inv = stub.Invoice(node_pb2.InvoiceRequest(
    amount_msat=node_pb2.primitives__pb2.AmountOrAny(
        amount=node_pb2.primitives__pb2.Amount(
            msat=int(sys.argv[1])
        )),
    label=f"label-{random.random()}",
    description=sys.argv[2]
))

print(inv.bolt11)