Updated: 2022-04-18

Introduction

This writeup is of interest to Lightning Network developers who want to make their own “autopilot”, but may also be of interest to curious node operators who want to know how CLBOSS treats nodes for making channels with.

One of the first things that was automated on the Lightning Network was to select nodes on the network and make channels with those nodes. After all, good liquidity to the rest of the network is vital for actual use of Lightning. Thus, lnd developers quickly implemented and deployed an “autopilot” that performed this task.

One can argue that CLBOSS is just the lame attempt of some cryptokid wannabe to implement autopilot for C-Lightning instead of just contributing to lnd which everyone is using anyway.

Now, one of the things that struck me with most autopilot implementations is that they select a node and immediately make a channel to it.

But an important figure of merit is how much uptime the selected node actually has.

Thus, CLBOSS also performs investigation, where it also checks the uptime of the node before it actually makes channels.

Investigation is separate from selection. Various modules propose nodes to make channels with. Then the proposals enter investigation. It is the latter part, the investigation, which this writeup will discuss.

In CLBOSS, we always periodically run the algorithms to select nodes to channel with, even if we have no spare onchain liquidity to actually make the channels. This is to allow investigation to take its time to determine the uptime of nodes. We should note in particular that judging uptime is time-consuming since we need to continuously check if the candidate is online or not; just because it is online now does not mean it actually is online >99.5% of the time, you need to check back later regularly. Selecting nodes from the gossip map is just pure computation in comparison to that, and is much less time-consuming.

Proposals and Patrons

When a module wants to propose a channel to a particular node, the rest of CLBOSS actually has an important question: Why?

Thus, CLBOSS has a concept of a channel proposal and its patron. The proposal is the actual node that some module is proposing to actually build a channel to. The patron is the reason why we think that the node is good to channel to — i.e. the patron is good for some module-specific reason, and the proposal has good liquidity to its patron.

This is important since making a channel to some proposal later raises this question: how much do we put into the channel?

CLBOSS determines how much to put into a channel to some proposal by determining the size of the channel(s) between the proposal and its patron.

Thus, a vital part of all the autopilot algorithms in CLBOSS is that every proposal ought to have a patron, because the size of the channel to the proposal is determined by the size of the channel(s) between the proposal and the patron.

In effect, we consider the patron as a sort of proxy for “the rest of the Lightning Network”. If the proposal only has, say, a 20mBTC channel to the patron, it is probably risky to put more than, say, 10mBTC into a channel with the proposal, since it is likely that the liquidity to “the rest of the Lightning Network” would be limited to 10mBTC.

Now you might say, maybe instead of having a patron, just have the proposing module indicate the amount that should be put in the channel.

The problem with that is, as was stated in the Introduction, we investigate proposals for their uptime, and uptime investigation takes time. The time from the initial proposal, to the time we actually build the channel, can be several days, weeks, or months. During that time, network conditions might have changed, and the original proposed amount may no longer be reasonable for the current liquidity of the proposed node.

Thus, having a patron lets us update in case the liquidity of the proposal changes in the future while we were investigating the proposal. When we have some onchain funds that are ready to put into channels, we query the liquidity between the proposal and its patron, so we get the most up-to-date information on how much funds it is reasonable to put into the new channel.

Patron Matchmaking

Modules that want to propose nodes for making channels need to provide the proposal together with its patron, as mentioned

However, at least one module is unable to provide a patron. Thus, the Boss::Mod::ChannelCandidateMatchmaker module takes such patronless proposals and finds a patron for them.

Now, we have mentioned a few times that CLBOSS will investigate proposals continuously. Obviously, such proposals-under-investigation need to be kept track of, so CLBOSS has a list of them stored on-disk.

When a patronless proposal is brought up, the matchmaker will ask for all existing proposals. It then shuffles this list of proposals.

Then, it attempts a getroute from the patronless proposal to the existing proposal. If a route is found from the patronless proposal to an existing proposal, we take the first hop of that route as the patron for the previously-patronless proposal. If no routes are found, then the module simply drops the patronless proposal.

Dowsing

As mentioned, the reason why we have a proposal and a patron is because we use the liquidity between them to determine how much to put in a channel with the proposal.

CLBOSS calls this computation dowsing, and the module Boss::Mod::Dowser is the one that computes dowsing.

Boss::Mod::Dowser works by repeatedly performing getroute between the proposal and its patron, specifying a maxhops parameter. The maxhops parameter limits the length of paths allowed. As of this writing (mid 2022) the limit is 3 hops.

Then, Boss::Mod::Dowser determines the capacity of the returned path. This is simply the smallest channel along the path.

The capacity of the path is then divided by the length of the path plus 1. For example, if there is a direct channel between the proposal and the patron, then its capacity is divided in half (route length is 1, plus 1, thus divide by 2).

Multiple routes are checked (as of this writing (mid 2022) the limit is 10 routes checked), and the capacity divided by the route length plus 1 are summed up. This is then the value returned by the Boss::Mod::Dowser to be used as the amount to put to a channel with the proposal.

Preinvestigation

As a quick check, CLBOSS preinvestigates proposals when they are first broadcast. This is simply a quick attempt to connect to the proposal node. If we fail to connect, we simply drop the proposal.

Modules should use the Boss::Msg::PreinvestigateChannelCandidates message to indicate nodes they think are good for making a channel with, unless they need to use the matchmaker, in which case they should use the Boss::Msg::ProposePatronlessChannelCandidate. The matchmaker will then emit the preinvestigate message on their behalf.

Notice that the Boss::Msg::PreinvestigateChannelCandidates message is plural, i.e. it is about preinvestigating channel candidates. More specifically, the message contains:

• An ordered list of proposal-patron pairs.
• A maximum number of passing proposals to add.

The preinvestigator will go over the ordered list one by one. If a proposal passes preinvestigation, we decrement the number of passing proposals to add and pass the proposal-patron pair to the investigator. If the maximum number of passing proposals drops to zero, this means the preinvestigator has completed the needed number of proposals and the rest of the proposals are simply dropped. If we reach the last proposal in the list, then we do nothing more.

A proposal passes preinvestigation if we are able to successfully connect to the proposal, and the dowser result for the proposal-patron pair is at least 5mBTC.

The preimvestigation stage means that channel selection modules do not have to consider uptime explicitly while selecting channel candidates. The channel selection module simply ranks its candidates according to whatever metric it uses, sorts them best first, and gives this ordered list of channel candidates to the preinvestigator. Then the preinvestigator looks for the best N candidates that are online right now, so that even if the first ones in the metric are somehow low-uptime (since a channel selection module might not use a metric that correlates with uptime) we give a chance to lower-ranked, but still reasonable, nodes that do have good uptime.

Now, a single connect attempt, like what the preinvestigator does, is not an accurate uptime measurement. But it is correlated with uptime: a high-uptime node is likely to be able to connect even just once, while a low-uptime node is likely to fail the connect even if the attempt is done just once. This is good enough for the preinvestigator, since any passing proposals will be given to the more in-depth investigator.

Another important point is that investigation gets a backseat at a certain phase in the lifetime of the node under management. Suppose a node operator starts a completely fresh C-Lightning node, adds CLBOSS as a plugin, and hands it over several hundred millibitcoins. If CLBOSS sits around investigating channel proposals for a few days before making even a single channel, most operators are going to assume CLBOSS is not working and uninstall it. So CLBOSS needs to get some channels up, ASAP, even with little time to do investigation.

Investigation is thus an online algorithm. An online algorithm will gets its inputs one-at-a-time, and can be queried, at any time, for its output. An online algorithm will often be able to give rough estimates at first, then when fed more data, will be able to give better and more accurate results. In the case of investigation, its input is the result of an attempt to connect to the proposal under investigation (i.e. whether the connect fails or not). At any time, we an ask investigation to give its best judgment, even though it might not have a lot of data yet.

As we will see in a later writeup, the initial channel selection modules tend to use metrics that correlate well with uptime. Thus, a quick preinvestigation check is helpful to further improve the uptime estimation of the initial channels. Once CLBOSS has some channels set up, it has time to do more in-depth investigation of other proposals, especially since it will use the reverse-the-direction trick using offchain-to-onchain swaps (discussed in a previous writeup) and will get more onchain funds to put into channels later, which our next batch of proposals, now better-investigated, will use.

Investigation

Once a proposal has passed through preinvestigation, we hand it over to Boss::Mod::ChannelCandidateInvestigator.

The investigator keeps a list of proposals under investigation on-disk, in the common CLBOSS database. Each candidate entry contains a proposal, a patron, and an onlineness score. Candidates must have unique proposal nodes.

The onlineness score starts at 0 and is bounded from -3 to 24.

The investigator primarily triggers on the Boss::Msg::TimerRandomHourly message. It then gathers up to 8 randomly-selected candidates (or all of them if there are less than 8).

The investigator then starts a connection to each of the selected candidate proposals. If connection succeeds, its onlineness is increased by one, otherwise the onlineness is decremented. If a node goes below -3, it is removed from the list of candidates. If the node goes above 24, then score saturates to 24.

Later, when CLBOSS detects that we have funds we can put into channels, we get all nodes with non-negative onlineness scores, sorted from highest to lowest score. If all available nodes have negative scores, however, we instead provide all of them (but still sorted according to score).

In addition, every random-hour, if there are less than 8 candidates with non-negative onlineness scores, we ask channel selection modules to provide more candidates. Even if there are 8 or more candidates with non-negative score, at random, the investigator might also ask for more as well, with the probability of asking for more going down with more non-negative score candidates. On the other hand, if there are more than 32 candidates already, we drop one selected at random.

Investigation is not performed if we are offline according to the Boss::Mod::InternetConnectionMonitor. Otherwise, if we are offline then investigation will always fail and we would unfairly demerit the selected candidates, when it is actually a problem on our side. However, the drop-candidates-if-too-many and get-more-candidates behavior still remains triggered at the random-hourly trigger even if we are offline, as channel selection can be performed offline as long as we have already downloaded the gossip map.