Apps powered by embattled crypto Ripple (XRP) faced problems earlier this week after two primary nodes holding up the ecosystem were pushed out of sync for more than five hours.
The issues were detailed in a Twitter thread by crypto analyst @WKahneman, who reported the problems were likely due to an overload of “trash” data.
This “trash” was really a spike in “trust line” and other token data that overwhelmed public nodes ‘S1’ and ‘S2’, both maintained by Ripple Labs itself.
According to @WKahneman, the problem stemmed from the fact that Ripple Labs is expecting those nodes to shoulder too much of the XRP Ledger (XRPL).
The issues also spilled over to other nodes such as explorer XRPLCluster.
Smaller exchanges also reported longer than expected transaction times.
Ripple-powered airdrops triggered bottlenecks
Ripple Labs has marketed XRPL for years by claiming it handles 1,500 transactions per second.
It’s reasonable to expect XRPL to be able to process whatever traffic can be thrown at it, considering how long Ripple Labs has been around (since 2012).
But rather than pure transaction volume, the sudden traffic came from a sharp rise in simultaneous users on the network.
The added usership coincided with surges in trustline requests (XRPL allows sending non-XRP tokens to addresses with open ‘trustlines’). Trustlines enable token airdrops.
A few days after this occurred, another XRP project disclosed another bug had forced nodes verifying XRPL’s full history to reboot.
“All 10 XRP Ledger Full History nodes (maintained by different operators at different geographical locations) restarted themselves,” wrote XRPL Labs.
“We will now have to wait for the Full History nodes to start, sync up with the network and start replying to requests again. This will take approximately 20 minutes. In the meantime, your transaction history may not show.”
XRPL Labs explained the reboot was most likely caused by a bug in the source code powering XRPL (so all nodes would’ve been affected).
“We have informed RippleX technical staff over a month ago about these issues, and the issues are being investigated,” added the group (our emphasis).
Apps that reported problems are more-or-less back online.
Still, the situation is peculiar given that Ripple Labs itself is in control of nodes S1 and S2.
One engineer working on an XRP wallet noted those nodes aren’t intended for “production” (live) usage, according to XRPL docs.
In reality, a permanent solution may be far away. Improvements to optimize XRPL’s underlying source code are required.
The fix in place appears to be an emergency patch rather than a long-lasting solution. So, builders in the XRPL ecosystem are likely to struggle with these bugs until that happens.
“A ledger [XRPL] that can’t handle 20 transactions per second without having nodes fall over and fee escalation bugs popping up needs fixing, agree?,” tweeted one engineer (our emphasis).
“Our systems just allow users to use the ledger [XRPL]. If using the ledger exposes technical debt, the problem is the technical debt.”
Follow us on Twitter for more informed crypto news.