Ethereum Developers Probe Incident That Led to Missing Blocks on ETH Mainnet
The Ethereum mainnet recently encountered a disruption marked by the absence of blocks, sparking concerns about the stability and performance of the network. As a result, Ethereum developers have initiated an investigation into the root causes of this incident.
The issue caught the attention of beaconchain.eth, an open-source Ethereum explorer, which highlighted that core developers are actively looking into the matter. In a tweet, beaconchain.eth advised users to closely monitor their nodes while the investigation is underway.
On the early morning of March 28, Ethereum developer Terence.eth suggested that the situation was gradually returning to normal. Terence stated that no action or update was required from any CL or EL client at that moment. When asked about the possible trigger for the issue, Terence mentioned that the blob sidecar was not released from the relayer. While the issue was unrelated to the inscription, it may have been compounded by the presence of additional blobs. Terence revealed that this issue had persisted for a week and was discussed during the recent Ethereum ACD call. While there may be further updates provided, Terence hinted at a post-mortem analysis of the incident.
In early March, the ETH mainnet underwent the Dencun mainnet upgrade, which introduced ephemeral data blobs through EIP-4844 (also known as “protodanksharding”). At the time of writing, it remains unclear whether this issue is related to the recent upgrade due to limited available information.
In a separate incident, Ethereum developers discovered a threat to the Ethereum network that existed from the Merge until the Dencun hard fork. Prior to the merge, different message size limits were implemented for RPC communication to safeguard clients against denial-of-service (DOS) attacks. However, it was subsequently found on Feb. 7 that it was possible to create a block exceeding the 5MB limit by including numerous transactions that were each below the 128KB limit and did not surpass 30 million gas.
This vulnerability meant that an attacker could generate multiple high-paying transactions and transmit them to the network, causing the majority of nodes (specifically geth) to reject blocks that a minority of nodes would approve. These rejected blocks would consequently be forked away, resulting in the proposer missing out on rewards. Fortunately, the bug has now been resolved in the client’s code.
Overall, these incidents highlight the ongoing efforts undertaken by Ethereum developers to ensure the network’s security, stability, and resilience.