The loss in finality meant that blocks could have been tampered with, and while it isn’t supposed to affect end-user experiences, it did lead to some inconveniences for some applications.
The Ethereum blockchain suffered two brief episodes last week where blocks weren’t finalizing – an unwanted bout of instability that presents risks to the blockchain’s security but isn’t considered dire.
There was a lot of confusion in terms of what the delay in “finality” meant for the functionality of the blockchain, prompting discussions about security concerns. So, it bears unpacking a bit.
You’re reading The Protocol, our weekly newsletter exploring the tech behind crypto, one block at a time. Sign up here to get it in your inbox every Wednesday.
The cause of the temporary loss of block finality remains under investigation, though Prysm, a provider of client software used to run a node on the blockchain, just released a new version, describing it as “the first full release following the recent mainnet issues,” with “critical fixes.”
When data blocks don’t finalize, there isn’t supposed to be any downtime or difference in end-user experience. That said, a loss in finality can lead to some security issues like reorgs.
Reorgs occur when a blockchain produces more than one block at the same time, usually because of a bug or an attack. This means that a validator node temporarily creates a new version of a blockchain, which makes it difficult to properly verify if a transaction has been successful, while the old version of the blockchain continues to exist.
However, snowball effects from this incident led to some end-user jolts. DYdX, a leading crypto exchange platform, had to temporarily pause deposits because of one of last week’s incidents, and Polygon’s zkEVM also experienced some delays with deposits.
The chart shows a spike in missed validator slots last week as the Ethereum blockchain temporarily stopped finalizing transactions. (Glassnode)
So how does finalization work?
In a proof-of-stake blockchain like Ethereum’s, validators first have to propose a block that contains transactions. Once those are proposed, other validators have to sign off on the block to permanently add it to the blockchain, which takes about 15 minutes. Once it is approved, or “attested,” by two-thirds of validators, the block eventually becomes finalized.
Thus, finality is the point where transactions on a blockchain are considered immutable. Finality is supposed to guarantee that transactions within a block cannot be altered.
If finality cannot be guaranteed, the blockchain enters an emergency state called the “inactivity leak,” where validators receive penalties for not reaching finalization. When the state is triggered, it acts as a way to incentive the blockchain to start finalizing again. The incident last week triggered Ethereum’s first-ever inactivity leak.
The Ethereum community has acknowledged that the current timeframe for blocks to be finalized is too long.
“Having a delay between a block’s proposal and finalization also creates an opportunity for short reorgs that an attacker could use to censor certain blocks or extract MEV,” the Ethereum website shared in a blog.
Ethereum co-founder Vitalik Buterin was writing about finality seven years ago, an indication of just how important an issue it is.
When the first loss of finality occurred on May 11, developers immediately shared it over Twitter, saying they were going to deploy extra help to figure out what was going on. After 25 minutes, the issue seemed to have been resolved and the chain resumed finalizing.
Roughly 24 hours later, the chain stopped finalizing again for about an hour, which caused outages for some infrastructure providers.
In the past, finalization has temporarily stopped because of bugs in client software used to run the blockchain. Ethereum has multiple clients in the event that there is a flaw or glitch in the software, so there are other options, and the activity on the blockchain can keep running.
How did this affect the applications?
Tim Beiko, protocol support lead at the Ethereum Foundation, told CoinDesk the incident is “definitely significant, but it’s not something where Ethereum’s security or soundness is at risk or compromised.”
“Within minutes, things were corrected and within like a day or two clients had software patches to make sure that this specific case did not come up again,” he said.
The developers are still looking to understand what caused the blockchain to stop finalizing, and are expected to discuss a post-mortem report in their upcoming Consensus Layer call.
Beiko told CoinDesk that the incident did not get to a point “where we began to test the very extreme fallbacks in the protocol to deal with this stuff.”
The incidents did affect several applications that run on top of the Ethereum blockchain.
Jordi Baylina, technical lead at Polygon, said that the finality stoppage meant that deposits onto the Polygon zkEVM chain were delayed, and since the chain relied on Infura, an infrastructure provider which also temporarily had an outage as a result of the loss in finality, issues for individuals using the zkEVM compounded.
“You need to wait for the finality in layer 1 deposit to be available in layer 2,” Balyina said. “So until you don’t have finality, you cannot use [the chain] or you have the risk of double spending in layer 2.”
DYdX paused its deposits temporarily today due to the lack of Ethereum finality and said it was “continuing to monitor and investigate this issue.”
Despite this, Ethereum developers emphasize that the network did not go down.
Recommended for you:
Crypto Adds Unsavory Footnote to Dreadful 2022: Year of the Rug Pull
Banks Will Dominate Stablecoins, and 2 Other Predictions About the Future of Money
What Fat Tails and Revolutionary Ages Mean for Digital Assets
“Today’s incident has been a great fire drill. It looks like two or three issues came together (as is often the case). The chain recovered gracefully and we discovered a few other issues that could be improved to make Ethereum more resilient,” tweeted Marius van der Wijden, a developer at the Ethereum Foundation.
From technical improvement efficiencies to novel market-based solutions, a plethora of projects are trying to improve the bitcoin mining’s environmental footprint.
The environmental impact of bitcoin mining was trending again in the last few weeks due to a Texas bill looking to limit its grid participation and a New York Times article that condemned the industry.
Bitcoin mining consumes about 145 terawatt hours (TWh) of electricity per year, says veri from Cambridge University’s Centre for Alternative Finance, which is about the power consumed by Sweden, according to the International Energy Agency.
To critics, bitcoin’s proof-of-work algorithm is wasteful by design as its electricity demand continuously grows. To industry advocates – it’s a feature, not a bug – as it secures the network while ensuring decentralization.
There is a third camp that sees bitcoin’s energy consumption as an opportunity. A slew of new products and companies are trying to mitigate the environmental impact with several innovative solutions that build on top of what is already in place to help miners become more sustainable.
Some of these projects are creating or taking advantage of market instruments that incentivize making bitcoin more “green,” while others are technical, centered around improving efficiencies and synergies around reusing the heat generated by the veri centers.
Renewable energy credits
One such product, offered by crypto lender BlockFills and fund Isla Verde Capital, aims to help not only miners but also investors to find a “green” solution for their energy usage.
The offering essentially is tradable environmental assets in the forms of carbon emissions offsets and Renewable Energy Credits (RECs). Carbon credits, assets that represent sequestered greenhouse gasses through projects such as reforestation, are as well known as they are criticized.
Renewable energy credits (RECs) represent ownership of the sustainability of electricity produced. These certificates represent 1 megawatt hour (MWh) of power produced from renewable sources such as wind, hydro and solar. They are usually separate to any power purchase agreements and are usually traded over the counter (OTC).
BlockFills and Isla Verde Capital tailor the purchase of RECs and carbon credits to miners’ needs, and later retire them, such that they can make claims about renewable energy sources.
The RECs are also geared towards bitcoin investors. “Massive asset managers” are now “looking at bitcoin, but they have these sustainability mandates that they must follow,” said BlockFills John Divine. The RECs can help them invest comfortably.
This might actually raise the price of RECs, “which directly incentivizes investment in renewable energy technology,” Divine said.
Incentivizing sustainability
Switzerland-based Block Green is another project that is trying to incentivize sustainable mining through a decentralized lending protocol. On their platform, liquidity providers looking for bitcoin-native investments can buy future hashrate over a specified period of time, or computing power.
The platform includes “know-your-miner” information about a company’s financials, operational veri, their energy sourcing and strategy. Block Green believes that market mechanisms on the platform will incentivize sustainable mining as liquidity providers will pick miners with sustainable operations, lowering their cost of capital.
“We are currently working with some of the largest miners in the U.S. and Canada and we have begun integrations with institutions such as custodians, exchanges and asset managers looking to give users access to transparent and scalable” returns on their bitcoin, said a spokesperson for the firm.
Tokenizing clean bitcoin
Another solution that is using financial incentives is offered by Clean Incentive and Sustainable Bitcoin Protocol (SBP). These companies are trying to promote investments in “clean” bitcoin by creating new, blockchain-based assets that miners can trade to capitalize on their use of renewables. Investors looking to verifiably own environmentally-conscious bitcoin are the right fit for these assets.
With SBP, miners can verify their use of clean energy with third-party auditors and be added to a registry. After that, for every block reward they get, they receive a Sustainable Bitcoin Mining Certificate, a blockchain-based asset which they can sell to institutional investors. SBP completed its first transaction of a sustainable bitcoin certificate in February.
Similarly, Clean Incentive looks to “collect, validate and tokenize ESG [environmental, social, and governance] attributes” from a network of miners, said its founder and CEO Casey Martinez, a veri scientist with experience in renewable energy.
The startup is still in stealth mode, but has already onboarded several miners, Martinez said Clean Incentive partnered with a small-scale miner from Canada, Ocean Falls Blockchain, in November.
Efficient cooling
Some of the more technical solutions that firms are providing include both hardware and software-related products.
Immersion cooling firm, LiquidStack, offers a hardware-based cooling solution that can cut the energy used by the computers in bitcoin mines by 40% and reduce their land use by one third, the company said in March.
For every megawatt (MW) of energy used for the actual computing in a veri center, LiquidStack’s solution uses 0.02 MW for cooling, whereas other options use 0.1 MW to 0.7 MW, LiquidStack said.
The firm, one of the earliest in the space, received funding from Trane Technologies (TT) in March, a 150-year old firm in the heating and cooling space which brought in $16 billion in revenues in 2022.
“What made LiquidStack attractive was its potential to improve sustainability for veri centers, including bitcoin mining, and its innovation,” said Amber Mulligan, VP of Strategic Sales and Marketing, Commercial HVAC Americas at Trane.
LiquidStack’s technology also makes heat reuse easier and more efficient, opening the door to a host of synergies for miners, said Mulligan, noting that because the heat is actually managed with liquids instead of traditional air cooling, capturing it and directing it to other uses is easier.
On the software-side, Vancouver-based mining services firm, Lincoin, has created a program that miners can use to more efficiently and profitably manage their operations, including their participation in demand response programs and heat reuse activities.
Recommended for you:
DeFi Giant MakerDAO Integrates Blockchain Veri Provider Chainlink for DAI Stablecoin
Cory Doctorow’s Web3 Rallying Cry: ‘Seize the Means of Computation’
Chainlink Unveils Crypto ‘Keepers’ and Anti-Fraud Blockchain Bridges
Join the Most Important Conversation in Crypto and Web3 in Austin, Texas April 26-28
Demand response is when a miner, or other energy consumer, shuts down their operations at times of peak demand, such that the grid can meet the consumption needs. Often miners get paid for this. Heat reuse refers to the practice of using excess heat from a mining operation for another activity, such as greenhouse farming.
The software, called Rails, integrates real-time veri from over 20,000 grid nodes in 9 deregulated electricity markets in the U.S. and Canada, said a press release.
“Large scale miners use Lincoin to monitor real-time profitability, manage and optimize their operations, streamline tasks and participate in grid ancillary services while smaller miners use Lincoin to innovate by managing heat in greenhouses, monetize their surplus solar energy generation, or simply mine intelligently,” said CEO Medi Naseri in an email interview with CoinDesk.
DWF Labs made headlines with more than $200 million of investments in crypto projects such as CryptoGPT or Synthetix. A closer examination reveals that many of their deals aren’t typical venture capital investments, but packaged with market making services, pledges to boost trading volume or even selling tokens directly for a project’s treasury. Industry experts claim red flags and conflict of interest, but the firm says it’s all a misunderstanding.
The giants of crypto venture capital are mostly a well-known group of firms that’ve been around for years, companies like A16Z, Paradigm, Pantera Capital and Digital Currency Group (CoinDesk’s parent).
So the quick and loud emergence of a firm called DWF Labs as a seemingly large player in the space over the past few months caught many by surprise. They announced through press releases and media organizations like CoinDesk and The Block a slew of investments in projects including $40 million for internet alternative provider Tomi, $40 million for artificial intelligence-related token Fetch.AI and $10 million in AI-focused crypto data project CryptoGPT.
But a closer examination reveals DWF, whose founders made their money as crypto high-frequency traders, isn’t exactly a venture capital firm – not always, at least.
While the recent slew of headlines refer to DWF’s partnerships with crypto projects as ‘investments,’ DWF Labs actually functions more similarly to an over-the-counter (OTC) trading desk. The company typically approaches a crypto project with a token, and offers to buy millions worth of the token at a discount to market value, according to conversations with several crypto projects that have worked with DWF.
But DWF Labs says it’s all a misunderstanding. “There might be some questions on the use of the word investment,” said DWF Labs Partner Stefano Virgilli. “When we use the word ‘investment’ – to us the most important thing is that if we’re purchasing the tokens and they’re using the funds to further develop, that’s an investment,” he added.
The controversy
Investments in crypto projects typically follow a venture capital model. Projects tap venture firms for capital via funding rounds (i.e. pre-seed, seed, Series A, etc.) and, in turn, the investors receive a portion of the project’s equity. In most cases, particularly in early stage investments where a project has not yet launched a token, investors will receive a Simple Agreement for Future Tokens (SAFT), a contract that outlines the tokens allocated to the investor if the project launches a token in the future.
DWF Labs’ investments are more ad hoc in nature and the company primarily selects for projects that have already launched a token.
While DWF Labs refers to itself as “a global web3 venture capital and market maker” or “multi-stage web3 investment firm” in press releases, the deals are often presented as “strategic partnerships” that can include token acquisitions, market making services, pledges to boost a token’s liquidity and trading volume, and additional support with marketing and media presence.
Even helping projects’ treasuries to sell their token holdings, according to the press release the firm distributed about its launch in September.
In the post, the firm said that “DWF Labs invests in digital asset companies and supports existing markets, enabling digital asset companies to sell their tokens for up-front capital without adverse price impact,” adding that “DWF Labs buys tokens with its own funds, allowing its corporate customers to sell tokens quickly.”
It is quite common in the crypto industry for market making firms to have venture capital arms. Jump Crypto and Wintermute, two heavyweights in the crypto market-making sector, both began as trading firms. But both have since expanded into cutting venture checks for projects, and even building their own pieces of core infrastructure (Jump has backed the Wormhole cross-chain bridge and Wintermute has launched its own decentralized exchange).
However, the industry standard is that these contracts should be separated. Even though the lines between the two divisions can be blurred sometimes by the market makers, some industry observers have grown concerned about DWF’s recent activity and seemingly packaging different services under partnerships.
“It’s a massive conflict of interest,” Walter Teng, research firm Fundstrat’s vice president of digital assets, told CoinDesk. “If you invest, you want the token’s price to go up. If you market make, you can manipulate the price to go up by spoofing.”
“All of their ‘investments’ are poorly disguised agency OTC (over-the-counter) trades,” a market making firm’s executive told CoinDesk, who asked not to be named due to company policy. “They make a big announcement about ‘partnerships, investments’ or some other nonsense, but in reality it is a way for token projects to sell their treasury without announcing that they are selling their treasury.”
DWF’s managing partner Andrei Grachev defended the firm’s token maneuvers in a recent tweet, calling it “dumb” if a market maker (MM) leaves all the acquired or borrowed assets in a wallet, because an “MM should create markets, provide depth, improve order execution instead of doing nothing and waiting when the market is skyrocketing to execute its call options.”
DWF Labs’ strategy
DWF Labs launched in September, as an investment-focused arm of Digital Wave Finance, a top high-frequency trading firm that trades spot and derivatives on over 40 exchanges, according to the firm’s press release.
Grachev told CoinDesk that DWF Labs’ funding comes from the money earned from profits of the high-frequency trading business. Grachev denied that the firm has received any funding from Russia, a rumor circling within the crypto industry.
Grachev said that the firm has multiple types of investments, some with token lock-ups, others without vesting period, and focuses on projects with tokens. “We prefer to have tokens but we also have several equity deals,” said. “But frankly with equity…it is not our strong side,” he said.
While he said that DWF Labs “usually do not include market making deals in our venture side,” later, he admitted that “we have pure investments without market making, we have market making [agreements] without investment, and we have [them] combined.”
“As a market maker, of course we support our portfolio. If we invest, we will provide much more liquidity to the project compared to if we don’t invest,” Grachev said.
When asked about DWF’s investment strategy and due diligence, Grachev talked about focusing on five sectors – TradFi, DeFi, GameFi, CEXs and artificial intelligence – and aiming to “have stakes in all major chains (…) in order to have access to their ecosystems.” The firm looks for projects with “life and traction,” he said, checking social media posts and what exchanges their token is listed on.
“If a project is listed on BitFinex, Coinbase or Binance, then the project is proven and good because these exchanges have very strict due diligence and very strict policies of listing,” he added.
Grachev also said DWF doesn’t usually participate in specific venture rounds. “We just approach them,” he says.
CoinDesk viewed a series of messages between DWF Labs and a crypto project that showed a member of the DWF Labs team offering to invest in the project and provide free market-making services. DWF told the project it could invest via a direct OTC purchase of liquid tokens from the project’s treasury, or with a lock-up period and market-making services.
Messages from the market maker to another project showed that DWF offered to buy tokens in daily tranches without any lock-up period at a discount or in one installment with a one-year lock-up at a steeper discount. According to the message, DWF promised to help list the token on Korean exchanges including Binance Korea which the firm has “good relationship” with, create options trading and “build narrative” leveraging DWF’s team and media presence.
There were several past announcements when DWF mashed investments and market-making deals.
One instance was its strategic partnership announcement with derivatives trading platform Synthetix. According to a press release on March 16, the firm said it acquired $15 million of the project’s native token SNX “aimed at boosting liquidity and market making,” adding a quote from Grachev that “we are thrilled to invest in Synthetix.”
Blockchain data shows that DFW’s wallet – labeled by crypto intelligence firm Nansen – received 5.3 millions of SNX directly from Synthetix’s treasury wallet between March 14 and March 16. Then, the firm transferred all tokens to Binance in multiple transactions between March 16 and 20.
Transactions between DWF Labs and Synthetix treasury. (Nansen)
In November, DWF announced a $10 million investment in the TON ecosystem. The firm’s press release said that the “strategic partnership” with the project extends to “an investment, token development, market creation and exchange listing.” The partnership also includes “50 seed investments scheduled over the next 12 months,” doubling the TON token’s trading volume in the first three months of the partnership, and developing an OTC market “to let buyers and sellers complete large transactions.”
Another case is the firm’s investment into web3 influencer platform So-Col. According to a story by crypto-focused publication The Block and cited on DWF’s website, DWF invested $1.5 million in “a round” by purchasing So-Col’s native token SIMP in February. Irene Zhao, So-Col’s founder, said that the tokens have a one-year vesting period ending in February 2024. The post does not mention other services besides investing.
However, Nansen’s blockchain data on the Ethereum blockchain shows that DWF’s crypto wallet received 3.3 million SIMP tokens between March 6 and March 24. Within the same period, DWF sent some 2.6 million tokens to KuCoin exchange, then transferred the rest to an unknown wallet on March 30. After the announcement on March 28, SIMP almost doubled from around 1.7 cents in a week, then started to plummet on April 4 towards 1 cent, per CoinGecko data.
CoinDesk reviewed Telegram messages of a So-Col representative saying that they decided to work with DWF Labs because besides serving as a market maker DWF also invested in the project directly helping to extend the startup’s runway.
Sending tokens to exchanges
Grachev said that DWF Labs keeps most of its funds and investments on centralized exchanges (CEXs) and transferring tokens to an exchange does not indicate the company will sell.
“We keep all of our inventory, almost all of our inventory, not only our investments but our own funds on exchanges,” he said.
However, keeping supposedly long-term investments on exchanges is a worrying sign for some industry experts, hidden from savvy blockchain analysts and traders whether DWF sells tokens or uses them for market making purposes.
“It’s a red flag,” a founder, who asked to remain anonymous, of a crypto analytics firm with former market making experience told CoinDesk. “They [DWF Labs] market them as an investment, and then claim to do ‘market making’ so they can keep funds on exchanges and just dump.”
It’s hard to opine on where a firm like DWF should draw the line between VC and market making. Perhaps, a page from the TradFi banking playbook could work. In that realm, investment banking and trading/research is separated by a so-called Chinese wall. Where that line might need to be drawn for crypto investment firms is unclear.
Recommended for you:
El Salvador Grants First Digital Asset License to Bitfinex
I Made an NFT Collection to Represent My Student Loan Debt
Stepn Parent Company Launches NFT Marketplace
Join the Most Important Conversation in Crypto and Web3 in Austin, Texas April 26-28
In the interview, Grachev admits his “biggest mistake” was not properly explaining his firm’s operating philosophy and investment process. “We need to be more open. I want [the community] to know how we work and then let people decide who is right and who is not right,” he said.
