About the background, ecology and future prospects of Data Availability War

DA (Data Availability) projects have sprung up in the market, among which highly competitive projects are Celestia, EigenLayer, Avail, NearDA and Covalent.
According to the analysis of Huobi Research Institute, DA of core content is not that difficult. The simplest one can do DA with a single machine, and the most complex one, such as Celestia, uses sampling mode for decentralization. To put it simply, DA is storage, and the cost of occupying it is very high. If you don't strictly require Ethernet-level security services, which DA you choose is a balance between cost and security.
The most important principle for using the DA layer is: the greater the value of the service, the DA should be as safe as possible.
This research report is Huobi Research Institute’s analysis of the background, ecology and follow-up prospects of the Data Availability War, including DA in the eyes of Buterin, and an inventory and sorting of various DA projects. Based on a comprehensive analysis of the DA track, Huobi Research Institute believes that future DAs may be decentralized, and 7-8 major DAs may be enough in the future market.

1. Data AvailabilityThe emergence of data availability problems

1.1 What is DA

Simply put, data availability means that the block producer All transaction data for is published to the network so that it can be downloaded by validators. If a block producer publishes the complete data and makes itavailable for download by validators, we say the data is available; if itwithholds some data so that validators cannot download the complete data, we say Just say the data is unavailable.

1.2  Two key points of DA issues: safety and cost

Therefore, combined with the definition of DA, we see that DA contains the following two aspects:

First, ensure The verification mechanism is carried out safely, and the second is to reduce the cost of publishing data.

· Issues of ensuring the security of the verification mechanism

In order to make the verification safe, the current L2 sequencer generally publishes L2 status data and transaction data on the strongly secure Ethereum, relying on Ethereum Square for settlement and data availability.

Therefore, the data availability layer is actually where L2 publishes transaction data. Currently, the mainstream L2 uses Ethereum as the data availability layer.

· Reducing the cost of publishing data

 L2 simply makes data availability and settlement happen on Ethereum. Although it has sufficient security, it also bears huge costs. This is also the second problem faced by L2, which is how to reduce the cost of publishing data.

2. Cost composition in DA & Cost reduction and efficiency improvement direction< /h2>

From the introduction in Chapter 1, we can see that an important issue of concern in DA is how to reduce costs.

p>

If you want to make L2 cheaper overall, you must reduce the cost of publishing data. So, how to reduce costs? There are two main approaches:

· Reducing the cost of publishing data on L1, such as Ethereum's upcoming EIP-4844 upgrade.

· Imitating Rollup, transaction execution is separated from L1, and data availability can also be separated from L1 to reduce costs. Just don’t use Ethereum as the data availability layer

So all parties have made a lot of efforts to reduce costs. Judging from the current DA solutions, Near DA has the lowest cost, about $0.0016/block. Followed by Celestia, EngenLayer, EIP4844, etc.

p>

3. DA in the eyes of Vitamin God

3.1 A solution that does not use Ethereum DA is not a true Layer 2 - the first counterattack of the Ethereum Foundation and Buterin

After Celestia became popular, Buterin once hinted that "Ethereum's layer 2 project must use data availability on ETH." Later, Dankrad Feist, a member of the Ethereum Foundation, also made remarks. He pointed out that it is not Rollup or Ethereum Layer 2 that does not use ETH as the DA layer (data availability layer).

In this case, both Arbitrum Nova and Mantle will be "removed" from the Layer 2 list because they only disclose transactions outside of ETH (an off-chain DA network called DAC) data.

At the same time, Dankrad also said that solutions such as Plasmas and state channels that do not require on-chain data availability (Data Availability) to ensure security are still considered Layer 2, but Validium (ZKRollup that does not use ETH as the DA layer) is not counted as Layer 2.

3.2 Use non-Ethereum as DA, then it is Ethereum Validium--V God’s compromise

Buterin later stated in a reply on his Twitter stream, "being a validium is a correct choice for many apps, and using good distributed DA guarantee systems can be a good way to increase the practical security of a validium"

At the same time, he believes that the core of rollup is an unconditional security guarantee: even if everyone is against you, you You can also withdraw your assets. This guarantee cannot be obtained if data availability depends on external systems.

3.3 About ENS and Data availability --- Vitalik wants to use ENS to gain control of DA

ENS domain name service will define a set of interaction logic. Users only need to enter a short domain name to automatically connect to the associated long address corresponding to the ENS smart contract, which solves the problem of complex and difficult to remember EOA addresses. Identify pain points. It should be said that this set of domain name services of ENS points to the future expansion market with larger user traffic, especially some Mass Adoption user groups. And layer2 is the future for Ethereum to expand and absorb large traffic.

V God believes that if ENS’ domain name resolution solution cannot cover layer 2, it will only stay in Ethernet At the level of the main network, it is difficult to open up the space for imagination. Based on this background, Buterin explained the importance of ENS on Twitter, "it needs to be affordable!" . ENS will naturally consider providing a complete set of data analysis solutions for layer2, so that users can directly analyze data on layer2 Perform domain name resolution and data search to reduce dependence on their respective layer 2 centralized gateways.

It is not difficult to see that for users to use the ENS domain name normally on layer 2, the prerequisite is to call and verify the global data on the Ethereum mainnet. This means that to enjoy the ENS set of services, you must use the orthodox Ethereum Data Availability capability, and those who take the shortcut of the hair chain based on OP Stack and put DA in third-party DA such as Celestia Layer2 on the platform is not compatible with ENS. At this point, it is not difficult to understand Vitalik’s intention. In short, Buterin’s deeper meaning is to use ENS to set a set of interoperability standards for layer2 platforms and at the same time tighten DA control.  

3.4 Vitalik talks about the return of Plasma

Vitalik mentioned in his article that Ethereum’s layer 2 expansion plans originally included: Plasma, Rollup, Validium, Parallel and other solutions.The expansion direction that Vitalik expects should be balanced development and adaptability to various applications. The scene is constructed with diversified layer 2, but the actual market structure is that the Rollup solution is the only one, and it is increasingly involution.

b. Plasma is equivalent to a side chain solution that regularly synchronizes Merkel status data to the main network. It is a data and The calculation depends on the expansion plan of the main network. In this case, Layer 2 can use a very centralized approach and design a very complex ledger model to efficiently expand, and can also reuse the system capabilities of the main network validator. Vitalik’s new article re-examines Plasma and introduces a ZK+Plasma expansion solution, which is obviously another political flag-waving issue related to layer 2.

3.5 Summary

The above various operations of V God can be summarized as follows:

DA’s demand is too strong, but it does not want to give the market to Celestia . Let’s talk about security first, and then bring up the issue of ENS. Finally, we found that the market is not paying for it, and there are still many people using first-party DA. It’s okay to just use Validium. A few days later, the old Plasma was revealed to guide the market. Exploration in the direction of ZK+Plasma. From its original intention, it is to continuously pull the DA market towards Ethereum.

4. Overview of DA solutions and DA projects

From the above content we can see that DA There are many layer options. From a broad perspective, it can be divided into two major parts: the upper chain and the lower chain.

· On-chain solutions

Referring to L2 still uses Ethereum as the DA layer and relies on Ethereum to reduce data availability costs. This means that Ethereum will serve as a real-time bulletin board in the future. The data on the board will be deleted after a period of time, and L2 must find a way to store backups of all data.

· Off-chain solution

That’s it No longer regard Ethereum as the DA layer, but instead look for more economical ways to obtain data availability. According to differences in decentralization and security, off-chain solutions can be divided into four types: Validium, Data Availability Committee (DAC), Volition, and general DA solutions.

4.2 Celestia

Celestia is the originator of modular public chains , which is developed based on the Cosmos SDK and focuses on data availability. It is a highly competitive leading DA project that has been launched on the mainnet.

· Technical Features

· Data Data availability sampling (DAS) 

DAS enables light nodes to verify data availability without downloading the entire block. Light nodes cannot verify data availability because they only download the block header. Celestia uses the 2-dimensional Reed-Solomon encoding scheme to re-encode the block data to implement DAS for light nodes. Data Availability Sampling (DAS) works by having light nodes conduct multiple rounds of random sampling of small portions of block data. As light nodes complete more rounds of block data sampling, the confidence that the data is available increases. Data is considered available once a light node successfully reaches a predetermined confidence level (e.g., 99%).

· Namespaced Merkle trees (NMT)

NMT enables the execution and settlement layers on Celestia to download only the transactions relevant to them. Celestia divides the data in the block into multiple namespaces. Each namespace corresponds to applications such as rollup built on Celestia. Each application only needs to download data related to itself to improve network efficiency.

· Celestia mainly generates revenue from applications in two ways:

· Pay for blob space: Rollup uses $TIA to pay and publish data to Celestia's blob space.

· Pay gas fees: Developers use $TIA as the gas token for Rollup, similar to ETH based on Ethereum Rollup.

· Development potential

· Project Already online, with high technical maturity

· Potentially rich airdrops, pledges from $TIA, such as project Dymension and Altlayer airdrop targets will choose $ For TIA pledgers, more Ethereum Layer 2 projects, modular public chains, and Cosmos ecological projects are likely to follow similar airdrop routes in the future.

· Ecological richness: Cooperate with cross-chain bridges, settlement layer solutions, defi projects, games, sequencers, etc.

· DA cooperation project parties continue to increase, including Manta, Eclipse, Caldera, and Snapchain. It also integrates with Arbitrum Orbit, Polygon CDK, Aevo derivatives trading platform, etc.

4.3 EigenDA

EigenLayer is an Ethereum-based The Restaking protocol allows users to re-pledge ETH, lsdETH and LP Token in other side chains, oracles, middleware, etc., as nodes and receive verification rewards, so that third-party projects can enjoy the security of the ETH main network , ETH pledgers can also get more benefits, achieving a win-win situation.

EigenDA is a decentralized data availability (DA) service built on Ethereum using EigenLayer Restaking, and will be the first active verification on EigenLayer Services (AVS). Unlike Celestia or Avail, EigenDA does not require bootstrapping a new validator set, and Ethereum validators are free to join if they choose.

· Technical features:

·  Enhance Ethereum's DA capabilities: Blob block data + KZG commitment

EigenDA mobilized Cancun's upgraded Blob block data + KZG commitment, Rollup chain can Perform erasure coding on the Blob Data to generate a KZG commitment and then publish it to the EigenDA contract. EigenDA nodes will guarantee the subsequent chain DA capabilities, which is equivalent to enhancing the DA capabilities of Ethereum. The key is that the entire process of EigenDA is completed around the existing infrastructure of Ethereum such as Blob and KZG, and its node verification work is also participated by Ethereum’s Validators.

· No autonomous consensus and P2P network

EigenDA node must be in Ethereum L1 Re-staking ETH (or rather, staking ETH derivatives) in the EigenLayer contract. EigenDA nodes are a subset of Ethereum validators. Subsequently, after DA purchasers (such as rollups, also known as decentralizers) receive the data blobs, they encode them with erasure codes and generate KZG commitments, publish and distribute them for node confirmation. The decentralizer then collects these signatures one by one, generates an aggregate signature, and publishes it to the EigenDA smart contract, which verifies the signature.

· Using custody certification method

EigenDA does not use data availability sampling method to To verify that the node actually stored the data, this is done using the escrow proof method. Anyone can submit a proof to the EigenDA smart contract and the proof will be verified by the smart contract. If validation succeeds, the lazy validator will be pruned.

· Development potential

· Multiple cooperation Project side, competing with Celestia: now integrated with multiple L2 projects such as Celo, Mantle, Fluent, Offshore, OP stack, etc.;

· Backed by Eigenlayer Diverse ecology, including sequencers, cross-chain bridges, oracles, etc.

4.4 Other DA projects

4.4.1 Avail

Avail can efficiently sort and record transactions, provide data storage and verification of data feasibility, and supports Ethereum Virtual machine (EVM) compatible blockchain allows Rollup to publish data directly to Avail, and its light client network verification mechanism (described below) allows Rollup on Avail to pass the light client network without relying on smart contracts and Base layer to verify status. And because of its modular nature, developers can store data in Avail and choose other networks for settlement.

· Consensus mechanism: BABE and GRANDPA consensus mechanisms inherited from Polkadot SDK

· Decentralization:

· Avail adopts Polkadot’s Nominated Proof of Stake (NPoS) and supports up to 1,000 verification nodes. NPoS also has effective The reward distribution can reduce the risk of equity concentration.

· Avail has the ability to sample data from its light client's P2P network. This feature provides Avail with an efficient and reliable backup mechanism. Data availability is ensured even in the event of a failure.

· Validity proof: Avail adopts KZG polynomial commitment

Current status: Mainnet Not yet online

4.4.2 Near DA

2023 On November 8, the NEAR Foundation announced the launch of the NEAR Data Availability (NEAR DA) layer, which provides powerful and cost-effective data availability for ETH rollup and Ethereum developers. Initial users include StarkNet’s Madara, Caldera, Fluent, Vistara, Dymension RollApps and Movement Labs.

· Security: Inherit the security of Near Network

· Cost Advantages: 100kB calldata on NEAR costs $0.0033

· Status: NEAR DA has been integrated with Polygon CDK for developers to build Ethereum ZK Rollups.

4.4.3 Covalent

Blockchain data query The service platform Covalent can standardize data from multiple blockchains, and its unified API allows developers to reuse queries in supported networks, solving the problem of difficulty in obtaining blockchain data.

After the Cancun upgrade, the Ethereum mainnet will only save the status data submitted by L2 for 1 month, and then discard it. In order to maintain the decentralization of the network and the establishment of the DAS light node mechanism, Celestia will also regularly discard the status data submitted by L2. However, Covalent launched the long-term DA service EWM (Ethereum Time Machine) at the end of last year, which permanently saves the state data submitted by L2 discarded by Ethereum. Covalent is responsible for reading L2 status data.

And Covalent will index and structure these data and integrate them into the on-chain data API service of its platform to provide professional blockchain data websites and government regulatory authorities with , artificial intelligence research team provides services and support.

· Current situation: As of December 2023, Covalent supports more than 210 blockchains and plans to support more than 1,000 blockchains by the end of 2024 . According to the Covalent data availability report released by Messari not long ago, Covalent has billions of data and is currently the most suitable to provide services for applications that require general and widely applicable data.

4.4.4 zkPorter

zkPorter is an Ethereum expansion Solution: The off-chain data availability solution optimized for decentralization launched by zkSync handles data availability through a hybrid approach that combines zkRollup and sharding ideas. zkPorter introduces an optional validator mechanism, where zkSync token holders stake their tokens to later verify and sign blocks. Although the product progress of zkPorter has not been disclosed to the outside world for a long time, judging from the fact that the Layer3 application chain GRVT on zkSync is scheduled to be launched on the main network in Q1 this year, and the project stores data on zkPorter, zkPorter has been speculated by the outside world or will be launched before then.

4.5 DA project comparison

Technical aspects ：

Performance:

Summary

· The most competitive project among the above DA projects is Celestia , EigenLayer, Avail and NearDA, while Covalent takes an unusual path, opening up unique requirements for its DA applications.

· Among all data availability implementation solutions, data availability sampling plus KZG polynomial commitment is the most mainstream solution, which can reduce node costs and improve Prove efficiency while ensuring data availability.

· From a technical perspective, Ethereum Danksharding and Celestia are the most decentralized.Because they both use sampling technology, this reduces the number of nodes. It can achieve large bandwidth while meeting the performance requirements. EigenDA is second, and also uses sampling, but EigenDA is parasitic on ether, and its number of nodes is a subset of ether. Other DA projects should not use sampling. For example, the degree of decentralization of NearDA is equivalent to the degree of decentralization of Near Protocol.

· The implementation threshold of the Optimistic proof selected by Celestia is lower than the KZG polynomial commitment, and the technology maturity is higher, but the future technology limit is Not as good as KZG polynomial commitment, compared with similar types of Avail and EigenDA, Celestia's development progress is currently faster and will be launched on the main network earlier. However, Celestia will also face direct competition from Ethereum after the Cancun upgrade.

· As a Layer2 developer, what lies ahead is nothing more than the Tradeoff of DA legitimacy and chain cost. DA legitimacy is relatively passive in the commercial market. It is suitable for some comprehensive layer2 projects that are more concerned about security consensus issues and have a certain brand heritage and market foundation. And some cutting-edge small layer2s, especially those based on OP Stack's one-click chain launch The chain will try its best to squeeze costs to a minimum. For them, third-party DAs such as Celesitia are naturally a better choice. However, Eigenlayer still cannot reduce the actual cost of developing L2 for the project side.

· However, for DA layer projects, their scenarios are single and to B mode. These projects cannot be as capable as a comprehensive public chain (such as Solana). Rely on ToC scene investment DAPP to obtain a large amount of liquidity. If it cannot successfully attract the adoption of many Rollup projects, its ecological construction will weaken. Moreover, Celestia seems to lack a very strong capital promoter. Although its technical narrative itself makes sense, if it cannot take root well within the Ethereum ecosystem, its grand blueprint is likely to become a castle in the air.

5. Modularization of DA layer and blockchain

The DA layer has always been there, whether it is BTC, ETH, or Solana. For example, in the Bitcoin network, data is stored directly on blocks. 15 years ago, Satoshi Nakamoto published the Bitcoin white paper and limited the block size to 1MB. The block size limits what each block can accommodate. The upper limit of transaction data. The later Segregated Witness (SegWit), Taproot, and Ordinal protocols were actually designed to enhance the data availability of the BTC network to a certain extent.

In the Pre-4844 Ethereum network, the data returned from Layer2 to Layer1 is stored in Calldata. Although Rollup safely transfers the calculation to L2, However, the storage is still in L1. Due to the limited storage capacity of L1, the capacity of Rollup on Ethereum is limited. Ethereum Blocksize is about 150K - 250K. Even if all space is used for Rollup, the storage space is limited, limiting the throughput of L2. Therefore, Ethereum adopts Proto-Danksharding to enhance data availability by introducing new transaction types including Blobs.

Therefore, it can be said that the DA layer is actually abstracted along with the expansion needs of each blockchain and the higher requirements for data availability. The first layer is the refinement of social division of labor in the development and evolution process of blockchain technology, similar to the refinement of social division of labor in the process of human development, while the modular blockchain decouples a certain functional level in the single blockchain and outsources it to Other blockchain networks are responsible for further dividing labor and improving efficiency.

Celestia proposes modular blockchain

Traditional monolith The architecture of the blockchain usually consists of four functional layers:

· Execution layer - The execution layer is mainly responsible for processing transactions and executing smart contracts. It includes transaction verification, execution and status updates.

· Data availability layer - The data availability layer in a modular blockchain is responsible for ensuring that data in the network can be accessed and verified. It usually includes functions such as data storage, transmission and verification to ensure the transparency and trust of the blockchain network.

· Consensus layer - responsible for the agreement between nodes to achieve consistency of data and transactions in the network. It verifies transactions and creates new blocks through a specific consensus algorithm such as Proof of Work (PoW) or Proof of Stake (PoS).

· Settlement layer - Responsible for completing the final settlement of transactions, ensuring that the transfer and records of assets are permanently stored on the blockchain, and determining the final outcome of the blockchain state.

Celestia first proposed the concept of modular blockchain, decoupling different functional levels from the middle section of a single blockchain, thus making the blockchain Able to focus on special functions at a certain level and reasonably divide labor to maximize overall usability.

This is not to say that the modularity of the blockchain must have advantages over the single blockchain, but I hope that we can use modular thinking to look at the area. The future development of the blockchain will allow for more possibilities and conjectures.

6. Conjectures about the future

6.1 Look at the combination of blockchains from a modular perspective

As mentioned in the previous paragraph, the differences between From a modular perspective, the blockchain can be divided into execution layer, data availability layer, consensus layer and settlement layer.

Here we take the Ethereum ecosystem as an example. The consensus layer is the Ethereum main chain.

< img src="https://img.jinse.cn/7182772_watermarknone.png" title="7182772" alt="blp2U5SDu3qE7HYDV1NqO2eVD6vB3JZe8UY1j3wk.png">

For Ether From the perspective of Fang Ecology, the number of possible modular blockchains in the future is = number of execution layer solutions * number of settlement layer solutions * number of DA layer solutions 

6.2 The DA layer will be decentralized

The DA layer can already spring up, and the core content is that DA is not that difficult. The simplest one can do DA with a single machine, and the most complex one can do decentralization under sampling like Celestia. Decentralization using sampling mode, the more nodes, the greater the bandwidth (imagine a p2p movie download network), this also has network effects, so I think there will not be too many such "decentralized sampling DA" in the end ". But there is no limit to other forms of DA, and there can be countless.

Here we can make an analogy, DA is storage, and the cost of occupancy is very high. If you don't strictly require Ethernet-level security services, which DA you choose is a balance between cost and security.
The principle is: the greater the value of the service, the DA should be as safe as possible.
Therefore, future DA may be decentralized. Even so, maybe 7-8 major DAs might be enough.

6.3 Ethereum may make a dedicated DA layer in ETH3.0

Ethereum is upgraded from 1.0 to 2.0, and Ethereum is divided into an execution layer and a consensus layer. The newly introduced Blobs will also be hung on the consensus layer of the beacon chain.

Perhaps in the future, during the next upgrade of Ethereum, with the advancement of technology and the increase in data availability, Ethereum will be able to improve the performance of the execution layer and the consensus layer. On the basis of this, a DA layer is specially loaded; or, in the next upgrade, a proprietary data storage chain similar to a sharding chain may appear. Of course, these are just conjectures. As for which direction it will develop, we will look forward to it together.

---------------------

About us

This speech was written by the Research team of HTX Ventures. HTX Ventures is the global investment arm of Huobi HTX, integrating investment, incubation and research to identify the best and most promising teams around the world. As a pioneer in the blockchain industry for ten years, HTX Ventures promotes the development of cutting-edge technologies and emerging business models in the industry, providing comprehensive support for cooperative projects, including financing, resources and strategic consulting, to establish a long-term blockchain ecosystem. Currently, HTX Ventures has supported more than 200 projects across multiple blockchain tracks, and some of these high-quality projects have been listed on Huobi. At the same time, HTX Ventures is one of the most active fund-of-funds (FOF) investors, working with the world's top blockchain funds such as IVC, Shima, and Animoca to build a blockchain ecosystem.

References

1.  Long push: jianshu’s analysis on Da https://twitter.com/jianshubiji

2. Long push: back hurdle A rising new narrative in the Kunshan upgrade era - DA War https://twitter.com/0xNing0x/status/1744350282509631590?s=20

3. The Ethereum Rollups (STARKNET) war is over, a new narrative DA: https://news.marsbit.co/20240220083034666673.html

4. First Class Warehouse Research Report: Modular Blockchain Celestia https://medium.com/@first.vip/%E5%A4%B4% E7%AD%89%E4%BB%93%E7%A0%94%E6%8A%A5-%E6%A8%A1%E5%9D%97%E5%8C%96%E5%8C%BA%E5 %9D%97%E9%93%BEcelestia-a2532da9d2be

5. EigenLayer in-depth research report: Ethereum’s middleware Agreement, leading the re-pledge narrativehttps://www.aicoin.com/article/360322.html

6 . Long tweet: DA solution https://twitter.com/i/web/status/1700728653750087945

7. What is EigenDA, the flagship product of EigenLayer, a re-pledge protocol? https://www.blocktempo.com/what-is-eigenda-that-eigenlayer-will-launch/

8. A glance at five factors of modular DA design: Celestia, Avail and EigenDA Who is the winner? https://foresightnews.pro/article/detail/52019

Disclaimer

1. HTX Ventures does not have any relationship with the projects or other third parties involved in this report that would affect the objectivity, independence, and fairness of the report.

2. The information and data cited in this report are all from compliance channels. The sources of the information and data are considered reliable by HTX Ventures and are true to them. Necessary checks have been made for authenticity, accuracy and completeness, but HTX Ventures does not make any guarantee as to its authenticity, accuracy or completeness.

3. The content of the report is for reference only, and the conclusions and opinions in the report do not constitute any investment advice on related digital assets. HTX Ventures does not assume any responsibility for losses caused by the use of the contents of this report, unless expressly provided by laws and regulations. Readers should not make investment decisions solely based on this report, nor should they lose the ability to make independent judgments based on this report.

4. The information, opinions and speculations contained in this report only reflect the judgment of the researchers on the day when this report is finalized. In the future, based on industry changes and updates of data information, There is the possibility of updating opinions and judgments.

5. The copyright of this report belongs only to HTX Ventures. If you need to quote the content of this report, please indicate the source. If you need to quote a large amount, please inform us in advance and use it within the permitted scope. Under no circumstances may any quotation, deletion or modification be made to this report that goes against the original intention.