v0.8.1

Summary

This is a patch release to fix a bug related to gRPC dial option (#711) in dispersal client library in v0.8.0, so users can retrieve the 4MiB blobs correctly.

Users/rollups: if you are using dispersal client to retrieve 4MiB blobs, please upgrade to v0.8.1. No impact otherwise.

Operators: no impact, and no action is needed.

Full Changelog: v0.8.0...v0.8.1

v0.8.0

Summary

This release is optional for now, but strongly recommended.

In particular, it’ll become required for operators to upgrade to v0.8.0 or higher before 2024/09/20 on Mainnet.

Two changes are needed to complete the upgrade:

• Upgrade image to v0.8.0 or higher
• Set the node environment variable “NODE_SRS_LOAD=8388608”

Both are included in operator setup repo: https://github.com/Layr-Labs/eigenda-operator-setup/releases/tag/v0.8.0

Notable features

• EigenDA users - Max Blob size raised from 2MiB to 4MiB
  • Not enabled for custom quorums with less than 8 operators
• Operators - Gnark chunk encoding to reduce data size by about 30%
  • Not enabled for now
  • Scheduled to enable after operators upgrade to v0.8.0 or later version
• NodeInfo API at Node: Operators will start making the Node software version information available to the Disperser
• Churn approval expiration reduced from 60 mins to 15 mins: Churner can approve a churn request every 15 mins now

Other notable changes

• Fixed a bug in blob state transition in Batcher (reset DISPERSING blobs back to PROCESSING at Batcher start)
• Improved performance logging

Extra details for blob size limit and chunk encoding

Increasing max blob size is a notable feature enabled in the release. This feature allows EigenDA users to send blobs of 4MiB, previously 2MiB. For operators, they will need to update the NODE_SRS_LOAD to support larger blobs. The resource utilization on the node is expected to be almost identical under the same level of traffic.

This release also contains chunk encoding changes that are not enabled for now, but are scheduled to enable at some future release, to give operators ample time to upgrade. When enabled after the release post 2024/09/20, the nodes with versions before v0.8.0 will not be able to parse the chunks from Disperser.

We will make an announcement in that future release when the changes are enabled.

What's Changed

• Simplify the bytes decoding of chunks by @jianoaix in #607
• Pull ejector out of dataapi by @ian-shim in #608
• Update eigenlayer-middleware by @0x0aa0 in #611
• Record args in mock transactor by @ian-shim in #616
• [1/N] Chunk encoding optimization: Support encoding format with gnark by @jianoaix in #617
• fix dataapi docker build by @ian-shim in #618
• Use variable rate interval for blob thoughput metrics based on time interval by @pschork in #621
• Add configuration setting for high-level EigenDA client for returning as soon as the blob has confirmed by @teddyknox in #620
• Log the batch validation latency breakdown by @jianoaix in #619
• [TxnManager] Make pk wallet compatible with broadcast check by @ian-shim in #626
• Log request serialization time by @jianoaix in #625
• [inabox] Set BATCHER_FINALIZATION_BLOCK_DELAY to 0 by @ian-shim in #631
• [4/N] Chunk encoding optimization: Add bundle encoding by @jianoaix in #632
• Adds NodeInfo endpoint to retriever and dispersal services for observability by @pschork in #630
• Refactor cpu prover by @bxue-l2 in #629
• [2/N] Chunk encoding optimization: bench mark the gnark based encoding by @jianoaix in #624
• [3/N] Chunk encoding optimization: Add support of new encoding format… by @jianoaix in #628
• [minibatch] Add feature flag on batcher by @ian-shim in #634
• Ignore project files for the Jetbrains GoLand IDE. by @cody-littley in #639
• Fix fft_fr.IsPowerOfTwo(). (#83) by @cody-littley in #638
• Remove ejector from dataapi by @ian-shim in #633
• [5/N] Chunk encoding optimization: Add support of new encoding at Node by @jianoaix in #636
• Refactor dispatcher timeout by @ian-shim in #641
• Collect Node Info during reachability check if port scan was successful by @pschork in #640
• Add a metric for DB writing throughput by @jianoaix in #644
• Inabox readme improvements by @cody-littley in #643
• Revert "Refactor dispatcher timeout" by @ian-shim in #645
• Minibatch store interface by @ian-shim in #635
• Update node interface to enable minibatching by @ian-shim in #646
• [minibatch] Create minibatch from encoding streamer by @ian-shim in #637
• Dispatcher method to disperse blobs to node via StoreBlobs endpoint by @ian-shim in #648
• Generate inclusion proof with blob index instead of hash by @ian-shim in #649
• Clean up EnableDualQuorums flag by @ian-shim in #656
• Add Minibatcher by @ian-shim in #655
• Fix typos by @nnsW3 in #659
• Code hygiene: return error instead of bool by @jianoaix in #660
• Doc hygiene: api docs, repo readme, codebase structure by @jianoaix in #662
• [dispatcher] Add method to send AttestBatch request to nodes by @ian-shim in #664
• [minibatcher] Fullbatch state management by @ian-shim in #665
• Minibatch dynamodb client by @pschork in #652
• [6/N] Chunk encoding optimization: Disperser/Retriever support for new chunk encoding by @jianoaix in #650
• Average blob throughput over 9m window by @pschork in #667
• Enable shadow PutItem blob metadata requests to separate dynamo table for minibatch preprod testing by @pschork in #669
• Added metrics utility for generator. by @cody-littley in #673
• Added test utilities. by @cody-littley in #672
• add configurable max blob size by @bxue-l2 in #675
• Log blob size on request by @ian-shim in #678
• [batcher] Recover blob state by @ian-shim in #657
• add grpc recv buffer by @bxue-l2 in #679
• Configuration for new generator functionality. by @cody-littley in #674
• Downgrade encoder error logs by @ian-shim in #682
• [7/N] Chunk encoding optimization: Node support for mixed encoding by @jianoaix in #654
• Added blob table data structure. by @cody-littley in #677
• Keep batch state in Minibatcher by @ian-shim in #668
• Code hygiene: more properly name the chunk encoding format field by @jianoaix in #661
• Fix the operator stake share basis point calculation by @jianoaix in #658
• [node] Add StoreBlobs endpoint for minibatch streaming by @ian-shim in #671
• Add BatchConfirmer by @ian-shim in #670
• Add batch read operation in dynamodb client by @ian-shim in #684
• Adjust the churn approval interval by @jianoaix in #651
• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696

New Contributors

• @nnsW3 made their first contribution in #659

Full Changelog: v0.7.4...v0.8.0

[Holesky] v0.8.0-rc.1

Summary

This pre-release is optional for now, but strongly recommended.

In particular, it’ll become required for operators to upgrade to v0.8.0 or higher before 2024/09/15 on testnet.

Two changes are needed to complete the upgrade:

• Upgrade image to v0.8.0 or higher
• Set the node environment variable “NODE_SRS_LOAD=8388608”

Both are included in release https://github.com/Layr-Labs/eigenda-operator-setup/releases/tag/v0.8.0-rc.1

Notable features

• EigenDA users - Max Blob size raised from 2MiB to 4MiB
  • Not enabled for custom quorums with less than 8 operators
• Operators - Gnark chunk encoding to reduce data size by about 30%
  • Not enabled for now
  • Scheduled to enable after operators upgrade to v0.8.0 or later version
• NodeInfo API at Node: Operators will start making the Node software version information available to the Disperser
• Churn approval expiration reduced from 60 mins to 15 mins: Churner can approve a churn request every 15 mins now

Other notable changes

• Fixed a bug in blob state transition in Batcher (reset DISPERSING blobs back to PROCESSING at Batcher start)
• Improved performance logging

Extra details for blob size limit and chunk encoding

Increasing max blob size is a notable feature enabled in the release. This feature allows EigenDA users to send blobs of 4MiB, previously 2MiB. For operators, they will need to update the NODE_SRS_LOAD to support larger blobs. The resource utilization on the node is expected to be almost identical under the same level of traffic.

This pre-release also contains chunk encoding changes that are not enabled for now, but are scheduled to enable at some future release, to give operators ample time to upgrade. When enabled after the release post 2024/09/15, the nodes with versions before v0.8.0 will not be able to parse the chunks from Disperser.

We will make an announcement in that future release when the changes are enabled.

What's Changed

• Simplify the bytes decoding of chunks by @jianoaix in #607
• Pull ejector out of dataapi by @ian-shim in #608
• Update eigenlayer-middleware by @0x0aa0 in #611
• Record args in mock transactor by @ian-shim in #616
• [1/N] Chunk encoding optimization: Support encoding format with gnark by @jianoaix in #617
• fix dataapi docker build by @ian-shim in #618
• Use variable rate interval for blob thoughput metrics based on time interval by @pschork in #621
• Add configuration setting for high-level EigenDA client for returning as soon as the blob has confirmed by @teddyknox in #620
• Log the batch validation latency breakdown by @jianoaix in #619
• [TxnManager] Make pk wallet compatible with broadcast check by @ian-shim in #626
• Log request serialization time by @jianoaix in #625
• [inabox] Set BATCHER_FINALIZATION_BLOCK_DELAY to 0 by @ian-shim in #631
• [4/N] Chunk encoding optimization: Add bundle encoding by @jianoaix in #632
• Adds NodeInfo endpoint to retriever and dispersal services for observability by @pschork in #630
• Refactor cpu prover by @bxue-l2 in #629
• [2/N] Chunk encoding optimization: bench mark the gnark based encoding by @jianoaix in #624
• [3/N] Chunk encoding optimization: Add support of new encoding format… by @jianoaix in #628
• [minibatch] Add feature flag on batcher by @ian-shim in #634
• Ignore project files for the Jetbrains GoLand IDE. by @cody-littley in #639
• Fix fft_fr.IsPowerOfTwo(). (#83) by @cody-littley in #638
• Remove ejector from dataapi by @ian-shim in #633
• [5/N] Chunk encoding optimization: Add support of new encoding at Node by @jianoaix in #636
• Refactor dispatcher timeout by @ian-shim in #641
• Collect Node Info during reachability check if port scan was successful by @pschork in #640
• Add a metric for DB writing throughput by @jianoaix in #644
• Inabox readme improvements by @cody-littley in #643
• Revert "Refactor dispatcher timeout" by @ian-shim in #645
• Minibatch store interface by @ian-shim in #635
• Update node interface to enable minibatching by @ian-shim in #646
• [minibatch] Create minibatch from encoding streamer by @ian-shim in #637
• Dispatcher method to disperse blobs to node via StoreBlobs endpoint by @ian-shim in #648
• Generate inclusion proof with blob index instead of hash by @ian-shim in #649
• Clean up EnableDualQuorums flag by @ian-shim in #656
• Add Minibatcher by @ian-shim in #655
• Fix typos by @nnsW3 in #659
• Code hygiene: return error instead of bool by @jianoaix in #660
• Doc hygiene: api docs, repo readme, codebase structure by @jianoaix in #662
• [dispatcher] Add method to send AttestBatch request to nodes by @ian-shim in #664
• [minibatcher] Fullbatch state management by @ian-shim in #665
• Minibatch dynamodb client by @pschork in #652
• [6/N] Chunk encoding optimization: Disperser/Retriever support for new chunk encoding by @jianoaix in #650
• Average blob throughput over 9m window by @pschork in #667
• Enable shadow PutItem blob metadata requests to separate dynamo table for minibatch preprod testing by @pschork in #669
• Added metrics utility for generator. by @cody-littley in #673
• Added test utilities. by @cody-littley in #672
• add configurable max blob size by @bxue-l2 in #675
• Log blob size on request by @ian-shim in #678
• [batcher] Recover blob state by @ian-shim in #657
• add grpc recv buffer by @bxue-l2 in #679
• Configuration for new generator functionality. by @cody-littley in #674
• Downgrade encoder error logs by @ian-shim in #682
• [7/N] Chunk encoding optimization: Node support for mixed encoding by @jianoaix in #654
• Added blob table data structure. by @cody-littley in #677
• Keep batch state in Minibatcher by @ian-shim in #668
• Code hygiene: more properly name the chunk encoding format field by @jianoaix in #661
• Fix the operator stake share basis point calculation by @jianoaix in #658
• [node] Add StoreBlobs endpoint for minibatch streaming by @ian-shim in #671
• Add BatchConfirmer by @ian-shim in #670
• Add batch read operation in dynamodb client by @ian-shim in #684
• Adjust the churn approval interval by @jianoaix in #651
• Downlevel more encoding streamer error log by @ian-shim in #689
• Check if blobs are expired before returning by @mooselumph in #695
• Add log for chunk encoding format by @jianoaix in #696

New Contributors

• @cody-littley made their first contribution in #639
• @nnsW3 made their first contribution in #659

Full Changelog: v0.7.4...v0.8.0-rc.1

hot fix disperser grpc recv buffer

v0.7.4-4

fix

v0.7.4

Summary

This release is optional and contains performance improvements as well as bug fixes.

Notable features that directly affect users are:

• Operators - ARM support: We now publish ARM docker images for the operator node.

• Operators - Improvements to storage: Added performance improvements on the node DB side resulting in 4x performance increase.

Other changes:

• Traffic generator now supports authenticated dispersals
• Fixes to the retriever service
• Improved logging on the operator node
• Preparing rewards integrations at the smart contract layer

What's Changed

• Add decode flag for GetBlob() by reworking codec layer by @teddyknox in #592
• Add nonsigning rate threshold control to ejector by @jianoaix in #601
• Few fixes to retriever server by @ian-shim in #599
• Auth traffic gen by @mooselumph in #566
• Adds support for multi-platform linux/amd64 & linux/arm64 docker images by @pschork in #603
• Add log for request data size breakdown by @jianoaix in #604
• Use accountKey as AccountID by @dmanc in #602
• Fix GitVersion regex to properly identify versioned tags when building in detached HEAD mode by @pschork in #605
• [finalizer] Update confirmation block number in finalizer by @ian-shim in #600
• Optimize the bytes encoding at node by @jianoaix in #606
• Rewards Integration by @0x0aa0 in #588

Full Changelog: v0.7.3...v0.7.4

[Holesky] v0.7.4-rc.1

Pre-release intended for Holesky testnet only

Summary

This pre-release is optional and contains performance improvements as well as bug fixes.

Notable features that directly affect users are:

• Operators - ARM support: We now publish ARM docker images for the operator node.

• Operators - Improvements to storage: Added performance improvements on the node DB side resulting in 4x performance increase.

Other changes:

• Traffic generator now supports authenticated dispersals
• Fixes to the retriever service
• Improved logging on the operator node
• Preparing rewards integrations at the smart contract layer

What's Changed

• Add decode flag for GetBlob() by reworking codec layer by @teddyknox in #592
• Add nonsigning rate threshold control to ejector by @jianoaix in #601
• Few fixes to retriever server by @ian-shim in #599
• Auth traffic gen by @mooselumph in #566
• Adds support for multi-platform linux/amd64 & linux/arm64 docker images by @pschork in #603
• Add log for request data size breakdown by @jianoaix in #604
• Use accountKey as AccountID by @dmanc in #602
• Fix GitVersion regex to properly identify versioned tags when building in detached HEAD mode by @pschork in #605
• [finalizer] Update confirmation block number in finalizer by @ian-shim in #600
• Optimize the bytes encoding at node by @jianoaix in #606
• Rewards Integration by @0x0aa0 in #588

Full Changelog: v0.7.3...v0.7.4-rc.1

v0.7.3

Summary

This is a patch release to fix a bug in v0.7.2, so the performance optimization (for request data deserialization) at Node will actually work.

Operators are highly recommended to upgrade to this version to get the benefits of this optimization, so you can keep up the signing rate.

Full Changelog: v0.7.2...v0.7.3

v0.7.2

Summary

This pre-release is optional and contains performance improvements as well as bug fixes.

Notable features that directly affect users are:

• Operators - 5x Blob Storage Performance Optimization: Parallel batch header deserialization improves storage performance by ~5x. This will further improve signing performance for nodes that often fail to return a signature within the timeout window.

• Operators - Additional e2e latency and storage metrics: Metrics for e2e attestation and chunk deserialization latencies added.

Other changes:

• Log operator state hash on batch validation failure
• Adds bucket level metrics to the rate limiter library
• Reachability check 404 error handling fixed
• Custom quorum support added to traffic generator

What's Changed

• Make ejector return the transaction hash of the ejection by @jianoaix in #565
• Support custom quorums in traffic generator by @ian-shim in #567
• fix some typos. by @hattizai in #557
• Log e2e attestation latency by @jianoaix in #569
• fix the misspelling words in the code. by @cocoyeal in #563
• Log reloaded allowlist by @ian-shim in #570
• Add store chunk request deserialization latency to metric by @jianoaix in #572
• Improved reachability check 404 handling by @pschork in #571
• Optimize the batch deserialization perf by @jianoaix in #573
• Add operatorId in attestation latency by @jianoaix in #574
• [node] Use URL builder to get reachability URL by @ian-shim in #575
• Add EigenDA client using handshake based authentication by @teddyknox in #551

New Contributors

• @hattizai made their first contribution in #557
• @cocoyeal made their first contribution in #563

Full Changelog: v0.7.1...v0.7.2

[Holesky] v0.7.2-rc.2

Pre-release intended for Holesky testnet only

Summary

This pre-release is optional and contains performance improvements as well as bug fixes.

Notable features that directly affect users are:

• Operators - 5x Blob Storage Performance Optimization: Parallel batch header deserialization improves storage performance by ~5x. This will further improve signing performance for nodes that often fail to return a signature within the timeout window.

• Operators - Additional e2e latency and storage metrics: Metrics for e2e attestation and chunk deserialization latencies added.

Other changes:

• Log operator state hash on batch validation failure
• Adds bucket level metrics to the rate limiter library
• Reachability check 404 error handling fixed
• Custom quorum support added to traffic generator

Full Changelog: v0.7.2-rc.1...v0.7.2-rc.2

[Holesky] v0.7.2-rc.1

Pre-release intended for Holesky testnet only

Summary

This pre-release is optional and contains performance improvements as well as bug fixes.

Notable features that directly affect users are:

• Operators - 5x Blob Storage Performance Optimization: Parallel batch header deserialization improves storage performance by ~5x. This will further improve signing performance for nodes that often fail to return a signature within the timeout window.

• Operators - Additional e2e latency and storage metrics: Metrics for e2e attestation and chunk deserialization latencies added.

Other changes:

• Reachability check 404 error handling fixed
• Custom quorum support added to traffic generator

What's Changed

• Make ejector return the transaction hash of the ejection by @jianoaix in #565
• Support custom quorums in traffic generator by @ian-shim in #567
• fix some typos. by @hattizai in #557
• Log e2e attestation latency by @jianoaix in #569
• fix the misspelling words in the code. by @cocoyeal in #563
• Log reloaded allowlist by @ian-shim in #570
• Add store chunk request deserialization latency to metric by @jianoaix in #572
• Improved reachability check 404 handling by @pschork in #571
• Optimize the batch deserialization perf by @jianoaix in #573
• Add operatorId in attestation latency by @jianoaix in #574
• [node] Use URL builder to get reachability URL by @ian-shim in #575
• Add EigenDA client using handshake based authentication by @teddyknox in #551

New Contributors

• @hattizai made their first contribution in #557
• @cocoyeal made their first contribution in #563

Full Changelog: v0.7.1...v0.7.2-rc.1