Spec compliant merge shares #261

evan-forbes · 2021-03-31T04:10:01Z

Description

This is the first draft for spec compliant merging of shares.

TODO:

testing all edge cases
integration test
some fuzzing

Closes: #234

types/shares_test.go

liamsi

Really excited about this!

types/block.go

types/shares.go

types/shares_test.go

codecov-io · 2021-04-01T04:20:57Z

Codecov Report

Merging #261 (1cb4030) into master (93c88a1) will decrease coverage by 0.11%.
The diff coverage is 82.77%.

@@            Coverage Diff             @@
##           master     #261      +/-   ##
==========================================
- Coverage   62.27%   62.16%   -0.12%     
==========================================
  Files         260      262       +2     
  Lines       23218    23362     +144     
==========================================
+ Hits        14460    14523      +63     
- Misses       7265     7317      +52     
- Partials     1493     1522      +29

Impacted Files	Coverage Δ
types/shares.go	`100.00% <ø> (+2.19%)`	⬆️
types/block.go	`77.40% <40.00%> (+0.78%)`	⬆️
types/share_merging.go	`76.47% <76.47%> (ø)`
types/share_splitting.go	`97.43% <97.43%> (ø)`
types/tx.go	`85.18% <100.00%> (ø)`
privval/signer_listener_endpoint.go	`82.35% <0.00%> (-7.06%)`	⬇️
privval/socket_listeners.go	`78.72% <0.00%> (-4.26%)`	⬇️
privval/secret_connection.go	`72.68% <0.00%> (-3.61%)`	⬇️
consensus/reactor.go	`73.58% <0.00%> (-3.53%)`	⬇️
privval/signer_endpoint.go	`75.75% <0.00%> (-3.04%)`	⬇️
... and 12 more

types/shares_test.go

evan-forbes · 2021-04-01T04:30:14Z

While most of the testing is finished, there still needs to be an integration test. I'm going to go ahead and mark this ready for review to start getting feedback in the mean time.

types/shares.go

liamsi

This looks amazing!

types/shares.go

types/shares_test.go

liamsi · 2021-04-01T12:29:41Z

@adlerjohn @evan-forbes now that this is (almost) ready, what do you two think about celestiaorg/celestia-specs#152 ?

Co-authored-by: Ismail Khoffi <[email protected]>

types/shares.go

adlerjohn · 2021-04-01T18:27:48Z

types/shares.go

+			// every other namespaceID should be a message
+			default:
+				msgShares = append(msgShares, share)
+			}


This logic doesn't seem to conform to specs. Potential namespace IDs: https://github.com/lazyledger/lazyledger-specs/blob/de5f4f74f56922e9fa735ef79d9e6e6492a2bad1/specs/consensus.md#reserved-namespace-ids. Namespaces up to and including NAMESPACE_ID_MAX_RESERVED = 255 are reserved, not messages: https://github.com/lazyledger/lazyledger-specs/blob/de5f4f74f56922e9fa735ef79d9e6e6492a2bad1/specs/consensus.md#constants.

There should also be a check somewhere that a reserved namespace ID that's up to 255 but not specifically reserved is invalid.

Thanks for pointing this out! added issue #67 in the lazyledger-app to add a check in MsgWirePayForMessage's ValidateBasic method, which get's called with every call to CheckTx.

Added a case for this along with a namespace var in consts.go to represent the max namespaceID https://github.com/lazyledger/lazyledger-core/blob/d118716769036925491d0412772693a55546ca2c/types/shares.go#L227

Should I also add the const NamespaceIDMaxReserved and use that to construct MaxReservedNamespaceID?

Should I also add the const NamespaceIDMaxReserved and use that to construct MaxReservedNamespaceID?

Nah, the setup you have now should be sufficient I think.

One additional check that needs to be somewhere (probably in a future PR so as not to overload this one) is that padding shares between a transaction and a message must have namespace ID TAIL_TRANSACTION_PADDING_NAMESPACE_ID (all the way at the bottom of https://github.com/lazyledger/lazyledger-specs/blob/de5f4f74f56922e9fa735ef79d9e6e6492a2bad1/specs/data_structures.md#arranging-available-data-into-shares, under the giant figures).

The non-interactive default rules may introduce empty shares that do not belong to any message (in the example above, the top-right share is empty). These are zeroes with namespace ID equal to the either TAIL_TRANSACTION_PADDING_NAMESPACE_ID if between a request with a reserved namespace ID and a message, or the namespace ID of the previous message if succeeded by a message. See the rationale doc for more info.

types/shares.go

Co-authored-by: John Adler <[email protected]>

evan-forbes · 2021-04-04T15:50:17Z

There were a few minor sneaky bugs. The first being that the implementation wasn't planning for the length delimiter occasionally getting split up between shares. ~~The second was that the start index (SHARE_RESERVED_BYTE) was wrong for the first share. To my understanding, it should always be zero.~~ (Nope. It was perfect the way it was.) Lastly, the evidence needed to be marshalled using *tmproto.Evidence instead of *tmproto.DuplicateVoteEvidence.

I also added fuzzing.

liamsi

This is so dope! Left some minor comments and suggestions.

types/block.go

types/consts.go

types/shares.go

liamsi · 2021-04-05T08:51:55Z

types/shares.go

+		return ss.peel(share, true)
+	}
+	return errors.New("failure to parse block data: transaction length exceeded data length")
+}


IMO, this can serve as a kind of blueprint for the pseudo-code that could end up in the spec.

types/shares.go

Co-authored-by: Ismail Khoffi <[email protected]>

…files

liamsi

❤️ This is awesome! 🚀

* start spec compliant share merging * refactor and finish unit testing * whoops * linter gods * fix initial changes and use constants * use constant * more polish * docs fix Co-authored-by: Ismail Khoffi <[email protected]> * review feedback: docs and out of range panic protection * review feedback: add panic protection from empty input * use constant instead of recalculating `ShareSize` Co-authored-by: John Adler <[email protected]> * don't redeclare existing var Co-authored-by: John Adler <[email protected]> * be more explicit with returned nil Co-authored-by: John Adler <[email protected]> * use constant instead of recalculating `ShareSize` Co-authored-by: John Adler <[email protected]> * review feedback: use consistent capitalization * stop accepting reserved namespaces as normal messages * use a descriptive var name for message length * linter and comparison fix * reorg tests, add test for parse delimiter, DataFromBlock and fix evidence marshal bug * catch error for linter * update test MakeShares to include length delimiters for the SHARE_RESERVED_BYTE * minor iteration change * refactor share splitting to fix bug * fix all bugs with third and final refactor * fix conflict * revert unnecessary changes * review feedback: better docs Co-authored-by: Ismail Khoffi <[email protected]> * reivew feedback: add comment for safeLen * review feedback: remove unnecessay comments * review feedback: split up share merging and splitting into their own files * review feedback: more descriptive var names * fix accidental change * add some constant docs * spelling error Co-authored-by: Ismail Khoffi <[email protected]> Co-authored-by: John Adler <[email protected]>

* start spec compliant share merging * refactor and finish unit testing * whoops * linter gods * fix initial changes and use constants * use constant * more polish * docs fix* review feedback: docs and out of range panic protection * review feedback: add panic protection from empty input * use constant instead of recalculating `ShareSize`* don't redeclare existing var* be more explicit with returned nil* use constant instead of recalculating `ShareSize`* review feedback: use consistent capitalization * stop accepting reserved namespaces as normal messages * use a descriptive var name for message length * linter and comparison fix * reorg tests, add test for parse delimiter, DataFromBlock and fix evidence marshal bug * catch error for linter * update test MakeShares to include length delimiters for the SHARE_RESERVED_BYTE * minor iteration change * refactor share splitting to fix bug * fix all bugs with third and final refactor * fix conflict * revert unnecessary changes * review feedback: better docs* reivew feedback: add comment for safeLen * review feedback: remove unnecessay comments * review feedback: split up share merging and splitting into their own files * review feedback: more descriptive var names * fix accidental change * add some constant docs * spelling error Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: John Adler <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]>

) * Basic DA functionality (#83) * move Messages field to the end of Block.Data * Add some constants for share computation and the NMT: - also a bunch of todos regarding shares computation * First (compiling) stab on creating shares * Test with Evidence and fix bug discovered by test * remove resolved todos * introduce split method * Introduce LenDelimitedMarshaler interface and some reformatting * Introduce TxLenDelimitedMarshaler * add some test cases * fix some comments * fix some comments & linter * Add reserved namespaces to params * Move ll-specific consts into a separate file (consts.go) * Add MarshalDelimited to HexBytes * Add tail-padding shares * Add ComputeShares method on Data to compute all shares * Fix compute the next square num and not the next power of two * lints * Unexport MakeShares function: - it's likely to change and it doesn't have to be part of the public API * lints 2 * First stab on computing row/column roots * fix rebase glitches: - move DA related constants out of params.go * refactor MakeBlock to take in interm. state roots and messages * refactor state.MakeBlock too * Add todos LenDelimitedMarshaler and extract appendShares logic * Simplify shares computation: remove LenDelimitedMarshaler abstraction * actually use DA header to compute the DataRoot everywhere (will lead to failing tests for sure) * WIP: Update block related core data structures in protobuf too * WIP: fix zero shares edge-case and get rid of Block.Data.hash (use dataAvailabilityHeader.Hash() instead) * Fixed tests, only 3 failing tests to go: TestReapMaxBytesMaxGas, TestTxFilter, TestMempoolFilters * Fix TestTxFilter: - the size of the wrapping Data{} proto message increased a few bytes * Fix Message proto and `DataFromProto` * Fix last 2 remaining tests related to the increased block/block.Data size * Use infectious lib instead of leopard * proto-lint: snake_case * some lints and minor changes * linter * panic if pushing to tree fails, extend Data.ToProto() * revert renaming in comment * add todo about refactoring as soon as the rsmt2d allows the user to choose the merkle tree * clean up some unused test helper functions * linter * still debugging the exact right number of bytes for max data... * Implement spec-compliant share splitting (#246) * Export block data compute shares. * Refactor to use ShareSize constant directly. * Change message splitting to prefix namespace ID. * Implement chunking for contiguous. * Add termination condition. * Rename append contiguous to split contiguous. * Update test for small tx. * Add test for two contiguous. * Make tx and msg adjusted share sizes exported constants. * Panic on hopefully-unreachable condition instead of silently skipping. * Update hardcoded response for block format. Co-authored-by: Ismail Khoffi <[email protected]> * fix overwrite bug (#251) * fix overwrite bug and stop splitting shares of size MsgShareSize * remove ineffectual code * review feedback: better docs Co-authored-by: Ismail Khoffi <[email protected]> * remove uneeded copy and only fix the source of the bug Co-authored-by: Ismail Khoffi <[email protected]> * fix overwrite bug while also being consistent with using NamespacedShares * update to the latest rsmt2d for the nmt wrapper Co-authored-by: Ismail Khoffi <[email protected]> * Spec compliant merge shares (#261) * start spec compliant share merging * refactor and finish unit testing * whoops * linter gods * fix initial changes and use constants * use constant * more polish * docs fix* review feedback: docs and out of range panic protection * review feedback: add panic protection from empty input * use constant instead of recalculating `ShareSize`* don't redeclare existing var* be more explicit with returned nil* use constant instead of recalculating `ShareSize`* review feedback: use consistent capitalization * stop accepting reserved namespaces as normal messages * use a descriptive var name for message length * linter and comparison fix * reorg tests, add test for parse delimiter, DataFromBlock and fix evidence marshal bug * catch error for linter * update test MakeShares to include length delimiters for the SHARE_RESERVED_BYTE * minor iteration change * refactor share splitting to fix bug * fix all bugs with third and final refactor * fix conflict * revert unnecessary changes * review feedback: better docs* reivew feedback: add comment for safeLen * review feedback: remove unnecessay comments * review feedback: split up share merging and splitting into their own files * review feedback: more descriptive var names * fix accidental change * add some constant docs * spelling error Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: John Adler <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]> * refactor to better accomodate real world use cases (celestia node) Co-authored-by: rene <[email protected]> * thank you linter Co-authored-by: Ismail Khoffi <[email protected]> Co-authored-by: John Adler <[email protected]> Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: rene <[email protected]>

* start spec compliant share merging * refactor and finish unit testing * whoops * linter gods * fix initial changes and use constants * use constant * more polish * docs fix* review feedback: docs and out of range panic protection * review feedback: add panic protection from empty input * use constant instead of recalculating `ShareSize`* don't redeclare existing var* be more explicit with returned nil* use constant instead of recalculating `ShareSize`* review feedback: use consistent capitalization * stop accepting reserved namespaces as normal messages * use a descriptive var name for message length * linter and comparison fix * reorg tests, add test for parse delimiter, DataFromBlock and fix evidence marshal bug * catch error for linter * update test MakeShares to include length delimiters for the SHARE_RESERVED_BYTE * minor iteration change * refactor share splitting to fix bug * fix all bugs with third and final refactor * fix conflict * revert unnecessary changes * review feedback: better docs* reivew feedback: add comment for safeLen * review feedback: remove unnecessay comments * review feedback: split up share merging and splitting into their own files * review feedback: more descriptive var names * fix accidental change * add some constant docs * spelling error Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: John Adler <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]>

) * Basic DA functionality (#83) * move Messages field to the end of Block.Data * Add some constants for share computation and the NMT: - also a bunch of todos regarding shares computation * First (compiling) stab on creating shares * Test with Evidence and fix bug discovered by test * remove resolved todos * introduce split method * Introduce LenDelimitedMarshaler interface and some reformatting * Introduce TxLenDelimitedMarshaler * add some test cases * fix some comments * fix some comments & linter * Add reserved namespaces to params * Move ll-specific consts into a separate file (consts.go) * Add MarshalDelimited to HexBytes * Add tail-padding shares * Add ComputeShares method on Data to compute all shares * Fix compute the next square num and not the next power of two * lints * Unexport MakeShares function: - it's likely to change and it doesn't have to be part of the public API * lints 2 * First stab on computing row/column roots * fix rebase glitches: - move DA related constants out of params.go * refactor MakeBlock to take in interm. state roots and messages * refactor state.MakeBlock too * Add todos LenDelimitedMarshaler and extract appendShares logic * Simplify shares computation: remove LenDelimitedMarshaler abstraction * actually use DA header to compute the DataRoot everywhere (will lead to failing tests for sure) * WIP: Update block related core data structures in protobuf too * WIP: fix zero shares edge-case and get rid of Block.Data.hash (use dataAvailabilityHeader.Hash() instead) * Fixed tests, only 3 failing tests to go: TestReapMaxBytesMaxGas, TestTxFilter, TestMempoolFilters * Fix TestTxFilter: - the size of the wrapping Data{} proto message increased a few bytes * Fix Message proto and `DataFromProto` * Fix last 2 remaining tests related to the increased block/block.Data size * Use infectious lib instead of leopard * proto-lint: snake_case * some lints and minor changes * linter * panic if pushing to tree fails, extend Data.ToProto() * revert renaming in comment * add todo about refactoring as soon as the rsmt2d allows the user to choose the merkle tree * clean up some unused test helper functions * linter * still debugging the exact right number of bytes for max data... * Implement spec-compliant share splitting (#246) * Export block data compute shares. * Refactor to use ShareSize constant directly. * Change message splitting to prefix namespace ID. * Implement chunking for contiguous. * Add termination condition. * Rename append contiguous to split contiguous. * Update test for small tx. * Add test for two contiguous. * Make tx and msg adjusted share sizes exported constants. * Panic on hopefully-unreachable condition instead of silently skipping. * Update hardcoded response for block format. Co-authored-by: Ismail Khoffi <[email protected]> * fix overwrite bug (#251) * fix overwrite bug and stop splitting shares of size MsgShareSize * remove ineffectual code * review feedback: better docs Co-authored-by: Ismail Khoffi <[email protected]> * remove uneeded copy and only fix the source of the bug Co-authored-by: Ismail Khoffi <[email protected]> * fix overwrite bug while also being consistent with using NamespacedShares * update to the latest rsmt2d for the nmt wrapper Co-authored-by: Ismail Khoffi <[email protected]> * Spec compliant merge shares (#261) * start spec compliant share merging * refactor and finish unit testing * whoops * linter gods * fix initial changes and use constants * use constant * more polish * docs fix* review feedback: docs and out of range panic protection * review feedback: add panic protection from empty input * use constant instead of recalculating `ShareSize`* don't redeclare existing var* be more explicit with returned nil* use constant instead of recalculating `ShareSize`* review feedback: use consistent capitalization * stop accepting reserved namespaces as normal messages * use a descriptive var name for message length * linter and comparison fix * reorg tests, add test for parse delimiter, DataFromBlock and fix evidence marshal bug * catch error for linter * update test MakeShares to include length delimiters for the SHARE_RESERVED_BYTE * minor iteration change * refactor share splitting to fix bug * fix all bugs with third and final refactor * fix conflict * revert unnecessary changes * review feedback: better docs* reivew feedback: add comment for safeLen * review feedback: remove unnecessay comments * review feedback: split up share merging and splitting into their own files * review feedback: more descriptive var names * fix accidental change * add some constant docs * spelling error Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: John Adler <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]> * refactor to better accomodate real world use cases (celestia node) Co-authored-by: rene <[email protected]> * thank you linter Co-authored-by: Ismail Khoffi <[email protected]> Co-authored-by: John Adler <[email protected]> Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: rene <[email protected]>

* start spec compliant share merging * refactor and finish unit testing * whoops * linter gods * fix initial changes and use constants * use constant * more polish * docs fix* review feedback: docs and out of range panic protection * review feedback: add panic protection from empty input * use constant instead of recalculating `ShareSize`* don't redeclare existing var* be more explicit with returned nil* use constant instead of recalculating `ShareSize`* review feedback: use consistent capitalization * stop accepting reserved namespaces as normal messages * use a descriptive var name for message length * linter and comparison fix * reorg tests, add test for parse delimiter, DataFromBlock and fix evidence marshal bug * catch error for linter * update test MakeShares to include length delimiters for the SHARE_RESERVED_BYTE * minor iteration change * refactor share splitting to fix bug * fix all bugs with third and final refactor * fix conflict * revert unnecessary changes * review feedback: better docs* reivew feedback: add comment for safeLen * review feedback: remove unnecessay comments * review feedback: split up share merging and splitting into their own files * review feedback: more descriptive var names * fix accidental change * add some constant docs * spelling error Co-authored-by: Hlib Kanunnikov <[email protected]> Co-authored-by: John Adler <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]>

@shahankhatch

* Initial commit * Add three timeouts and align pseudocode better with existing algorithm * Align protocol with Tendermint code and add find valid value mechanism * Prepare to Nuke Develop (#47) * state -> step * vote -> v * New version of the algorithm and the proof * New version of the algorithm and the proofs * Added algorithm description * Add algorithm description * Add introduction * Add conclusion * Add conclusion file * fix warnings (caption was defined twice) - only the latter is used anyways (centers captions) - this makes it possible to autom. building the paper * Update grammar * s/state_p/step_p * Address Ismail's comments * intro: language fixes * definitions: language fixes * consensus: various fixes * proof: some fixes * try to improve reviewability * \eq -> = * textwrap to 79 * various minor fixes * proof: fix itemization * proof: more minor fixes * proof: timeouts are functions * proof: fixes to lemma6 * Intro changes and improve title page * Add Marko and Ming to acks * add readme * Format algorithm correctly Clarify condition semantic and timeouts Improve descriptions * patform -> platform * Ensure that rules are mutually exclusive - various clarifications and small improvements * Release v0.6 * small nits for smoother readability * This PR is to create signed commits to be able to merge (#50) Signed-off-by: Marko Baricevic <[email protected]> * Add consesnus and blockchain specs, (#52) - Open questions - Do we want to split lite client work from consesnsus - From the blockchain spec, is encoding nessecary in the spec Signed-off-by: Marko Baricevic <[email protected]> * Add ABCI SPEC (#51) - move the abci spec from tendermint to spec repo Signed-off-by: Marko Baricevic <[email protected]> * spec/consensus/signing: add more details about nil and amnesia (#54) - Add more details about nil votes and about amnesia attacks Signed-off-by: Marko Baricevic <[email protected]> * Add Section for P2P (#53) * Add Section for P2P - moved over the section on p2p Signed-off-by: Marko Baricevic <[email protected]> * add some more files Signed-off-by: Marko Baricevic <[email protected]> * Fix model section * Add non-recursive specification of Bisection algorithm - Fix timing issues by introducing Delta parameter * spec: update spec with tendermint updates (#62) * spec: update spec with tendermint updates - this in preperation of deleting the spec folder in docs in tendermint/tendermint Signed-off-by: Marko Baricevic <[email protected]> * spec: added in reactors & p2p Signed-off-by: Marko Baricevic <[email protected]> * spec: update readme in spec to comply with docs site Signed-off-by: Marko Baricevic <[email protected]> * docs: addded more changes from tednermint/tendermint Signed-off-by: Marko Baricevic <[email protected]> * reflect breaking changes made to Commit (#63) * reflect breaking changes made to Commit PR: tendermint/tendermint#4146 Issue: tendermint/tendermint#1648 * types: rename Commit#Precommits to Signatures * update BlockIDFlagAbsent comment * remove iota * Clean up error conditions and simplify pseudocode * Apply suggestions from code review Co-Authored-By: Anca Zamfir <[email protected]> * Add spec doc about unconditional_peer, persistent_peers_max_dial of ADR-050 (#68) * Add spec doc about unconditional_peer_ids, persistent_peers_max_dial_period of ADR-050 * Add indefinitely dialing condition * Add sr25519 amino documentation (#67) * sr25519 amino * Update spec/blockchain/encoding.md Co-Authored-By: Marko <[email protected]> * some suggestions for pseuodocode changes * Improved error handling * Add explanation on difference between trusted models * Address reviewer's comments * Addressing reviewer's comments * Separating algorithm from proofs * Intermediate commit (aligning spec with the code) * Removing Store from API and providing end-to-end timing guarantees * Address reviewer comment's. Intermediate commit * light client dir and readmes * titles * add redirects * add diagram * detection TODO * fix image * update readme * Aligh the correctness arguments with the pseudocode changes * lite->light * Fix link in readme ./light -> ./light-client * p2p: Merlin based malleability fixes (#72) * Update the secret connection spec with the use of merlin to eliminte handshake malleability * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]> * docs: update specs to remove cmn (#77) - cmn was remvoed in favor of sub pkgs. cmn.kvpair is now kv.pair Signed-off-by: Marko Baricevic <[email protected]> * evidence: Add time to evidence params (#69) * evidence: Add time to evidence params - this pr is grouped together with tendermint/tendermint#4254, once that PR is merged then this one can be as well. Signed-off-by: Marko Baricevic <[email protected]> * remove note Signed-off-by: Marko Baricevic <[email protected]> * Apply suggestions from code review Co-Authored-By: Anton Kaliaev <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]> * update link to the pex reactor * add markdown link checker * changed tab spacing * removed folder-path flag * first attempt at fixing all links * second attempt at fixing all links * codeowners: add code owners (#82) * codeowners: add code owners - added some codeowners please comment if youd like to be added as well. Signed-off-by: Marko Baricevic <[email protected]> * remove comment of repo maintainers * remove .idea dir (#83) Signed-off-by: Marko Baricevic <[email protected]> * RFC-001: configurable block retention (#84) * Added RFC for truncated block history coordination * Clarified minimum block retention * Added hard checks on block retention and snapshot interval, and made some minor tweaks * Genesis parameters are immutable * Use local config for snapshot interval * Reordered parameter descriptions * Clarified local config option for snapshot-interval * rewrite for ABCI commit response * Renamed RFC * add block retention diagram * Removed retain_blocks table * fix image numbers * resolved open questions * image quality * accept RFC-001 (#86) * abci: add basic description of ABCI Commit.ResponseHeight (#85) Documentation for block pruning, once it's merged: tendermint/tendermint#4588. Minimum documentation, for now - we probably shouldn't encourage using this feature too much until we release state sync. * abci: add MaxAgeNumBlocks/MaxAgeDuration to EvidenceParams (#87) * abci: update MaxAgeNumBlocks & MaxAgeDuration docs (#88) * document state sync ABCI interface and P2P protocol (#90) The corresponding Tendermint PRs are tendermint/tendermint#4704 and tendermint/tendermint#4705. * Revert "document state sync ABCI interface and P2P protocol (#90)" (#92) This reverts commit 9842b4b. * blockchain: change validator set sorting method (#91) * abci: specify sorting of RequestInitChain.Validators * blockchain: change validator sorting method Refs tendermint/tendermint#2478 * reactors/pex: specify hash function (#94) https://github.com/tendermint/tendermint/pull/4810/files * document state sync ABCI interface and P2P protocol (#93) * Revert "Revert "document state sync ABCI interface and P2P protocol (#90)" (#92)" This reverts commit 90797ce. * update with new enum case * fix links Co-authored-by: Erik Grinaker <[email protected]> * Update evidence params with MaxNum (#95) evidence params now includes maxNum which is the maximum number of evidence that can be committed on a single block * reactors/pex: masked IP is used as group key (#96) * spec: add ProofTrialPeriod to EvidenceParam (#99) * spec: modify Header.LastResultsHash (#97) Refs: tendermint/tendermint#1007 PR: tendermint/tendermint#4845 * spec: link to abci server implementations (#100) * spec: update evidence in blockchain.md (#108) now evidence reflects the actual evidence present in the tendermint repo * abci: add AppVersion to ConsensusParams (#106) * abci: tweak node sync estimate (#115) * spec/abci: expand on Validator#Address (#118) Refs tendermint/tendermint#3732 * blockchain: rename to core (#123) * blockchain: remove duplicate evidence sections (#124) * spec/consensus: canonical vs subjective commit Refs tendermint/tendermint#2769 * Apply suggestions from code review Co-authored-by: Igor Konnov <[email protected]> * update spec with the removal of phantom validator evidence (#126) * bring blockchain back * add correct links * spec: revert event hashing (#132) * Evidence time is sourced from block time (#138) * RFC-002: non-zero genesis (#119) * abci: add ResponseInitChain.app_hash (#140) * update hashing of empty inputs, and initial block LastResultsHash (#141) * update evidence verification (#139) * accept RFC-002 (#142) * add description of arbitrary initial height (#135) * update ResponseInitChain.app_hash description (#143) * remove unused directories and update README (#145) This change removes unused directories (`papers` and `research`) and updates the README to reflect our strategy for merging the informalsystems/tendermint-rs specs into this repository. Partially addresses #121. * ci: add markdown linter (#146) * ci: add dependabot config (#148) * build(deps): bump gaurav-nelson/github-action-markdown-link-check from 0.6.0 to 1.0.7 (#149) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 0.6.0 to 1.0.7. Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: add sections to abci (#150) * spec: update abci events (#151) * spec: extract light-client to its own directory (#152) Co-authored-by: Callum Waters <[email protected]> * spec: remove evidences (#153) * add a stale bot (#134) * Current versions of light client specs from tendermint-rs (#158) * current versions of light client specs from tendermint-rs * markdown lint * linting * links * links * links Co-authored-by: Marko Baricevic <[email protected]> * Fastsync spec from tendermint-rs (#157) * fastsync spec from tendermint-rs * fixed broken link * fixed linting * more fixes * markdown lint * move fast_sync to rust-spec Co-authored-by: Marko Baricevic <[email protected]> * Update README.md (#160) * spec/reactors/mempool: batch txs per peer (#155) * spec/reactors/mempool: batch txs per peer Refs tendermint/tendermint#625 * update * spec: Light client attack detector (#164) * start with new detection and evidence spec * more definitions at top * sketch of functions * pre post draft * evidence proof * typo * evidence theory polished * some TODOs resolved * more TODOs * links * second to last revision before PR * links * I will read once more and then make a PR * removed peer handling definitions * secondary * ready to review * detector ready for review * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md * skip-trace * PossibleCommit explained * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * comments by Zarko * renamed and changed link in README Co-authored-by: Zarko Milosevic <[email protected]> * fixed an overlooked conflict (#167) * describe valset sorting according to v0.34 requirements (#169) * evidence: update data structures (#165) * fix markdown linter (#172) * TLA+ specs from MBT revision (#173) * remove setOption (#181) * spec: protobuf changes (#156) Co-authored-by: Anton Kaliaev <[email protected]> * first check latest with secondary (#184) * Extending the blockchain specification (in the light client) to produce different ratios of faults (#183) * cleaning unused definitions * introduced the ratio of faulty processes * Update README.md (#185) * build(deps): bump gaurav-nelson/github-action-markdown-link-check from 1.0.7 to 1.0.8 (#188) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.7 to 1.0.8. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](gaurav-nelson/github-action-markdown-link-check@1.0.7...e3c371c) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * spec: update light client verification to match supervisor (#171) * VDD renaming of verification spec + links fixed * latest() * backwards * added TODOs * link in old file to new name * better text * revision done. needs one more round of reading * renamed constants in 001 according to TLA+ and impl * ready for PR * forgot linting * Update rust-spec/lightclient/verification/verification_002_draft.md * Update rust-spec/lightclient/verification/verification_002_draft.md * added lightstore function needed for supervisor * added lightstore functions for supervisor * ident * Update rust-spec/lightclient/verification/verification_002_draft.md * github: issue template for proposals (#190) * Sequential Supervisor (#186) * move from tendermint-rs but needs discussion * markdown lint * TODO links replaced * links * links * links lint * Update rust-spec/lightclient/supervisor/supervisor.md * Update rust-spec/lightclient/supervisor/supervisor.md * Update rust-spec/lightclient/supervisor/supervisor.md * Update rust-spec/lightclient/supervisor/supervisor.md * moved peer handling definitions to supervisor * polishing * rename * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * changes to maintain StateVerified again * ready for changes in verification * start of supervisor * module name * fixed * more details * supevisor completed. Now I have to add function to verification * ready for review * tla comment * removed issues * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * intro text fixed * indentation * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * comment to entry points Co-authored-by: Marko Baricevic <[email protected]> * RFC: adopt zip 215 (#144) Co-authored-by: Robert Zaremba <[email protected]> * Core: move validation & data structures together (#176) Co-authored-by: Callum Waters <[email protected]> * docs: make blockchain not viewable (#211) * evidence: update data structures to reflect added support of abci evidence (#213) * encoding: add secp, ref zip215, tables (#212) * Detector English Spec ready (#215) Add detector English spec * add Ivy proofs (#210) * add Ivy proofs * fix docker-compose command * Light client detector spec in TLA+ and refactoring of light client verification TLA+ spec (#216) Add light client detector spec in TLA+ * abci: lastcommitinfo.round extra sentence (#221) * abci: add abci_version to requestInfo (#223) * BFT requires _less than_ 1/3 faulty validators (#228) Thanks fo spotting the imprecision in the text, @shahankhatch ! * Draft of evidence handling for discussion (#225) * start with accountability deliverable * problem statement * draft function * quite complete draft. ready to discuss with Igor * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * ready for TLA+ to take over * isolate * isolateamnesiatodos * Update isolate-attackers_001_draft.md * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * The TLA+ specification of the attackers detection (#231) * the working attackers isolation spec, needs more comments * the TLA+ spec of the attackers isolation * build(deps): bump gaurav-nelson/github-action-markdown-link-check (#233) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.8 to 1.0.11. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](gaurav-nelson/github-action-markdown-link-check@1.0.8...2a60e0f) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Computing attack types (#232) Add light attack evidence handling * Update README.md (#234) * p2p: update frame size (#235) Reflect the change made in tendermint/tendermint#5805 The MTU (Maximum Transmission Unit) for Ethernet is 1500 bytes. The IP header and the TCP header take up 20 bytes each at least (unless optional header fields are used) and thus the max for (non-Jumbo frame) Ethernet is 1500 - 20 -20 = 1460 Source: https://stackoverflow.com/a/3074427/820520 * build(deps): bump gaurav-nelson/github-action-markdown-link-check (#239) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.11 to 1.0.12. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](gaurav-nelson/github-action-markdown-link-check@1.0.11...0fe4911) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * layout: add section titles (#240) * reactors: remove bcv1 (#241) * abci: rewrite to proto interface (#237) * Update supervisor_001_draft.md (#243) * spec: remove reactor section (#242) Co-authored-by: Tess Rinearson <[email protected]> * non-critical bugfix in the TLA+ spec (found by new version of apalache) (#244) * params: remove block timeiota (#248) * proto: add files (#246) Co-authored-by: Erik Grinaker <[email protected]> * proto: modify height int64 to uint64 (#253) * abci: note on concurrency (#258) Co-authored-by: Marko <[email protected]> * spec: merge rust-spec (#252) * Fix list of RFCs (#266) * readme: cleanup (#262) * modify readme * add rfc and proto * add rust=spec back to avoid breakage * lint readme * genesis: Explain fields in genesis file (#270) * describe the genesis * Update spec/core/genesis.md Co-authored-by: Dev Ojha <[email protected]> * Apply suggestions from code review Co-authored-by: Callum Waters <[email protected]> * add wording on app_state * Update spec/core/genesis.md Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Dev Ojha <[email protected]> Co-authored-by: Callum Waters <[email protected]> * p2p: links (#268) * fix links * fix more links * Proposer-based timestamp specification (#261) * added proposer-based timestamp spec * Update spec/consensus/proposer-based-timestamp/pbts_001_draft.md Co-authored-by: Aleksandr Bezobchuk <[email protected]> * Update spec/consensus/proposer-based-timestamp/pbts_001_draft.md Co-authored-by: Aleksandr Bezobchuk <[email protected]> * Update spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md Co-authored-by: Marko <[email protected]> * Update spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md * Update spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md Co-authored-by: Callum Waters <[email protected]> * fixes from PR Co-authored-by: Josef Widder <[email protected]> Co-authored-by: Aleksandr Bezobchuk <[email protected]> Co-authored-by: Marko <[email protected]> Co-authored-by: Callum Waters <[email protected]> * abci: reorder sidebar (#282) * ABCI++ RFC (#254) * ABCI++ RFC This commit adds an RFC for ABCI++, which is a collection of three new phases of communication between the consensus engine and the application. Co-authored-by: Sunny Aggarwal <[email protected]> * Fix bugs pointed out by @liamsi * Update rfc/004-abci++.md Co-authored-by: Federico Kunze <[email protected]> * Fix markdown lints * Update rfc/004-abci++.md Co-authored-by: Ismail Khoffi <[email protected]> * Update rfc/004-abci++.md Co-authored-by: Tess Rinearson <[email protected]> * Update rfc/004-abci++.md Co-authored-by: Tess Rinearson <[email protected]> * Add information about the rename in the context section * Bold RFC * Add example for self-authenticating vote data * More exposition of the term IPC * Update pros / negatives * Fix sentence fragment * Add desc for no-ops Co-authored-by: Sunny Aggarwal <[email protected]> Co-authored-by: Federico Kunze <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]> Co-authored-by: Tess Rinearson <[email protected]> * RFC: ReverseSync - fetching historical data (#224) * core: update a few sections (#284) * p2p: update state sync messages for reverse sync (#285) * Update README.md (#286) * rpc: define spec for RPC (#276) * add rpc spec and support outline * add json * add more routes remove unneeded ones * add rest of rpc endpoints * add jsonrpc calls * add more jsonrpc calls * fix blockchain * cleanup unused links and add links to repos * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> * add missing param from consensus param * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> * fix cast and add doc to readme Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Marko Baricevic <[email protected]> * A few improvements to the Ivy proof (#288) * Avoid quantifier alternation cycle The problematic quantifier alternation cycle arose because the definition of accountability_violation was unfolded. This commit also restructures the induction proof for clarity. * add count_lines.sh * fix typo and add forgotten complete=fo in comment Co-authored-by: Giuliano <[email protected]> * Fixed a broken link (#291) * fix message type for block-sync (#298) * lint: fix lint errors (#301) * build(deps): bump actions/stale from 3 to 3.0.18 (#300) Bumps [actions/stale](https://github.com/actions/stale) from 3 to 3.0.18. - [Release notes](https://github.com/actions/stale/releases) - [Commits](actions/stale@v3...v3.0.18) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump actions/stale from 3.0.18 to 3.0.19 (#302) Bumps [actions/stale](https://github.com/actions/stale) from 3.0.18 to 3.0.19. - [Release notes](https://github.com/actions/stale/releases) - [Commits](actions/stale@v3.0.18...v3.0.19) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * rename HasVote to ReceivedVote (#289) * add a changelog to track changes (#303) * add a changelog to track changes * Update CHANGELOG.md Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Callum Waters <[email protected]> * rpc: clarify timestamps (#304) * clarify timestamps * changelog entry * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Callum Waters <[email protected]> * rpc: add chunked genesis endpoint (#299) * rpc: add chunked genesis endpoint * fix lint * feedback * add info about error * fix lint Co-authored-by: marbar3778 <[email protected]> * update ResponseCheckTx (#306) * rpc: Add totalGasUSed to block_results response (#308) * Add C++ code generation and test scenario (#310) * add parameters to byzantine send action * make net not trusted it's not necessary since for proofs Ivy will assume that the environment does not break action preconditions * use require instead of assume it seems that assume is not checked when other isolates call! * add comment * add comment * run with random seed * make domain model extractable to C++ * substitute require for assume assumes in an action are not checked when the action is called! I.e. they place no requirement on the caller; we're just assuming that the caller is going to do the right thing. This wasn't very important here but it leade to a minor inconsistency slipping through. * make the net isolate not trusted there was no need for it * add tendermint_test.ivy contains a simple test scenario that show that the specification is no vacuuous * update comment * add comments * throw if trying to parse nset value in the repl * add comment * minor refactoring * add new pex messages (#312) * build(deps): bump gaurav-nelson/github-action-markdown-link-check (#313) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.12 to 1.0.13. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](gaurav-nelson/github-action-markdown-link-check@1.0.12...1.0.13) --- updated-dependencies: - dependency-name: gaurav-nelson/github-action-markdown-link-check dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update spec to reference currently used timestamp type (#317) * build(deps): bump actions/stale from 3.0.19 to 4 (#319) Bumps [actions/stale](https://github.com/actions/stale) from 3.0.19 to 4. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@v3.0.19...v4) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * address discrepancies between spec and implementation (#322) * update proto files for release (#318) * stale bot: ignore issues (#325) * evidence: add section explaining evidence (#324) * statesync: new messages for gossiping consensus params (#328) * rpc: update peer format in specification in NetInfo operation (#331) * Update supervisor_001_draft.md (#334) * core: text cleanup (#332) * abci: clarify what abci stands for (#336) * abci: clarify what abci stands for * link to abci type protos. * abci: clarify connection use in-process (#337) * abci: clarify connection use in-process * Update abci.md * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * invert abci explanations * lint++ * lint++ * lint++ * lint++ Co-authored-by: M. J. Fromberger <[email protected]> * proto: move proto files under the correct directory related to their package name (#344) * abci.md fixup (#339) * abci: points of clarification ahead of v0.1.0 * lint++ * typo * lint++ * double word score * grammar * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * pr feedback * wip * update non-zero status code docs * fix event description * update CheckTx description Co-authored-by: M. J. Fromberger <[email protected]> * Update supervisor_001_draft.md (#333) * Update supervisor_001_draft.md If the only node in the *FullNodes* set is the primary, that was just deemed faulty, we can't find honest primary. * Update supervisor_001_draft.md * light: update initialization description (#320) * apps.md fixups (#341) * wip * wip * wip * remove comments in favor of gh comments * wip * udpates to language, should must etc * Apply suggestions from code review Co-authored-by: M. J. Fromberger <[email protected]> * remove tendermint cache description Co-authored-by: M. J. Fromberger <[email protected]> * proto: add tendermint go changes (#349) * add missed proto files * add abci changes * rename blockchain to blocksync * Update proto/tendermint/abci/types.proto Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Callum Waters <[email protected]> * fix mockery generation script (#9094) Signed-off-by: Marko Baricevic <[email protected]> Co-authored-by: Milosevic, Zarko <[email protected]> Co-authored-by: Milosevic, Zarko <[email protected]> Co-authored-by: Zarko Milosevic <[email protected]> Co-authored-by: Marko <[email protected]> Co-authored-by: Zarko Milosevic <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]> Co-authored-by: Anca Zamfir <[email protected]> Co-authored-by: dongsamb <[email protected]> Co-authored-by: Sunny Aggarwal <[email protected]> Co-authored-by: Anca Zamfir <[email protected]> Co-authored-by: Ethan Buchman <[email protected]> Co-authored-by: Zarko Milosevic <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]> Co-authored-by: Zaki Manian <[email protected]> Co-authored-by: Erik Grinaker <[email protected]> Co-authored-by: Tess Rinearson <[email protected]> Co-authored-by: Alexander Simmerl <[email protected]> Co-authored-by: Igor Konnov <[email protected]> Co-authored-by: Sean Braithwaite <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Josef Widder <[email protected]> Co-authored-by: Andrey Kuprianov <[email protected]> Co-authored-by: Igor Konnov <[email protected]> Co-authored-by: Sam Hart <[email protected]> Co-authored-by: Robert Zaremba <[email protected]> Co-authored-by: Giuliano <[email protected]> Co-authored-by: Shahan Khatchadourian <[email protected]> Co-authored-by: Dev Ojha <[email protected]> Co-authored-by: istoilkovska <[email protected]> Co-authored-by: Aleksandr Bezobchuk <[email protected]> Co-authored-by: Sam Kleinman <[email protected]> Co-authored-by: Sunny Aggarwal <[email protected]> Co-authored-by: Federico Kunze <[email protected]> Co-authored-by: Marko Baricevic <[email protected]> Co-authored-by: Giuliano <[email protected]> Co-authored-by: Jordan Sexton <[email protected]> Co-authored-by: MengXiangJian <[email protected]> Co-authored-by: Yixin Luo <[email protected]> Co-authored-by: crypto-facs <[email protected]> Co-authored-by: Giuliano <[email protected]> Co-authored-by: William Banfield <[email protected]> Co-authored-by: Mateusz Górski <[email protected]> Co-authored-by: M. J. Fromberger <[email protected]> Co-authored-by: Thane Thomson <[email protected]>

* mempool: reduce lock contention during CheckTx (backport #8983) (#8985) A manual cherry-pick of 9e64c95. The way this was originally structured, we reacquired the lock after issuing the initial ABCI CheckTx call, only to immediately release it. Restructure the code so that this redundant acquire is no longer necessary. * mempool: release lock during app connection flush (#8986) A manual backport of #8984. This case is symmetric to what we did for CheckTx calls, where we release the mempool mutex to ensure callbacks can fire during call setup. We also need this behaviour for application flush, for the same reason: The caller holds the lock by contract from the Mempool interface. * Prepare changelog for v0.34.20-rc1. (#8966) * config: remove obsolete mempool v1 warning (#8987) * build(deps): Bump google.golang.org/grpc from 1.47.0 to 1.48.0 (#8991) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.47.0 to 1.48.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.47.0...v1.48.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sam Kleinman <[email protected]> * mempool: ensure evicted transactions are removed from the cache (backport #9000) (#9004) This is a manual cherry-pick of commit b94470a6a42e8ffe7e7467521de5f51eb937c454. In the original implementation transactions evicted for priority were also removed from the cache. In addition, remove expired transactions from the cache. Related: - Add Has method to cache implementations. - Update tests to exercise this condition. * mempool: ensure async requests are flushed to the server (#9010) In the v0.34 line, the socket and gRPC clients require explicit flushes to ensure that the client and server have received an async request. Add these calls explicitly where required in the backport of the priority mempool. In addition, the gRPC client's flush plumbing was not fully hooked up in the v0.34 line, so this change includes that update as well. * Prepare changelog for Release v0.34.20 (#9032) * build(deps): Bump github.com/golangci/golangci-lint (#9036) Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.46.2 to 1.47.0. - [Release notes](https://github.com/golangci/golangci-lint/releases) - [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md) - [Commits](https://github.com/golangci/golangci-lint/compare/v1.46.2...v1.47.0) --- updated-dependencies: - dependency-name: github.com/golangci/golangci-lint dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * mempool: rework lock discipline to mitigate callback deadlocks (backport #9030) (#9033) (manual cherry-pick of commit 22ed610083cb8275a954406296832149c4cc1dcd) * build(deps): Bump github.com/golangci/golangci-lint (#9043) * build(deps): Bump github.com/BurntSushi/toml from 1.1.0 to 1.2.0 (#9062) Bumps [github.com/BurntSushi/toml](https://github.com/BurntSushi/toml) from 1.1.0 to 1.2.0. - [Release notes](https://github.com/BurntSushi/toml/releases) - [Commits](https://github.com/BurntSushi/toml/compare/v1.1.0...v1.2.0) --- updated-dependencies: - dependency-name: github.com/BurntSushi/toml dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): Bump github.com/golangci/golangci-lint (#9071) Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.47.1 to 1.47.2. - [Release notes](https://github.com/golangci/golangci-lint/releases) - [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md) - [Commits](https://github.com/golangci/golangci-lint/compare/v1.47.1...v1.47.2) --- updated-dependencies: - dependency-name: github.com/golangci/golangci-lint dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * cli: add command to manually reindex tx and block events (backport: #6676) (#9034) * fix mockery generation script (#9094) (#9114) * build(deps): Bump github.com/bufbuild/buf from 1.4.0 to 1.7.0 (#9137) * backport: Fix unsafe-reset-all for working with default home (#9103) (#9113) * config: p2p.external-address (backport #9107) (#9153) * remove old proto workflow (#9167) * build(deps): Bump github.com/golangci/golangci-lint (#9188) Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.47.2 to 1.48.0. - [Release notes](https://github.com/golangci/golangci-lint/releases) - [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md) - [Commits](https://github.com/golangci/golangci-lint/compare/v1.47.2...v1.48.0) --- updated-dependencies: - dependency-name: github.com/golangci/golangci-lint dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Backport of sam/abci-responses (#9090) (#9159) *backport of sam/abci-responses Co-authored-by: William Banfield <[email protected]> * Bump linter to 1.47 (#9218) *bump linter to 1.47 Co-authored-by: William Banfield <[email protected]> * chore: Bump go to 1.18 (#9212) * update to 1.18 Co-authored-by: William Banfield <[email protected]> * Small update to toml.go for abci-responses (#9232) * update to toml * update default (#9235) * Sync codeowners with main (#9245) * Update to ABCILastResponseskey (#9253) * update last responses key * cli: Enable reindex-event cmd (#9268) I noticed today that this wasn't enabled. Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> * spec: migrate v0.7.1. into v0.34 (#9262) * Initial commit * Add three timeouts and align pseudocode better with existing algorithm * Align protocol with Tendermint code and add find valid value mechanism * Prepare to Nuke Develop (#47) * state -> step * vote -> v * New version of the algorithm and the proof * New version of the algorithm and the proofs * Added algorithm description * Add algorithm description * Add introduction * Add conclusion * Add conclusion file * fix warnings (caption was defined twice) - only the latter is used anyways (centers captions) - this makes it possible to autom. building the paper * Update grammar * s/state_p/step_p * Address Ismail's comments * intro: language fixes * definitions: language fixes * consensus: various fixes * proof: some fixes * try to improve reviewability * \eq -> = * textwrap to 79 * various minor fixes * proof: fix itemization * proof: more minor fixes * proof: timeouts are functions * proof: fixes to lemma6 * Intro changes and improve title page * Add Marko and Ming to acks * add readme * Format algorithm correctly Clarify condition semantic and timeouts Improve descriptions * patform -> platform * Ensure that rules are mutually exclusive - various clarifications and small improvements * Release v0.6 * small nits for smoother readability * This PR is to create signed commits to be able to merge (#50) Signed-off-by: Marko Baricevic <[email protected]> * Add consesnus and blockchain specs, (#52) - Open questions - Do we want to split lite client work from consesnsus - From the blockchain spec, is encoding nessecary in the spec Signed-off-by: Marko Baricevic <[email protected]> * Add ABCI SPEC (#51) - move the abci spec from tendermint to spec repo Signed-off-by: Marko Baricevic <[email protected]> * spec/consensus/signing: add more details about nil and amnesia (#54) - Add more details about nil votes and about amnesia attacks Signed-off-by: Marko Baricevic <[email protected]> * Add Section for P2P (#53) * Add Section for P2P - moved over the section on p2p Signed-off-by: Marko Baricevic <[email protected]> * add some more files Signed-off-by: Marko Baricevic <[email protected]> * Fix model section * Add non-recursive specification of Bisection algorithm - Fix timing issues by introducing Delta parameter * spec: update spec with tendermint updates (#62) * spec: update spec with tendermint updates - this in preperation of deleting the spec folder in docs in tendermint/tendermint Signed-off-by: Marko Baricevic <[email protected]> * spec: added in reactors & p2p Signed-off-by: Marko Baricevic <[email protected]> * spec: update readme in spec to comply with docs site Signed-off-by: Marko Baricevic <[email protected]> * docs: addded more changes from tednermint/tendermint Signed-off-by: Marko Baricevic <[email protected]> * reflect breaking changes made to Commit (#63) * reflect breaking changes made to Commit PR: https://github.com/tendermint/tendermint/pull/4146 Issue: https://github.com/tendermint/tendermint/issues/1648 * types: rename Commit#Precommits to Signatures * update BlockIDFlagAbsent comment * remove iota * Clean up error conditions and simplify pseudocode * Apply suggestions from code review Co-Authored-By: Anca Zamfir <[email protected]> * Add spec doc about unconditional_peer, persistent_peers_max_dial of ADR-050 (#68) * Add spec doc about unconditional_peer_ids, persistent_peers_max_dial_period of ADR-050 * Add indefinitely dialing condition * Add sr25519 amino documentation (#67) * sr25519 amino * Update spec/blockchain/encoding.md Co-Authored-By: Marko <[email protected]> * some suggestions for pseuodocode changes * Improved error handling * Add explanation on difference between trusted models * Address reviewer's comments * Addressing reviewer's comments * Separating algorithm from proofs * Intermediate commit (aligning spec with the code) * Removing Store from API and providing end-to-end timing guarantees * Address reviewer comment's. Intermediate commit * light client dir and readmes * titles * add redirects * add diagram * detection TODO * fix image * update readme * Aligh the correctness arguments with the pseudocode changes * lite->light * Fix link in readme ./light -> ./light-client * p2p: Merlin based malleability fixes (#72) * Update the secret connection spec with the use of merlin to eliminte handshake malleability * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> * Update spec/p2p/peer.md Co-Authored-By: Anton Kaliaev <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]> * docs: update specs to remove cmn (#77) - cmn was remvoed in favor of sub pkgs. cmn.kvpair is now kv.pair Signed-off-by: Marko Baricevic <[email protected]> * evidence: Add time to evidence params (#69) * evidence: Add time to evidence params - this pr is grouped together with https://github.com/tendermint/tendermint/pull/4254, once that PR is merged then this one can be as well. Signed-off-by: Marko Baricevic <[email protected]> * remove note Signed-off-by: Marko Baricevic <[email protected]> * Apply suggestions from code review Co-Authored-By: Anton Kaliaev <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]> * update link to the pex reactor * add markdown link checker * changed tab spacing * removed folder-path flag * first attempt at fixing all links * second attempt at fixing all links * codeowners: add code owners (#82) * codeowners: add code owners - added some codeowners please comment if youd like to be added as well. Signed-off-by: Marko Baricevic <[email protected]> * remove comment of repo maintainers * remove .idea dir (#83) Signed-off-by: Marko Baricevic <[email protected]> * RFC-001: configurable block retention (#84) * Added RFC for truncated block history coordination * Clarified minimum block retention * Added hard checks on block retention and snapshot interval, and made some minor tweaks * Genesis parameters are immutable * Use local config for snapshot interval * Reordered parameter descriptions * Clarified local config option for snapshot-interval * rewrite for ABCI commit response * Renamed RFC * add block retention diagram * Removed retain_blocks table * fix image numbers * resolved open questions * image quality * accept RFC-001 (#86) * abci: add basic description of ABCI Commit.ResponseHeight (#85) Documentation for block pruning, once it's merged: tendermint/tendermint#4588. Minimum documentation, for now - we probably shouldn't encourage using this feature too much until we release state sync. * abci: add MaxAgeNumBlocks/MaxAgeDuration to EvidenceParams (#87) * abci: update MaxAgeNumBlocks & MaxAgeDuration docs (#88) * document state sync ABCI interface and P2P protocol (#90) The corresponding Tendermint PRs are tendermint/tendermint#4704 and tendermint/tendermint#4705. * Revert "document state sync ABCI interface and P2P protocol (#90)" (#92) This reverts commit 9842b4b0fb703e609ad233af9683adc773bc95ef. * blockchain: change validator set sorting method (#91) * abci: specify sorting of RequestInitChain.Validators * blockchain: change validator sorting method Refs https://github.com/tendermint/tendermint/issues/2478 * reactors/pex: specify hash function (#94) https://github.com/tendermint/tendermint/pull/4810/files * document state sync ABCI interface and P2P protocol (#93) * Revert "Revert "document state sync ABCI interface and P2P protocol (#90)" (#92)" This reverts commit 90797cef90da762db7f7a14ce834c745140f7bcd. * update with new enum case * fix links Co-authored-by: Erik Grinaker <[email protected]> * Update evidence params with MaxNum (#95) evidence params now includes maxNum which is the maximum number of evidence that can be committed on a single block * reactors/pex: masked IP is used as group key (#96) * spec: add ProofTrialPeriod to EvidenceParam (#99) * spec: modify Header.LastResultsHash (#97) Refs: https://github.com/tendermint/tendermint/issues/1007 PR: https://github.com/tendermint/tendermint/pull/4845 * spec: link to abci server implementations (#100) * spec: update evidence in blockchain.md (#108) now evidence reflects the actual evidence present in the tendermint repo * abci: add AppVersion to ConsensusParams (#106) * abci: tweak node sync estimate (#115) * spec/abci: expand on Validator#Address (#118) Refs https://github.com/tendermint/tendermint/issues/3732 * blockchain: rename to core (#123) * blockchain: remove duplicate evidence sections (#124) * spec/consensus: canonical vs subjective commit Refs https://github.com/tendermint/tendermint/issues/2769 * Apply suggestions from code review Co-authored-by: Igor Konnov <[email protected]> * update spec with the removal of phantom validator evidence (#126) * bring blockchain back * add correct links * spec: revert event hashing (#132) * Evidence time is sourced from block time (#138) * RFC-002: non-zero genesis (#119) * abci: add ResponseInitChain.app_hash (#140) * update hashing of empty inputs, and initial block LastResultsHash (#141) * update evidence verification (#139) * accept RFC-002 (#142) * add description of arbitrary initial height (#135) * update ResponseInitChain.app_hash description (#143) * remove unused directories and update README (#145) This change removes unused directories (`papers` and `research`) and updates the README to reflect our strategy for merging the informalsystems/tendermint-rs specs into this repository. Partially addresses #121. * ci: add markdown linter (#146) * ci: add dependabot config (#148) * build(deps): bump gaurav-nelson/github-action-markdown-link-check from 0.6.0 to 1.0.7 (#149) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 0.6.0 to 1.0.7. Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * docs: add sections to abci (#150) * spec: update abci events (#151) * spec: extract light-client to its own directory (#152) Co-authored-by: Callum Waters <[email protected]> * spec: remove evidences (#153) * add a stale bot (#134) * Current versions of light client specs from tendermint-rs (#158) * current versions of light client specs from tendermint-rs * markdown lint * linting * links * links * links Co-authored-by: Marko Baricevic <[email protected]> * Fastsync spec from tendermint-rs (#157) * fastsync spec from tendermint-rs * fixed broken link * fixed linting * more fixes * markdown lint * move fast_sync to rust-spec Co-authored-by: Marko Baricevic <[email protected]> * Update README.md (#160) * spec/reactors/mempool: batch txs per peer (#155) * spec/reactors/mempool: batch txs per peer Refs https://github.com/tendermint/tendermint/issues/625 * update * spec: Light client attack detector (#164) * start with new detection and evidence spec * more definitions at top * sketch of functions * pre post draft * evidence proof * typo * evidence theory polished * some TODOs resolved * more TODOs * links * second to last revision before PR * links * I will read once more and then make a PR * removed peer handling definitions * secondary * ready to review * detector ready for review * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * Update rust-spec/lightclient/detection/detection.md * skip-trace * PossibleCommit explained * Update rust-spec/lightclient/detection/detection.md Co-authored-by: Zarko Milosevic <[email protected]> * comments by Zarko * renamed and changed link in README Co-authored-by: Zarko Milosevic <[email protected]> * fixed an overlooked conflict (#167) * describe valset sorting according to v0.34 requirements (#169) * evidence: update data structures (#165) * fix markdown linter (#172) * TLA+ specs from MBT revision (#173) * remove setOption (#181) * spec: protobuf changes (#156) Co-authored-by: Anton Kaliaev <[email protected]> * first check latest with secondary (#184) * Extending the blockchain specification (in the light client) to produce different ratios of faults (#183) * cleaning unused definitions * introduced the ratio of faulty processes * Update README.md (#185) * build(deps): bump gaurav-nelson/github-action-markdown-link-check from 1.0.7 to 1.0.8 (#188) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.7 to 1.0.8. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](https://github.com/gaurav-nelson/github-action-markdown-link-check/compare/1.0.7...e3c371c731b2f494f856dc5de7f61cea4d519907) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * spec: update light client verification to match supervisor (#171) * VDD renaming of verification spec + links fixed * latest() * backwards * added TODOs * link in old file to new name * better text * revision done. needs one more round of reading * renamed constants in 001 according to TLA+ and impl * ready for PR * forgot linting * Update rust-spec/lightclient/verification/verification_002_draft.md * Update rust-spec/lightclient/verification/verification_002_draft.md * added lightstore function needed for supervisor * added lightstore functions for supervisor * ident * Update rust-spec/lightclient/verification/verification_002_draft.md * github: issue template for proposals (#190) * Sequential Supervisor (#186) * move from tendermint-rs but needs discussion * markdown lint * TODO links replaced * links * links * links lint * Update rust-spec/lightclient/supervisor/supervisor.md * Update rust-spec/lightclient/supervisor/supervisor.md * Update rust-spec/lightclient/supervisor/supervisor.md * Update rust-spec/lightclient/supervisor/supervisor.md * moved peer handling definitions to supervisor * polishing * rename * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * changes to maintain StateVerified again * ready for changes in verification * start of supervisor * module name * fixed * more details * supevisor completed. Now I have to add function to verification * ready for review * tla comment * removed issues * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * intro text fixed * indentation * Update rust-spec/lightclient/supervisor/supervisor_001_draft.md * comment to entry points Co-authored-by: Marko Baricevic <[email protected]> * RFC: adopt zip 215 (#144) Co-authored-by: Robert Zaremba <[email protected]> * Core: move validation & data structures together (#176) Co-authored-by: Callum Waters <[email protected]> * docs: make blockchain not viewable (#211) * evidence: update data structures to reflect added support of abci evidence (#213) * encoding: add secp, ref zip215, tables (#212) * Detector English Spec ready (#215) Add detector English spec * add Ivy proofs (#210) * add Ivy proofs * fix docker-compose command * Light client detector spec in TLA+ and refactoring of light client verification TLA+ spec (#216) Add light client detector spec in TLA+ * abci: lastcommitinfo.round extra sentence (#221) * abci: add abci_version to requestInfo (#223) * BFT requires _less than_ 1/3 faulty validators (#228) Thanks fo spotting the imprecision in the text, @shahankhatch ! * Draft of evidence handling for discussion (#225) * start with accountability deliverable * problem statement * draft function * quite complete draft. ready to discuss with Igor * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * Update isolate-attackers_001_draft.md * ready for TLA+ to take over * isolate * isolateamnesiatodos * Update isolate-attackers_001_draft.md * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * Update rust-spec/lightclient/attacks/isolate-attackers_001_draft.md Co-authored-by: Igor Konnov <[email protected]> * The TLA+ specification of the attackers detection (#231) * the working attackers isolation spec, needs more comments * the TLA+ spec of the attackers isolation * build(deps): bump gaurav-nelson/github-action-markdown-link-check (#233) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.8 to 1.0.11. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](https://github.com/gaurav-nelson/github-action-markdown-link-check/compare/1.0.8...2a60e0fe41b5361f446ccace6621a1a2a5c324cf) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Computing attack types (#232) Add light attack evidence handling * Update README.md (#234) * p2p: update frame size (#235) Reflect the change made in https://github.com/tendermint/tendermint/pull/5805 The MTU (Maximum Transmission Unit) for Ethernet is 1500 bytes. The IP header and the TCP header take up 20 bytes each at least (unless optional header fields are used) and thus the max for (non-Jumbo frame) Ethernet is 1500 - 20 -20 = 1460 Source: https://stackoverflow.com/a/3074427/820520 * build(deps): bump gaurav-nelson/github-action-markdown-link-check (#239) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.11 to 1.0.12. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](https://github.com/gaurav-nelson/github-action-markdown-link-check/compare/1.0.11...0fe4911067fa322422f325b002d2038ba5602170) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * layout: add section titles (#240) * reactors: remove bcv1 (#241) * abci: rewrite to proto interface (#237) * Update supervisor_001_draft.md (#243) * spec: remove reactor section (#242) Co-authored-by: Tess Rinearson <[email protected]> * non-critical bugfix in the TLA+ spec (found by new version of apalache) (#244) * params: remove block timeiota (#248) * proto: add files (#246) Co-authored-by: Erik Grinaker <[email protected]> * proto: modify height int64 to uint64 (#253) * abci: note on concurrency (#258) Co-authored-by: Marko <[email protected]> * spec: merge rust-spec (#252) * Fix list of RFCs (#266) * readme: cleanup (#262) * modify readme * add rfc and proto * add rust=spec back to avoid breakage * lint readme * genesis: Explain fields in genesis file (#270) * describe the genesis * Update spec/core/genesis.md Co-authored-by: Dev Ojha <[email protected]> * Apply suggestions from code review Co-authored-by: Callum Waters <[email protected]> * add wording on app_state * Update spec/core/genesis.md Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Dev Ojha <[email protected]> Co-authored-by: Callum Waters <[email protected]> * p2p: links (#268) * fix links * fix more links * Proposer-based timestamp specification (#261) * added proposer-based timestamp spec * Update spec/consensus/proposer-based-timestamp/pbts_001_draft.md Co-authored-by: Aleksandr Bezobchuk <[email protected]> * Update spec/consensus/proposer-based-timestamp/pbts_001_draft.md Co-authored-by: Aleksandr Bezobchuk <[email protected]> * Update spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md Co-authored-by: Marko <[email protected]> * Update spec/consensus/proposer-based-timestamp/pbts-algorithm_001_draft.md * Update spec/consensus/proposer-based-timestamp/pbts-sysmodel_001_draft.md Co-authored-by: Callum Waters <[email protected]> * fixes from PR Co-authored-by: Josef Widder <[email protected]> Co-authored-by: Aleksandr Bezobchuk <[email protected]> Co-authored-by: Marko <[email protected]> Co-authored-by: Callum Waters <[email protected]> * abci: reorder sidebar (#282) * ABCI++ RFC (#254) * ABCI++ RFC This commit adds an RFC for ABCI++, which is a collection of three new phases of communication between the consensus engine and the application. Co-authored-by: Sunny Aggarwal <[email protected]> * Fix bugs pointed out by @liamsi * Update rfc/004-abci++.md Co-authored-by: Federico Kunze <[email protected]> * Fix markdown lints * Update rfc/004-abci++.md Co-authored-by: Ismail Khoffi <[email protected]> * Update rfc/004-abci++.md Co-authored-by: Tess Rinearson <[email protected]> * Update rfc/004-abci++.md Co-authored-by: Tess Rinearson <[email protected]> * Add information about the rename in the context section * Bold RFC * Add example for self-authenticating vote data * More exposition of the term IPC * Update pros / negatives * Fix sentence fragment * Add desc for no-ops Co-authored-by: Sunny Aggarwal <[email protected]> Co-authored-by: Federico Kunze <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]> Co-authored-by: Tess Rinearson <[email protected]> * RFC: ReverseSync - fetching historical data (#224) * core: update a few sections (#284) * p2p: update state sync messages for reverse sync (#285) * Update README.md (#286) * rpc: define spec for RPC (#276) * add rpc spec and support outline * add json * add more routes remove unneeded ones * add rest of rpc endpoints * add jsonrpc calls * add more jsonrpc calls * fix blockchain * cleanup unused links and add links to repos * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> * add missing param from consensus param * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> * fix cast and add doc to readme Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Marko Baricevic <[email protected]> * A few improvements to the Ivy proof (#288) * Avoid quantifier alternation cycle The problematic quantifier alternation cycle arose because the definition of accountability_violation was unfolded. This commit also restructures the induction proof for clarity. * add count_lines.sh * fix typo and add forgotten complete=fo in comment Co-authored-by: Giuliano <[email protected]> * Fixed a broken link (#291) * fix message type for block-sync (#298) * lint: fix lint errors (#301) * build(deps): bump actions/stale from 3 to 3.0.18 (#300) Bumps [actions/stale](https://github.com/actions/stale) from 3 to 3.0.18. - [Release notes](https://github.com/actions/stale/releases) - [Commits](https://github.com/actions/stale/compare/v3...v3.0.18) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): bump actions/stale from 3.0.18 to 3.0.19 (#302) Bumps [actions/stale](https://github.com/actions/stale) from 3.0.18 to 3.0.19. - [Release notes](https://github.com/actions/stale/releases) - [Commits](https://github.com/actions/stale/compare/v3.0.18...v3.0.19) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * rename HasVote to ReceivedVote (#289) * add a changelog to track changes (#303) * add a changelog to track changes * Update CHANGELOG.md Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Callum Waters <[email protected]> * rpc: clarify timestamps (#304) * clarify timestamps * changelog entry * Update spec/rpc/README.md Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Callum Waters <[email protected]> * rpc: add chunked genesis endpoint (#299) * rpc: add chunked genesis endpoint * fix lint * feedback * add info about error * fix lint Co-authored-by: marbar3778 <[email protected]> * update ResponseCheckTx (#306) * rpc: Add totalGasUSed to block_results response (#308) * Add C++ code generation and test scenario (#310) * add parameters to byzantine send action * make net not trusted it's not necessary since for proofs Ivy will assume that the environment does not break action preconditions * use require instead of assume it seems that assume is not checked when other isolates call! * add comment * add comment * run with random seed * make domain model extractable to C++ * substitute require for assume assumes in an action are not checked when the action is called! I.e. they place no requirement on the caller; we're just assuming that the caller is going to do the right thing. This wasn't very important here but it leade to a minor inconsistency slipping through. * make the net isolate not trusted there was no need for it * add tendermint_test.ivy contains a simple test scenario that show that the specification is no vacuuous * update comment * add comments * throw if trying to parse nset value in the repl * add comment * minor refactoring * add new pex messages (#312) * build(deps): bump gaurav-nelson/github-action-markdown-link-check (#313) Bumps [gaurav-nelson/github-action-markdown-link-check](https://github.com/gaurav-nelson/github-action-markdown-link-check) from 1.0.12 to 1.0.13. - [Release notes](https://github.com/gaurav-nelson/github-action-markdown-link-check/releases) - [Commits](https://github.com/gaurav-nelson/github-action-markdown-link-check/compare/1.0.12...1.0.13) --- updated-dependencies: - dependency-name: gaurav-nelson/github-action-markdown-link-check dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * update spec to reference currently used timestamp type (#317) * build(deps): bump actions/stale from 3.0.19 to 4 (#319) Bumps [actions/stale](https://github.com/actions/stale) from 3.0.19 to 4. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/stale/compare/v3.0.19...v4) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * address discrepancies between spec and implementation (#322) * update proto files for release (#318) * stale bot: ignore issues (#325) * evidence: add section explaining evidence (#324) * statesync: new messages for gossiping consensus params (#328) * rpc: update peer format in specification in NetInfo operation (#331) * Update supervisor_001_draft.md (#334) * core: text cleanup (#332) * abci: clarify what abci stands for (#336) * abci: clarify what abci stands for * link to abci type protos. * abci: clarify connection use in-process (#337) * abci: clarify connection use in-process * Update abci.md * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * invert abci explanations * lint++ * lint++ * lint++ * lint++ Co-authored-by: M. J. Fromberger <[email protected]> * proto: move proto files under the correct directory related to their package name (#344) * abci.md fixup (#339) * abci: points of clarification ahead of v0.1.0 * lint++ * typo * lint++ * double word score * grammar * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * Update spec/abci/abci.md Co-authored-by: M. J. Fromberger <[email protected]> * pr feedback * wip * update non-zero status code docs * fix event description * update CheckTx description Co-authored-by: M. J. Fromberger <[email protected]> * Update supervisor_001_draft.md (#333) * Update supervisor_001_draft.md If the only node in the *FullNodes* set is the primary, that was just deemed faulty, we can't find honest primary. * Update supervisor_001_draft.md * light: update initialization description (#320) * apps.md fixups (#341) * wip * wip * wip * remove comments in favor of gh comments * wip * udpates to language, should must etc * Apply suggestions from code review Co-authored-by: M. J. Fromberger <[email protected]> * remove tendermint cache description Co-authored-by: M. J. Fromberger <[email protected]> * proto: add tendermint go changes (#349) * add missed proto files * add abci changes * rename blockchain to blocksync * Update proto/tendermint/abci/types.proto Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Callum Waters <[email protected]> * fix mockery generation script (#9094) Signed-off-by: Marko Baricevic <[email protected]> Co-authored-by: Milosevic, Zarko <[email protected]> Co-authored-by: Milosevic, Zarko <[email protected]> Co-authored-by: Zarko Milosevic <[email protected]> Co-authored-by: Marko <[email protected]> Co-authored-by: Zarko Milosevic <[email protected]> Co-authored-by: Anton Kaliaev <[email protected]> Co-authored-by: Anca Zamfir <[email protected]> Co-authored-by: dongsamb <[email protected]> Co-authored-by: Sunny Aggarwal <[email protected]> Co-authored-by: Anca Zamfir <[email protected]> Co-authored-by: Ethan Buchman <[email protected]> Co-authored-by: Zarko Milosevic <[email protected]> Co-authored-by: Ismail Khoffi <[email protected]> Co-authored-by: Zaki Manian <[email protected]> Co-authored-by: Erik Grinaker <[email protected]> Co-authored-by: Tess Rinearson <[email protected]> Co-authored-by: Alexander Simmerl <[email protected]> Co-authored-by: Igor Konnov <[email protected]> Co-authored-by: Sean Braithwaite <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Josef Widder <[email protected]> Co-authored-by: Andrey Kuprianov <[email protected]> Co-authored-by: Igor Konnov <[email protected]> Co-authored-by: Sam Hart <[email protected]> Co-authored-by: Robert Zaremba <[email protected]> Co-authored-by: Giuliano <[email protected]> Co-authored-by: Shahan Khatchadourian <[email protected]> Co-authored-by: Dev Ojha <[email protected]> Co-authored-by: istoilkovska <[email protected]> Co-authored-by: Aleksandr Bezobchuk <[email protected]> Co-authored-by: Sam Kleinman <[email protected]> Co-authored-by: Sunny Aggarwal <[email protected]> Co-authored-by: Federico Kunze <[email protected]> Co-authored-by: Marko Baricevic <[email protected]> Co-authored-by: Giuliano <[email protected]> Co-authored-by: Jordan Sexton <[email protected]> Co-authored-by: MengXiangJian <[email protected]> Co-authored-by: Yixin Luo <[email protected]> Co-authored-by: crypto-facs <[email protected]> Co-authored-by: Giuliano <[email protected]> Co-authored-by: William Banfield <[email protected]> Co-authored-by: Mateusz Górski <[email protected]> Co-authored-by: M. J. Fromberger <[email protected]> Co-authored-by: Thane Thomson <[email protected]> * config: Move `discard_abci_responses` flag into its own storage section (#9275) * config: Move discard_abci_responses flag into its own storage section Signed-off-by: Thane Thomson <[email protected]> * Update config comment to highlight space saving tradeoff Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> * docs: Minor recommendations prior to v0.34.21 release (#9267) * Make reindex-event cmd docs consistent with other commands Signed-off-by: Thane Thomson <[email protected]> * Add warning regarding DiscardABCIResponses to BlockResults Go API Signed-off-by: Thane Thomson <[email protected]> * Update OpenAPI spec to reflect discard_abci_responses change Signed-off-by: Thane Thomson <[email protected]> * Add release highlights to CHANGELOG_PENDING Signed-off-by: Thane Thomson <[email protected]> * Add pending changelog entry for #9033 Signed-off-by: Thane Thomson <[email protected]> * Format pending changelog entries consistently Signed-off-by: Thane Thomson <[email protected]> * Correct and simplify comment wording Signed-off-by: Thane Thomson <[email protected]> * Remove changelog entry regarding storage section Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> * release: prepare v0.34.21 (#9285) * docs: Update v0.34.x to prepare for v0.37 (#9244) * Ignore generated/copied RPC docs Signed-off-by: Thane Thomson <[email protected]> * Sync vuepress config with main Signed-off-by: Thane Thomson <[email protected]> * Sync docs package-lock.json with main Signed-off-by: Thane Thomson <[email protected]> * Sync docs redirects with main Signed-off-by: Thane Thomson <[email protected]> * Sync docs versions with main Signed-off-by: Thane Thomson <[email protected]> * Update OpenAPI version to v0.34 Signed-off-by: Thane Thomson <[email protected]> * Sync DOCS_README with main Signed-off-by: Thane Thomson <[email protected]> * Update all v0.34.x docs references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update v0.34 OpenAPI references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update repo doc links from master to main Signed-off-by: Thane Thomson <[email protected]> * Update code comment references from master to main Signed-off-by: Thane Thomson <[email protected]> * Update repo root doc links from master to main Signed-off-by: Thane Thomson <[email protected]> * Update repo root doc links for docs.tendermint.com from master to main Signed-off-by: Thane Thomson <[email protected]> * Build v0.34.x as "latest" Signed-off-by: Thane Thomson <[email protected]> * Explicitly mark v0.34 docs as latest in version selector Signed-off-by: Thane Thomson <[email protected]> * Add nav link to main and clearly mark as unstable Signed-off-by: Thane Thomson <[email protected]> * Direct all docs.tendermint.com links to v0.34 on v0.34.x Signed-off-by: Thane Thomson <[email protected]> * Update all relevant links on v0.34.x branch to be v0.34-specific Signed-off-by: Thane Thomson <[email protected]> * Update changelog refs to docs.tendermint.com Signed-off-by: Thane Thomson <[email protected]> * Update remaining GH master link to main Signed-off-by: Thane Thomson <[email protected]> * Sync docs build and nav config with main Signed-off-by: Thane Thomson <[email protected]> * Migrate spec links to GitHub repo from docs site Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> * build(deps): Bump google.golang.org/grpc from 1.48.0 to 1.49.0 (#9320) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.48.0 to 1.49.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.48.0...v1.49.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * test: add the loadtime tool (Backport #9342) (#9358) * test: add the loadtime tool (#9342) This pull request adds the loadtime tool. This tool leverages the tm-load-test framework. Using the framework means that the only real logic that needs to be written is the logic for Tx generation. The framework does the rest. The tool writes a set of metadata into the transaction, including the current transaction rate, number of connections, specified size of the transaction, and the current time. * lint * test: add the loadtime report tool (backport #9351) (#9365) * test: add the loadtime report tool (#9351) This pull request adds the report tool and modifies the loadtime libraries to better support its use. (cherry picked from commit 8655080a0ff26c001025b4c1af009f39c90cbb9e) * add nolint Co-authored-by: William Banfield <[email protected]> Co-authored-by: William Banfield <[email protected]> * build(deps): Bump google.golang.org/protobuf from 1.28.0 to 1.28.1 (#9362) Bumps [google.golang.org/protobuf](https://github.com/protocolbuffers/protobuf-go) from 1.28.0 to 1.28.1. - [Release notes](https://github.com/protocolbuffers/protobuf-go/releases) - [Changelog](https://github.com/protocolbuffers/protobuf-go/blob/master/release.bash) - [Commits](https://github.com/protocolbuffers/protobuf-go/compare/v1.28.0...v1.28.1) --- updated-dependencies: - dependency-name: google.golang.org/protobuf dependency-type: direct:production update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sam Kleinman <[email protected]> * build(deps): Bump github.com/golangci/golangci-lint (#9363) Bumps [github.com/golangci/golangci-lint](https://github.com/golangci/golangci-lint) from 1.42.1 to 1.49.0. - [Release notes](https://github.com/golangci/golangci-lint/releases) - [Changelog](https://github.com/golangci/golangci-lint/blob/master/CHANGELOG.md) - [Commits](https://github.com/golangci/golangci-lint/compare/v1.42.1...v1.49.0) --- updated-dependencies: - dependency-name: github.com/golangci/golangci-lint dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Sam Kleinman <[email protected]> Co-authored-by: Thane Thomson <[email protected]> * ci: Remove "(WARNING: BETA SOFTWARE)" tagline from all upcoming releases (backport #9371) (#9373) * ci: Remove "(WARNING: BETA SOFTWARE)" tagline from all upcoming releases (#9371) This is by no means a signal that we offer any additional guarantees with our software. This warning seems somewhat pointless given that: 1. Our open source license clearly states that we offer no warranties with this software. 2. We are clearly still pre-1.0. It also doesn't make sense to append "(WARNING: BETA SOFTWARE)" to pre-releases such as alpha releases, which are to be considered _more_ unstable than beta releases. --- #### PR checklist - [x] Tests written/updated, or no tests needed - [x] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [x] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed (cherry picked from commit d7645628f13031746be27507d1673e132cf1f6e6) # Conflicts: # .goreleaser.yml * Resolve conflicts Signed-off-by: Thane Thomson <[email protected]> * Sync root docs with main Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> Co-authored-by: Thane Thomson <[email protected]> * add separated runs by UUID (backport #9367) (#9380) * add separated runs by UUID (#9367) This _should_ be the last piece needed for this tool. This allows the tool to generate reports on multiple experimental runs that may have been performed against the same chain. The `load` tool has been updated to generate a `UUID` on startup to uniquely identify each experimental run. The `report` tool separates all of the results it reads by `UUID` and performs separate calculations for each discovered experiment. Sample output is as follows ``` Experiment ID: 6bd7d1e8-d82c-4dbe-a1b3-40ab99e4fa30 Connections: 1 Rate: 1000 Size: 1024 Total Valid Tx: 9000 Total Negative Latencies: 0 Minimum Latency: 86.632837ms Maximum Latency: 1.151089602s Average Latency: 813.759361ms Standard Deviation: 225.189977ms Experiment ID: 453960af-6295-4282-aed6-367fc17c0de0 Connections: 1 Rate: 1000 Size: 1024 Total Valid Tx: 9000 Total Negative Latencies: 0 Minimum Latency: 79.312992ms Maximum Latency: 1.162446243s Average Latency: 422.755139ms Standard Deviation: 241.832475ms Total Invalid Tx: 0 ``` closes: #9352 #### PR checklist - [ ] Tests written/updated, or no tests needed - [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [ ] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed (cherry picked from commit 1067ba15719b89a74c89bcbec065062d2d0159d8) # Conflicts: # go.mod * fix merge conflict * fix lint Co-authored-by: William Banfield <[email protected]> Co-authored-by: William Banfield <[email protected]> * print all versions of tendermint and its sub protocols (#9329) (#9387) (cherry picked from commit ffce25327386bdc8d311a47609b7722146b0e91d) Co-authored-by: Marko <[email protected]> * Add redirect link for tutorial (backport #9385) (#9390) * add redirect links (#9385) (cherry picked from commit 43ebbed9c299324a54eb5d562cf6f50a6d2cf65f) # Conflicts: # docs/.vuepress/redirects * Fix conflicts Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> Co-authored-by: samricotta <[email protected]> Co-authored-by: Thane Thomson <[email protected]> * test: generate uuid on startup for load tool (#9383) (#9393) the `NewClient` method is called by the load test framework for each connection. This means that if multiple connections are instantiated, each connection will erroneously have its own UUID. This PR changes the UUID generation to happen at the _beginning_ of the script instead of on client creation so that each experimental run shares a UUID. Caught while preparing the script for production readiness. #### PR checklist - [ ] Tests written/updated, or no tests needed - [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [ ] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed (cherry picked from commit 59a711eabe90e91f91d61bc7b8fc0fab8b88d89c) Co-authored-by: William Banfield <[email protected]> * build(deps): Bump github.com/spf13/viper from 1.12.0 to 1.13.0 (#9402) * feat: support HTTPS inside websocket (backport #9416) (#9423) * state: restore previous error message (#9435) (#9441) * build(deps): Bump github.com/bufbuild/buf from 1.7.0 to 1.8.0 (#9448) Bumps [github.com/bufbuild/buf](https://github.com/bufbuild/buf) from 1.7.0 to 1.8.0. - [Release notes](https://github.com/bufbuild/buf/releases) - [Changelog](https://github.com/bufbuild/buf/blob/main/CHANGELOG.md) - [Commits](https://github.com/bufbuild/buf/compare/v1.7.0...v1.8.0) --- updated-dependencies: - dependency-name: github.com/bufbuild/buf dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * build(deps): Bump bufbuild/buf-setup-action from 1.6.0 to 1.8.0 (#9449) Bumps [bufbuild/buf-setup-action](https://github.com/bufbuild/buf-setup-action) from 1.6.0 to 1.8.0. - [Release notes](https://github.com/bufbuild/buf-setup-action/releases) - [Commits](https://github.com/bufbuild/buf-setup-action/compare/v1.6.0...v1.8.0) --- updated-dependencies: - dependency-name: bufbuild/buf-setup-action dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Thane Thomson <[email protected]> * Sync Vote.Verify() in spec with implementation (#9466) (#9477) * config: Add missing storage section when generating config (backport #9483) (#9488) * config: Add missing storage section when generating config (#9483) (cherry picked from commit b7f1e1f218ffea7a164608037bc8e0bc8f57b37c) * Add pending changelog entry Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> Co-authored-by: Thane Thomson <[email protected]> * loadtime: add block time to the data point (backport #9484) (#9490) * loadtime: add block time to the data point (#9484) This pull request adds the block time as the unix time since the epoch to the `report` tool's csv output. ```csv ... a7a8b903-1136-4da1-97aa-d25da7b4094f,1614226790,1663707084905417366,4,200,1024 a7a8b903-1136-4da1-97aa-d25da7b4094f,1614196724,1663707084905417366,4,200,1024 a7a8b903-1136-4da1-97aa-d25da7b4094f,1613097336,1663707084905417366,4,200,1024 a7a8b903-1136-4da1-97aa-d25da7b4094f,1609365168,1663707084905417366,4,200,1024 a7a8b903-1136-4da1-97aa-d25da7b4094f,1617199169,1663707084905417366,4,200,1024 a7a8b903-1136-4da1-97aa-d25da7b4094f,1615197134,1663707084905417366,4,200,1024 a7a8b903-1136-4da1-97aa-d25da7b4094f,1610399447,1663707084905417366,4,200,1024 ... ``` #### PR checklist - [ ] Tests written/updated, or no tests needed - [ ] `CHANGELOG_PENDING.md` updated, or no changelog entry needed - [ ] Updated relevant documentation (`docs/`) and code comments, or no documentation updates needed (cherry picked from commit 5fe1a72416722f8045b863fa0c7c045de583b6a1) * lint fix Co-authored-by: William Banfield <[email protected]> Co-authored-by: William Banfield <[email protected]> * Update apps.md (#9461) typo: "later" to "latter" * build(deps): Bump actions/stale from 5 to 6 (#9492) Bumps [actions/stale](https://github.com/actions/stale) from 5 to 6. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](https://github.com/actions/stale/compare/v5...v6) --- updated-dependencies: - dependency-name: actions/stale dependency-type: direct:production update-type: version-update:semver-major ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Extend the load report tool to include transactions' hashes (backport #9509) (#9514) * Extend the load report tool to include transactions' hashes (#9509) * Add transaction hash to raw data * Add hash in formatted output * Cosmetic (cherry picked from commit cdd3479f20b15c7dab0c683e2d5dddb7e7b95721) # Conflicts: # test/loadtime/cmd/report/main.go * Resolve conflict * Appease linter Co-authored-by: Sergio Mena <[email protected]> * security/p2p: prevent peers who errored being added to the peer_set (backport #9500) (#9516) * security/p2p: prevent peers who errored being added to the peer_set (#9500) * Mark failed removal of peer to address security bug Co-authored-by: Callum Waters <[email protected]> (cherry picked from commit c0bdb2423acef508372a3750d0db3e0dd9982178) * Changelong entry and added missing functions for implementations of Peer Co-authored-by: Jasmina Malicevic <[email protected]> * indexer: move deduplication functionality purely to the kvindexer (backport #9473) (#9521) * blocksync: retry requests after timeout (backport #9518) (#9534) * blocksync: retry requests after timeout (#9518) * blocksync: retry requests after timeout * Minimize changes to re-send block request after timeout * TO REVERT: reduce queue capacity * Add reset * Revert "TO REVERT: reduce queue capacity" This reverts commit dd0fee56924c958bed2ab7733e1917eb88fb5957. * 30 seconds * don't reset the timer * Update blocksync/pool.go Co-authored-by: Callum Waters <[email protected]> Co-authored-by: Sergio Mena <[email protected]> Co-authored-by: Callum Waters <[email protected]> (cherry picked from commit a371b1e3a8ea7603ada20e21bd6b4d5bf9f664f2) * Add changelog entry Co-authored-by: William Banfield <[email protected]> Co-authored-by: Sergio Mena <[email protected]> * build(deps): Bump google.golang.org/grpc from 1.49.0 to 1.50.0 (#9527) Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from 1.49.0 to 1.50.0. - [Release notes](https://github.com/grpc/grpc-go/releases) - [Commits](https://github.com/grpc/grpc-go/compare/v1.49.0...v1.50.0) --- updated-dependencies: - dependency-name: google.golang.org/grpc dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * Fix TX payload for DO testnets (#9540) (#9543) * Added print * Fix unmarshall * Fix unmarshalling * Simplified steps to unmarshall * minor * Use 'encoding/hex' * Forget about C, this is Go! * gosec warning * Set maximum payload size * nosec annotation (cherry picked from commit b42c439776811a6fcab8e22fb97fb44cee5701b3) Co-authored-by: Sergio Mena <[email protected]> * QA Process report for v0.37.x (and baseline for v0.34.x) (backport #9499) (#9578) * QA Process report for v0.37.x (and baseline for v0.34.x) (#9499) * 1st version. 200 nodes. Missing rotating node * Small fixes * Addressed @jmalicevic's comment * Explain in method how to set the tmint version to test. Improve result section * 1st version of how to run the 'rotating node' testnet * Apply suggestions from @williambanfield Co-authored-by: William Banfield <[email protected]> * Addressed @williambanfield's comments * Added reference to Unix load metric * Added total TXs * Fixed some 'png's that got swapped. Excluded '.*-node-exporter' processes from memory plots * Report for rotating node * Adressed remaining comments from @williambanfield * Cosmetic * Addressed some of @thanethomson's comments * Re-executed the 200 node tests and updated the corresponding sections of the report * Ignore Python virtualenv directories Signed-off-by: Thane Thomson <[email protected]> * Add latency vs throughput script Signed-off-by: Thane Thomson <[email protected]> * Add README for latency vs throughput script Signed-off-by: Thane Thomson <[email protected]> * Fix local links to folders Signed-off-by: Thane Thomson <[email protected]> * v034: only have one level-1 heading Signed-off-by: Thane Thomson <[email protected]> * Adjust headings Signed-off-by: Thane Thomson <[email protected]> * v0.37.x: add links to issues/PRs Signed-off-by: Thane Thomson <[email protected]> * v0.37.x: add note about bug being present in v0.34 Signed-off-by: Thane Thomson <[email protected]> * method: adjust heading depths Signed-off-by: Thane Thomson <[email protected]> * Show data points on latency vs throughput plot Signed-off-by: Thane Thomson <[email protected]> * Add latency vs throughput plots Signed-off-by: Thane Thomson <[email protected]> * Correct mentioning of v0.34.21 and add heading Signed-off-by: Thane Thomson <[email protected]> * Refactor latency vs throughput script Update the latency vs throughput script to rather generate plots from the "raw" CSV output from the loadtime reporting tool as opposed to the separated CSV files from the experimental method. Also update the relevant documentation, and regenerate the images from the raw CSV data (resulting in pretty much the same plots as the previous ones). Signed-off-by: Thane Thomson <[email protected]> * Remove unused default duration const Signed-off-by: Thane Thomson <[email protected]> * Adjust experiment start time to be more accurate and re-plot latency vs throughput Signed-off-by: Thane Thomson <[email protected]> * Addressed @williambanfield's comments * Apply suggestions from code review Co-authored-by: William Banfield <[email protected]> * Apply suggestions from code review Co-authored-by: William Banfield <[email protected]> * scripts: Update latency vs throughput readme for clarity Signed-off-by: Thane Thomson <[email protected]> Signed-off-by: Thane Thomson <[email protected]> Co-authored-by: W…

evan-forbes commented Mar 31, 2021

View reviewed changes

types/shares_test.go Outdated Show resolved Hide resolved

liamsi reviewed Mar 31, 2021

View reviewed changes

types/block.go Outdated Show resolved Hide resolved

types/shares.go Show resolved Hide resolved

types/shares.go Outdated Show resolved Hide resolved

types/shares_test.go Outdated Show resolved Hide resolved

liamsi mentioned this pull request Mar 31, 2021

Reconsider serializing block data into shares celestiaorg/celestia-specs#152

Closed

evan-forbes added 2 commits March 31, 2021 14:25

start spec compliant share merging

494db15

refactor and finish unit testing

0d15df4

evan-forbes force-pushed the evan/merge-shares branch from 5a2e0b2 to 0d15df4 Compare April 1, 2021 03:56

evan-forbes added 4 commits March 31, 2021 22:57

whoops

3f0d808

linter gods

8535122

fix initial changes and use constants

8bae88b

use constant

cbb5d76

more polish

b523831

evan-forbes commented Apr 1, 2021

View reviewed changes

types/shares_test.go Outdated Show resolved Hide resolved

evan-forbes marked this pull request as ready for review April 1, 2021 04:30

evan-forbes requested a review from tac0turtle as a code owner April 1, 2021 04:30

evan-forbes requested review from liamsi and adlerjohn and removed request for tac0turtle April 1, 2021 04:30

evan-forbes self-assigned this Apr 1, 2021

evan-forbes commented Apr 1, 2021

View reviewed changes

types/shares.go Outdated Show resolved Hide resolved

liamsi reviewed Apr 1, 2021

View reviewed changes

evan-forbes and others added 3 commits April 1, 2021 08:11

docs fix

f0f38a6

Co-authored-by: Ismail Khoffi <[email protected]>

review feedback: docs and out of range panic protection

014a00e

review feedback: add panic protection from empty input

0896a3b

adlerjohn reviewed Apr 1, 2021

View reviewed changes

evan-forbes and others added 2 commits April 1, 2021 16:30

use constant instead of recalculating ShareSize

96eafc7

Co-authored-by: John Adler <[email protected]>

don't redeclare existing var

c3d897a

Co-authored-by: John Adler <[email protected]>

evan-forbes added 3 commits April 4, 2021 10:57

Merge branch 'master' into evan/merge-shares

632bb37

fix conflict

3ca2afa

revert unnecessary changes

0f930fd

liamsi reviewed Apr 5, 2021

View reviewed changes

evan-forbes and others added 8 commits April 5, 2021 10:11

review feedback: better docs

91c3989

Co-authored-by: Ismail Khoffi <[email protected]>

reivew feedback: add comment for safeLen

043812d

review feedback: remove unnecessay comments

76d7a4b

review feedback: split up share merging and splitting into their own …

1cb4030

…files

review feedback: more descriptive var names

a88db3b

fix accidental change

2aad8fd

add some constant docs

887aa08

spelling error

40a57c1

liamsi approved these changes Apr 5, 2021

View reviewed changes

evan-forbes merged commit 13575a6 into master Apr 6, 2021

evan-forbes deleted the evan/merge-shares branch April 6, 2021 03:51

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Spec compliant merge shares #261

Spec compliant merge shares #261

evan-forbes commented Mar 31, 2021 •

edited

Loading

liamsi left a comment

codecov-io commented Apr 1, 2021 •

edited

Loading

evan-forbes commented Apr 1, 2021

liamsi left a comment

liamsi commented Apr 1, 2021

adlerjohn Apr 1, 2021 •

edited

Loading

evan-forbes Apr 1, 2021 •

edited

Loading

adlerjohn Apr 2, 2021

adlerjohn Apr 2, 2021 •

edited

Loading

evan-forbes commented Apr 4, 2021 •

edited

Loading

liamsi left a comment •

edited

Loading

liamsi Apr 5, 2021

liamsi left a comment

Spec compliant merge shares #261

Spec compliant merge shares #261

Conversation

evan-forbes commented Mar 31, 2021 • edited Loading

Description

liamsi left a comment

Choose a reason for hiding this comment

codecov-io commented Apr 1, 2021 • edited Loading

Codecov Report

evan-forbes commented Apr 1, 2021

liamsi left a comment

Choose a reason for hiding this comment

liamsi commented Apr 1, 2021

adlerjohn Apr 1, 2021 • edited Loading

Choose a reason for hiding this comment

evan-forbes Apr 1, 2021 • edited Loading

Choose a reason for hiding this comment

adlerjohn Apr 2, 2021

Choose a reason for hiding this comment

adlerjohn Apr 2, 2021 • edited Loading

Choose a reason for hiding this comment

evan-forbes commented Apr 4, 2021 • edited Loading

liamsi left a comment • edited Loading

Choose a reason for hiding this comment

liamsi Apr 5, 2021

Choose a reason for hiding this comment

liamsi left a comment

Choose a reason for hiding this comment

evan-forbes commented Mar 31, 2021 •

edited

Loading

codecov-io commented Apr 1, 2021 •

edited

Loading

adlerjohn Apr 1, 2021 •

edited

Loading

evan-forbes Apr 1, 2021 •

edited

Loading

adlerjohn Apr 2, 2021 •

edited

Loading

evan-forbes commented Apr 4, 2021 •

edited

Loading

liamsi left a comment •

edited

Loading