Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature] #2373: kagami swarm #3475

Merged
merged 39 commits into from
Jun 8, 2023
Merged

[feature] #2373: kagami swarm #3475

merged 39 commits into from
Jun 8, 2023

Conversation

0x009922
Copy link
Contributor

@0x009922 0x009922 commented May 12, 2023

Description

This PR introduces kagami swarm command and some related refactoring and fixes.

This command allows to generate a directory with everything needed to run docker compose up in it. It allows to specify variable number of peers and source of the images.

Examples:

  • Generate docker-compose configuration in ./compose-iroha directory with 2 peers using GitHub source:

    kagami swarm --build-from-github --peers 2 --outdir ./compose-iroha

    Generated files:

    [1.1M]  ./compose-iroha
    ├── [761K]  config
    │   ├── [2.1K]  config.json
    │   ├── [ 13K]  genesis.json
    │   └── [746K]  validator.wasm
    ├── [2.1K]  docker-compose.yml
    └── [323K]  iroha-cloned
        └── ...
    
    docker-compose.yml
    version: '3.8'
    services:
      iroha0:
        build: ./iroha-cloned
        environment:
          IROHA_PUBLIC_KEY: ed0120F958FCF0FCD15BD3A3C50F5A3DA9DF97D1B49D9BD6E482A109AE8082BCC54198
          IROHA_PRIVATE_KEY: '{"digest_function":"ed25519","payload":"a9ba80952c6f1a87e1039931c5355c026f0e6d2eff3ab5ba38c4411933c5226df958fcf0fcd15bd3a3c50f5a3da9df97d1b49d9bd6e482a109ae8082bcc54198"}'
          TORII_P2P_ADDR: iroha0:1337
          TORII_API_URL: iroha0:8080
          TORII_TELEMETRY_URL: iroha0:8180
          IROHA_GENESIS_ACCOUNT_PUBLIC_KEY: ed01200D80056A4CB7E32F6E0593C8708C03340D3D1E7F1050E635DBE0A821C236E8E4
          IROHA_GENESIS_ACCOUNT_PRIVATE_KEY: '{"digest_function":"ed25519","payload":"3e51c0c06622a95a27d4936d706318fc7430c652136d5348d55db0d47835783c0d80056a4cb7e32f6e0593c8708c03340d3d1e7f1050e635dbe0a821c236e8e4"}'
          SUMERAGI_TRUSTED_PEERS: '[{"address":"iroha1:1338","public_key":"ed0120CFD105A9754B91389D726A3A3CA38CC8002562F7F9D54DB8F4AA5C6ABCFDE558"}]'
        ports:
        - 1337:1337
        - 8080:8080
        - 8180:8180
        volumes:
        - ./config:/config
        init: true
        command: iroha --submit-genesis
      iroha1:
        build: ./iroha-cloned
        environment:
          IROHA_PUBLIC_KEY: ed0120CFD105A9754B91389D726A3A3CA38CC8002562F7F9D54DB8F4AA5C6ABCFDE558
          IROHA_PRIVATE_KEY: '{"digest_function":"ed25519","payload":"28a79c99529fb3375ed8a0663a7a98ea8349aa7179c1c4dbab255e21fab9442bcfd105a9754b91389d726a3a3ca38cc8002562f7f9d54db8f4aa5c6abcfde558"}'
          TORII_P2P_ADDR: iroha1:1338
          TORII_API_URL: iroha1:8081
          TORII_TELEMETRY_URL: iroha1:8181
          IROHA_GENESIS_ACCOUNT_PUBLIC_KEY: ed01200D80056A4CB7E32F6E0593C8708C03340D3D1E7F1050E635DBE0A821C236E8E4
          IROHA_GENESIS_ACCOUNT_PRIVATE_KEY: '{"digest_function":"ed25519","payload":"3e51c0c06622a95a27d4936d706318fc7430c652136d5348d55db0d47835783c0d80056a4cb7e32f6e0593c8708c03340d3d1e7f1050e635dbe0a821c236e8e4"}'
          SUMERAGI_TRUSTED_PEERS: '[{"address":"iroha0:1337","public_key":"ed0120F958FCF0FCD15BD3A3C50F5A3DA9DF97D1B49D9BD6E482A109AE8082BCC54198"}]'
        ports:
        - 1338:1338
        - 8081:8081
        - 8181:8181
        volumes:
        - ./config:/config
        init: true
  • Generated docker-compose configuration based on a locally installed Iroha, with a single peer and without default
    configuration:

    kagami swarm --build ~/Git/iroha --peers 1 --outdir ~/compose-iroha --no-default-configuration

    Generated files:

    [1.1K]  ~/compose-iroha
    ├── [  64]  config
    └── [ 980]  docker-compose.yml
    
    docker-compose.yml`
    version: '3.8'
    services:
      iroha0:
        build: ../Git/iroha
        environment:
          IROHA_PUBLIC_KEY: ed01208A880AB596D8368C3319230978E4292285791865A5E7DDE9415BD48B8F0CBC3C
          IROHA_PRIVATE_KEY: '{"digest_function":"ed25519","payload":"2689e60e1321ac59050e38113908b66dda9975c3da4659c552e10de4f1a1fbe58a880ab596d8368c3319230978e4292285791865a5e7dde9415bd48b8f0cbc3c"}'
          TORII_P2P_ADDR: iroha0:1337
          TORII_API_URL: iroha0:8080
          TORII_TELEMETRY_URL: iroha0:8180
          IROHA_GENESIS_ACCOUNT_PUBLIC_KEY: ed0120CD1EE423ECB2D0A5B26C99CE4326C9D17CE79B38863751646D661759A0E5E625
          IROHA_GENESIS_ACCOUNT_PRIVATE_KEY: '{"digest_function":"ed25519","payload":"fdc2b6f906307b37c0c67a92f7a32ea41a92088da060b710f62718dc00fbd4c0cd1ee423ecb2d0a5b26c99ce4326c9d17ce79b38863751646d661759a0e5e625"}'
          SUMERAGI_TRUSTED_PEERS: '[]'
        ports:
          - 1337:1337
          - 8080:8080
          - 8180:8180
        volumes:
          - ./config:/config
        init: true
        command: iroha --submit-genesis

Linked issue

Closes #2373

Checklist

  • Unit tests
  • Try with docker-compose up
  • Update doc comments
  • Self review
  • Peers review

To Do in separate PRs

@0x009922 0x009922 self-assigned this May 12, 2023
@github-actions github-actions bot added the iroha2-dev The re-implementation of a BFT hyperledger in RUST label May 12, 2023
@appetrosyan appetrosyan self-assigned this May 15, 2023
@ilchu ilchu self-assigned this May 15, 2023
@0x009922 0x009922 marked this pull request as ready for review May 22, 2023 00:19
@codecov
Copy link

codecov bot commented May 22, 2023

Codecov Report

Merging #3475 (f08c12b) into iroha2-dev (a4d5c9f) will decrease coverage by 5.55%.
The diff coverage is 32.86%.

❗ Current head f08c12b differs from pull request most recent head 702ce69. Consider uploading reports for the commit 702ce69 to get more accurate results

@@              Coverage Diff               @@
##           iroha2-dev    #3475      +/-   ##
==============================================
- Coverage       62.33%   56.78%   -5.55%     
==============================================
  Files             169      161       -8     
  Lines           31218    32249    +1031     
==============================================
- Hits            19459    18312    -1147     
- Misses          11759    13937    +2178     
Impacted Files Coverage Δ
cli/src/lib.rs 0.56% <0.00%> (-68.87%) ⬇️
cli/src/main.rs 0.66% <0.00%> (-0.44%) ⬇️
cli/src/samples.rs 0.00% <0.00%> (-61.85%) ⬇️
cli/src/style.rs 0.00% <0.00%> (ø)
cli/src/torii/mod.rs 0.00% <0.00%> (-27.66%) ⬇️
cli/src/torii/routing.rs 0.00% <0.00%> (-57.11%) ⬇️
cli/src/torii/utils.rs 0.00% <0.00%> (-84.85%) ⬇️
client/src/http_default.rs 0.00% <0.00%> (-38.94%) ⬇️
client/src/lib.rs 3.84% <0.00%> (-96.16%) ⬇️
client_cli/src/main.rs 0.24% <0.00%> (-0.02%) ⬇️
... and 105 more

... and 50 files with indirect coverage changes

config/base/src/lib.rs Outdated Show resolved Hide resolved
config/base/derive/src/proxy.rs Outdated Show resolved Hide resolved
config/base/src/lib.rs Outdated Show resolved Hide resolved
config/src/iroha.rs Outdated Show resolved Hide resolved
config/src/sumeragi.rs Show resolved Hide resolved
core/test_network/src/lib.rs Outdated Show resolved Hide resolved
tools/kagami/src/swarm.rs Outdated Show resolved Hide resolved
tools/kagami/src/swarm.rs Outdated Show resolved Hide resolved
tools/kagami/src/swarm.rs Outdated Show resolved Hide resolved
tools/kagami/src/swarm.rs Show resolved Hide resolved
tools/kagami/src/swarm.rs Show resolved Hide resolved
@0x009922 0x009922 requested a review from ilchu May 26, 2023 01:52
Copy link
Contributor

@ilchu ilchu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM besides one stylistic point, will approve after fix.

tools/kagami/src/swarm.rs Outdated Show resolved Hide resolved
ilchu
ilchu previously approved these changes May 31, 2023
appetrosyan
appetrosyan previously approved these changes Jun 7, 2023
0x009922 and others added 22 commits June 8, 2023 12:30
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Co-authored-by: Ilia Churin <[email protected]>
Signed-off-by: 0x009922 <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: Dmitry Balashov <[email protected]>
@0x009922 0x009922 dismissed stale reviews from appetrosyan and ilchu via 702ce69 June 8, 2023 03:32
@0x009922 0x009922 merged commit 5a62f17 into iroha2-dev Jun 8, 2023
@0x009922 0x009922 deleted the 2373-swarm branch June 8, 2023 08:10
mversic pushed a commit that referenced this pull request Oct 17, 2023
* [refactor]: bump `clap`, scaffold `swarm` subcommand

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: transform `algorithm` to `ValueEnum`

Signed-off-by: Dmitry Balashov <[email protected]>

* [feature]: implement basic flow

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: clear dead code

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: safer paths manipulations

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: sort generated services

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: update snapshot

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: skip empty command serialisation; chores

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: define consts

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: pass workspace check; lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chores

Signed-off-by: Dmitry Balashov <[email protected]>

* [feat]: impl basic UI

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: fix shallow git clone

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: update ui

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: simplify resolution reporting

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: doc comments, naming, etc

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: move `swarm` mod

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chores; update doc comments

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: fix config path in docker image

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: clippy lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: specify all trusted peers; set genesis key pair only for the genesis peer

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: add fixme

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: chores

Signed-off-by: Dmitry Balashov <[email protected]>

* [test]: fix `kagami` and `iroha_config_base` tests

Signed-off-by: Dmitry Balashov <[email protected]>

* [fix]: use `SocketAddr`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: allow any docker image; rename args

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: fix lints

Signed-off-by: Dmitry Balashov <[email protected]>

* Apply suggestions from code review

Co-authored-by: Ilia Churin <[email protected]>
Signed-off-by: 0x009922 <[email protected]>

* [chore]: remove extra doc

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: remove inaccurate comment

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `.dir()`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: derive `Display` for `AlgorithmArg`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: use `Error::FieldDeserialization`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: apply lints

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: move secp256k1 seed len check to `iroha_crypto`

Signed-off-by: Dmitry Balashov <[email protected]>

* [refactor]: expand `swarm::key_gen` module

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: fix format

Signed-off-by: Dmitry Balashov <[email protected]>

* [chore]: capitalise error messages

Signed-off-by: Dmitry Balashov <[email protected]>

---------

Signed-off-by: Dmitry Balashov <[email protected]>
Signed-off-by: 0x009922 <[email protected]>
Co-authored-by: Ilia Churin <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
iroha2-dev The re-implementation of a BFT hyperledger in RUST
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants