Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ion sync upstream #2

Merged
merged 63 commits into from
Jul 9, 2024
Merged

Ion sync upstream #2

merged 63 commits into from
Jul 9, 2024

Conversation

ice-proteus
Copy link

No description provided.

neodix42 and others added 30 commits March 12, 2024 19:11
@neodix42
Add standalone emulator library to release (ton-blockchain#917)
79a75d5 
* Add standalone emulator.so/dylyib/dll to release artifacts

* sync emulator names
@neodix42 @xqueezeme
Improve windows build (ton-blockchain#921)
4969176 
* fix windows build

* exclude test-bigint execution on windows

---------

Co-authored-by: My Name <[email protected]>
@EmelyanenkoK @SpyCheese
Fix running VM in liteserver and tonlib (ton-blockchain#933)
fc69377 
Co-authored-by: SpyCheese <[email protected]>
@dungeon-master-666 @EmelyanenkoK
Fix linker error: undefined reference to LiteServerCacheImpl::MAX_CAC…
69de1cb 
…HE_SIZE (ton-blockchain#930)

Co-authored-by: EmelyanenkoK <[email protected]>
@EmelyanenkoK @dungeon-master-666
Tonlib improvement fixes (ton-blockchain#934)
f159264 
* 3.6. Fix directory traversal in KeyValueDir

* 3.9. Fix buffer_to_hex reversing nibbles

* 3.5. Fix error handling at blocks.getBlockHeader

* 3.11. Fix query.forget

* 3.12. Fix error handling in RemoteRunSmcMethod

* 4.1. Delete unused files

* 3.10. Use named constants instead hardcoded constants

* 3.4. Fix response block header verification

* 3.1. Check proof of blocks.getShards response

* fix td::buffer_to_hex + test

* 3.2. Add proof check for listBlockTransactions response in RunEmulator actor

* 3.8. Add proof checking for getLibraries method

* fix regression tests

* 3.3 Add proof checking for lookupBlock method

* Add publishers to proof of getLibrariesWithProof response  (ton-blockchain#25)

* fix missing return, fix requesting mc block

* Fix requesting lookupBlock with client mc blk == mc ref block

* Fix duplicating lib data in proof and data, add mode 2 for not including the data

* Migration of LastBlockStorage with fixed td::buffer_to_hex

---------

Co-authored-by: ms <[email protected]>
Co-authored-by: Marat <[email protected]>
@EmelyanenkoK @SpyCheese
Don't serialize expired persistent states (ton-blockchain#935)
0a82a19 
Co-authored-by: SpyCheese <[email protected]>
@dungeon-master-666
Build fix: 'all_of' identifier not found (ton-blockchain#936)
14f319d
@EmelyanenkoK
Merge pull request ton-blockchain#937 from ton-blockchain/master
bf9848c 
Merge master
@dungeon-master-666
Save rocksdb statistics to file every minute (ton-blockchain#932)
7a6bfa7 
* Save rocksdb statistics to file every minute

* Add flag to disable collecting rocksdb statistics
@dungeon-master-666
Mode 644 for db_stats.txt (ton-blockchain#938)
0feaaf5
@EmelyanenkoK @SpyCheese
LS queries to nonfinal blocks (ton-blockchain#941)
9452c36 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Exp/compress candidates (ton-blockchain#942)
0bcebe8 
* Compress block candidates in validator-session

* Compress blocks in full-node (disabled for now)

---------

Co-authored-by: SpyCheese <[email protected]>
@iliavy
Fix typo in variable name (ton-blockchain#940)
43ad994
@omahs
Fix typos (ton-blockchain#926)
438e59a 
* fix typos

* fix typo
@EmelyanenkoK @SpyCheese
liteServer.getOutMsgQueueSizes method (ton-blockchain#943)
b076143 
Co-authored-by: SpyCheese <[email protected]>
@dungeon-master-666
Collect statistics for .pack files (ton-blockchain#944)
10487b1 
* Statistics for .pack files

* optimizations

* fix typo

* fix erasing packages
@neodix42 @SpyCheese
lz4 support in multiple builds (ton-blockchain#946)
b784924 
* Compress block candidates in validator-session

* Compress blocks in full-node (disabled for now)

* test pipeline with lz4

* tonlib compilation required lz4;
try win compile;

* install lz4 on mac.

* wip, test builds

* remove FindLZ4.cmake

* fix typo

* fix wasm lz4 path

* increase groovy timeout to 120 sec

* add lz4 for android and emscripten builds

* add lz4 for android and emscripten builds

* fix win build include path for lz4

* add precompiled lz4 for android

* cleanup

* adjust android include dir for lz4

* fix path for android arm of lz4

* cleanup

* minor fix

---------

Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK
Update recent_changelog.md for April2024 update
576cf9d
@dungeon-master-666
Check pack_statistics for nullptr (ton-blockchain#950)
46ca0e6
@EmelyanenkoK @SpyCheese
Add custom overlays for external messages (ton-blockchain#949)
0434ead 
* Private overlay for external messages

* Improve ext msg overlays

* Manage from validator console
* Bypass out queue size limit for high-priority messages
* Shuffle messages in get_external_messages

* Cleanup mempool when creating validator group

* Improve private overlays for externals

1. Allow using validator adnl ids in addition to fullnode ids
2. Set priority per sender, not per overlay
3. Require the same overlay name for all nodes
4. Enable lz4 in private block overlay

* Fix typo, add debug logs

* Enable lz4 in private block overlay by config

Change proto_version for lz4 in catchain overlays to 4

* Add logs for broadcasts in fullnode

---------

Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Rename private ext msg overlays to custom overlays, change interface …
f7907bd 
…a bit (ton-blockchain#952)

Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Set temp packages ttl to 1h (ton-blockchain#953)
8390d88 
This leaves only 3 last temp packages

Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Fix setting inited_ in FullNodeCustomOverlay (ton-blockchain#954)
cc4244f 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Fix setting due_payment in storage phase (ton-blockchain#957)
b8111d8 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Fix prepare_vm_c7 (ton-blockchain#958)
a2bd695 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK
Merge pull request ton-blockchain#959 from ton-blockchain/testnet
8acf37c 
Merge developer branch
@EmelyanenkoK
Add changelog for April update
2b45b09
@neodix42
add usage of liblz4-dev to Dockerfile (ton-blockchain#960)
c073560
@EmelyanenkoK
Merge pull request ton-blockchain#961 from ton-blockchain/testnet
4cfe1d1 
Merge developer branch
@EmelyanenkoK @SpyCheese
Use shard client timestamp to determine expiration of temp archives (t…
190aa6b 
…on-blockchain#965)

Co-authored-by: SpyCheese <[email protected]>
EmelyanenkoK and others added 19 commits May 14, 2024 17:11
@EmelyanenkoK @SpyCheese
Blockchain explorer: use liteServer.runSmcMethod (ton-blockchain#994)
3a802fa 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Change default values for celldb flags (ton-blockchain#996)
3827409 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Automatically disable celldb direct I/O if cache is small (ton-blockc…
7a74888 
…hain#997)

Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Add candidates cache (ton-blockchain#1000)
539d5dd 
* Broadcast shardchain block candidates in private overlays, generate proof links from candidates

* Disable shardchain block broadcasts in private overlays

* Send block candidate broadcasts to custom overlays, allow non-validators to receive candidates

---------

Co-authored-by: SpyCheese <[email protected]>
@neodix42 @EmelyanenkoK
Use jemalloc (ton-blockchain#987)
db505f4 
* use jemalloc

* install system jemalloc in github action

* check if jemalloc is actually used

---------

Co-authored-by: neodiX <[email protected]>
@EmelyanenkoK @SpyCheese
Fix choosing neighbours in private overlays (ton-blockchain#1004)
d80ce8d 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Limit checked external messages per address (ton-blockchain#1005)
ceefac7 
* Limit checked external messages per address

* Change max_ext_msg_per_addr_time_window; cleanup mempool by timer

---------

Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Compress block candidate broadcasts (ton-blockchain#1007)
8a4d44d 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Fix TL scheme for liteServer.nonfinal.getValidatorGroups (ton-blockch…
229d6a8 
…ain#1008)

Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK @SpyCheese
Command to disable state serializer (ton-blockchain#1011)
74801d0 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK
Rollback celldb default flag values and decrease default TTLs (ton-bl…
3d7e85b 
…ockchain#1012)

* Rollback celldb default flag values and decrease default TTLs

* Fix description
@EmelyanenkoK @SpyCheese
Fix loading key block config in FullNodeImpl (ton-blockchain#1013)
0301e9f 
Co-authored-by: SpyCheese <[email protected]>
@EmelyanenkoK
Update validator list on each keyblock
f9b6d21
@neodix42
Change fift path separator for FIFTPATH and -I argument (ton-blockcha…
5186c47 
…in#1014)

* add github action for macOS 14 (arm64, M1)

* add github action (portable) for macOS 14 (arm64, M1)

* rename macOS arm64 output artifact

* Colon cannot be used as a path separator for FIFTPATH or -I argument in fift on Windows when absolute paths are used (e.g. C:\path\lib:C:\path\smartcont).
Suggestion to use @ as a new path separator on Windows.

---------

Co-authored-by: neodiX <[email protected]>
@neodix42
Use jemalloc in portable artifacts (ton-blockchain#1003)
0a0a92c 
* use jemalloc in portable builds;
also avoid mixing musl with glibc artifacts in nix builds;

* minor fix for ubuntu arm nix build
@neodix42
Add github actions for macOS 14 (arm64, shared and portable) (ton-blo…
ec00ccc 
…ckchain#1001)

* add github action for macOS 14 (arm64, M1)

* add github action (portable) for macOS 14 (arm64, M1)

* rename macOS arm64 output artifact
@EmelyanenkoK
Add changelog
7841d75
@EmelyanenkoK
Merge pull request ton-blockchain#1016 from ton-blockchain/testnet
5c392e0 
Merge developer branch
@ice-hermes @ice-proteus
rebranding ton to ion
8d66fac
@ice-proteus ice-proteus requested a review from ice-hermes June 14, 2024 07:44
@ice-charon
Copy link
Collaborator

@ice-proteus can you resolve conflicts in the hidden folder .github on top level branch.

@ice-proteus
Copy link
Author

ice-proteus commented Jun 20, 2024 via email

@ice-charon
Copy link
Collaborator

ice-charon commented Jun 20, 2024 •
edited
Loading

Hey @ice-proteus you can see below review requested there is running tests, below them there is unresolved conflicts.
Usually you have to fetch master and merge it over your branch

git fetch origin master
git merge FETCH_HEAD

Then you have to force push changes.
You can experimenting, if you have problems I can do it for you.
Also can you change this line https://github.com/ice-blockchain/ion/blob/master/crypto/CMakeLists.txt#L565 to

share/ion

That's why 4 tests are unsuccessful.

@ice-proteus
Copy link
Author

ice-proteus commented Jun 21, 2024 via email

@ice-charon
Copy link
Collaborator

Ok, I see. I changed the line from crypto/CMakeLists.txt to share/ion. I see that the conflicts from the .github folder involve some extra libraries. Resolving the conflicts would mean keeping these extra libraries?

Correct.

Windows build is broken because of MSVC is version 19.40.33811.0 which based on this calculation
https://github.com/ice-blockchain/ion/blob/master/CMake/FindSodium.cmake#L128-L133
is 1940 / 10 - 50 = 144 but in libsodium there is no v144 directory, there is v142 and v143 so we need a one version older Windows machine, @ice-hermes

@ice-hermes ice-hermes force-pushed the ion-sync-upstream branch from d00a523 to 14a5451 Compare July 9, 2024 07:47
@ice-hermes ice-hermes merged commit 2366e24 into master Jul 9, 2024
10 of 11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ice-hermes ice-hermes Awaiting requested review from ice-hermes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@ice-proteus @ice-charon @ice-hermes @neodix42 @EmelyanenkoK @dungeon-master-666 @iliavy @omahs