feat!: change CompileAPI to ecosystem based compilers [APE-1322] #1626

bilbeyt · 2023-08-25T07:48:25Z

What I did

Changed CompilerAPI so that we can select which compilers to be used in compilation process, so same extensions can be used as plugin as long as you dont use them at the same time for ecosystem configurations. .build/ folder now generates summary per ecosystem.

fixes: #

How to verify it

Run tests.

Checklist

All changes are completed
New test cases have been added
Documentation has been updated

fubuloubu · 2023-08-25T13:41:10Z

Maintainer note: Changed name to include exclamation point to show that it's a breaking change, scheduled for v0.7

fubuloubu · 2023-08-25T13:44:11Z

src/ape/api/projects.py

@@ -153,7 +153,8 @@ def contracts(self) -> Dict[str, ContractType]:

    @property
    def _cache_folder(self) -> Path:
-        folder = self.contracts_folder.parent / ".build"
+        current_ecosystem = self.network_manager.network.ecosystem.name
+        folder = self.contracts_folder.parent / ".build" / current_ecosystem


Hmm, so with this change it would have to rebuild for every different ecosystem

On the one hand, this may actually be pretty useful for networks like arbitrum that don't support newer VM rules yet (current suggestion is downgrade VM rules to Paris), but would need a way for the ecosystem to basically declare it's okay with another ecosystem type's VM rules

Also, I believe ethPM-types will have to change to allow compilation artifacts for multiple ecosystems

what is needed for ethpm-types ?

Currently the data structure is ContractType.name -> ContractType`, but doesn't have a way to delineate between different productions of bytecode using different VM rules/compiler config

Should be something more like ContractType.*bytecode fields should allow multiple bytecode productions, indexed by compiler settings e.g. ContractType.runtimeBytecode[Compiler.name] -> Bytecode
https://github.com/ApeWorX/ethpm-types/blob/main/ethpm_types/contract_type.py#L79

This may take some thinking

but do we really need this? now if we dont use duplicate plugins per ecosystem, lets say for ethereum, if we dont have solidity and zksolc then contract types shouldnt have any problem in my opinion.

The other option is back to what I said before, there's a .build/{ecosystem}/ folder that the ecosystem plugin creates mirror copies of the types that actually end up in the published/cached contract manifest, and those mirrors are kept in sync every time you try to do something on that ecosystem. since, at least in this instance, the goal is that you are using the same source code for that ecosystem, it's not really necessary to have the transpiled code tracked alongside the EVM code (or add it as an additional field to ethpm_types.ContractType, which should accept additional properties not defined in the model)

I couldnt really understand this. Manifest and abi files will be synced as we have network option and usage now. Can you give more details on this? What should i do?

Pulling this piece out for 0.7.0.
#1742

src/ape/managers/compilers.py

tests/integration/cli/projects/with-contracts/ape-config.yaml

github-actions · 2023-10-06T01:48:59Z

This pull request is considered stale because it has been open 30 days with no activity. Remove stale label, add a comment, or make a new commit, otherwise this PR will be closed in 5 days.

github-actions · 2023-10-12T01:48:13Z

This PR was closed because it has been inactive for 35 days.

github-actions · 2023-11-12T01:50:20Z

This pull request is considered stale because it has been open 30 days with no activity. Remove stale label, add a comment, or make a new commit, otherwise this PR will be closed in 5 days.

NotPeopling2day · 2023-11-17T12:26:40Z

.github/workflows/test.yaml

@@ -95,7 +95,7 @@ jobs:
            pip install .[test]

        - name: Run Tests
-          run: pytest -m "not fuzzing" -s --cov=src -n auto --dist loadscope
+          run: ape test -m "not fuzzing" -s --cov=src --dist loadscope


Please revert back to pytest, ape test is used for running ape projects not the framework or plugins.

NotPeopling2day · 2023-11-17T12:26:58Z

.github/workflows/test.yaml

@@ -117,4 +117,4 @@ jobs:
            pip install .[test]

        - name: Run Tests
-          run: pytest -m "fuzzing" --no-cov -s
+          run: ape test -m "fuzzing" --no-cov -s


Here as well, back to pytest

NotPeopling2day · 2023-11-17T13:46:41Z

src/ape/api/projects.py

@@ -153,7 +153,8 @@ def contracts(self) -> Dict[str, ContractType]:

    @property
    def _cache_folder(self) -> Path:
-        folder = self.contracts_folder.parent / ".build"
+        current_ecosystem = self.network_manager.network.ecosystem.name
+        folder = self.contracts_folder.parent / ".build" / current_ecosystem


Pulling this piece out for 0.7.0.
#1742

antazoey · 2023-11-18T11:27:29Z

closing as it seems we want to go in a different direction (modifying ethpm-types and using a single manifest)

vany365 changed the title ~~Change CompileAPI to ecosystem based compilers~~ Change CompileAPI to ecosystem based compilers [APE-1322] Aug 25, 2023

bilbeyt force-pushed the compiler_changes branch from 656ce9c to 739d2bd Compare August 25, 2023 07:49

bilbeyt changed the title ~~Change CompileAPI to ecosystem based compilers [APE-1322]~~ feat: change CompileAPI to ecosystem based compilers [APE-1322] Aug 25, 2023

bilbeyt force-pushed the compiler_changes branch 4 times, most recently from 8165591 to 5ba6a4d Compare August 25, 2023 08:40

fubuloubu changed the title ~~feat: change CompileAPI to ecosystem based compilers [APE-1322]~~ feat!: change CompileAPI to ecosystem based compilers [APE-1322] Aug 25, 2023

fubuloubu reviewed Aug 25, 2023

View reviewed changes

antazoey reviewed Aug 25, 2023

View reviewed changes

src/ape/managers/compilers.py Outdated Show resolved Hide resolved

tests/integration/cli/projects/with-contracts/ape-config.yaml Outdated Show resolved Hide resolved

antazoey force-pushed the main branch from 45b0779 to 6bdce6a Compare August 28, 2023 21:05

bilbeyt force-pushed the compiler_changes branch 6 times, most recently from 241b6ef to a78463d Compare August 30, 2023 11:18

bilbeyt added 3 commits August 30, 2023 13:19

feat: change CompileAPI to ecosystem based compilers

d06a7ba

tests: adapt tests for new compilerapi

b2f92cf

ci: disable multiprocess testing

c828c4a

bilbeyt force-pushed the compiler_changes branch from a78463d to c828c4a Compare August 30, 2023 11:19

github-actions bot added the stale No activity for 30 days label Oct 6, 2023

github-actions bot added the inactive no recent activity, closed label Oct 12, 2023

github-actions bot closed this Oct 12, 2023

antazoey reopened this Oct 12, 2023

github-actions bot removed inactive no recent activity, closed stale No activity for 30 days labels Oct 13, 2023

github-actions bot added the stale No activity for 30 days label Nov 12, 2023

NotPeopling2day removed the stale No activity for 30 days label Nov 13, 2023

NotPeopling2day reviewed Nov 17, 2023

View reviewed changes

antazoey closed this Nov 18, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat!: change CompileAPI to ecosystem based compilers [APE-1322] #1626

feat!: change CompileAPI to ecosystem based compilers [APE-1322] #1626

bilbeyt commented Aug 25, 2023

fubuloubu commented Aug 25, 2023

fubuloubu Aug 25, 2023

bilbeyt Aug 29, 2023

fubuloubu Aug 29, 2023

bilbeyt Aug 31, 2023

fubuloubu Aug 31, 2023

bilbeyt Sep 4, 2023

NotPeopling2day Nov 17, 2023

github-actions bot commented Oct 6, 2023

github-actions bot commented Oct 12, 2023

github-actions bot commented Nov 12, 2023

NotPeopling2day Nov 17, 2023

NotPeopling2day Nov 17, 2023

NotPeopling2day Nov 17, 2023

antazoey commented Nov 18, 2023

feat!: change CompileAPI to ecosystem based compilers [APE-1322] #1626

feat!: change CompileAPI to ecosystem based compilers [APE-1322] #1626

Conversation

bilbeyt commented Aug 25, 2023

What I did

How to verify it

Checklist

fubuloubu commented Aug 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Oct 6, 2023

github-actions bot commented Oct 12, 2023

github-actions bot commented Nov 12, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

antazoey commented Nov 18, 2023