Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core/vm: implement full EOF suite #6382

Draft
wants to merge 162 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 111 commits
Commits
Show all changes
162 commits
Select commit Hold shift + click to select a range
efa68c4
core/vm: add eof container
lightclient Dec 6, 2022
a3ad895
Fix compilation
yperbasis Dec 19, 2022
59d3209
core/vm: add eof execution semantics
lightclient Dec 8, 2022
7549553
GasFastishStep -> GasSwiftStep
yperbasis Dec 20, 2022
b1b4382
PC is still valid in EOF
yperbasis Dec 20, 2022
8ab6795
Merge branch 'devel' into eof
yperbasis Dec 20, 2022
abc43bc
binary.BigEndian.Uint16 instead of parseUint16
yperbasis Dec 20, 2022
f911a1d
Merge branch 'devel' into eof
yperbasis Dec 21, 2022
c124cff
Merge branch 'devel' into eof
yperbasis Dec 21, 2022
05419ea
Post-merge fix
yperbasis Dec 21, 2022
a2127ef
Revert "PC is still valid in EOF"
yperbasis Dec 23, 2022
d53b419
Merge branch 'devel' into eof
yperbasis Dec 29, 2022
eada5ac
core/vm: add eof code validation
lightclient Dec 22, 2022
5c56bc2
Merge branch 'devel' into eof
yperbasis Dec 30, 2022
1aa0a3e
core/vm: properly validated deployed code if eof
lightclient Dec 29, 2022
260aeea
Use hasEOFByte
yperbasis Dec 30, 2022
e301fe6
Merge branch 'devel' into eof
yperbasis Dec 30, 2022
f01fcf3
core/vm: fix relative jump destination validation
lightclient Dec 29, 2022
8bdd69a
core/vm: do code validation on eof
lightclient Dec 29, 2022
c1c226b
Move jump tables to Config
yperbasis Dec 30, 2022
67a18be
core/vm: fix linter and account for overflowing rjumpv case
lightclient Dec 29, 2022
609ab11
core/vm: add tests for deploying eof containers
lightclient Dec 29, 2022
edd7dd7
core/vm: disallow eof deploying legacy code
lightclient Dec 29, 2022
83aa4da
core/vm: parse relative args as ints in validation
lightclient Dec 30, 2022
16770c6
Generic Min/Max
yperbasis Dec 30, 2022
c465602
Merge branch 'devel' into eof
yperbasis Dec 30, 2022
55486c2
core/vm: add invalid to jump table
lightclient Dec 30, 2022
4bd4986
core/vm: reorder stack height checks in eof validation
lightclient Dec 30, 2022
aa09b84
core/vm: follow rjumpv branches correctly
lightclient Dec 30, 2022
d5f21e1
Merge branch 'devel' into eof
yperbasis Jan 2, 2023
4b153f1
core/vm: make eof parser more paranoid
holiman Dec 30, 2022
21c78f6
core/vm: fix on eof
holiman Dec 30, 2022
6a01a12
core/vm: fix uint16 overflow
holiman Dec 30, 2022
fc5d555
core/vm: tests for codeBitmap/eofCodeBitmap
holiman Dec 30, 2022
37c4da3
core/vm: bound check rjumpv before code analysis
lightclient Dec 31, 2022
065f27b
core/vm: fix fuzzer crashes
lightclient Dec 31, 2022
0661575
core/vm: more fixes
lightclient Dec 31, 2022
cfc44a4
core/vm: more fixes
lightclient Dec 31, 2022
bd27294
core/vm: bound type section inputs and outputs to 127
lightclient Jan 1, 2023
d57f259
core/vm: exit stack validation on terminal op
lightclient Jan 2, 2023
8c51bce
core/vm: disallow legacy initcode deploying eof bytecode
lightclient Jan 2, 2023
09d25e6
core/vm: reduce max_stack_height limit to 1023
lightclient Jan 2, 2023
9e1f307
fix compilation
yperbasis Jan 2, 2023
a645765
core/vm: validate retf outputs
lightclient Jan 2, 2023
45f8c88
Switch to Min from erigon-lib
yperbasis Jan 2, 2023
55f3f52
core/vm: account for input stack height in max stack height calculation
lightclient Jan 2, 2023
0e44325
Remove JUMPF
yperbasis Jan 2, 2023
8adb290
Add new opcodes to stringToOp
yperbasis Jan 2, 2023
42f0f5a
core/vm: increment pos after callf in stack validation
lightclient Jan 2, 2023
abeb3bc
core/vm: flip comparison operator in stack validation for callf
lightclient Jan 2, 2023
44ddbef
core/vm: clean up eof unmarshaling errors plus a few other fixes
lightclient Jan 2, 2023
fc44b19
core/vm: account for callf return value in stack analysis and some ot…
lightclient Jan 3, 2023
5125803
Extract ErrInvalidMagic
yperbasis Jan 3, 2023
df33857
core/vm: fix lints
lightclient Jan 3, 2023
00de003
core/vm: clean up eof validation
lightclient Jan 3, 2023
a247ea7
core/vm: account for callf inputs and add some comments
lightclient Jan 3, 2023
a4d3e66
core/vm: ban eof code from running legacy initcode
lightclient Jan 4, 2023
07ebcaa
core/vm: uncomment analysis tests
lightclient Jan 4, 2023
c347ca7
core/vm: don't consume all gas if eof initcode validation fails
lightclient Jan 4, 2023
e5f5025
Revert "core/vm: disallow legacy initcode deploying eof bytecode"
yperbasis Jan 5, 2023
e5cb68b
core/vm: check for full magic in initcode
lightclient Jan 4, 2023
1b3e8f5
EOF postponed to Cancun
yperbasis Jan 5, 2023
817d7b5
Merge branch 'devel' into eof
yperbasis Jan 6, 2023
e7dce33
Merge branch 'devel' into eof
yperbasis Jan 11, 2023
0e00801
core/vm: clean stop after retf
lightclient Jan 6, 2023
1585d66
core/vm: add custom eof errors
lightclient Jan 6, 2023
8a0269c
core: reorder magic check
lightclient Jan 10, 2023
ce4f529
core/vm: don't update caller nonce on creation if eof validation fails
lightclient Jan 11, 2023
9f518d4
core/vm: clean up eof bitmap analysis
lightclient Jan 11, 2023
f878abc
core/vm: remove custom parser error
lightclient Jan 11, 2023
a9833f6
Merge branch 'devel' into eof
yperbasis Jan 12, 2023
ddc61df
Merge branch 'devel' into eof
yperbasis Jan 17, 2023
8d11000
Post-merge fix
yperbasis Jan 17, 2023
5b7d661
core/vm: change formatter in error msg for invalid magic
lightclient Jan 17, 2023
e22dc93
core/vm: bump nonce for create tx with invalid eof
lightclient Jan 17, 2023
f17351c
core/vm: don't double bump nonce when deployed eof is invalid in crea…
lightclient Jan 19, 2023
2f04fc6
eof: merge with devel
racytech Sep 20, 2023
0bab944
eof: eip-7480: data section access instructions
racytech Sep 25, 2023
51c690a
interpreter: fix after merge
racytech Sep 25, 2023
d851acc
merge with devel
racytech Sep 25, 2023
60d03e9
change isCancun -> isPrague
racytech Oct 2, 2023
6696280
Merge branch 'eof_devel' into eof
racytech Oct 2, 2023
6579a53
updated eof
racytech Oct 2, 2023
4f79f72
fix code/vm/runtime TestBlockhash test
racytech Oct 2, 2023
17036b5
Merge branch 'eof_devel' into eof
racytech Oct 4, 2023
3cf7d3c
change gas prices for DATALOAD and DATALOADN
racytech Oct 5, 2023
f0a07b7
merge with devel
racytech Oct 9, 2023
2f6ef36
fix TestValidateCode
racytech Oct 9, 2023
9bfabb1
Merge branch 'devel' into eof
racytech Oct 17, 2023
bcbf3d6
merge with devel, post-merge fix
racytech Oct 23, 2023
43481a5
post-merge fix
racytech Oct 23, 2023
7efac9d
JUMPF implement start
racytech Oct 23, 2023
df3f49c
Merge branch 'devel' into eof
racytech Oct 27, 2023
e46ba9d
Merge branch 'devel' into eof
racytech Nov 17, 2023
1292572
Merge branch 'devel' into eof
racytech Nov 21, 2023
a48e4fe
CREATE3 impl
racytech Nov 21, 2023
3e66353
RETURNCONTRACT start
racytech Nov 22, 2023
55ef3ca
RETURNCONTRACT todos and notes
racytech Nov 22, 2023
ffc22c7
merge with devel & fixes after merge
racytech May 22, 2024
504f1f8
merge with main
May 26, 2024
3033d4f
restart work on eof
racytech May 27, 2024
cf49325
eof tests added
racytech May 29, 2024
72bc21a
merge with main
racytech May 29, 2024
07540e9
fix after merge
racytech May 29, 2024
dc59486
container unmarshalling fix, eip3540 tests pass
racytech May 30, 2024
d01c72b
merge with main
racytech May 31, 2024
1588472
pass eip6206 tests
racytech May 31, 2024
cbdc926
merge with devel
racytech Jun 3, 2024
d8357a4
start 7480 tests
racytech Jun 7, 2024
a03a9c2
merge with main
racytech Jun 7, 2024
171e2fc
refactor control flow
racytech Jun 12, 2024
a37c87e
remove min and max
racytech Jun 18, 2024
97ff1f2
merge with main
racytech Jun 18, 2024
c10c8a7
add pragueEOF time flag
racytech Jun 18, 2024
0015708
add new tests & fix stack validation
racytech Jun 18, 2024
52542cb
merge with main
racytech Jun 19, 2024
2544197
add validate_instruction, validate_rjumps, fix stack validation, all …
racytech Jun 19, 2024
98b23eb
merge with main
racytech Jun 24, 2024
37f96ec
update tests
racytech Jun 24, 2024
55b68a6
fix validation tests
racytech Jun 25, 2024
2b1dac2
merge with main
racytech Jul 1, 2024
de2dab0
start EOFCREATE impl
racytech Jul 1, 2024
88f035e
update tests, fix failed tests
racytech Jul 3, 2024
f87d19e
start implementing all instructions
racytech Jul 9, 2024
6e3866b
start TXCREATE
racytech Jul 11, 2024
2735fdf
merge with main
racytech Jul 16, 2024
6bbc9f1
TXCREATE impl
racytech Jul 17, 2024
7bc4ede
evmone-go init
racytech Jul 23, 2024
1d6086a
evmone-go build fix
racytech Jul 26, 2024
2114b7b
pre-merge
racytech Jul 26, 2024
0a1241b
merge with main
racytech Jul 26, 2024
5186b8d
merge with main
racytech Jul 29, 2024
176f5a3
test exec-specs
racytech Aug 1, 2024
f1d9af7
merge with main
racytech Aug 1, 2024
284e343
update tests
racytech Aug 1, 2024
557cd79
fix failed validation tests
racytech Aug 2, 2024
1d8becc
update tests to eof-prague
racytech Aug 2, 2024
85e1950
merge with eof
racytech Aug 2, 2024
5275f96
start fix prague eof evm
racytech Aug 2, 2024
80bb570
state_tests: fix dupn, swapn, exchange
racytech Aug 7, 2024
f69aa25
state_tests: fix rjumpv
racytech Aug 8, 2024
8b6ee48
state_tests: fix callf
racytech Aug 8, 2024
20f8208
state_tests: fix jumpf, does not pass yet
racytech Aug 8, 2024
f8819a2
state_tests: fix data_section/data_opcodes
racytech Aug 12, 2024
d32627b
state_tests: fix data_section/datacopy_memory_expansion
racytech Aug 15, 2024
85065ee
merge with main
racytech Aug 15, 2024
8855338
state_tests: fix eip3540_eof_v1/calls
racytech Aug 21, 2024
8d4b4bb
state_tests: fix eip3540_eof_v1/execution_function
racytech Aug 23, 2024
2d1af74
state_tests: fix eip7069_extcall/returndataload
racytech Aug 23, 2024
6297dc5
state_tests: fix eip7069_extcall/calldata
racytech Aug 23, 2024
91ee246
state_tests: pass eip7069_extcall
racytech Aug 23, 2024
c33faf2
update tests
racytech Aug 29, 2024
8621c0f
merge with main
racytech Aug 29, 2024
e55d911
save before merge
racytech Sep 11, 2024
3204038
merge with main
racytech Sep 11, 2024
df28487
update tests to v1.1.0
racytech Sep 23, 2024
2109837
state_tests: fix EOFCREATE/eofcreate_failures
racytech Oct 1, 2024
8a4f64f
state_tests: fix EOFCREATE (un/marshalling, legacy <-> EOF)
racytech Oct 2, 2024
ff3dce6
state_tests: fix JUMPF
racytech Oct 8, 2024
55c6422
update tests
racytech Oct 22, 2024
a50e84e
merge with main
racytech Oct 22, 2024
338b3cf
fix validation tests
racytech Oct 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions core/state_transition.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package core

import (
"errors"
"fmt"

"github.com/holiman/uint256"
Expand Down Expand Up @@ -412,6 +413,13 @@ func (st *StateTransition) TransitionDb(refunds bool, gasBailout bool) (*Executi
// It does get incremented inside the `Create` call, after the computation
// of the contract's address, but before the execution of the code.
ret, _, st.gasRemaining, vmerr = st.evm.Create(sender, st.data, st.gasRemaining, st.value)
// Special case for EOF, if the initcode or deployed code is
// invalid, the tx is considered valid (so update nonce), but
// is to be treated as an exceptional abort (so burn all gas).
if errors.Is(vmerr, vm.ErrInvalidEOFInitcode) {
st.gasRemaining = 0
st.state.SetNonce(msg.From(), st.state.GetNonce(sender.Address())+1)
}
} else {
// Increment the nonce for the next transaction
st.state.SetNonce(msg.From(), st.state.GetNonce(sender.Address())+1)
Expand Down
128 changes: 128 additions & 0 deletions core/vm/analysis.go
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,50 @@

package vm

const (
set2BitsMask = uint16(0b11)
set3BitsMask = uint16(0b111)
set4BitsMask = uint16(0b1111)
set5BitsMask = uint16(0b1_1111)
set6BitsMask = uint16(0b11_1111)
set7BitsMask = uint16(0b111_1111)
)

// bitvec is a bit vector which maps bytes in a program.
// An unset bit means the byte is an opcode, a set bit means
// it's data (i.e. argument of PUSHxx).
type bitvec []byte

func (bits bitvec) set1(pos uint64) {
bits[pos/8] |= 1 << (pos % 8)
}

func (bits bitvec) setN(flag uint16, pos uint64) {
a := flag << (pos % 8)
bits[pos/8] |= byte(a)
if b := byte(a >> 8); b != 0 {
bits[pos/8+1] = b
}
}

func (bits bitvec) set8(pos uint64) {
a := byte(0xFF << (pos % 8))
bits[pos/8] |= a
bits[pos/8+1] = ^a
}

func (bits bitvec) set16(pos uint64) {
a := byte(0xFF << (pos % 8))
bits[pos/8] |= a
bits[pos/8+1] = 0xFF
bits[pos/8+2] = ^a
}

// codeSegment checks if the position is in a code segment.
func (bits *bitvec) codeSegment(pos uint64) bool {
return (((*bits)[pos/8] >> (pos % 8)) & 1) == 0
}

// codeBitmap collects data locations in code.
func codeBitmap(code []byte) []uint64 {
// The bitmap is 4 bytes longer than necessary, in case the code
Expand All @@ -41,3 +85,87 @@ func codeBitmap(code []byte) []uint64 {
}
return bits
}

// eofCodeBitmap collects data locations in code.
func eofCodeBitmap(code []byte) bitvec {
// The bitmap is 4 bytes longer than necessary, in case the code
// ends with a PUSH32, the algorithm will push zeroes onto the
// bitvector outside the bounds of the actual code.
bits := make(bitvec, len(code)/8+1+4)
return eofCodeBitmapInternal(code, bits)
}

// eofCodeBitmapInternal is the internal implementation of codeBitmap for EOF
// code validation.
func eofCodeBitmapInternal(code, bits bitvec) bitvec {
for pc := uint64(0); pc < uint64(len(code)); {
var (
op = OpCode(code[pc])
numbits uint8
)
pc++

switch {
case op >= PUSH1 && op <= PUSH32:
numbits = uint8(op - PUSH1 + 1)
case op == RJUMP || op == RJUMPI:
numbits = 2
case op == RJUMPV:
// RJUMPV is unique as it has a variable sized operand.
// The total size is determined by the count byte which
// immediate proceeds RJUMPV. Truncation will be caught
// in other validation steps -- for now, just return a
// valid bitmap for as much of the code as is
// available.
end := uint64(len(code))
if pc >= end {
// Count missing, no more bits to mark.
return bits
}
numbits = code[pc]*2 + 1
if pc+uint64(numbits) > end {
// Jump table is truncated, mark as many bits
// as possible.
numbits = uint8(end - pc)
}
default:
// Op had no immediate operand, continue.
continue
}

if numbits >= 8 {
for ; numbits >= 16; numbits -= 16 {
bits.set16(pc)
pc += 16
}
for ; numbits >= 8; numbits -= 8 {
bits.set8(pc)
pc += 8
}
}
switch numbits {
case 1:
bits.set1(pc)
pc += 1
case 2:
bits.setN(set2BitsMask, pc)
pc += 2
case 3:
bits.setN(set3BitsMask, pc)
pc += 3
case 4:
bits.setN(set4BitsMask, pc)
pc += 4
case 5:
bits.setN(set5BitsMask, pc)
pc += 5
case 6:
bits.setN(set6BitsMask, pc)
pc += 6
case 7:
bits.setN(set7BitsMask, pc)
pc += 7
}
}
return bits
}
52 changes: 52 additions & 0 deletions core/vm/analysis_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
package vm

import (
"encoding/binary"
"testing"

"github.com/holiman/uint256"
Expand All @@ -25,6 +26,18 @@ import (
"github.com/ledgerwatch/erigon/crypto"
)

func bitvecToUint64Slice(b bitvec) []uint64 {
n := (len(b) + 7) / 8
padded := make([]byte, n*8)
copy(padded, b)

res := make([]uint64, n)
for i := 0; i < n; i++ {
res[i] = binary.LittleEndian.Uint64(padded[i*8:])
}
return res
}

func TestJumpDestAnalysis(t *testing.T) {
t.Parallel()
tests := []struct {
Expand All @@ -49,6 +62,28 @@ func TestJumpDestAnalysis(t *testing.T) {
if ret[test.which] != test.exp {
t.Fatalf("expected %x, got %02x", test.exp, ret[test.which])
}
retEof := bitvecToUint64Slice(eofCodeBitmap(test.code))
if retEof[test.which] != test.exp {
t.Fatalf("eof expected %x, got %02x", test.exp, retEof[test.which])
}
}
}

func TestEOFAnalysis(t *testing.T) {
tests := []struct {
code []byte
exp byte
which int
}{
{[]byte{byte(RJUMP), 0x01, 0x01, 0x01}, 0b0000_0110, 0},
{[]byte{byte(RJUMPI), byte(RJUMP), byte(RJUMP), byte(RJUMPI)}, 0b0011_0110, 0},
{[]byte{byte(RJUMPV), 0x02, byte(RJUMP), 0x00, byte(RJUMPI), 0x00}, 0b0011_1110, 0},
}
for i, test := range tests {
ret := eofCodeBitmap(test.code)
if ret[test.which] != test.exp {
t.Fatalf("test %d: expected %x, got %02x", i, test.exp, ret[test.which])
}
}
}

Expand Down Expand Up @@ -101,3 +136,20 @@ func BenchmarkJumpDest(b *testing.B) {
b.StopTimer()
}
}

func TestCodeAnalysis(t *testing.T) {
for _, tc := range []string{
"5e30303030",
} {
eofCodeBitmap(libcommon.FromHex(tc))
codeBitmap(libcommon.FromHex(tc))
}
}

func FuzzCodeAnalysis(f *testing.F) {
f.Add(libcommon.FromHex("5e30303030"))
f.Fuzz(func(t *testing.T, data []byte) {
eofCodeBitmap(data)
codeBitmap(data)
})
}
31 changes: 26 additions & 5 deletions core/vm/contract.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,10 +51,11 @@ type Contract struct {
analysis []uint64 // Locally cached result of JUMPDEST analysis
skipAnalysis bool

Code []byte
CodeHash libcommon.Hash
CodeAddr *libcommon.Address
Input []byte
Code []byte
Container *Container
CodeHash libcommon.Hash
CodeAddr *libcommon.Address
Input []byte

Gas uint64
value *uint256.Int
Expand Down Expand Up @@ -184,10 +185,30 @@ func (c *Contract) Value() *uint256.Int {
return c.value
}

// IsEOF returns whether the contract is EOF.
func (c *Contract) IsEOF() bool {
return c.Container != nil
}

func (c *Contract) CodeAt(section uint64) []byte {
if c.Container == nil {
return c.Code
}
return c.Container.Code[section]
}

func (c *Contract) Data() []byte {
// if c.Container == nil {
// return nil
// }
return c.Container.Data
}

// SetCallCode sets the code of the contract and address of the backing data
// object
func (c *Contract) SetCallCode(addr *libcommon.Address, hash libcommon.Hash, code []byte) {
func (c *Contract) SetCallCode(addr *libcommon.Address, hash libcommon.Hash, code []byte, container *Container) {
c.Code = code
c.Container = container
c.CodeHash = hash
c.CodeAddr = addr
}
Expand Down
Loading
Loading