Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat/1.0.0 beta2 #104

Closed
wants to merge 82 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
01bddb2
feat: add draft of config file
aimxhaisse Nov 20, 2023
50912fa
feat: adapt unit tests to new config file
aimxhaisse Nov 20, 2023
9965494
fix: remove unused test file
aimxhaisse Nov 20, 2023
9041295
feat: add unit test for empty config file
aimxhaisse Nov 20, 2023
56d703c
feat: add unit test for filled config
aimxhaisse Nov 20, 2023
53a2d0e
feat: move config to pydantic
aimxhaisse Nov 20, 2023
0708036
feat: update authors
aimxhaisse Nov 20, 2023
9b9d02d
fix: dead import that worked locally
aimxhaisse Nov 21, 2023
2609fbd
fix: remove legacy exception check
aimxhaisse Nov 21, 2023
70a9b70
feat: pretty print config errors
aimxhaisse Nov 21, 2023
f14b68b
feat: make BeaconType compulsory with a default value
aimxhaisse Nov 21, 2023
5e726f9
feat: add unit test for null config file
aimxhaisse Nov 21, 2023
35eb724
feat: add test for invalid config
aimxhaisse Nov 21, 2023
cc0ac17
feat: remove dead code
aimxhaisse Nov 21, 2023
467833c
feat: add new config setting for beacon timeout
aimxhaisse Nov 21, 2023
4d8fafc
feat: introduce labels for queue size and active validator count
aimxhaisse Nov 22, 2023
cc26e5f
feat: add dummy gitignore for local configs
aimxhaisse Apr 3, 2024
07236a4
feat: add new alternate entrypoint to later support labeling processing
aimxhaisse Apr 3, 2024
c23d6c1
feat: add dedicated prometheus metric class
aimxhaisse Apr 3, 2024
5956b36
feat: add basic handling of the main processing loop and introduced w…
aimxhaisse Apr 4, 2024
38700e3
feat: normalize public key for comparisons
aimxhaisse Apr 4, 2024
3f8e3fa
feat: add beacon clock to synchronize the different steps of the watcher
aimxhaisse Apr 4, 2024
1f00692
feat: add ability to process every slot
aimxhaisse Apr 4, 2024
f5b2dc7
feat: add expose of validator status count per label
aimxhaisse Apr 4, 2024
0dbb2ec
feat: start metrics server
aimxhaisse Apr 4, 2024
0bfe45c
fix: epoch
aimxhaisse Apr 4, 2024
3dc3ace
feat: add processing of rewards by scope
aimxhaisse Apr 4, 2024
dab0dba
fix: simplify the main loop
aimxhaisse Apr 4, 2024
2d6dfdb
feat: compute ideal/actual consensus rewards per scope
aimxhaisse Apr 4, 2024
7d882b5
feat: now processing missed attestations
aimxhaisse Apr 5, 2024
da466bb
feat: exporting status in protocol format instead of camelCased
aimxhaisse Apr 5, 2024
d76b3d9
feat: add tracking of missed attestations and consecutive missed atte…
aimxhaisse Apr 5, 2024
f4a038f
feat: rework handle of ETH price
aimxhaisse Apr 15, 2024
6ce63a5
feat: actually process missed attestations
aimxhaisse Apr 15, 2024
3002bb5
feat: add handling of head/finalized blocks in the processing
aimxhaisse Apr 17, 2024
e5f3841
feat: export prometheus metrics about block proposals
aimxhaisse Apr 17, 2024
f53a0c9
fix: skew a bit the clock to have the data we want
aimxhaisse Apr 17, 2024
c04e491
feat: add handling of future block proposals
aimxhaisse Apr 17, 2024
df141b7
feat: do not account non-active validators for missed attestations/bl…
aimxhaisse May 13, 2024
720f2d7
feat: strip down beacon class to the minimum
aimxhaisse May 13, 2024
4477d54
feat: move beacon clock to a dedicated file
aimxhaisse May 13, 2024
7ec4360
feat: temporarily delete unit tests to have a basic working CI
aimxhaisse May 14, 2024
d7741a0
feat: add unit test to check existence of a block in a slot
aimxhaisse May 14, 2024
90972af
fix: do not log every missed attestations
aimxhaisse May 15, 2024
3e2b9e5
feat: remove legacy missed attestations
aimxhaisse May 15, 2024
0e21c70
feat: add unit tests for config
aimxhaisse May 24, 2024
85566e3
feat: add new label for the watched network
aimxhaisse May 27, 2024
38de7d0
feat: get rid of legacy beacon type config
aimxhaisse May 27, 2024
c208573
feat: update unit test for the new network setting
aimxhaisse May 27, 2024
5b924b7
feat: populate the network label
aimxhaisse May 27, 2024
37d9ef8
feat: space out config reloading
aimxhaisse May 27, 2024
c5be53f
feat: add label unwatched for keys we don't monitor
aimxhaisse May 27, 2024
41d2f2a
fix: labelling and status metrics regardless of whether or not the va…
aimxhaisse May 30, 2024
2ed2cb9
feat: handle slashes
aimxhaisse May 30, 2024
ba6c4a3
feat: speed up parsing by bypassing python requests JSON's parser
aimxhaisse Jun 3, 2024
1f5686b
feat: reduce processing time of watched metrics
aimxhaisse Jun 3, 2024
1187b32
feat: use google python style for private class members
aimxhaisse Jun 6, 2024
0a42173
feat: parallelize the bulk of processing
aimxhaisse Jun 6, 2024
35b2ad0
fix: unit tests
aimxhaisse Jun 20, 2024
ad8da27
feat: remove unused configuration files
aimxhaisse Jun 21, 2024
0fee9f9
feat: rework the documentation of the watcher
aimxhaisse Jun 21, 2024
087352e
feat: add images
aimxhaisse Jun 21, 2024
a573230
feat: rework a bit the documentation
aimxhaisse Jun 21, 2024
91413c2
feat: add the breakdown dashboard
aimxhaisse Jun 21, 2024
6ec1ca1
feat: add overview dashboard
aimxhaisse Jun 21, 2024
694931e
feat: add logging of proposed/missed blocks
aimxhaisse Jun 21, 2024
e53d277
feat: prepare for v1.0.0-beta.1
aimxhaisse Jun 21, 2024
a8e6ac2
feat: bump dependencies
aimxhaisse Jun 21, 2024
8b323d8
fix: exported grafana
aimxhaisse Jun 21, 2024
d93a259
fix: nit
aimxhaisse Jun 21, 2024
83c3b73
feat: introduce fast-path in cpp
aimxhaisse Jun 24, 2024
eb9750f
feat: introduce the C++ validator type to allow fast-cast
aimxhaisse Jun 24, 2024
ecb386f
feat: adapt Dockerfile to cpp builds
aimxhaisse Jun 24, 2024
c5f33c8
feat: draft implementation of fast cpp processing
aimxhaisse Jun 24, 2024
eb3941c
feat: handle reward slot without block in it
aimxhaisse Jun 24, 2024
0eafebf
fix: init of config on first run
aimxhaisse Jun 24, 2024
1916986
fix: do not rely on implicit references of pybind objects in python
aimxhaisse Jun 25, 2024
3be75b8
fix: config issue with default values
aimxhaisse Jun 25, 2024
98c316d
feat: emojis
aimxhaisse Jun 25, 2024
40537f7
fix: adjust a bit emojis
aimxhaisse Jun 25, 2024
d8f8177
fix: use cpp17 instead of cpp23 to facilitate the build
aimxhaisse Jun 25, 2024
866a98f
fix: watched keys without labels
aimxhaisse Jun 25, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
etc/
3 changes: 2 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
Makefile
__pycache__
.mypy_cache
.coverage
.idea
coverage.xml
coverage.xml
26 changes: 12 additions & 14 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,19 +1,17 @@
FROM python:3.11.6-bookworm as builder
WORKDIR /app

COPY . /app
RUN pip install --upgrade pip
RUN pip install --no-cache-dir --progress-bar off .
FROM python:3.12-bookworm as builder

# Use Python 3.11 from Debian 12
FROM gcr.io/distroless/python3-debian12:nonroot
RUN pip install poetry

COPY --from=builder /app /app
COPY --from=builder /usr/local/lib/python3.11/site-packages /usr/local/lib/python3.11/site-packages
COPY --from=builder /usr/local/bin/eth-validator-watcher /usr/local/bin/eth-validator-watcher
WORKDIR /app

COPY liveness_check.py /usr/local/bin/liveness_check.py
COPY eth_validator_watcher /app/eth_validator_watcher
COPY pyproject.toml /app/pyproject.toml
COPY README.md /app/README.md
COPY poetry.lock /app/poetry.lock
COPY tests /app/tests
COPY build.py /app/build.py

ENV PYTHONPATH=/usr/local/lib/python3.11/site-packages
RUN poetry build
RUN poetry install

ENTRYPOINT [ "python", "/usr/local/bin/eth-validator-watcher" ]
ENTRYPOINT [ "poetry", "run", "eth-validator-watcher" ]
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
all:
eth-validator-watcher 2>&1 | tee -a /var/log/ethereum-watcher/watcher.log

test:
poetry run pytest
312 changes: 103 additions & 209 deletions README.md

Large diffs are not rendered by default.

19 changes: 19 additions & 0 deletions build.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
from pybind11.setup_helpers import Pybind11Extension, build_ext


def build(setup_kwargs):
ext_modules = [
Pybind11Extension(
"eth_validator_watcher_ext", [
"eth_validator_watcher/mod.cc"
],
extra_compile_args=['-O3', '-pthread'],
language='c++',
cxx_std=17
)
]
setup_kwargs.update({
"ext_modules": ext_modules,
"cmd_class": {"build_ext": build_ext},
"zip_safe": False,
})
Binary file removed docs/img/grafana_dashboard.png
Binary file not shown.
Binary file added docs/img/watcher-breakdown.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/watcher-overview.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
3 changes: 3 additions & 0 deletions etc/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
config.dev.yaml
config.py
mainnet.yaml
20 changes: 20 additions & 0 deletions etc/config.local.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
# Example config file for the Ethereum watcher.

beacon_url: http://localhost:5051/
beacon_type: other
execution_url: ~
web3signer_url: ~
default_fee_recipient: ~
slack_channel: ~
slack_token: ~
relays: ~
liveness_file: ~

# This mapping is reloaded dynamically at the beginning of each
# epoch. If the new mapping is invalid the watcher will crash, be sure
# to use atomic filesystem operations to have a completely updated
# configuration file if you dynamically watch keys.
watched_keys:
- public_key: '0x832b8286f5d6535fd941c6c4ed8b9b20d214fc6aa726ce4fba1c9dbb4f278132646304f550e557231b6932aa02cf08d3'
labels: ['google']
fee_recipient: ~
Loading
Loading