Skip to content

Commit

Permalink
chore: re-organize and document profiling scripts
Browse files Browse the repository at this point in the history
  • Loading branch information
davide-baldo committed Nov 28, 2024
1 parent d8098bd commit ebefe6e
Show file tree
Hide file tree
Showing 8 changed files with 141 additions and 25 deletions.
29 changes: 16 additions & 13 deletions tools/profile/README.md
Original file line number Diff line number Diff line change
@@ -1,22 +1,25 @@
# Profiling

This directory contains tools for profiling ockam.
This directory contains scripts for profiling ockam.
Each script simulates a portal in conjunction with a speed test called `iperf3`.

Two scenarios for performance profiling:
- `portal.perf` - local portal, within one node
- `portal_two_nodes.perf` - two nodes, one inlet and outlet
- `relay_port.perf` - one node, one inlet and outlet passing through a relay
The scenarios are:
- `portal` - local portal, within one node
- `portal_two_nodes` - two nodes, one inlet and outlet
- `portal_relay` - one node, one inlet and outlet passing through the project relay

And one scenario for heap profiling:
- `portal.valgrind.dhat` - local portal, within one node
Each comes with different variants:
- `baseline` - no profiling, useful for a quick benchmark
- `cpu` - profile CPU usage
- `allocations` - profile memory allocations

## Running the performance tests

To run the performance tests, simply run `tools/profile/SCRIPT` from the ockam
git root.
To run the performance tests, simply run `tools/profile/<scenario>.<variant>` from the ockam
git root. The script uses the ports 5500 and 8200, and expects an environment without
any other node (otherwise the script might get stuck waiting for a stopped node).

## OS Compatibility
The performance scripts are currently compatible only with Linux since they use `perf`.
On MacOS, a similar approach should be doable with `dtrace`, but is not yet implemented.

Heap profiling with valgrind is compatible with both Linux and MacOS.
CPU profiling is supported on Linux and MacOS.
Allocation profiling should work on both MacOS and Linux, but MacOS requires the
binary to be signed with an extra capability.
Original file line number Diff line number Diff line change
Expand Up @@ -38,14 +38,14 @@ fi


sleep 1
"${OCKAM}" tcp-outlet create --to 5000 --at portal
"${OCKAM}" tcp-inlet create --from 8000 --to /secure/api/service/outlet --at portal
"${OCKAM}" tcp-outlet create --to 5500 --at portal
"${OCKAM}" tcp-inlet create --from 8200 --to /secure/api/service/outlet --at portal

iperf3 --server --port 5000 --one-off &
iperf3 --server --port 5500 --one-off &
iperf3_server_pid=$!

sleep 0.3 # wait for server to start
iperf3 --zerocopy --client 127.0.0.1 --port 8000 --time 60
iperf3 --zerocopy --client 127.0.0.1 --port 8200 --time 60

kill ${iperf3_server_pid}
"${OCKAM}" node delete portal -y
Expand Down
32 changes: 32 additions & 0 deletions tools/profile/portal.baseline
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
#!/bin/bash

if ! [ -x "$(command -v iperf3)" ]; then
echo 'Error: iperf3 is not installed.' >&2
exit 1
fi

set -e

if [ -z "${OCKAM}" ]; then
RUSTFLAGS="-C force-frame-pointers=yes" cargo build --profile profiling -p ockam_command -F ockam_vault/aws-lc
OCKAM=target/profiling/ockam
fi

"${OCKAM}" node delete portal -y >/dev/null 2>&1 || true
export OCKAM_LOG_LEVEL=info
export OCKAM_OPENTELEMETRY_EXPORT=0

"${OCKAM}" node create portal -f &

sleep 2
"${OCKAM}" tcp-outlet create --to 5500 --at portal
"${OCKAM}" tcp-inlet create --from 8200 --to /secure/api/service/outlet --at portal

iperf3 --server --port 5500 --one-off &
iperf3_server_pid=$!

sleep 0.3 # wait for server to start
iperf3 --zerocopy --client 127.0.0.1 --port 8200 --time 60

kill ${iperf3_server_pid} || true
"${OCKAM}" node delete portal -y
File renamed without changes.
42 changes: 42 additions & 0 deletions tools/profile/portal_relay.baseline
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
#!/bin/bash

if ! [ -x "$(command -v iperf3)" ]; then
echo 'Error: iperf3 is not installed.' >&2
exit 1
fi

set -e

if [ -z "${OCKAM}" ]; then
RUSTFLAGS="-C force-frame-pointers=yes" cargo build --profile profiling -p ockam_command
OCKAM=target/profiling/ockam
fi

"${OCKAM}" node delete portal -y >/dev/null 2>&1 || true
export OCKAM_LOG_LEVEL=info
export OCKAM_OPENTELEMETRY_EXPORT=0

"${OCKAM}" node create portal -f &

sleep 1
"${OCKAM}" tcp-outlet create --to 5500 --at portal
"${OCKAM}" relay create --to portal
"${OCKAM}" tcp-inlet create --from 8200 --to /project/default/service/forward_to_default/secure/api/service/outlet --at portal

iperf3 --server --port 5500 --one-off &
iperf3_server_pid=$!

sleep 0.3 # wait for server to start
iperf3 --zerocopy --client 127.0.0.1 --port 8200 --time 60

kill ${iperf3_server_pid}
"${OCKAM}" node delete portal -y

echo "Waiting for perf to finish writing /tmp/ockam.perf..."
wait ${perf_pid}

echo "Converting perf file to firefox profiler format, could take up to few minutes..."
perf script -F +pid --input /tmp/ockam.perf > /tmp/ockam.perf.firefox

echo "You can use firefox web profiler to open /tmp/ockam.perf.firefox file."
echo "https://profiler.firefox.com/"
Original file line number Diff line number Diff line change
Expand Up @@ -19,19 +19,21 @@ fi

"${OCKAM}" node delete portal -y >/dev/null 2>&1 || true
export OCKAM_LOG_LEVEL=info
export OCKAM_OPENTELEMETRY_EXPORT=0

perf record --call-graph dwarf -F 99 --output /tmp/ockam.perf -- "${OCKAM}" node create portal -f &
perf_pid=$!

sleep 1
"${OCKAM}" tcp-outlet create --to 5000 --at portal
"${OCKAM}" tcp-outlet create --to 5500 --at portal
"${OCKAM}" relay create --to portal
"${OCKAM}" tcp-inlet create --from 8000 --to /project/default/service/forward_to_default/secure/api/service/outlet --at portal
"${OCKAM}" tcp-inlet create --from 8200 --to /project/default/service/forward_to_default/secure/api/service/outlet --at portal

iperf3 --server --port 5000 --one-off &
iperf3 --server --port 5500 --one-off &
iperf3_server_pid=$!

sleep 0.3 # wait for server to start
iperf3 --zerocopy --client 127.0.0.1 --port 8000 --time 60
iperf3 --zerocopy --client 127.0.0.1 --port 8200 --time 60

kill ${iperf3_server_pid}
"${OCKAM}" node delete portal -y
Expand Down
35 changes: 35 additions & 0 deletions tools/profile/portal_two_nodes.baseline
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
#!/bin/bash

if ! [ -x "$(command -v iperf3)" ]; then
echo 'Error: iperf3 is not installed.' >&2
exit 1
fi

set -e

if [ -z "${OCKAM}" ]; then
RUSTFLAGS="-C force-frame-pointers=yes" cargo build --profile profiling -p ockam_command -F ockam_vault/aws-lc
OCKAM=target/profiling/ockam
fi

"${OCKAM}" node delete portal -y >/dev/null 2>&1 || true
export OCKAM_LOG_LEVEL=info
export OCKAM_OPENTELEMETRY_EXPORT=0

"${OCKAM}" node create inlet -f &
"${OCKAM}" node create outlet -f &

sleep 1
"${OCKAM}" tcp-outlet create --to 5500 --at outlet
"${OCKAM}" tcp-inlet create --from 8200 --to /node/outlet/secure/api/service/outlet --at inlet

iperf3 --server --port 5500 --one-off &
iperf3_server_pid=$!

sleep 0.3 # wait for server to start
iperf3 --zerocopy --client 127.0.0.1 --port 8200 --time 60

kill ${iperf3_server_pid}
"${OCKAM}" node delete inlet -y
"${OCKAM}" node delete outlet -y

Original file line number Diff line number Diff line change
Expand Up @@ -19,18 +19,20 @@ fi

"${OCKAM}" node delete portal -y >/dev/null 2>&1 || true
export OCKAM_LOG_LEVEL=info
export OCKAM_OPENTELEMETRY_EXPORT=0

perf record --call-graph dwarf -F 99 --output /tmp/ockam.inlet.perf -- "${OCKAM}" node create inlet -f &
perf record --call-graph dwarf -F 99 --output /tmp/ockam.outlet.perf -- "${OCKAM}" node create outlet -f &

sleep 1
"${OCKAM}" tcp-outlet create --to 5000 --at outlet
"${OCKAM}" tcp-inlet create --from 8000 --to /node/outlet/secure/api/service/outlet --at inlet
"${OCKAM}" tcp-outlet create --to 5500 --at outlet
"${OCKAM}" tcp-inlet create --from 8200 --to /node/outlet/secure/api/service/outlet --at inlet

iperf3 --server --port 5000 --one-off &
iperf3 --server --port 5500 --one-off &
iperf3_server_pid=$!

sleep 0.3 # wait for server to start
iperf3 --zerocopy --client 127.0.0.1 --port 8000 --time 60
iperf3 --zerocopy --client 127.0.0.1 --port 8200 --time 60

kill ${iperf3_server_pid}
"${OCKAM}" node delete inlet -y
Expand Down

0 comments on commit ebefe6e

Please sign in to comment.