LLVM Opt Benchmark

LLVM Opt Benchmark is an LLVM IR dataset for data-driven compiler optimization research. This repository is also used by LLVM developers to evaluate the impact of their patches on real-world applications.

Don't submit PR to add new benchmarks. You can request new open-source C/C++/Rust repos here.

Please cite this work with the following BibTex entry:

@misc{opt-benchmark,
  title = {LLVM Opt Benchmark},
  url = {https://github.com/dtcxzyw/llvm-opt-benchmark},
  author = {Yingwei Zheng},
  year = {2023},
}

Online services (previously hosted by PLCT Lab, ISCAS/currently hosted by SUSTech ARiSE Lab)

Special Acknowledgement: Thank @goldsteinn for providing additional computational resources to meet the growing demand for testing!

• Fuzzy DAG matching

  Please file an issue to provide LLVM IR with a single function. I will add the grep label to trigger CI.

  Example: #1072

• Middle-end optimization pre-commit testing

  Ping me if you want to see what is affected by your PR. It is useful for reviewers to find potential performance regressions and new optimization opportunities.

  For convenience, some senior middle-end maintainers are authorized to request pre-commit tests in #1312. Some basic PR editing commands are also supported by leaving a comment starts with /:

  • /close : Close the PR
  • /reopen: Reopen the PR
  • /add-label labels: Add labels (separated by comma). Available labels: reviewed, regression, crash, hang and miscompilation.
  • /remove-label labels: Remove labels.

• Codegen pre-commit testing

  See also llvm-codegen-benchmark.

• Weekly coverage report:

  https://dtcxzyw.github.io/llvm-opt-benchmark/

Benchmark List

Currently, this repository contains the following libraries/applications:

Name	Language	Stars	Last Updated
abc	C		2024-03-06
bdwgc	C		2024-03-04
brotli	C		2023-12-08
c3c	C		2024-04-16
chibicc	C		2020-12-07
cjson	C		2023-12-26
clamav	C		2024-04-17
cmake	C		2024-03-08
coremark	C		2023-01-24
cpython	C		2023-12-17
curl	C		2024-01-19
darktable	C		2024-03-29
flac	C		2023-11-24
freetype	C		2024-06-23
git	C		2024-01-16
graphviz	C		2024-03-08
hdf5	C		2024-05-23
hwloc	C		2024-02-08
jemalloc	C		2023-10-25
jq	C		2024-03-01
kcp	C		2023-05-07
libdeflate	C		2023-11-25
libevent	C		2023-11-26
libjpeg-turbo	C		2024-07-09
libpng	C		2024-06-17
libquic	C		2016-09-22
libsodium	C		2024-01-17
libuv	C		2023-12-12
libwebp	C		2024-06-25
linux	C		2024-02-29
lua	C		2023-11-01
luajit	C		2023-12-11
lvgl	C		2024-10-25
lz4	C		2023-12-12
memcached	C		2023-10-15
mimalloc	C		2023-06-12
miniaudio	C		2023-11-15
nanosvg	C		2023-12-30
nuklear	C		2023-12-12
nuttx	C		2024-03-04
ompi	C		2024-02-28
oniguruma	C		2023-10-11
openblas	C		2024-03-11
openssl	C		2023-12-15
osqp	C		2024-04-22
php-src	C		2024-03-02
portaudio	C		2024-08-25
postgres	C		2024-03-04
qemu	C		2023-12-12
qoi	C		2024-01-18
quickjs	C		2024-03-03
raylib	C		2024-03-05
redis	C		2023-12-15
riscv-isa-sim	C		2024-03-03
ruby	C		2024-03-04
slurm	C		2024-03-08
sqlite	C		2024-03-03
stb	C		2023-12-14
sundials	C		2024-02-29
wireshark	C		2024-03-07
wolfssl	C		2024-01-22
yyjson	C		2023-12-14
zlib	C		2023-11-14
zstd	C		2023-12-13
abseil-cpp	C++		2023-12-15
annoy	C++		2023-08-20
arrow	C++		2023-12-16
assimp	C++		2023-12-11
boost	C++		2024-10-25
box2d	C++		2023-02-20
bullet3	C++		2023-11-28
casadi	C++		2024-03-06
ceres-solver	C++		2024-02-01
cpp-httplib	C++		2023-12-15
crow	C++		2024-04-19
csmith	C++		2023-11-02
cvc5	C++		2023-12-15
cxxopts	C++		2023-11-25
double-conversion	C++		2023-11-01
draco	C++		2024-02-06
duckdb	C++		2023-12-15
eastl	C++		2023-08-16
entt	C++		2023-11-30
faiss	C++		2024-03-04
flatbuffers	C++		2023-12-17
fmt	C++		2023-12-15
folly	C++		2023-12-16
g2o	C++		2024-08-31
glog	C++		2024-02-21
glslang	C++		2024-06-25
gromacs	C++		2024-04-23
grpc	C++		2023-12-15
gsl	C++		2023-10-18
harfbuzz	C++		2024-01-15
hermes	C++		2023-12-15
hyperscan	C++		2023-04-19
icu	C++		2023-12-15
imgui	C++		2023-12-14
ipopt	C++		2024-01-18
json	C++		2023-12-14
jsonnet	C++		2024-04-17
libphonenumber	C++		2024-02-23
libzmq	C++		2023-12-19
lief	C++		2024-03-02
lightgbm	C++		2024-07-14
llama.cpp	C++		2023-12-16
llvm-project	C++		2024-09-03
lodepng	C++		2023-12-03
luau	C++		2024-06-20
meshlab	C++		2024-02-13
meshoptimizer	C++		2023-12-10
minetest	C++		2024-03-26
mitsuba3	C++		2024-03-22
mixbox	C++		2022-12-16
mold	C++		2023-12-24
msdfgen	C++		2024-01-06
msgpack-c	C++		2024-04-17
nanobind	C++		2024-03-11
ncnn	C++		2024-09-04
nghttp2	C++		2024-01-21
ninja	C++		2024-02-29
nix	C++		2024-03-06
node	C++		2023-12-17
nori	C++		2023-11-15
open_spiel	C++		2024-08-27
opencc	C++		2023-10-15
opencolorio	C++		2023-12-07
opencv	C++		2024-07-16
openexr	C++		2023-12-09
openimageio	C++		2023-12-13
openjdk	C++		2024-07-16
openusd	C++		2024-07-24
openvdb	C++		2023-12-06
ozz-animation	C++		2024-04-13
pbrt-v4	C++		2023-10-18
pcg-cpp	C++		2022-04-08
pocketpy	C++		2024-06-20
proj	C++		2024-04-17
protobuf	C++		2023-12-15
proxy	C++		2024-05-22
proxygen	C++		2023-12-16
pugixml	C++		2023-11-16
pybind11	C++		2024-02-13
quantlib	C++		2024-09-10
quest	C++		2023-09-22
re2	C++		2023-12-14
readerwriterqueue	C++		2023-03-05
recastnavigation	C++		2024-01-28
rocksdb	C++		2023-12-15
sentencepiece	C++		2024-05-22
simdjson	C++		2023-12-15
snappy	C++		2023-07-12
soc-simulator	C++		2024-06-25
spdlog	C++		2023-11-04
stockfish	C++		2024-03-03
taskflow	C++		2023-12-15
tev	C++		2024-01-12
tinygltf	C++		2024-03-27
tinympc	C++		2024-05-08
tinyobjloader	C++		2023-08-24
tinyrenderer	C++		2023-02-10
tomlplusplus	C++		2023-11-23
vcpkg-tool	C++		2024-03-05
velox	C++		2023-12-15
verilator	C++		2024-03-28
wasmedge	C++		2024-07-15
xgboost	C++		2024-07-15
yalantinglibs	C++		2023-12-17
yaml-cpp	C++		2023-12-14
yoga	C++		2023-12-17
yosys	C++		2024-03-11
z3	C++		2023-12-16
zfp	C++		2024-02-20
zxing-cpp	C++		2024-04-23
actix-web	Rust		2024-04-15
anki	Rust		2024-06-24
clap	Rust		2024-03-01
coreutils	Rust		2024-04-23
delta-rs	Rust		2024-04-23
diesel	Rust		2024-03-01
egg	Rust		2024-08-30
html5ever	Rust		2023-09-06
hyper	Rust		2024-03-02
image	Rust		2024-02-22
influxdb	Rust		2024-03-01
json	Rust		2024-01-11
just	Rust		2024-04-01
log	Rust		2024-02-29
logos	Rust		2024-06-10
meilisearch	Rust		2024-06-25
mini-lsm	Rust		2024-02-26
mio	Rust		2024-03-01
nom	Rust		2024-04-21
ockam	Rust		2024-04-22
pyo3	Rust		2024-06-24
qdrant	Rust		2024-03-19
rand	Rust		2024-02-18
rayon	Rust		2024-02-27
regex	Rust		2024-01-10
ring	Rust		2024-03-03
ripgrep	Rust		2024-03-27
ropey	Rust		2024-04-08
rust-analyzer	Rust		2024-04-22
rust-base64	Rust		2024-03-01
rustfmt	Rust		2024-03-04
rustls	Rust		2024-03-07
serde	Rust		2024-01-08
smol	Rust		2024-03-04
softposit-rs	Rust		2022-12-14
statrs	Rust		2024-06-24
syn	Rust		2024-01-13
tokenizers	Rust		2024-05-06
tokio	Rust		2024-03-04
tree-sitter	Rust		2024-03-08
turborepo	Rust		2024-10-03
typst	Rust		2024-03-25
unicode-normalization	Rust		2024-03-03
wasmtime	Rust		2024-04-22
zed	Rust		2024-10-04