Benchmarks

This page collects data on benchmark measurements of Lingua Franca programs. The hardware on which these tests are run is detailed below.

Internal Benchmarks

These are programs that are not part of any standard benchmark suite, but which we use to measure the impact of design changes.

• test/C/TimeLimit.lf: This program sends a counting sequence of length 10,000,002 from a source to a destination. Is is meant to measure the basic overhead of reaction invocation. Mac1: 705 msec (Oct. 13, 2020)
• test/C/TimeLimitThreaded.lf: This version of the previous test uses the threaded runtime. There is no exploitable parallelism, so no speedup is expected. Mac1 (with threads = 8): 1.1 sec (Oct. 13, 2020)

Savina Benchmarks

The Savina benchmark suite for actor languages and frameworks provides a number of useful patterns for measuring performance of Lingua Franca programs. See the paper on Savina. None of these benchmarks has any notion of time, and not all comparisons with the actor implementations will be fair.

• benchmark/C/Savina/PingPong.lf: This benchmark tests a feedback interaction where Ping sends a message to Pong, which responds, triggering Ping to send another message. This gets repeated 1,000,000 times. This version of the benchmark runs in the unthreaded runtime. Based on [https://www.scala-lang.org/old/node/54]. Mac1: 105 msec (Oct. 13, 2020)
• benchmark/C/Savina/PingPongThreaded.lf: Version of the previous benchmark runs in the threaded runtime with 8 worker threads. There is no parallelism, so no speedup is expected. Mac1: 165 msec (Oct. 13, 2020)
• benchmark/C/Savina/PingPongMultiThreaded.lf: Version of the previous benchmark runs in the threaded runtime with 8 worker threads and has two independent Ping-Pong benchmarks running together. There is parallelism, so speedup is expected, but instead, we get dramatic slowdown! Mac1: 32.6 sec (Oct. 13, 2020)

Hardware

Mac1: Model Name: MacBook Pro Model Identifier: MacBookPro15,1 Processor Name: Intel Core i9 Processor Speed: 2.3 GHz Number of Processors: 1 Total Number of Cores: 8 L2 Cache (per Core): 256 KB L3 Cache: 16 MB Hyper-Threading Technology: Enabled Memory: 32 GB Operating System: macOS 10.14.6