NOTE: Requires Rust nightly

Extism requires a mutable reference for a plugin to be "callable" (word I made up). As Rust allows only one mutable reference at a time, it imposes usage restrictions with performance penalties.

These benchmark compare performance between various invocation strategies.

Native rendering of the same template using Tera is also included for baseline performance.

Benchmarks:

To run, cargo bench

1. no_compile ./benches/extism.rs: compiles plugin on every run
2. no_compile_ref ./benches/extism.rs: compiles plugin on every run, and reuses reference to compiled plugin.
3. pre_compiled ./benches/extism.rs: pre-compiles plugin before run, and initializes "callable" plugin from it. Most practical situation.
4. pre_compiled_ref ./benches/extism.rs: pre-compiles plugin before run, initializes "callable" plugin from it, and uses reference to the "callable" plugin on every run. Most ideal method, but unfeasible.

Results

Benchmark was run on a dirty computer with tonnes of other processes. So please use results for comparison between implementations, and not as absolute figures of performance.

18:01 atm@lab extism-bench ±|master ✗|→ cargo bench
    Finished `bench` profile [optimized] target(s) in 0.13s
     Running unittests src/lib.rs (target/release/deps/a-d1b90df92d897fe5)

running 1 test
test template::tests::tera_test ... ignored

test result: ok. 0 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in 0.00s

     Running benches/extism.rs (target/release/deps/extism-513f2c5db113092d)

running 4 tests
test no_compile       ... bench:  32,611,308.70 ns/iter (+/- 309,115.92)
test no_compile_ref   ... bench:  10,439,343.80 ns/iter (+/- 93,946.20)
test pre_compiled     ... bench:  12,135,763.40 ns/iter (+/- 309,283.24)
test pre_compiled_ref ... bench:  10,386,191.60 ns/iter (+/- 507,092.77)

test result: ok. 0 passed; 0 failed; 0 ignored; 4 measured; 0 filtered out; finished in 19.84s

     Running benches/rust_template.rs (target/release/deps/rust_template-b11982f9d9e50cf9)

running 1 test
test tera ... bench:       2,918.54 ns/iter (+/- 43.23)

test result: ok. 0 passed; 0 failed; 0 ignored; 1 measured; 0 filtered out; finished in 0.34s

JavaScript Benchmark

Uses nunjucks, typescript, Performance API, and very, very sketchy math. See js/

18:45 atm@lab js ±|master ✗|→ npx tsc src/index.ts  && node src/index.js
mean: 297,756.12 ns
std: 196,902.6 ns
18:45 atm@lab js ±|master ✗|→