Why comnum#

Four properties, each measured from the files of the repository rather than claimed: the numbers below are recomputed by tools/highlights.py and checked by just check (see Updating at the end).

  • Lightweight: one run-time dependency in Python (numpy), 9 direct crates in Rust and no C, C++ or Fortran library to build; 27 MB installed with its dependencies (the benchmark’s environment: komm 22 MB, OptiCommPy 321 MB, Sionna 1004 MB).

  • Fast: compiled Rust, Monte Carlo runs on every core; on one CPU thread, the links of the benchmark take 2.1 to 33 times as long with komm, 2.3 to 4.7 times as long with OptiCommPy and 1.1 to 7.4 times as long with Sionna as with comnum.

  • Safe: unsafe code forbidden in every crate, invalid inputs are errors (one error type), results independent of the thread count; 750 unit tests, 198 validation tests, 276 Python tests.

  • Validated: compared with 139 papers, standards and reference tools (88 fully reproduced, 47 partly), 120 inconsistencies found in them; 104 validation cards rerun a comparison without the paper.

Lightweight#

The Python package needs numpy only at run time (matplotlib is optional, for the plots). The Rust library uses 9 crates of the Rust ecosystem: ndarray, num-complex, rayon, rustfft, serde, serde_json, serde_norway, thiserror, toml. It links no BLAS, LAPACK or FFTW and builds no C, C++ or Fortran code: the special functions, the quadratures, the linear algebra and the solvers are written in the library, so that cargo build works on any platform with Rust alone and the results do not change with the version of a numerical library. In Cargo.lock, which also covers the bindings and the benchmarks, the only crates that bind a system library are windows-sys, the operating-system interfaces of Windows.

What a user installs to run each library of the benchmark: the run-time requirements (without extras), every distribution they pull in, and the size on disk of the whole, measured in the benchmark’s environment (benchmarks/libraries/footprint.py).

Library

Direct requirements

Distributions pulled in

Installed size

comnum 0.2.0

numpy

1

27 MB

komm 0.37.0

numpy, tqdm

2

22 MB

OptiCommPy 0.11.0

matplotlib, mpl-scatter-density, numba, numpy, prettytable, scipy, simple-pid, sympy, tqdm

23

321 MB

Sionna 2.2.0

h5py, importlib-resources, matplotlib, numpy, scipy, sionna-rt, torch

51

1004 MB

Sionna and OptiCommPy are heavier for reasons of their own: PyTorch (GPU, automatic differentiation, learning) and the ray tracer for Sionna, numba and SciPy for OptiCommPy.

Fast#

Every block is compiled Rust; a Python call processes whole arrays, and the Monte Carlo runner (comnum.run_spec, Chain.run, the comnum command) runs the trials on every core, with results independent of the thread count. Times of the benchmark (median of five runs; the rules, the limits of the comparison and the code are on that page), as the time of the other library over the best time of comnum on the same link (above 1: comnum is faster):

Case

komm (1 thread / all)

OptiCommPy (1 thread / all)

Sionna (1 thread / all)

AWGN: QPSK

33× / 147×

3.2× / 8.8×

3.3× / 9.6×

AWGN: 16-QAM

31× / 125×

4.7× / 8.9×

7.4× / 17×

flat Rayleigh: BPSK

—

—

1.1× / 4.0×

flat Rayleigh: QPSK

—

—

1.6× / 4.7×

4 × 4 MIMO, 16-QAM: zero forcing

—

—

1.4× / 4.6×

4 × 4 MIMO, 16-QAM: MMSE

—

—

1.7× / 5.6×

4 × 4 MIMO, 16-QAM: ML (exhaustive)

—

—

3.9× / 8.7×

coding: convolutional (133, 171), soft Viterbi

2.1× / 2.0×

—

4.1× / 3.3×

coding: 5G NR LDPC (1024, 2048)

—

—

4.5× / 2.5×

coding: LDPC (1024, 512), same LLRs

—

4.7× / 14×

7.2× / 3.9×

fibre, 10 × 80 km: split-step Fourier

—

2.3× / 2.3×

—

Measured on Apple M6 (12 cores, 24 GB, macOS 27.0), comnum commit f20118f9160d. The CPU only: Sionna is built for GPUs, where its times would differ.

Safe#

Rust checks memory safety at compile time, and unsafe code is forbidden in every crate of the library (unsafe_code = "forbid" in the workspace lints; the Python extension excepted, whose PyO3 macros expand to it). An invalid input — a negative variance, lengths that differ, a parameter out of range — returns an error of one shared type, raised as ValueError in Python, instead of a wrong result or a crash. Random blocks draw from explicit, reproducible streams: the same seed gives the same result on 1 or 12 threads. Clippy runs in its pedantic mode with every warning an error.

Tests

Count

Unit tests of the crates

750

Validation tests (against papers, standards and tools)

198

Python tests (functions; many are parametrised)

276

Validated#

Every module is compared with an external reference: the figures and tables of papers, the tables of standards, the recorded outputs of reference implementations (OptiCommPy, GNPy, AFF3CT, Sionna, py3gpp), in tests named after the reference. The comparisons that are rerun without reading the paper again are written as validation cards (just card <id>).

Evidence

Count

Documents and tools compared

139

— fully reproduced

88

— partly reproduced

47

— implemented, not yet compared

4

Inconsistencies found in the documents

120

Validation cards

104

Modules at maturity A / B / C

70 / 3 / 2

The comparisons, paper by paper, are in docs/LITERATURE.md and the evidence per module in Maturity.

Updating#

The numbers of this page come from files that scripts regenerate:

just bench-libraries               # the benchmark and the footprint (quiet machine)
python tools/literature_table.py   # after editing validation/literature.yaml
python tools/maturity_table.py     # after editing validation/maturity.yaml
python tools/highlights.py         # this page and the front pages

just check fails when one of these pages is older than its sources.