Benchmarks, every number ======================== Every point, interval, timing and command of the :doc:`benchmark `, by scenario and by machine. The same links simulated with comnum and with the Python libraries that offer them — `komm `_, `OptiCommPy `_ (with and without numba) and `Sionna `_ — with the same parameters and the same number of samples: do the results agree (with the closed form when there is one, with each other otherwise), and how long does each library take? The :doc:`comparisons ` page compares the outputs of single blocks value by value; this one runs whole links. A feature a library does not offer is marked *not available*, with the reason: no substitute is written for it. The scripts import every library as a user would (the GPL-3.0 code of OptiCommPy is imported by the benchmark scripts only, never copied). .. generated by tools/benchmarks_page.py from benchmarks/libraries/results/*.json: do not edit Rules ----- - **Same work.** Same parameters, same number of bits, vectors, frames or samples for every library; every library draws its own bits and noise with its own generator and independent seeds, from the same distributions, so the results agree statistically, not bit for bit (the input field of the split-step case and the LLRs of the case "same LLRs" are identical in every library). - **Confidence intervals.** 95 % intervals: Wilson's for independent errors; for coded frames and MIMO vectors, whose errors come in bursts, the interval of the mean of the per-frame (per-vector) error counts. A result *agrees* when every point lies within four standard errors of the closed form (or of comnum, when there is no closed form; points without errors on both sides are skipped). - **Timing.** ``time.perf_counter``; one warm-up run first, reported apart as *first call* (it contains the numba compilation, from an empty cache, and the lazy initialisations of PyTorch); then the median of five timed runs. The times are per unit of work (bit, vector, frame, propagation). - **Threads.** Every library runs in its own process. *1 thread*: ``OMP_NUM_THREADS``, ``OPENBLAS_NUM_THREADS``, ``MKL_NUM_THREADS``, ``VECLIB_MAXIMUM_THREADS``, ``NUMBA_NUM_THREADS`` and ``RAYON_NUM_THREADS`` set to 1, ``torch.set_num_threads(1)``, ``threads=1`` for ``comnum.run_spec`` and ``--threads 1`` for the binary. *All threads*: the defaults of every library (every core; ``threads=0`` for comnum). The functions of comnum are single-threaded; its Monte Carlo runner (``run_spec``, the binary) runs the trials in parallel. - **Precision.** What every library computes in is stated next to its times: comnum in float64 (float32 on request for the split-step), komm in float64, Sionna in float32 by default, OptiCommPy in float64 except its Gray constellations (complex64) and its LDPC decoder (float32 by default). - **Sionna** 2.x is built on PyTorch (not TensorFlow); it runs eagerly, under ``torch.inference_mode()``, without ``torch.compile``: on the CPU in the modes *1 thread* and *all threads* (``config.device = "cpu"``: Sionna's own default is the first CUDA device as soon as it sees one), and on the GPU in the mode *GPU* (``--threads gpu``: ``config.device = "cuda:0"``, the device every block of Sionna creates its tensors on, the result brought back to the host inside the timed call). **OptiCommPy** has a CuPy copy of its split-step (``optic.models.modelsGPU.ssfm``), run in the mode *GPU* of the split-step scenario, in complex128 and complex64, the transfers to and from the device inside the timed call; its other blocks are CPU only. comnum and komm have no GPU path: in the GPU column they do not run, and the comparison is a library on the device against comnum on the CPU of the same machine. - **OptiCommPy without numba** means ``NUMBA_DISABLE_JIT=1``: numba's own switch, which runs every ``@njit`` function of OptiCommPy as plain Python (OptiCommPy has no option of its own). Its results are those of the compiled code; it is timed only. Limits of this comparison ------------------------- - **The CPU of a laptop, and a GPU.** Sionna is designed for GPUs, large batches, learning and automatic differentiation. The first set of results runs it on the CPU of the same machine as comnum, eagerly, without ``torch.compile``: its home ground is elsewhere. A second set, when present below, runs the same scripts on a machine with a GPU, Sionna (and OptiCommPy's split-step) on the device and comnum on that machine's CPU; comnum has no GPU path, so that comparison is one of a CPU library against an accelerator, which is what a user choosing between the two faces. - **Python libraries only.** komm, OptiCommPy and Sionna are NumPy, numba and PyTorch code. Specialised C++ libraries (AFF3CT, SIMD decoders) are not timed here: the decoders of comnum are timed against those of AFF3CT, one thread, in :doc:`comparisons `. - **Precision.** comnum computes in float64, Sionna in float32 by default: the faster library is not always doing less work, but the two are not strictly the same computation. - **One machine, cases chosen by comnum.** The scenarios are the classical links that several libraries offer; other machines, sizes and cases may rank the libraries differently. The scripts are published to be rerun. - **What this benchmark changed in comnum.** Over flat Rayleigh fading the first run found the comnum functions 3.4 times slower than Sionna on one thread (293 against 87 ns per bit for BPSK): the linear detector built a dozen 1 × 1 matrices for every symbol. It now has a single-stream path (the same filter, as scalars), with the same error counts at every point, and the binding passes the vectors without a copy. - **Why Sionna is fast on a CPU.** It processes the whole batch in one call per operation: the cost of Python is paid once per operation, not once per symbol, and the kernels of PyTorch are vectorised C++ working in float32. Libraries --------- .. list-table:: :header-rows: 1 :widths: 22 63 15 * - Name on this page - What runs - Version * - comnum (functions) - comnum from Python: the functions of the comnum package, called block by block - 0.2.0 * - comnum.run_spec - comnum from Python: comnum.run_spec, the parallel Monte Carlo runner of Rust - 0.2.0 * - comnum binary (Rust) - comnum from Rust: the comnum command-line binary on the same specification - 0.2.0 * - komm - komm (NumPy) - 0.37.0 * - OptiCommPy (numba) - OptiCommPy with numba (its @njit kernels compiled) - 0.11.0 * - OptiCommPy (no numba) - OptiCommPy without numba (NUMBA_DISABLE_JIT=1: the same @njit kernels run as plain Python) - 0.11.0 * - Sionna - Sionna PHY (PyTorch, CPU, eager mode) - 2.2.0 Machine and versions -------------------- .. list-table:: :header-rows: 0 :widths: 30 70 * - CPU - Apple M6 * - Cores - 12 logical (2 super, 4 performance, 6 efficiency) * - Memory - 24.0 GB * - System - macOS 27.0 * - Python - 3.12.15 * - Rust - rustc 1.99.1 (58cf8f9c3 2026-10-13) * - comnum commit - ``f20118f9160d`` * - Generated - 2026-10-30T08:05:23+00:00 * - comnum - 0.2.0 * - komm - 0.37.0 * - OptiCommPy - 0.11.0 * - sionna - 2.2.0 * - torch - 2.15.0 * - numba - 0.68.1 * - numpy - 2.5.4 * - scipy - 1.18.1 Reproducing ----------- On a quiet machine (the timings are wrong on a loaded one), from the root of the repository: .. code-block:: bash uv venv --python 3.12 benchmarks/libraries/.venv source benchmarks/libraries/.venv/bin/activate uv pip install -r benchmarks/libraries/requirements.txt maturin develop --release # comnum, from this checkout cd benchmarks/libraries python run_all.py # builds target/release/comnum first python ../../tools/benchmarks_page.py ``python run_all.py --quick`` runs everything at small sizes in about two minutes, to check that it works (its timings mean nothing). The versions are pinned in `requirements.txt `_; see `benchmarks/libraries/README.md `_. Uncoded QPSK and 16-QAM over AWGN --------------------------------- Scenario 1: uncoded Gray QPSK and 16-QAM over AWGN, BER against the exact closed form. Parameters: - Eb/N0 (dB): 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 - bits per point: 10^7 (quick: 2·10^5), in blocks of 10^6 (quick: 10^5) - timing: one block of 10^6 bits (quick: 2·10^5) at Eb/N0 = 6 dB - theory: exact BER of Gray square QAM, Cho & Yoon, IEEE Trans. Commun. 50(7), 2002 Script: `benchmarks/libraries/s1_awgn.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s1_awgn.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - comnum binary (Rust) - komm - OptiCommPy (numba) - OptiCommPy (no numba) - Sionna * - QPSK - yes - yes - yes - yes - yes - yes - yes * - 16-QAM - yes - yes - yes - yes - yes - yes - yes .. rubric:: QPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 1.90 - yes - binomial * - comnum.run_spec - float64 - theory - 2.53 - yes - binomial * - comnum binary (Rust) - float64 - theory - 2.53 - yes - binomial * - komm - float64 - theory - 2.37 - yes - binomial * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - theory - 3.31 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.13 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - comnum binary (Rust) - komm - OptiCommPy (numba) - Sionna * - 0 - 7.86e-2 - 7.86e-2 [7.84e-2, 7.88e-2] - 7.87e-2 [7.85e-2, 7.89e-2] - 7.87e-2 [7.85e-2, 7.89e-2] - 7.86e-2 [7.85e-2, 7.88e-2] - 7.86e-2 [7.84e-2, 7.88e-2] - 7.86e-2 [7.84e-2, 7.88e-2] * - 1 - 5.63e-2 - 5.63e-2 [5.62e-2, 5.65e-2] - 5.63e-2 [5.61e-2, 5.64e-2] - 5.63e-2 [5.61e-2, 5.64e-2] - 5.64e-2 [5.63e-2, 5.65e-2] - 5.63e-2 [5.61e-2, 5.64e-2] - 5.61e-2 [5.60e-2, 5.63e-2] * - 2 - 3.75e-2 - 3.75e-2 [3.74e-2, 3.76e-2] - 3.74e-2 [3.73e-2, 3.76e-2] - 3.74e-2 [3.73e-2, 3.76e-2] - 3.75e-2 [3.74e-2, 3.77e-2] - 3.75e-2 [3.73e-2, 3.76e-2] - 3.76e-2 [3.75e-2, 3.77e-2] * - 3 - 2.29e-2 - 2.28e-2 [2.27e-2, 2.29e-2] - 2.30e-2 [2.29e-2, 2.31e-2] - 2.30e-2 [2.29e-2, 2.31e-2] - 2.29e-2 [2.28e-2, 2.29e-2] - 2.29e-2 [2.28e-2, 2.30e-2] - 2.29e-2 [2.28e-2, 2.30e-2] * - 4 - 1.25e-2 - 1.25e-2 [1.25e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.25e-2, 1.26e-2] * - 5 - 5.95e-3 - 5.95e-3 [5.90e-3, 6.00e-3] - 5.96e-3 [5.92e-3, 6.01e-3] - 5.96e-3 [5.92e-3, 6.01e-3] - 5.98e-3 [5.93e-3, 6.03e-3] - 5.93e-3 [5.89e-3, 5.98e-3] - 5.93e-3 [5.88e-3, 5.98e-3] * - 6 - 2.39e-3 - 2.39e-3 [2.36e-3, 2.42e-3] - 2.41e-3 [2.38e-3, 2.44e-3] - 2.41e-3 [2.38e-3, 2.44e-3] - 2.36e-3 [2.33e-3, 2.39e-3] - 2.43e-3 [2.40e-3, 2.46e-3] - 2.39e-3 [2.36e-3, 2.42e-3] * - 7 - 7.73e-4 - 7.56e-4 [7.39e-4, 7.73e-4] - 7.77e-4 [7.60e-4, 7.95e-4] - 7.77e-4 [7.60e-4, 7.95e-4] - 7.64e-4 [7.47e-4, 7.81e-4] - 8.02e-4 [7.84e-4, 8.20e-4] - 7.88e-4 [7.70e-4, 8.05e-4] * - 8 - 1.91e-4 - 1.84e-4 [1.76e-4, 1.92e-4] - 1.93e-4 [1.84e-4, 2.01e-4] - 1.93e-4 [1.84e-4, 2.01e-4] - 1.93e-4 [1.84e-4, 2.01e-4] - 1.90e-4 [1.82e-4, 1.99e-4] - 1.82e-4 [1.73e-4, 1.90e-4] * - 9 - 3.36e-5 - 3.21e-5 [2.88e-5, 3.58e-5] - 3.41e-5 [3.07e-5, 3.79e-5] - 3.41e-5 [3.07e-5, 3.79e-5] - 3.36e-5 [3.02e-5, 3.74e-5] - 3.42e-5 [3.08e-5, 3.80e-5] - 3.57e-5 [3.22e-5, 3.96e-5] * - 10 - 3.87e-6 - 4.00e-6 [2.94e-6, 5.45e-6] - 3.20e-6 [2.27e-6, 4.52e-6] - 3.20e-6 [2.27e-6, 4.52e-6] - 2.40e-6 [1.61e-6, 3.57e-6] - 4.10e-6 [3.02e-6, 5.56e-6] - 3.70e-6 [2.68e-6, 5.10e-6] .. rubric:: 16-QAM: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.15 - yes - binomial * - comnum.run_spec - float64 - theory - 1.64 - yes - binomial * - comnum binary (Rust) - float64 - theory - 1.64 - yes - binomial * - komm - float64 - theory - 2.08 - yes - binomial * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - theory - 1.42 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.41 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - comnum binary (Rust) - komm - OptiCommPy (numba) - Sionna * - 0 - 1.41e-1 - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] * - 1 - 1.19e-1 - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] * - 2 - 9.77e-2 - 9.78e-2 [9.76e-2, 9.80e-2] - 9.77e-2 [9.75e-2, 9.79e-2] - 9.77e-2 [9.75e-2, 9.79e-2] - 9.77e-2 [9.75e-2, 9.79e-2] - 9.78e-2 [9.76e-2, 9.80e-2] - 9.75e-2 [9.73e-2, 9.77e-2] * - 3 - 7.75e-2 - 7.75e-2 [7.74e-2, 7.77e-2] - 7.74e-2 [7.73e-2, 7.76e-2] - 7.74e-2 [7.73e-2, 7.76e-2] - 7.76e-2 [7.74e-2, 7.77e-2] - 7.75e-2 [7.73e-2, 7.76e-2] - 7.74e-2 [7.73e-2, 7.76e-2] * - 4 - 5.86e-2 - 5.87e-2 [5.85e-2, 5.88e-2] - 5.86e-2 [5.84e-2, 5.87e-2] - 5.86e-2 [5.84e-2, 5.87e-2] - 5.86e-2 [5.85e-2, 5.88e-2] - 5.85e-2 [5.84e-2, 5.87e-2] - 5.85e-2 [5.84e-2, 5.87e-2] * - 5 - 4.19e-2 - 4.18e-2 [4.17e-2, 4.19e-2] - 4.19e-2 [4.17e-2, 4.20e-2] - 4.19e-2 [4.17e-2, 4.20e-2] - 4.19e-2 [4.18e-2, 4.20e-2] - 4.20e-2 [4.19e-2, 4.21e-2] - 4.20e-2 [4.19e-2, 4.21e-2] * - 6 - 2.79e-2 - 2.79e-2 [2.78e-2, 2.80e-2] - 2.79e-2 [2.78e-2, 2.80e-2] - 2.79e-2 [2.78e-2, 2.80e-2] - 2.79e-2 [2.78e-2, 2.80e-2] - 2.78e-2 [2.77e-2, 2.79e-2] - 2.79e-2 [2.78e-2, 2.81e-2] * - 7 - 1.70e-2 - 1.70e-2 [1.69e-2, 1.71e-2] - 1.70e-2 [1.69e-2, 1.71e-2] - 1.70e-2 [1.69e-2, 1.71e-2] - 1.70e-2 [1.69e-2, 1.70e-2] - 1.70e-2 [1.69e-2, 1.70e-2] - 1.70e-2 [1.69e-2, 1.71e-2] * - 8 - 9.25e-3 - 9.24e-3 [9.18e-3, 9.30e-3] - 9.24e-3 [9.18e-3, 9.30e-3] - 9.24e-3 [9.18e-3, 9.30e-3] - 9.31e-3 [9.25e-3, 9.37e-3] - 9.22e-3 [9.16e-3, 9.28e-3] - 9.24e-3 [9.19e-3, 9.30e-3] * - 9 - 4.39e-3 - 4.35e-3 [4.30e-3, 4.39e-3] - 4.42e-3 [4.38e-3, 4.47e-3] - 4.42e-3 [4.38e-3, 4.47e-3] - 4.39e-3 [4.35e-3, 4.43e-3] - 4.40e-3 [4.36e-3, 4.44e-3] - 4.37e-3 [4.33e-3, 4.41e-3] * - 10 - 1.75e-3 - 1.75e-3 [1.73e-3, 1.78e-3] - 1.78e-3 [1.75e-3, 1.80e-3] - 1.78e-3 [1.75e-3, 1.80e-3] - 1.73e-3 [1.71e-3, 1.76e-3] - 1.76e-3 [1.73e-3, 1.79e-3] - 1.76e-3 [1.73e-3, 1.78e-3] .. rubric:: QPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 6.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 7.58 - 7.56 - 1.00 - 0.0198 s * - comnum.run_spec - float64 - 9.60 - 1.71 - 1.27 - 0.00968 s * - comnum binary (Rust) - float64 - 12.9 - 4.57 - 1.70 - 0.0136 s * - komm - float64 - 249 - 252 - 32.9 - 0.253 s * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - 24.2 - 15.1 - 3.19 - 0.921 s * - OptiCommPy (no numba) - complex64 symbols (its Gray constellation), float64 noise - 373 - 373 - 49.2 - 0.0376 s * - Sionna - float32 (config.precision = 'single') - 25.2 - 16.4 - 3.33 - 0.0377 s .. rubric:: 16-QAM: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 6.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 4.44 - 4.39 - 1.00 - 0.00451 s * - comnum.run_spec - float64 - 5.41 - 1.11 - 1.22 - 0.00539 s * - comnum binary (Rust) - float64 - 8.14 - 3.76 - 1.83 - 0.0086 s * - komm - float64 - 136 - 139 - 30.6 - 0.137 s * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - 20.9 - 9.85 - 4.70 - 0.0213 s * - OptiCommPy (no numba) - complex64 symbols (its Gray constellation), float64 noise - 192 - 191 - 43.2 - 0.0192 s * - Sionna - float32 (config.precision = 'single') - 32.9 - 19.1 - 7.41 - 0.0362 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib comnum --threads 1 --work results/_work --out results/_work/s1_awgn_comnum_1.json python s1_awgn.py --lib comnum --threads all --work results/_work --out results/_work/s1_awgn_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib comnum-spec --threads 1 --work results/_work --out results/_work/s1_awgn_comnum-spec_1.json python s1_awgn.py --lib comnum-spec --threads all --work results/_work --out results/_work/s1_awgn_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib comnum-rust --threads 1 --work results/_work --out results/_work/s1_awgn_comnum-rust_1.json python s1_awgn.py --lib comnum-rust --threads all --work results/_work --out results/_work/s1_awgn_comnum-rust_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib komm --threads 1 --work results/_work --out results/_work/s1_awgn_komm_1.json python s1_awgn.py --lib komm --threads all --work results/_work --out results/_work/s1_awgn_komm_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib opticommpy --threads 1 --work results/_work --out results/_work/s1_awgn_opticommpy_1.json python s1_awgn.py --lib opticommpy --threads all --work results/_work --out results/_work/s1_awgn_opticommpy_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 NUMBA_DISABLE_JIT=1 python s1_awgn.py --lib opticommpy-nojit --threads 1 --work results/_work --out results/_work/s1_awgn_opticommpy-nojit_1.json --timing-only NUMBA_DISABLE_JIT=1 python s1_awgn.py --lib opticommpy-nojit --threads all --work results/_work --out results/_work/s1_awgn_opticommpy-nojit_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib sionna --threads 1 --work results/_work --out results/_work/s1_awgn_sionna_1.json python s1_awgn.py --lib sionna --threads all --work results/_work --out results/_work/s1_awgn_sionna_all.json --timing-only BPSK and QPSK over flat Rayleigh fading --------------------------------------- Scenario 2: BPSK and Gray QPSK over flat Rayleigh fading (one coefficient per symbol, perfectly known at the receiver), BER against the closed form. Parameters: - Eb/N0 (dB): 0, 5, 10, 15, 20, 25, 30 - bits per point: 10^7 (quick: 2·10^5), in blocks of 10^6 (quick: 10^5) - channel: y = h x + w, h ~ CN(0, 1) independent for every symbol, known at the receiver - timing: one block of 10^6 bits (quick: 2·10^5) at Eb/N0 = 10 dB - theory: BPSK: (1 - sqrt(G / (1 + G))) / 2; Gray QPSK: the same per bit (Proakis & Salehi, 5th ed., Sec. 13.3) Script: `benchmarks/libraries/s2_rayleigh.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s2_rayleigh.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - komm - OptiCommPy (numba) - OptiCommPy (no numba) - Sionna * - BPSK - yes - yes - not available: no fading channel - not available: no fading channel - not available: no fading channel - yes * - QPSK - yes - yes - not available: no fading channel - not available: no fading channel - not available: no fading channel - yes .. rubric:: BPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.22 - yes - binomial * - comnum.run_spec - float64 - theory - 1.78 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.35 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.46e-1 - 1.46e-1 [1.46e-1, 1.47e-1] - 1.46e-1 [1.46e-1, 1.47e-1] - 1.47e-1 [1.46e-1, 1.47e-1] * - 5 - 6.42e-2 - 6.43e-2 [6.41e-2, 6.44e-2] - 6.42e-2 [6.41e-2, 6.44e-2] - 6.42e-2 [6.40e-2, 6.43e-2] * - 10 - 2.33e-2 - 2.33e-2 [2.32e-2, 2.34e-2] - 2.32e-2 [2.32e-2, 2.33e-2] - 2.33e-2 [2.32e-2, 2.34e-2] * - 15 - 7.72e-3 - 7.73e-3 [7.67e-3, 7.78e-3] - 7.76e-3 [7.71e-3, 7.82e-3] - 7.75e-3 [7.69e-3, 7.80e-3] * - 20 - 2.48e-3 - 2.51e-3 [2.48e-3, 2.54e-3] - 2.46e-3 [2.43e-3, 2.49e-3] - 2.52e-3 [2.49e-3, 2.55e-3] * - 25 - 7.89e-4 - 8.08e-4 [7.91e-4, 8.26e-4] - 7.83e-4 [7.66e-4, 8.00e-4] - 7.88e-4 [7.71e-4, 8.06e-4] * - 30 - 2.50e-4 - 2.52e-4 [2.42e-4, 2.62e-4] - 2.41e-4 [2.31e-4, 2.51e-4] - 2.43e-4 [2.34e-4, 2.53e-4] .. rubric:: QPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.08 - yes - binomial * - comnum.run_spec - float64 - theory - 1.17 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 3.13 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.46e-1 - 1.46e-1 [1.46e-1, 1.46e-1] - 1.46e-1 [1.46e-1, 1.47e-1] - 1.46e-1 [1.46e-1, 1.47e-1] * - 5 - 6.42e-2 - 6.41e-2 [6.39e-2, 6.42e-2] - 6.42e-2 [6.40e-2, 6.43e-2] - 6.42e-2 [6.40e-2, 6.43e-2] * - 10 - 2.33e-2 - 2.34e-2 [2.33e-2, 2.34e-2] - 2.33e-2 [2.32e-2, 2.34e-2] - 2.31e-2 [2.30e-2, 2.32e-2] * - 15 - 7.72e-3 - 7.77e-3 [7.72e-3, 7.82e-3] - 7.73e-3 [7.67e-3, 7.78e-3] - 7.67e-3 [7.61e-3, 7.72e-3] * - 20 - 2.48e-3 - 2.51e-3 [2.48e-3, 2.54e-3] - 2.50e-3 [2.47e-3, 2.53e-3] - 2.46e-3 [2.43e-3, 2.50e-3] * - 25 - 7.89e-4 - 8.02e-4 [7.84e-4, 8.19e-4] - 7.81e-4 [7.64e-4, 7.99e-4] - 7.92e-4 [7.75e-4, 8.10e-4] * - 30 - 2.50e-4 - 2.53e-4 [2.44e-4, 2.63e-4] - 2.46e-4 [2.37e-4, 2.56e-4] - 2.51e-4 [2.41e-4, 2.61e-4] .. rubric:: BPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 10.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 76.5 - 77.2 - 1.00 - 0.0814 s * - comnum.run_spec - float64 - 99.7 - 18.0 - 1.30 - 0.1 s * - Sionna - float32 (config.precision = 'single') - 87.6 - 72.6 - 1.15 - 0.099 s .. rubric:: QPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 10.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 39.1 - 39.4 - 1.00 - 0.0404 s * - comnum.run_spec - float64 - 50.9 - 9.96 - 1.30 - 0.0514 s * - Sionna - float32 (config.precision = 'single') - 63.0 - 46.7 - 1.61 - 0.0628 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib comnum --threads 1 --work results/_work --out results/_work/s2_rayleigh_comnum_1.json python s2_rayleigh.py --lib comnum --threads all --work results/_work --out results/_work/s2_rayleigh_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib comnum-spec --threads 1 --work results/_work --out results/_work/s2_rayleigh_comnum-spec_1.json python s2_rayleigh.py --lib comnum-spec --threads all --work results/_work --out results/_work/s2_rayleigh_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib sionna --threads 1 --work results/_work --out results/_work/s2_rayleigh_sionna_1.json python s2_rayleigh.py --lib sionna --threads all --work results/_work --out results/_work/s2_rayleigh_sionna_all.json --timing-only 4 x 4 MIMO, 16-QAM, i.i.d. Rayleigh fading: ZF, MMSE and ML detection --------------------------------------------------------------------- Scenario 3: 4 x 4 spatial multiplexing of Gray 16-QAM over i.i.d. Rayleigh fading (one channel matrix per vector, known at the receiver), detected by zero forcing, MMSE and maximum likelihood. Parameters: - Eb/N0 (dB): 0, 4, 8, 12, 16 - vectors per point: linear detectors and sphere: 10^5 (quick: 5·10^3); exhaustive ML: 2·10^4 (quick: 10^3) - timing: 10^4 vectors at Eb/N0 = 12 dB (exhaustive ML: 10^3; quick: 2·10^3 and 2·10^2) - theory: ZF: 16-QAM over SISO Rayleigh fading (Winters, Salz & Gitlin, IEEE Trans. Commun. 42(2/3/4), 1994) Script: `benchmarks/libraries/s3_mimo.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s3_mimo.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - komm - OptiCommPy (numba) - OptiCommPy (no numba) - Sionna * - zero forcing - yes - yes - not available: no MIMO detection - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - MMSE - yes - yes - not available: no MIMO detection - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - ML (exhaustive) - yes - yes - not available: no MIMO detection - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - ML (sphere decoder) - yes - yes - not available: no MIMO detection - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - not available: no sphere decoder (its exhaustive ML detector is the case ml) .. rubric:: zero forcing: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 1.88 - yes - per-frame * - comnum.run_spec - float64 - theory - 2.98 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 1.79 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.98e-1 - 1.98e-1 [1.97e-1, 1.99e-1] - 1.98e-1 [1.97e-1, 1.98e-1] - 1.98e-1 [1.97e-1, 1.99e-1] * - 4 - 1.20e-1 - 1.19e-1 [1.18e-1, 1.20e-1] - 1.19e-1 [1.19e-1, 1.20e-1] - 1.19e-1 [1.19e-1, 1.20e-1] * - 8 - 6.20e-2 - 6.16e-2 [6.10e-2, 6.23e-2] - 6.14e-2 [6.10e-2, 6.18e-2] - 6.26e-2 [6.19e-2, 6.32e-2] * - 12 - 2.83e-2 - 2.82e-2 [2.78e-2, 2.87e-2] - 2.81e-2 [2.79e-2, 2.84e-2] - 2.83e-2 [2.78e-2, 2.87e-2] * - 16 - 1.20e-2 - 1.21e-2 [1.18e-2, 1.24e-2] - 1.19e-2 [1.17e-2, 1.20e-2] - 1.19e-2 [1.16e-2, 1.22e-2] .. rubric:: MMSE: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 3.63 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.57 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.39e-1 [1.38e-1, 1.40e-1] - 1.38e-1 [1.38e-1, 1.39e-1] - 1.38e-1 [1.38e-1, 1.39e-1] * - 4 - 8.27e-2 [8.21e-2, 8.33e-2] - 8.40e-2 [8.36e-2, 8.44e-2] - 8.33e-2 [8.28e-2, 8.39e-2] * - 8 - 4.31e-2 [4.26e-2, 4.35e-2] - 4.29e-2 [4.26e-2, 4.32e-2] - 4.32e-2 [4.28e-2, 4.37e-2] * - 12 - 1.97e-2 [1.94e-2, 2.00e-2] - 1.99e-2 [1.97e-2, 2.01e-2] - 1.96e-2 [1.93e-2, 1.99e-2] * - 16 - 8.42e-3 [8.20e-3, 8.64e-3] - 8.33e-3 [8.19e-3, 8.47e-3] - 8.23e-3 [8.02e-3, 8.45e-3] .. rubric:: ML (exhaustive): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 2.53 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.09 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (320,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.25e-1 [1.23e-1, 1.26e-1] - 1.25e-1 [1.24e-1, 1.26e-1] - 1.25e-1 [1.23e-1, 1.27e-1] * - 4 - 4.03e-2 [3.90e-2, 4.16e-2] - 3.99e-2 [3.92e-2, 4.06e-2] - 4.05e-2 [3.92e-2, 4.17e-2] * - 8 - 4.12e-3 [3.68e-3, 4.56e-3] - 4.77e-3 [4.54e-3, 5.02e-3] - 4.47e-3 [4.02e-3, 4.93e-3] * - 12 - 2.34e-4 [1.24e-4, 3.45e-4] - 2.28e-4 [1.81e-4, 2.87e-4] - 1.59e-4 [7.61e-5, 2.43e-4] * - 16 - 0 [0, 9.38e-6] - 0 [0, 1.20e-5] - 1.25e-5 [0, 3.70e-5] .. rubric:: ML (sphere decoder): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 2.84 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec * - 0 - 1.24e-1 [1.23e-1, 1.25e-1] - 1.25e-1 [1.24e-1, 1.25e-1] * - 4 - 3.97e-2 [3.92e-2, 4.03e-2] - 4.06e-2 [4.03e-2, 4.09e-2] * - 8 - 4.44e-3 [4.23e-3, 4.64e-3] - 4.62e-3 [4.51e-3, 4.73e-3] * - 12 - 1.91e-4 [1.49e-4, 2.32e-4] - 1.95e-4 [1.75e-4, 2.18e-4] * - 16 - 3.13e-6 [0, 6.37e-6] - 1.25e-5 [8.09e-6, 1.93e-5] .. rubric:: zero forcing: time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 0.76 - 0.763 - 1.00 - 0.00823 s * - comnum.run_spec - float64 - 0.776 - 0.175 - 1.02 - 0.00794 s * - Sionna - float32 (config.precision = 'single') - 1.06 - 0.796 - 1.40 - 0.0148 s .. rubric:: MMSE: time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 0.761 - 0.76 - 1.00 - 0.00759 s * - comnum.run_spec - float64 - 0.778 - 0.174 - 1.02 - 0.00782 s * - Sionna - float32 (config.precision = 'single') - 1.28 - 0.98 - 1.68 - 0.0129 s .. rubric:: ML (exhaustive): time Median of 5 runs, µs per vector; workload: 1000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 393 - 394 - 1.00 - 0.393 s * - comnum.run_spec - float64 - 393 - 105 - 1 - 0.398 s * - Sionna - float32 (config.precision = 'single') - 1519 - 908 - 3.87 - 1.57 s .. rubric:: ML (sphere decoder): time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 1.02 - 1.02 - 1.00 - 0.0108 s * - comnum.run_spec - float64 - 1.03 - 0.209 - 1.01 - 0.0104 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib comnum --threads 1 --work results/_work --out results/_work/s3_mimo_comnum_1.json python s3_mimo.py --lib comnum --threads all --work results/_work --out results/_work/s3_mimo_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib comnum-spec --threads 1 --work results/_work --out results/_work/s3_mimo_comnum-spec_1.json python s3_mimo.py --lib comnum-spec --threads all --work results/_work --out results/_work/s3_mimo_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib sionna --threads 1 --work results/_work --out results/_work/s3_mimo_sionna_1.json python s3_mimo.py --lib sionna --threads all --work results/_work --out results/_work/s3_mimo_sionna_all.json --timing-only Channel coding over BPSK and AWGN --------------------------------- Scenario 4: channel coding over BPSK and AWGN. Parameters: - convolutional (133, 171), soft Viterbi: K = 7, (133, 171), k = 1024 + 6 tail bits, soft Viterbi; Eb/N0 = 1, 2, 3, 4 dB - 5G NR LDPC (1024, 2048): 5G NR LDPC, k = 1024, n = 2048 (base graph 2, Z = 104), flooding sum-product, 50 iterations; Eb/N0 = 1, 1.5, 2, 2.5 dB - LDPC (1024, 512), same LLRs: PEG (1024, 512), flooding sum-product, 50 iterations, identical LLRs; Eb/N0 = 1.5, 2, 2.5, 3 dB - frames per point: 2000 (quick: 100) - timing: 200 frames (quick: 20) at Eb/N0 = 3 dB (conv) or 2 dB (LDPC); OptiCommPy without numba: 2 frames Script: `benchmarks/libraries/s4_coding.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s4_coding.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - comnum binary (Rust) - komm - OptiCommPy (numba) - OptiCommPy (no numba) - Sionna * - convolutional (133, 171), soft Viterbi - yes - yes - yes - yes - not available: no convolutional code - not available: no convolutional code - yes * - 5G NR LDPC (1024, 2048) - yes - not available: the specification format has no 5G NR LDPC code (NrLdpc: Python and Rust API) - not available: the specification format has no 5G NR LDPC code (NrLdpc: Python and Rust API) - not available: no LDPC code - not available: no 5G NR LDPC code (its decoder takes a parity-check matrix: see ldpc_same_llr) - not available: no 5G NR LDPC code (its decoder takes a parity-check matrix: see ldpc_same_llr) - yes * - LDPC (1024, 512), same LLRs - yes - not available: the runner draws its own channel (this case feeds the same LLRs to every decoder) - not available: the runner draws its own channel (this case feeds the same LLRs to every decoder) - not available: no LDPC (belief-propagation) decoder - yes - yes - yes .. rubric:: convolutional (133, 171), soft Viterbi: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 1.35 - 1.33 - yes - binomial * - comnum binary (Rust) - float64 - comnum - 1.35 - 1.33 - yes - binomial * - komm - float64 - comnum - 0.974 - 1.81 - yes - per-frame * - Sionna - float32 (config.precision = 'single') - comnum - 1.41 - 2.53 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - union bound - comnum (functions) - comnum.run_spec - comnum binary (Rust) - komm - Sionna * - 1 - 3.05e-1 - 4.02e-2 [3.91e-2, 4.13e-2] - 4.02e-2 [3.99e-2, 4.05e-2] - 4.02e-2 [3.99e-2, 4.05e-2] - 4.05e-2 [3.94e-2, 4.17e-2] - 4.03e-2 [3.91e-2, 4.14e-2] * - 2 - 1.65e-2 - 4.97e-3 [4.64e-3, 5.30e-3] - 5.16e-3 [5.06e-3, 5.26e-3] - 5.16e-3 [5.06e-3, 5.26e-3] - 5.21e-3 [4.86e-3, 5.55e-3] - 5.32e-3 [4.97e-3, 5.67e-3] * - 3 - 6.27e-4 - 3.31e-4 [2.59e-4, 4.02e-4] - 2.98e-4 [2.76e-4, 3.23e-4] - 2.98e-4 [2.76e-4, 3.23e-4] - 3.71e-4 [3.04e-4, 4.38e-4] - 3.64e-4 [2.94e-4, 4.34e-4] * - 4 - 2.02e-5 - 2.54e-5 [8.46e-6, 4.23e-5] - 1.32e-5 [9.06e-6, 1.92e-5] - 1.32e-5 [9.06e-6, 1.92e-5] - 2.00e-5 [3.66e-6, 3.64e-5] - 3.08e-5 [9.04e-6, 5.25e-5] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - comnum binary (Rust) - komm - Sionna * - 1 - 9.73e-1 [9.66e-1, 9.80e-1] - 9.77e-1 [9.69e-1, 9.83e-1] - 9.77e-1 [9.69e-1, 9.83e-1] - 9.82e-1 [9.75e-1, 9.87e-1] - 9.85e-1 [9.79e-1, 9.89e-1] * - 2 - 5.05e-1 [4.84e-1, 5.27e-1] - 4.84e-1 [4.63e-1, 5.06e-1] - 4.84e-1 [4.63e-1, 5.06e-1] - 5.01e-1 [4.79e-1, 5.23e-1] - 5.13e-1 [4.91e-1, 5.35e-1] * - 3 - 5.95e-2 [5.00e-2, 7.07e-2] - 6.05e-2 [5.09e-2, 7.18e-2] - 6.05e-2 [5.09e-2, 7.18e-2] - 7.15e-2 [6.10e-2, 8.36e-2] - 6.70e-2 [5.69e-2, 7.88e-2] * - 4 - 6.00e-3 [3.44e-3, 1.05e-2] - 5.00e-3 [2.72e-3, 9.18e-3] - 5.00e-3 [2.72e-3, 9.18e-3] - 5.00e-3 [2.72e-3, 9.18e-3] - 5.50e-3 [3.07e-3, 9.82e-3] .. rubric:: 5G NR LDPC (1024, 2048): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - Sionna - float32 (config.precision = 'single') - comnum - 0.893 - 1.21 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - Sionna * - 1 - 8.79e-3 [7.46e-3, 1.01e-2] - 9.30e-3 [7.94e-3, 1.07e-2] * - 1.5 - 3.47e-5 [0, 1.03e-4] - 9.38e-5 [0, 2.04e-4] * - 2 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] * - 2.5 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - Sionna * - 1 - 9.40e-2 [8.20e-2, 1.08e-1] - 1.05e-1 [9.28e-2, 1.20e-1] * - 1.5 - 5.00e-4 [8.83e-5, 2.83e-3] - 1.50e-3 [5.10e-4, 4.40e-3] * - 2 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] * - 2.5 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] .. rubric:: LDPC (1024, 512), same LLRs: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - comnum - 0.076 - 0 - yes - per-frame * - Sionna - float32 (config.precision = 'single') - comnum - 0.0906 - 0.0381 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - OptiCommPy (numba) - Sionna * - 1.5 - 1.53e-2 [1.39e-2, 1.66e-2] - 1.53e-2 [1.40e-2, 1.67e-2] - 1.54e-2 [1.40e-2, 1.67e-2] * - 2 - 1.37e-3 [9.56e-4, 1.78e-3] - 1.37e-3 [9.55e-4, 1.78e-3] - 1.37e-3 [9.57e-4, 1.78e-3] * - 2.5 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] * - 3 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - OptiCommPy (numba) - Sionna * - 1.5 - 2.21e-1 [2.03e-1, 2.40e-1] - 2.21e-1 [2.03e-1, 2.40e-1] - 2.21e-1 [2.04e-1, 2.40e-1] * - 2 - 2.30e-2 [1.73e-2, 3.05e-2] - 2.30e-2 [1.73e-2, 3.05e-2] - 2.30e-2 [1.73e-2, 3.05e-2] * - 2.5 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] * - 3 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] .. rubric:: Same LLRs: the decisions frame by frame The decoders received identical LLRs. Over all the points: the frames where a library makes as many bit errors as comnum, and the frames it decodes (or fails to decode) as comnum does. .. list-table:: :header-rows: 1 * - Library - Points - Frames - Same bit errors - Same frame outcome * - OptiCommPy (numba) - 4 - 8000 - 7996 (99.95 %) - 8000 (100.00 %) * - Sionna - 4 - 8000 - 7980 (99.75 %) - 7999 (99.99 %) .. rubric:: convolutional (133, 171), soft Viterbi: decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 3.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 0.276 - 0.277 - 1.00 - 0.000821 s * - komm - float64 - 0.566 - 0.564 - 2.05 - 0.0188 s * - Sionna - float32 (config.precision = 'single') - 1.13 - 0.916 - 4.10 - 0.0305 s .. rubric:: 5G NR LDPC (1024, 2048): decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 1.70 - 1.68 - 1.00 - 0.00361 s * - Sionna - float32 (config.precision = 'single') - 7.74 - 4.23 - 4.54 - 0.0213 s .. rubric:: LDPC (1024, 512), same LLRs: decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 0.316 - 0.323 - 1.00 - 0.000748 s * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - 1.47 - 4.53 - 4.66 - 1.47 s * - OptiCommPy (no numba) - float32 (decodeLDPC default prec = np.float32) - 42.4 - 42.0 - 134 - 0.082 s * - Sionna - float32 (config.precision = 'single') - 2.29 - 1.26 - 7.25 - 0.00648 s .. rubric:: convolutional (133, 171), soft Viterbi: whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 3.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 0.325 - 0.325 - 1.00 - 0.000821 s * - comnum.run_spec - float64 - 0.324 - 0.0513 - 0.997 - 0.0651 s * - comnum binary (Rust) - float64 - 0.336 - 0.0688 - 1.03 - 0.0696 s * - komm - float64 - 0.687 - 0.687 - 2.11 - 0.0188 s * - Sionna - float32 (config.precision = 'single') - 1.23 - 1.01 - 3.79 - 0.0305 s .. rubric:: 5G NR LDPC (1024, 2048): whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 1.75 - 1.74 - 1.00 - 0.00361 s * - Sionna - float32 (config.precision = 'single') - 7.84 - 4.28 - 4.48 - 0.0213 s .. rubric:: LDPC (1024, 512), same LLRs: whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions) - float64 - 0.356 - 0.363 - 1.00 - 0.000748 s * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - 1.44 - 4.31 - 4.04 - 1.47 s * - OptiCommPy (no numba) - float32 (decodeLDPC default prec = np.float32) - 37.0 - 36.5 - 104 - 0.082 s * - Sionna - float32 (config.precision = 'single') - 2.35 - 1.32 - 6.60 - 0.00648 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib comnum --threads 1 --work results/_work --out results/_work/s4_coding_comnum_1.json python s4_coding.py --lib comnum --threads all --work results/_work --out results/_work/s4_coding_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib comnum-spec --threads 1 --work results/_work --out results/_work/s4_coding_comnum-spec_1.json python s4_coding.py --lib comnum-spec --threads all --work results/_work --out results/_work/s4_coding_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib comnum-rust --threads 1 --work results/_work --out results/_work/s4_coding_comnum-rust_1.json python s4_coding.py --lib comnum-rust --threads all --work results/_work --out results/_work/s4_coding_comnum-rust_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib komm --threads 1 --work results/_work --out results/_work/s4_coding_komm_1.json python s4_coding.py --lib komm --threads all --work results/_work --out results/_work/s4_coding_komm_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib opticommpy --threads 1 --work results/_work --out results/_work/s4_coding_opticommpy_1.json python s4_coding.py --lib opticommpy --threads all --work results/_work --out results/_work/s4_coding_opticommpy_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 NUMBA_DISABLE_JIT=1 python s4_coding.py --lib opticommpy-nojit --threads 1 --work results/_work --out results/_work/s4_coding_opticommpy-nojit_1.json --timing-only NUMBA_DISABLE_JIT=1 python s4_coding.py --lib opticommpy-nojit --threads all --work results/_work --out results/_work/s4_coding_opticommpy-nojit_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib sionna --threads 1 --work results/_work --out results/_work/s4_coding_sionna_1.json python s4_coding.py --lib sionna --threads all --work results/_work --out results/_work/s4_coding_sionna_all.json --timing-only Split-step Fourier propagation, 16-QAM, 32 GBd, 10 x 80 km ---------------------------------------------------------- Scenario 5: single-polarisation split-step Fourier propagation of 32 GBd 16-QAM over 10 x 80 km of standard single-mode fibre, a fixed step of 1 km (80 steps per span), noiseless amplification at the end of every span (gain e^{alpha L}). Parameters: - fibre: alpha = 0.2 dB/km, D = 17 ps/nm/km, gamma = 1.3 /W/km, fc = 193.1 THz - link: 10 x 80 km, ideal (noiseless) amplification after every span - step: 1 km (800 steps); reference: comnum at 0.05 km (16000 steps) - signal: 16-QAM, 32 GBd, RRC roll-off 0.1, 4 samples per symbol, 4 dBm; 2^14 symbols (quick: 2^11) - timing: one propagation over the 10 spans Script: `benchmarks/libraries/s5_ssfm.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s5_ssfm.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - komm - OptiCommPy (numba) - OptiCommPy (no numba) - Sionna * - split-step Fourier - yes - not available: no fibre model - yes - yes - yes .. rubric:: Output fields NMSE of the output field, ``sum |a - b|^2 / sum |b|^2``, against comnum in float64 at the same step and against the step-converged reference (comnum, 0.05 km). The NMSE against the reference is the error of the 1 km step itself. .. list-table:: :header-rows: 1 * - Field - NMSE vs comnum (float64, 1 km) - NMSE vs reference * - comnum, float32 - 6.35e-9 - 6.14e-8 * - comnum, float64 - 0 - 3.69e-8 * - OptiCommPy, float64 - 5.72e-26 - 3.69e-8 * - Sionna, float64 - 4.83e-26 - 3.69e-8 * - Sionna, float64, its own convention (not conjugated) - 2.03e0 - 2.03e0 * - Sionna, float32 - 1.02e-8 - 7.34e-8 * - Sionna, float32, its own convention (not conjugated) - 2.03e0 - 2.03e0 .. rubric:: split-step Fourier: time Median of 5 runs, s per propagation; workload: 65536 samples, 10 x 80 km, 1 km steps. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - Relative - First call * - comnum (functions), f64 - float64 - 0.556 - 0.554 - 1.00 - 0.561 s * - comnum (functions), f32 - float32 - 0.365 - 0.364 - 0.656 - 0.365 s * - OptiCommPy (numba), f64 - float64 (prec = np.complex128) - 1.28 - 1.28 - 2.31 - 1.29 s * - OptiCommPy (no numba), f64 - float64 (prec = np.complex128) - 1.28 - 1.28 - 2.30 - 1.29 s * - Sionna, single - float32 (precision = 'single', the default) - 1.00 - 0.872 - 1.80 - 0.999 s * - Sionna, double - float64 (precision = 'double') - 1.30 - 1.11 - 2.34 - 1.29 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib comnum --threads 1 --work results/_work --out results/_work/s5_ssfm_comnum_1.json python s5_ssfm.py --lib comnum --threads all --work results/_work --out results/_work/s5_ssfm_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib opticommpy --threads 1 --work results/_work --out results/_work/s5_ssfm_opticommpy_1.json python s5_ssfm.py --lib opticommpy --threads all --work results/_work --out results/_work/s5_ssfm_opticommpy_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 NUMBA_DISABLE_JIT=1 python s5_ssfm.py --lib opticommpy-nojit --threads 1 --work results/_work --out results/_work/s5_ssfm_opticommpy-nojit_1.json --timing-only NUMBA_DISABLE_JIT=1 python s5_ssfm.py --lib opticommpy-nojit --threads all --work results/_work --out results/_work/s5_ssfm_opticommpy-nojit_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib sionna --threads 1 --work results/_work --out results/_work/s5_ssfm_sionna_1.json python s5_ssfm.py --lib sionna --threads all --work results/_work --out results/_work/s5_ssfm_sionna_all.json --timing-only On another machine: h100 (NVIDIA H100 80GB HBM3) ================================================ The same scripts, run on a second machine (``run_all.py --results results/h100``): Sionna (and, for the split-step, OptiCommPy) on its GPU in the column *GPU*, and every library on its CPU in the other columns. The *Relative* column stays the 1-thread time over that of comnum (functions) on 1 thread of this machine. This machine is a `RunPod `_ Secure Cloud pod with one NVIDIA H100 SXM (80 GB HBM3), the PyTorch template (torch 2.13, CUDA 13.0): a container with a CPU quota on a shared host. Machine and versions -------------------- .. list-table:: :header-rows: 0 :widths: 30 70 * - CPU - Intel(R) Xeon(R) Platinum 8480+ * - Cores - 224 logical; 24 available to the process (a container's CPU quota: the thread pools of the mode *all threads* are sized to it) * - Memory - 2015.3 GB * - GPU - NVIDIA H100 80GB HBM3, 81559 MiB, driver 595.91.07 * - System - Ubuntu 24.04.5 LTS * - Python - 3.12.3 * - Rust - — * - comnum commit - ``45a7c78f80fc`` * - Generated - 2026-11-06T17:48:24+00:00 * - comnum - 0.3.1 * - komm - not installed * - OptiCommPy - 0.11.0 * - sionna - 2.2.0 * - torch - 2.13.0+cu130 * - numba - 0.68.1 * - numpy - 2.5.2 * - scipy - 1.18.1 Uncoded QPSK and 16-QAM over AWGN --------------------------------- Scenario 1: uncoded Gray QPSK and 16-QAM over AWGN, BER against the exact closed form. Parameters: - Eb/N0 (dB): 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 - bits per point: 10^7 (quick: 2·10^5), in blocks of 10^6 (quick: 10^5) - timing: one block of 10^6 bits (quick: 2·10^5) at Eb/N0 = 6 dB - theory: exact BER of Gray square QAM, Cho & Yoon, IEEE Trans. Commun. 50(7), 2002 Script: `benchmarks/libraries/s1_awgn.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s1_awgn.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - QPSK - yes - yes - yes - yes * - 16-QAM - yes - yes - yes - yes .. rubric:: QPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 1.90 - yes - binomial * - comnum.run_spec - float64 - theory - 2.53 - yes - binomial * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - theory - 3.31 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.13 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - 0 - 7.86e-2 - 7.86e-2 [7.84e-2, 7.88e-2] - 7.87e-2 [7.85e-2, 7.89e-2] - 7.86e-2 [7.84e-2, 7.88e-2] - 7.86e-2 [7.84e-2, 7.88e-2] * - 1 - 5.63e-2 - 5.63e-2 [5.62e-2, 5.65e-2] - 5.63e-2 [5.61e-2, 5.64e-2] - 5.63e-2 [5.61e-2, 5.64e-2] - 5.61e-2 [5.60e-2, 5.63e-2] * - 2 - 3.75e-2 - 3.75e-2 [3.74e-2, 3.76e-2] - 3.74e-2 [3.73e-2, 3.76e-2] - 3.75e-2 [3.73e-2, 3.76e-2] - 3.76e-2 [3.75e-2, 3.77e-2] * - 3 - 2.29e-2 - 2.28e-2 [2.27e-2, 2.29e-2] - 2.30e-2 [2.29e-2, 2.31e-2] - 2.29e-2 [2.28e-2, 2.30e-2] - 2.29e-2 [2.28e-2, 2.30e-2] * - 4 - 1.25e-2 - 1.25e-2 [1.25e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.25e-2, 1.26e-2] * - 5 - 5.95e-3 - 5.95e-3 [5.90e-3, 6.00e-3] - 5.96e-3 [5.92e-3, 6.01e-3] - 5.93e-3 [5.89e-3, 5.98e-3] - 5.93e-3 [5.88e-3, 5.98e-3] * - 6 - 2.39e-3 - 2.39e-3 [2.36e-3, 2.42e-3] - 2.41e-3 [2.38e-3, 2.44e-3] - 2.43e-3 [2.40e-3, 2.46e-3] - 2.39e-3 [2.36e-3, 2.42e-3] * - 7 - 7.73e-4 - 7.56e-4 [7.39e-4, 7.73e-4] - 7.77e-4 [7.60e-4, 7.95e-4] - 8.02e-4 [7.84e-4, 8.20e-4] - 7.88e-4 [7.70e-4, 8.05e-4] * - 8 - 1.91e-4 - 1.84e-4 [1.76e-4, 1.92e-4] - 1.93e-4 [1.84e-4, 2.01e-4] - 1.90e-4 [1.82e-4, 1.99e-4] - 1.82e-4 [1.73e-4, 1.90e-4] * - 9 - 3.36e-5 - 3.21e-5 [2.88e-5, 3.58e-5] - 3.41e-5 [3.07e-5, 3.79e-5] - 3.42e-5 [3.08e-5, 3.80e-5] - 3.57e-5 [3.22e-5, 3.96e-5] * - 10 - 3.87e-6 - 4.00e-6 [2.94e-6, 5.45e-6] - 3.20e-6 [2.27e-6, 4.52e-6] - 4.10e-6 [3.02e-6, 5.56e-6] - 3.70e-6 [2.68e-6, 5.10e-6] .. rubric:: 16-QAM: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.15 - yes - binomial * - comnum.run_spec - float64 - theory - 1.64 - yes - binomial * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - theory - 1.42 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.41 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - 0 - 1.41e-1 - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] * - 1 - 1.19e-1 - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] * - 2 - 9.77e-2 - 9.78e-2 [9.76e-2, 9.80e-2] - 9.77e-2 [9.75e-2, 9.79e-2] - 9.78e-2 [9.76e-2, 9.80e-2] - 9.75e-2 [9.73e-2, 9.77e-2] * - 3 - 7.75e-2 - 7.75e-2 [7.74e-2, 7.77e-2] - 7.74e-2 [7.73e-2, 7.76e-2] - 7.75e-2 [7.73e-2, 7.76e-2] - 7.74e-2 [7.73e-2, 7.76e-2] * - 4 - 5.86e-2 - 5.87e-2 [5.85e-2, 5.88e-2] - 5.86e-2 [5.84e-2, 5.87e-2] - 5.85e-2 [5.84e-2, 5.87e-2] - 5.85e-2 [5.84e-2, 5.87e-2] * - 5 - 4.19e-2 - 4.18e-2 [4.17e-2, 4.19e-2] - 4.19e-2 [4.17e-2, 4.20e-2] - 4.20e-2 [4.19e-2, 4.21e-2] - 4.20e-2 [4.19e-2, 4.21e-2] * - 6 - 2.79e-2 - 2.79e-2 [2.78e-2, 2.80e-2] - 2.79e-2 [2.78e-2, 2.80e-2] - 2.78e-2 [2.77e-2, 2.79e-2] - 2.79e-2 [2.78e-2, 2.81e-2] * - 7 - 1.70e-2 - 1.70e-2 [1.69e-2, 1.71e-2] - 1.70e-2 [1.69e-2, 1.71e-2] - 1.70e-2 [1.69e-2, 1.70e-2] - 1.70e-2 [1.69e-2, 1.71e-2] * - 8 - 9.25e-3 - 9.24e-3 [9.18e-3, 9.30e-3] - 9.24e-3 [9.18e-3, 9.30e-3] - 9.22e-3 [9.16e-3, 9.28e-3] - 9.24e-3 [9.19e-3, 9.30e-3] * - 9 - 4.39e-3 - 4.35e-3 [4.30e-3, 4.39e-3] - 4.42e-3 [4.38e-3, 4.47e-3] - 4.40e-3 [4.36e-3, 4.44e-3] - 4.37e-3 [4.33e-3, 4.41e-3] * - 10 - 1.75e-3 - 1.75e-3 [1.73e-3, 1.78e-3] - 1.78e-3 [1.75e-3, 1.80e-3] - 1.76e-3 [1.73e-3, 1.79e-3] - 1.76e-3 [1.73e-3, 1.78e-3] .. rubric:: QPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 6.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 36.1 - 36.9 - — - 1.00 - 0.0419 s * - comnum.run_spec - float64 - 34.0 - 6.32 - — - 0.942 - 0.0345 s * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - 86.7 - 50.3 - — - 2.40 - 2.94 s * - Sionna - float32 (config.precision = 'single') - 98.2 - 17.7 - 0.548 - 2.72 - 0.116 s .. rubric:: 16-QAM: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 6.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 19.5 - 20.0 - — - 1.00 - 0.0205 s * - comnum.run_spec - float64 - 19.9 - 4.25 - — - 1.02 - 0.0219 s * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - 87.5 - 33.7 - — - 4.48 - 0.0959 s * - Sionna - float32 (config.precision = 'single') - 172 - 23.1 - 0.702 - 8.82 - 0.206 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib comnum --threads 1 --work h100/_work --out h100/_work/s1_awgn_comnum_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s1_awgn.py --lib comnum --threads all --work h100/_work --out h100/_work/s1_awgn_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib comnum-spec --threads 1 --work h100/_work --out h100/_work/s1_awgn_comnum-spec_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s1_awgn.py --lib comnum-spec --threads all --work h100/_work --out h100/_work/s1_awgn_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib opticommpy --threads 1 --work h100/_work --out h100/_work/s1_awgn_opticommpy_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s1_awgn.py --lib opticommpy --threads all --work h100/_work --out h100/_work/s1_awgn_opticommpy_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib sionna --threads 1 --work h100/_work --out h100/_work/s1_awgn_sionna_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s1_awgn.py --lib sionna --threads all --work h100/_work --out h100/_work/s1_awgn_sionna_all.json --timing-only OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s1_awgn.py --lib sionna --threads gpu --work h100/_work --out h100/_work/s1_awgn_sionna_gpu.json --timing-only BPSK and QPSK over flat Rayleigh fading --------------------------------------- Scenario 2: BPSK and Gray QPSK over flat Rayleigh fading (one coefficient per symbol, perfectly known at the receiver), BER against the closed form. Parameters: - Eb/N0 (dB): 0, 5, 10, 15, 20, 25, 30 - bits per point: 10^7 (quick: 2·10^5), in blocks of 10^6 (quick: 10^5) - channel: y = h x + w, h ~ CN(0, 1) independent for every symbol, known at the receiver - timing: one block of 10^6 bits (quick: 2·10^5) at Eb/N0 = 10 dB - theory: BPSK: (1 - sqrt(G / (1 + G))) / 2; Gray QPSK: the same per bit (Proakis & Salehi, 5th ed., Sec. 13.3) Script: `benchmarks/libraries/s2_rayleigh.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s2_rayleigh.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - BPSK - yes - yes - not available: no fading channel - yes * - QPSK - yes - yes - not available: no fading channel - yes .. rubric:: BPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.22 - yes - binomial * - comnum.run_spec - float64 - theory - 1.78 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.35 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.46e-1 - 1.46e-1 [1.46e-1, 1.47e-1] - 1.46e-1 [1.46e-1, 1.47e-1] - 1.47e-1 [1.46e-1, 1.47e-1] * - 5 - 6.42e-2 - 6.43e-2 [6.41e-2, 6.44e-2] - 6.42e-2 [6.41e-2, 6.44e-2] - 6.42e-2 [6.40e-2, 6.43e-2] * - 10 - 2.33e-2 - 2.33e-2 [2.32e-2, 2.34e-2] - 2.32e-2 [2.32e-2, 2.33e-2] - 2.33e-2 [2.32e-2, 2.34e-2] * - 15 - 7.72e-3 - 7.73e-3 [7.67e-3, 7.78e-3] - 7.76e-3 [7.71e-3, 7.82e-3] - 7.75e-3 [7.70e-3, 7.80e-3] * - 20 - 2.48e-3 - 2.51e-3 [2.48e-3, 2.54e-3] - 2.46e-3 [2.43e-3, 2.49e-3] - 2.52e-3 [2.49e-3, 2.55e-3] * - 25 - 7.89e-4 - 8.08e-4 [7.91e-4, 8.26e-4] - 7.83e-4 [7.66e-4, 8.00e-4] - 7.88e-4 [7.71e-4, 8.06e-4] * - 30 - 2.50e-4 - 2.52e-4 [2.42e-4, 2.62e-4] - 2.41e-4 [2.31e-4, 2.51e-4] - 2.43e-4 [2.34e-4, 2.53e-4] .. rubric:: QPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.08 - yes - binomial * - comnum.run_spec - float64 - theory - 1.17 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 3.13 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.46e-1 - 1.46e-1 [1.46e-1, 1.46e-1] - 1.46e-1 [1.46e-1, 1.47e-1] - 1.46e-1 [1.46e-1, 1.47e-1] * - 5 - 6.42e-2 - 6.41e-2 [6.39e-2, 6.42e-2] - 6.42e-2 [6.40e-2, 6.43e-2] - 6.42e-2 [6.40e-2, 6.43e-2] * - 10 - 2.33e-2 - 2.34e-2 [2.33e-2, 2.34e-2] - 2.33e-2 [2.32e-2, 2.34e-2] - 2.31e-2 [2.30e-2, 2.32e-2] * - 15 - 7.72e-3 - 7.77e-3 [7.72e-3, 7.82e-3] - 7.73e-3 [7.67e-3, 7.78e-3] - 7.67e-3 [7.61e-3, 7.72e-3] * - 20 - 2.48e-3 - 2.51e-3 [2.48e-3, 2.54e-3] - 2.50e-3 [2.47e-3, 2.53e-3] - 2.46e-3 [2.43e-3, 2.50e-3] * - 25 - 7.89e-4 - 8.02e-4 [7.84e-4, 8.19e-4] - 7.81e-4 [7.64e-4, 7.99e-4] - 7.92e-4 [7.75e-4, 8.10e-4] * - 30 - 2.50e-4 - 2.53e-4 [2.44e-4, 2.63e-4] - 2.46e-4 [2.37e-4, 2.56e-4] - 2.51e-4 [2.41e-4, 2.61e-4] .. rubric:: BPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 10.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 273 - 249 - — - 1.00 - 0.264 s * - comnum.run_spec - float64 - 291 - 20.7 - — - 1.07 - 0.305 s * - Sionna - float32 (config.precision = 'single') - 327 - 196 - 8.11 - 1.20 - 0.348 s .. rubric:: QPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 10.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 133 - 136 - — - 1.00 - 0.124 s * - comnum.run_spec - float64 - 151 - 10.9 - — - 1.14 - 0.154 s * - Sionna - float32 (config.precision = 'single') - 217 - 106 - 4.46 - 1.64 - 0.215 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib comnum --threads 1 --work h100/_work --out h100/_work/s2_rayleigh_comnum_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s2_rayleigh.py --lib comnum --threads all --work h100/_work --out h100/_work/s2_rayleigh_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib comnum-spec --threads 1 --work h100/_work --out h100/_work/s2_rayleigh_comnum-spec_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s2_rayleigh.py --lib comnum-spec --threads all --work h100/_work --out h100/_work/s2_rayleigh_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib sionna --threads 1 --work h100/_work --out h100/_work/s2_rayleigh_sionna_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s2_rayleigh.py --lib sionna --threads all --work h100/_work --out h100/_work/s2_rayleigh_sionna_all.json --timing-only OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s2_rayleigh.py --lib sionna --threads gpu --work h100/_work --out h100/_work/s2_rayleigh_sionna_gpu.json --timing-only 4 x 4 MIMO, 16-QAM, i.i.d. Rayleigh fading: ZF, MMSE and ML detection --------------------------------------------------------------------- Scenario 3: 4 x 4 spatial multiplexing of Gray 16-QAM over i.i.d. Rayleigh fading (one channel matrix per vector, known at the receiver), detected by zero forcing, MMSE and maximum likelihood. Parameters: - Eb/N0 (dB): 0, 4, 8, 12, 16 - vectors per point: linear detectors and sphere: 10^5 (quick: 5·10^3); exhaustive ML: 2·10^4 (quick: 10^3) - timing: 10^4 vectors at Eb/N0 = 12 dB (exhaustive ML: 10^3; quick: 2·10^3 and 2·10^2) - theory: ZF: 16-QAM over SISO Rayleigh fading (Winters, Salz & Gitlin, IEEE Trans. Commun. 42(2/3/4), 1994) Script: `benchmarks/libraries/s3_mimo.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s3_mimo.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - zero forcing - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - MMSE - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - ML (exhaustive) - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - ML (sphere decoder) - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - not available: no sphere decoder (its exhaustive ML detector is the case ml) .. rubric:: zero forcing: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 1.88 - yes - per-frame * - comnum.run_spec - float64 - theory - 2.98 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 1.79 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.98e-1 - 1.98e-1 [1.97e-1, 1.99e-1] - 1.98e-1 [1.97e-1, 1.98e-1] - 1.98e-1 [1.97e-1, 1.99e-1] * - 4 - 1.20e-1 - 1.19e-1 [1.18e-1, 1.20e-1] - 1.19e-1 [1.19e-1, 1.20e-1] - 1.19e-1 [1.19e-1, 1.20e-1] * - 8 - 6.20e-2 - 6.16e-2 [6.10e-2, 6.23e-2] - 6.14e-2 [6.10e-2, 6.18e-2] - 6.26e-2 [6.19e-2, 6.32e-2] * - 12 - 2.83e-2 - 2.82e-2 [2.78e-2, 2.87e-2] - 2.81e-2 [2.79e-2, 2.84e-2] - 2.83e-2 [2.78e-2, 2.87e-2] * - 16 - 1.20e-2 - 1.21e-2 [1.18e-2, 1.24e-2] - 1.19e-2 [1.17e-2, 1.20e-2] - 1.19e-2 [1.16e-2, 1.22e-2] .. rubric:: MMSE: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 3.63 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.57 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.39e-1 [1.38e-1, 1.40e-1] - 1.38e-1 [1.38e-1, 1.39e-1] - 1.38e-1 [1.38e-1, 1.39e-1] * - 4 - 8.27e-2 [8.21e-2, 8.33e-2] - 8.40e-2 [8.36e-2, 8.44e-2] - 8.33e-2 [8.28e-2, 8.39e-2] * - 8 - 4.31e-2 [4.26e-2, 4.35e-2] - 4.29e-2 [4.26e-2, 4.32e-2] - 4.32e-2 [4.28e-2, 4.37e-2] * - 12 - 1.97e-2 [1.94e-2, 2.00e-2] - 1.99e-2 [1.97e-2, 2.01e-2] - 1.96e-2 [1.93e-2, 1.99e-2] * - 16 - 8.42e-3 [8.20e-3, 8.64e-3] - 8.33e-3 [8.19e-3, 8.47e-3] - 8.23e-3 [8.02e-3, 8.45e-3] .. rubric:: ML (exhaustive): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 2.53 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.09 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (320,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.25e-1 [1.23e-1, 1.26e-1] - 1.25e-1 [1.24e-1, 1.26e-1] - 1.25e-1 [1.23e-1, 1.27e-1] * - 4 - 4.03e-2 [3.90e-2, 4.16e-2] - 3.99e-2 [3.92e-2, 4.06e-2] - 4.05e-2 [3.92e-2, 4.17e-2] * - 8 - 4.12e-3 [3.68e-3, 4.56e-3] - 4.77e-3 [4.54e-3, 5.02e-3] - 4.47e-3 [4.02e-3, 4.93e-3] * - 12 - 2.34e-4 [1.24e-4, 3.45e-4] - 2.28e-4 [1.81e-4, 2.87e-4] - 1.59e-4 [7.61e-5, 2.43e-4] * - 16 - 0 [0, 9.38e-6] - 0 [0, 1.20e-5] - 1.25e-5 [0, 3.70e-5] .. rubric:: ML (sphere decoder): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 2.84 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec * - 0 - 1.24e-1 [1.23e-1, 1.25e-1] - 1.25e-1 [1.24e-1, 1.25e-1] * - 4 - 3.97e-2 [3.92e-2, 4.03e-2] - 4.06e-2 [4.03e-2, 4.09e-2] * - 8 - 4.44e-3 [4.23e-3, 4.64e-3] - 4.62e-3 [4.51e-3, 4.73e-3] * - 12 - 1.91e-4 [1.49e-4, 2.32e-4] - 1.95e-4 [1.75e-4, 2.18e-4] * - 16 - 3.13e-6 [0, 6.37e-6] - 1.25e-5 [8.09e-6, 1.93e-5] .. rubric:: zero forcing: time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 2.13 - 2.11 - — - 1.00 - 0.0241 s * - comnum.run_spec - float64 - 2.28 - 0.355 - — - 1.07 - 0.0209 s * - Sionna - float32 (config.precision = 'single') - 5.09 - 1.86 - 0.143 - 2.40 - 0.0513 s .. rubric:: MMSE: time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 2.16 - 2.12 - — - 1.00 - 0.0265 s * - comnum.run_spec - float64 - 2.42 - 0.341 - — - 1.12 - 0.0231 s * - Sionna - float32 (config.precision = 'single') - 4.34 - 2.41 - 0.158 - 2.01 - 0.052 s .. rubric:: ML (exhaustive): time Median of 5 runs, µs per vector; workload: 1000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 969 - 984 - — - 1.00 - 0.985 s * - comnum.run_spec - float64 - 1015 - 55.8 - — - 1.05 - 0.986 s * - Sionna - float32 (config.precision = 'single') - 13203 - 2919 - 61.8 - 13.6 - 13.1 s .. rubric:: ML (sphere decoder): time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 2.65 - 2.67 - — - 1.00 - 0.0283 s * - comnum.run_spec - float64 - 3.17 - 0.435 - — - 1.20 - 0.0299 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib comnum --threads 1 --work h100/_work --out h100/_work/s3_mimo_comnum_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s3_mimo.py --lib comnum --threads all --work h100/_work --out h100/_work/s3_mimo_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib comnum-spec --threads 1 --work h100/_work --out h100/_work/s3_mimo_comnum-spec_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s3_mimo.py --lib comnum-spec --threads all --work h100/_work --out h100/_work/s3_mimo_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib sionna --threads 1 --work h100/_work --out h100/_work/s3_mimo_sionna_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s3_mimo.py --lib sionna --threads all --work h100/_work --out h100/_work/s3_mimo_sionna_all.json --timing-only OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s3_mimo.py --lib sionna --threads gpu --work h100/_work --out h100/_work/s3_mimo_sionna_gpu.json --timing-only Channel coding over BPSK and AWGN --------------------------------- Scenario 4: channel coding over BPSK and AWGN. Parameters: - convolutional (133, 171), soft Viterbi: K = 7, (133, 171), k = 1024 + 6 tail bits, soft Viterbi; Eb/N0 = 1, 2, 3, 4 dB - 5G NR LDPC (1024, 2048): 5G NR LDPC, k = 1024, n = 2048 (base graph 2, Z = 104), flooding sum-product, 50 iterations; Eb/N0 = 1, 1.5, 2, 2.5 dB - LDPC (1024, 512), same LLRs: PEG (1024, 512), flooding sum-product, 50 iterations, identical LLRs; Eb/N0 = 1.5, 2, 2.5, 3 dB - frames per point: 2000 (quick: 100) - timing: 200 frames (quick: 20) at Eb/N0 = 3 dB (conv) or 2 dB (LDPC); OptiCommPy without numba: 2 frames Script: `benchmarks/libraries/s4_coding.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s4_coding.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - convolutional (133, 171), soft Viterbi - yes - yes - not available: no convolutional code - yes * - 5G NR LDPC (1024, 2048) - yes - not available: the specification format has no 5G NR LDPC code (NrLdpc: Python and Rust API) - not available: no 5G NR LDPC code (its decoder takes a parity-check matrix: see ldpc_same_llr) - yes * - LDPC (1024, 512), same LLRs - yes - not available: the runner draws its own channel (this case feeds the same LLRs to every decoder) - yes - yes .. rubric:: convolutional (133, 171), soft Viterbi: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 1.35 - 1.33 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.41 - 2.53 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - union bound - comnum (functions) - comnum.run_spec - Sionna * - 1 - 3.05e-1 - 4.02e-2 [3.91e-2, 4.13e-2] - 4.02e-2 [3.99e-2, 4.05e-2] - 4.03e-2 [3.91e-2, 4.14e-2] * - 2 - 1.65e-2 - 4.97e-3 [4.64e-3, 5.30e-3] - 5.16e-3 [5.06e-3, 5.26e-3] - 5.32e-3 [4.97e-3, 5.67e-3] * - 3 - 6.27e-4 - 3.31e-4 [2.59e-4, 4.02e-4] - 2.98e-4 [2.76e-4, 3.23e-4] - 3.64e-4 [2.94e-4, 4.34e-4] * - 4 - 2.02e-5 - 2.54e-5 [8.46e-6, 4.23e-5] - 1.32e-5 [9.06e-6, 1.92e-5] - 3.08e-5 [9.04e-6, 5.25e-5] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 1 - 9.73e-1 [9.66e-1, 9.80e-1] - 9.77e-1 [9.69e-1, 9.83e-1] - 9.85e-1 [9.79e-1, 9.89e-1] * - 2 - 5.05e-1 [4.84e-1, 5.27e-1] - 4.84e-1 [4.63e-1, 5.06e-1] - 5.13e-1 [4.91e-1, 5.35e-1] * - 3 - 5.95e-2 [5.00e-2, 7.07e-2] - 6.05e-2 [5.09e-2, 7.18e-2] - 6.70e-2 [5.69e-2, 7.88e-2] * - 4 - 6.00e-3 [3.44e-3, 1.05e-2] - 5.00e-3 [2.72e-3, 9.18e-3] - 5.50e-3 [3.07e-3, 9.82e-3] .. rubric:: 5G NR LDPC (1024, 2048): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - Sionna - float32 (config.precision = 'single') - comnum - 0.893 - 1.21 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - Sionna * - 1 - 8.79e-3 [7.46e-3, 1.01e-2] - 9.31e-3 [7.94e-3, 1.07e-2] * - 1.5 - 3.47e-5 [0, 1.03e-4] - 9.38e-5 [0, 2.04e-4] * - 2 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] * - 2.5 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - Sionna * - 1 - 9.40e-2 [8.20e-2, 1.08e-1] - 1.05e-1 [9.28e-2, 1.20e-1] * - 1.5 - 5.00e-4 [8.83e-5, 2.83e-3] - 1.50e-3 [5.10e-4, 4.40e-3] * - 2 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] * - 2.5 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] .. rubric:: LDPC (1024, 512), same LLRs: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - comnum - 0.076 - 0 - yes - per-frame * - Sionna - float32 (config.precision = 'single') - comnum - 0.0795 - 0.0381 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - OptiCommPy (numba) - Sionna * - 1.5 - 1.53e-2 [1.39e-2, 1.66e-2] - 1.53e-2 [1.40e-2, 1.67e-2] - 1.53e-2 [1.40e-2, 1.67e-2] * - 2 - 1.37e-3 [9.56e-4, 1.78e-3] - 1.37e-3 [9.55e-4, 1.78e-3] - 1.37e-3 [9.56e-4, 1.78e-3] * - 2.5 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] * - 3 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - OptiCommPy (numba) - Sionna * - 1.5 - 2.21e-1 [2.03e-1, 2.40e-1] - 2.21e-1 [2.03e-1, 2.40e-1] - 2.21e-1 [2.04e-1, 2.40e-1] * - 2 - 2.30e-2 [1.73e-2, 3.05e-2] - 2.30e-2 [1.73e-2, 3.05e-2] - 2.30e-2 [1.73e-2, 3.05e-2] * - 2.5 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] * - 3 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] .. rubric:: Same LLRs: the decisions frame by frame The decoders received identical LLRs. Over all the points: the frames where a library makes as many bit errors as comnum, and the frames it decodes (or fails to decode) as comnum does. .. list-table:: :header-rows: 1 * - Library - Points - Frames - Same bit errors - Same frame outcome * - OptiCommPy (numba) - 4 - 8000 - 7996 (99.95 %) - 8000 (100.00 %) * - Sionna - 4 - 8000 - 7981 (99.76 %) - 7999 (99.99 %) .. rubric:: convolutional (133, 171), soft Viterbi: decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 3.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 0.71 - 0.703 - — - 1.00 - 0.00199 s * - Sionna - float32 (config.precision = 'single') - 3.55 - 2.85 - 0.8 - 5.00 - 0.0987 s .. rubric:: 5G NR LDPC (1024, 2048): decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 7.81 - 7.79 - — - 1.00 - 0.016 s * - Sionna - float32 (config.precision = 'single') - 32.8 - 4.73 - 0.135 - 4.20 - 0.0476 s .. rubric:: LDPC (1024, 512), same LLRs: decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 1.51 - 1.51 - — - 1.00 - 0.00307 s * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - 7.88 - 6.99 - — - 5.23 - 5.34 s * - Sionna - float32 (config.precision = 'single') - 6.39 - 0.873 - 0.114 - 4.24 - 0.0188 s .. rubric:: convolutional (133, 171), soft Viterbi: whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 3.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 0.896 - 0.895 - — - 1.00 - 0.00199 s * - comnum.run_spec - float64 - 0.882 - 0.0723 - — - 0.984 - 0.177 s * - Sionna - float32 (config.precision = 'single') - 3.84 - 3.12 - 1.31 - 4.29 - 0.0987 s .. rubric:: 5G NR LDPC (1024, 2048): whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 7.96 - 8.20 - — - 1.00 - 0.016 s * - Sionna - float32 (config.precision = 'single') - 33.8 - 4.79 - 0.132 - 4.24 - 0.0476 s .. rubric:: LDPC (1024, 512), same LLRs: whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 1.59 - 1.55 - — - 1.00 - 0.00307 s * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - 7.46 - 6.36 - — - 4.68 - 5.34 s * - Sionna - float32 (config.precision = 'single') - 6.62 - 1.09 - 0.26 - 4.15 - 0.0188 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib comnum --threads 1 --work h100/_work --out h100/_work/s4_coding_comnum_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s4_coding.py --lib comnum --threads all --work h100/_work --out h100/_work/s4_coding_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib comnum-spec --threads 1 --work h100/_work --out h100/_work/s4_coding_comnum-spec_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s4_coding.py --lib comnum-spec --threads all --work h100/_work --out h100/_work/s4_coding_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib opticommpy --threads 1 --work h100/_work --out h100/_work/s4_coding_opticommpy_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s4_coding.py --lib opticommpy --threads all --work h100/_work --out h100/_work/s4_coding_opticommpy_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib sionna --threads 1 --work h100/_work --out h100/_work/s4_coding_sionna_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s4_coding.py --lib sionna --threads all --work h100/_work --out h100/_work/s4_coding_sionna_all.json --timing-only OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s4_coding.py --lib sionna --threads gpu --work h100/_work --out h100/_work/s4_coding_sionna_gpu.json --timing-only Split-step Fourier propagation, 16-QAM, 32 GBd, 10 x 80 km ---------------------------------------------------------- Scenario 5: single-polarisation split-step Fourier propagation of 32 GBd 16-QAM over 10 x 80 km of standard single-mode fibre, a fixed step of 1 km (80 steps per span), noiseless amplification at the end of every span (gain e^{alpha L}). Parameters: - fibre: alpha = 0.2 dB/km, D = 17 ps/nm/km, gamma = 1.3 /W/km, fc = 193.1 THz - link: 10 x 80 km, ideal (noiseless) amplification after every span - step: 1 km (800 steps); reference: comnum at 0.05 km (16000 steps) - signal: 16-QAM, 32 GBd, RRC roll-off 0.1, 4 samples per symbol, 4 dBm; 2^14 symbols (quick: 2^11) - timing: one propagation over the 10 spans Script: `benchmarks/libraries/s5_ssfm.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s5_ssfm.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - OptiCommPy (numba) - Sionna * - split-step Fourier - yes - yes - yes .. rubric:: Output fields NMSE of the output field, ``sum |a - b|^2 / sum |b|^2``, against comnum in float64 at the same step and against the step-converged reference (comnum, 0.05 km). The NMSE against the reference is the error of the 1 km step itself. .. list-table:: :header-rows: 1 * - Field - NMSE vs comnum (float64, 1 km) - NMSE vs reference * - comnum, float32 - 1.14e-8 - 7.45e-8 * - comnum, float64 - 0 - 3.69e-8 * - OptiCommPy, float64 - 2.24e-26 - 3.69e-8 * - Sionna, float64 - 4.17e-26 - 3.69e-8 * - Sionna, float64, its own convention (not conjugated) - 2.03e0 - 2.03e0 * - Sionna, float32 - 2.61e-8 - 1.05e-7 * - Sionna, float32, its own convention (not conjugated) - 2.03e0 - 2.03e0 .. rubric:: split-step Fourier: time Median of 5 runs, s per propagation; workload: 65536 samples, 10 x 80 km, 1 km steps. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions), f64 - float64 - 1.21 - 1.20 - — - 1.00 - 1.21 s * - comnum (functions), f32 - float32 - 0.627 - 0.637 - — - 0.519 - 0.673 s * - OptiCommPy (numba), f64 - float64 (prec = np.complex128) - 3.56 - 3.65 - 0.0929 - 2.95 - 3.87 s * - OptiCommPy (numba), f32 - float32 (prec = cp.complex64) - — - — - 0.0938 - — - 1.78 s * - Sionna, single - float32 (precision = 'single', the default) - 7.31 - 1.12 - 0.167 - 6.05 - 7.39 s * - Sionna, double - float64 (precision = 'double') - 6.99 - 2.26 - 0.171 - 5.79 - 7.37 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib comnum --threads 1 --work h100/_work --out h100/_work/s5_ssfm_comnum_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s5_ssfm.py --lib comnum --threads all --work h100/_work --out h100/_work/s5_ssfm_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib opticommpy --threads 1 --work h100/_work --out h100/_work/s5_ssfm_opticommpy_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s5_ssfm.py --lib opticommpy --threads all --work h100/_work --out h100/_work/s5_ssfm_opticommpy_all.json --timing-only OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s5_ssfm.py --lib opticommpy --threads gpu --work h100/_work --out h100/_work/s5_ssfm_opticommpy_gpu.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib sionna --threads 1 --work h100/_work --out h100/_work/s5_ssfm_sionna_1.json OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s5_ssfm.py --lib sionna --threads all --work h100/_work --out h100/_work/s5_ssfm_sionna_all.json --timing-only OMP_NUM_THREADS=24 OPENBLAS_NUM_THREADS=24 MKL_NUM_THREADS=24 VECLIB_MAXIMUM_THREADS=24 NUMBA_NUM_THREADS=24 RAYON_NUM_THREADS=24 python s5_ssfm.py --lib sionna --threads gpu --work h100/_work --out h100/_work/s5_ssfm_sionna_gpu.json --timing-only On another machine: rtx4090 (NVIDIA GeForce RTX 4090) ===================================================== The same scripts, run on a second machine (``run_all.py --results results/rtx4090``): Sionna (and, for the split-step, OptiCommPy) on its GPU in the column *GPU*, and every library on its CPU in the other columns. The *Relative* column stays the 1-thread time over that of comnum (functions) on 1 thread of this machine. This machine is a `RunPod `_ Secure Cloud pod with one NVIDIA GeForce RTX 4090 (24 GB), the PyTorch template (torch 2.13, CUDA 13.0): a container with a CPU quota on a shared host. Machine and versions -------------------- .. list-table:: :header-rows: 0 :widths: 30 70 * - CPU - AMD EPYC 7K62 48-Core Processor * - Cores - 96 logical; 11 available to the process (a container's CPU quota: the thread pools of the mode *all threads* are sized to it) * - Memory - 251.5 GB * - GPU - NVIDIA GeForce RTX 4090, 24564 MiB, driver 595.91.07 * - System - Ubuntu 24.04.5 LTS * - Python - 3.12.3 * - Rust - — * - comnum commit - ``45a7c78f80fc`` * - Generated - 2026-11-06T17:38:20+00:00 * - comnum - 0.3.1 * - komm - not installed * - OptiCommPy - 0.11.0 * - sionna - 2.2.0 * - torch - 2.13.0+cu130 * - numba - 0.68.1 * - numpy - 2.5.2 * - scipy - 1.18.1 Uncoded QPSK and 16-QAM over AWGN --------------------------------- Scenario 1: uncoded Gray QPSK and 16-QAM over AWGN, BER against the exact closed form. Parameters: - Eb/N0 (dB): 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 - bits per point: 10^7 (quick: 2·10^5), in blocks of 10^6 (quick: 10^5) - timing: one block of 10^6 bits (quick: 2·10^5) at Eb/N0 = 6 dB - theory: exact BER of Gray square QAM, Cho & Yoon, IEEE Trans. Commun. 50(7), 2002 Script: `benchmarks/libraries/s1_awgn.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s1_awgn.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - QPSK - yes - yes - yes - yes * - 16-QAM - yes - yes - yes - yes .. rubric:: QPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 1.90 - yes - binomial * - comnum.run_spec - float64 - theory - 2.53 - yes - binomial * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - theory - 3.31 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.13 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - 0 - 7.86e-2 - 7.86e-2 [7.84e-2, 7.88e-2] - 7.87e-2 [7.85e-2, 7.89e-2] - 7.86e-2 [7.84e-2, 7.88e-2] - 7.86e-2 [7.84e-2, 7.88e-2] * - 1 - 5.63e-2 - 5.63e-2 [5.62e-2, 5.65e-2] - 5.63e-2 [5.61e-2, 5.64e-2] - 5.63e-2 [5.61e-2, 5.64e-2] - 5.61e-2 [5.60e-2, 5.63e-2] * - 2 - 3.75e-2 - 3.75e-2 [3.74e-2, 3.76e-2] - 3.74e-2 [3.73e-2, 3.76e-2] - 3.75e-2 [3.73e-2, 3.76e-2] - 3.76e-2 [3.75e-2, 3.77e-2] * - 3 - 2.29e-2 - 2.28e-2 [2.27e-2, 2.29e-2] - 2.30e-2 [2.29e-2, 2.31e-2] - 2.29e-2 [2.28e-2, 2.30e-2] - 2.29e-2 [2.28e-2, 2.30e-2] * - 4 - 1.25e-2 - 1.25e-2 [1.25e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.24e-2, 1.26e-2] - 1.25e-2 [1.25e-2, 1.26e-2] * - 5 - 5.95e-3 - 5.95e-3 [5.90e-3, 6.00e-3] - 5.96e-3 [5.92e-3, 6.01e-3] - 5.93e-3 [5.89e-3, 5.98e-3] - 5.93e-3 [5.88e-3, 5.98e-3] * - 6 - 2.39e-3 - 2.39e-3 [2.36e-3, 2.42e-3] - 2.41e-3 [2.38e-3, 2.44e-3] - 2.43e-3 [2.40e-3, 2.46e-3] - 2.39e-3 [2.36e-3, 2.42e-3] * - 7 - 7.73e-4 - 7.56e-4 [7.39e-4, 7.73e-4] - 7.77e-4 [7.60e-4, 7.95e-4] - 8.02e-4 [7.84e-4, 8.20e-4] - 7.88e-4 [7.70e-4, 8.05e-4] * - 8 - 1.91e-4 - 1.84e-4 [1.76e-4, 1.92e-4] - 1.93e-4 [1.84e-4, 2.01e-4] - 1.90e-4 [1.82e-4, 1.99e-4] - 1.82e-4 [1.73e-4, 1.90e-4] * - 9 - 3.36e-5 - 3.21e-5 [2.88e-5, 3.58e-5] - 3.41e-5 [3.07e-5, 3.79e-5] - 3.42e-5 [3.08e-5, 3.80e-5] - 3.57e-5 [3.22e-5, 3.96e-5] * - 10 - 3.87e-6 - 4.00e-6 [2.94e-6, 5.45e-6] - 3.20e-6 [2.27e-6, 4.52e-6] - 4.10e-6 [3.02e-6, 5.56e-6] - 3.70e-6 [2.68e-6, 5.10e-6] .. rubric:: 16-QAM: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.15 - yes - binomial * - comnum.run_spec - float64 - theory - 1.64 - yes - binomial * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - theory - 1.42 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.41 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - 0 - 1.41e-1 - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] - 1.41e-1 [1.41e-1, 1.41e-1] * - 1 - 1.19e-1 - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] - 1.19e-1 [1.19e-1, 1.19e-1] * - 2 - 9.77e-2 - 9.78e-2 [9.76e-2, 9.80e-2] - 9.77e-2 [9.75e-2, 9.79e-2] - 9.78e-2 [9.76e-2, 9.80e-2] - 9.75e-2 [9.73e-2, 9.77e-2] * - 3 - 7.75e-2 - 7.75e-2 [7.74e-2, 7.77e-2] - 7.74e-2 [7.73e-2, 7.76e-2] - 7.75e-2 [7.73e-2, 7.76e-2] - 7.74e-2 [7.73e-2, 7.76e-2] * - 4 - 5.86e-2 - 5.87e-2 [5.85e-2, 5.88e-2] - 5.86e-2 [5.84e-2, 5.87e-2] - 5.85e-2 [5.84e-2, 5.87e-2] - 5.85e-2 [5.84e-2, 5.87e-2] * - 5 - 4.19e-2 - 4.18e-2 [4.17e-2, 4.19e-2] - 4.19e-2 [4.17e-2, 4.20e-2] - 4.20e-2 [4.19e-2, 4.21e-2] - 4.20e-2 [4.19e-2, 4.21e-2] * - 6 - 2.79e-2 - 2.79e-2 [2.78e-2, 2.80e-2] - 2.79e-2 [2.78e-2, 2.80e-2] - 2.78e-2 [2.77e-2, 2.79e-2] - 2.79e-2 [2.78e-2, 2.81e-2] * - 7 - 1.70e-2 - 1.70e-2 [1.69e-2, 1.71e-2] - 1.70e-2 [1.69e-2, 1.71e-2] - 1.70e-2 [1.69e-2, 1.70e-2] - 1.70e-2 [1.69e-2, 1.71e-2] * - 8 - 9.25e-3 - 9.24e-3 [9.18e-3, 9.30e-3] - 9.24e-3 [9.18e-3, 9.30e-3] - 9.22e-3 [9.16e-3, 9.28e-3] - 9.24e-3 [9.19e-3, 9.30e-3] * - 9 - 4.39e-3 - 4.35e-3 [4.30e-3, 4.39e-3] - 4.42e-3 [4.38e-3, 4.47e-3] - 4.40e-3 [4.36e-3, 4.44e-3] - 4.37e-3 [4.33e-3, 4.41e-3] * - 10 - 1.75e-3 - 1.75e-3 [1.73e-3, 1.78e-3] - 1.78e-3 [1.75e-3, 1.80e-3] - 1.76e-3 [1.73e-3, 1.79e-3] - 1.76e-3 [1.73e-3, 1.78e-3] .. rubric:: QPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 6.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 50.4 - 50.7 - — - 1.00 - 0.0588 s * - comnum.run_spec - float64 - 42.2 - 8.56 - — - 0.836 - 0.0421 s * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - 105 - 73.0 - — - 2.09 - 3.3 s * - Sionna - float32 (config.precision = 'single') - 123 - 48.4 - 0.748 - 2.44 - 0.148 s .. rubric:: 16-QAM: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 6.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 27.1 - 27.2 - — - 1.00 - 0.0273 s * - comnum.run_spec - float64 - 29.0 - 5.16 - — - 1.07 - 0.0271 s * - OptiCommPy (numba) - complex64 symbols (its Gray constellation), float64 noise - 100 - 51.0 - — - 3.71 - 0.108 s * - Sionna - float32 (config.precision = 'single') - 219 - 63.1 - 0.832 - 8.10 - 0.25 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib comnum --threads 1 --work runpod/_work --out runpod/_work/s1_awgn_comnum_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s1_awgn.py --lib comnum --threads all --work runpod/_work --out runpod/_work/s1_awgn_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib comnum-spec --threads 1 --work runpod/_work --out runpod/_work/s1_awgn_comnum-spec_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s1_awgn.py --lib comnum-spec --threads all --work runpod/_work --out runpod/_work/s1_awgn_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib opticommpy --threads 1 --work runpod/_work --out runpod/_work/s1_awgn_opticommpy_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s1_awgn.py --lib opticommpy --threads all --work runpod/_work --out runpod/_work/s1_awgn_opticommpy_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s1_awgn.py --lib sionna --threads 1 --work runpod/_work --out runpod/_work/s1_awgn_sionna_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s1_awgn.py --lib sionna --threads all --work runpod/_work --out runpod/_work/s1_awgn_sionna_all.json --timing-only OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s1_awgn.py --lib sionna --threads gpu --work runpod/_work --out runpod/_work/s1_awgn_sionna_gpu.json --timing-only BPSK and QPSK over flat Rayleigh fading --------------------------------------- Scenario 2: BPSK and Gray QPSK over flat Rayleigh fading (one coefficient per symbol, perfectly known at the receiver), BER against the closed form. Parameters: - Eb/N0 (dB): 0, 5, 10, 15, 20, 25, 30 - bits per point: 10^7 (quick: 2·10^5), in blocks of 10^6 (quick: 10^5) - channel: y = h x + w, h ~ CN(0, 1) independent for every symbol, known at the receiver - timing: one block of 10^6 bits (quick: 2·10^5) at Eb/N0 = 10 dB - theory: BPSK: (1 - sqrt(G / (1 + G))) / 2; Gray QPSK: the same per bit (Proakis & Salehi, 5th ed., Sec. 13.3) Script: `benchmarks/libraries/s2_rayleigh.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s2_rayleigh.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - BPSK - yes - yes - not available: no fading channel - yes * - QPSK - yes - yes - not available: no fading channel - yes .. rubric:: BPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.22 - yes - binomial * - comnum.run_spec - float64 - theory - 1.78 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 2.35 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.46e-1 - 1.46e-1 [1.46e-1, 1.47e-1] - 1.46e-1 [1.46e-1, 1.47e-1] - 1.47e-1 [1.46e-1, 1.47e-1] * - 5 - 6.42e-2 - 6.43e-2 [6.41e-2, 6.44e-2] - 6.42e-2 [6.41e-2, 6.44e-2] - 6.42e-2 [6.40e-2, 6.43e-2] * - 10 - 2.33e-2 - 2.33e-2 [2.32e-2, 2.34e-2] - 2.32e-2 [2.32e-2, 2.33e-2] - 2.33e-2 [2.32e-2, 2.34e-2] * - 15 - 7.72e-3 - 7.73e-3 [7.67e-3, 7.78e-3] - 7.76e-3 [7.71e-3, 7.82e-3] - 7.75e-3 [7.70e-3, 7.80e-3] * - 20 - 2.48e-3 - 2.51e-3 [2.48e-3, 2.54e-3] - 2.46e-3 [2.43e-3, 2.49e-3] - 2.52e-3 [2.49e-3, 2.55e-3] * - 25 - 7.89e-4 - 8.08e-4 [7.91e-4, 8.26e-4] - 7.83e-4 [7.66e-4, 8.00e-4] - 7.88e-4 [7.71e-4, 8.06e-4] * - 30 - 2.50e-4 - 2.52e-4 [2.42e-4, 2.62e-4] - 2.41e-4 [2.31e-4, 2.51e-4] - 2.43e-4 [2.34e-4, 2.53e-4] .. rubric:: QPSK: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 2.08 - yes - binomial * - comnum.run_spec - float64 - theory - 1.17 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 3.13 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (10,000,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.46e-1 - 1.46e-1 [1.46e-1, 1.46e-1] - 1.46e-1 [1.46e-1, 1.47e-1] - 1.46e-1 [1.46e-1, 1.47e-1] * - 5 - 6.42e-2 - 6.41e-2 [6.39e-2, 6.42e-2] - 6.42e-2 [6.40e-2, 6.43e-2] - 6.42e-2 [6.40e-2, 6.43e-2] * - 10 - 2.33e-2 - 2.34e-2 [2.33e-2, 2.34e-2] - 2.33e-2 [2.32e-2, 2.34e-2] - 2.31e-2 [2.30e-2, 2.32e-2] * - 15 - 7.72e-3 - 7.77e-3 [7.72e-3, 7.82e-3] - 7.73e-3 [7.67e-3, 7.78e-3] - 7.67e-3 [7.61e-3, 7.72e-3] * - 20 - 2.48e-3 - 2.51e-3 [2.48e-3, 2.54e-3] - 2.50e-3 [2.47e-3, 2.53e-3] - 2.46e-3 [2.43e-3, 2.50e-3] * - 25 - 7.89e-4 - 8.02e-4 [7.84e-4, 8.19e-4] - 7.81e-4 [7.64e-4, 7.99e-4] - 7.92e-4 [7.75e-4, 8.10e-4] * - 30 - 2.50e-4 - 2.53e-4 [2.44e-4, 2.63e-4] - 2.46e-4 [2.37e-4, 2.56e-4] - 2.51e-4 [2.41e-4, 2.61e-4] .. rubric:: BPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 10.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 303 - 295 - — - 1.00 - 0.317 s * - comnum.run_spec - float64 - 322 - 48.9 - — - 1.06 - 0.347 s * - Sionna - float32 (config.precision = 'single') - 437 - 487 - 8.61 - 1.44 - 0.447 s .. rubric:: QPSK: time Median of 5 runs, ns per bit; workload: 1000000 bits at Eb/N0 = 10.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 156 - 151 - — - 1.00 - 0.156 s * - comnum.run_spec - float64 - 166 - 25.5 - — - 1.06 - 0.162 s * - Sionna - float32 (config.precision = 'single') - 289 - 298 - 3.67 - 1.86 - 0.32 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib comnum --threads 1 --work runpod/_work --out runpod/_work/s2_rayleigh_comnum_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s2_rayleigh.py --lib comnum --threads all --work runpod/_work --out runpod/_work/s2_rayleigh_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib comnum-spec --threads 1 --work runpod/_work --out runpod/_work/s2_rayleigh_comnum-spec_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s2_rayleigh.py --lib comnum-spec --threads all --work runpod/_work --out runpod/_work/s2_rayleigh_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s2_rayleigh.py --lib sionna --threads 1 --work runpod/_work --out runpod/_work/s2_rayleigh_sionna_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s2_rayleigh.py --lib sionna --threads all --work runpod/_work --out runpod/_work/s2_rayleigh_sionna_all.json --timing-only OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s2_rayleigh.py --lib sionna --threads gpu --work runpod/_work --out runpod/_work/s2_rayleigh_sionna_gpu.json --timing-only 4 x 4 MIMO, 16-QAM, i.i.d. Rayleigh fading: ZF, MMSE and ML detection --------------------------------------------------------------------- Scenario 3: 4 x 4 spatial multiplexing of Gray 16-QAM over i.i.d. Rayleigh fading (one channel matrix per vector, known at the receiver), detected by zero forcing, MMSE and maximum likelihood. Parameters: - Eb/N0 (dB): 0, 4, 8, 12, 16 - vectors per point: linear detectors and sphere: 10^5 (quick: 5·10^3); exhaustive ML: 2·10^4 (quick: 10^3) - timing: 10^4 vectors at Eb/N0 = 12 dB (exhaustive ML: 10^3; quick: 2·10^3 and 2·10^2) - theory: ZF: 16-QAM over SISO Rayleigh fading (Winters, Salz & Gitlin, IEEE Trans. Commun. 42(2/3/4), 1994) Script: `benchmarks/libraries/s3_mimo.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s3_mimo.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - zero forcing - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - MMSE - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - ML (exhaustive) - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - yes * - ML (sphere decoder) - yes - yes - not available: no MIMO detection (only the 2 x 2 adaptive equalisers of polarisation multiplexing) - not available: no sphere decoder (its exhaustive ML detector is the case ml) .. rubric:: zero forcing: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - theory - 1.88 - yes - per-frame * - comnum.run_spec - float64 - theory - 2.98 - yes - binomial * - Sionna - float32 (config.precision = 'single') - theory - 1.79 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - theory - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.98e-1 - 1.98e-1 [1.97e-1, 1.99e-1] - 1.98e-1 [1.97e-1, 1.98e-1] - 1.98e-1 [1.97e-1, 1.99e-1] * - 4 - 1.20e-1 - 1.19e-1 [1.18e-1, 1.20e-1] - 1.19e-1 [1.19e-1, 1.20e-1] - 1.19e-1 [1.19e-1, 1.20e-1] * - 8 - 6.20e-2 - 6.16e-2 [6.10e-2, 6.23e-2] - 6.14e-2 [6.10e-2, 6.18e-2] - 6.26e-2 [6.19e-2, 6.32e-2] * - 12 - 2.83e-2 - 2.82e-2 [2.78e-2, 2.87e-2] - 2.81e-2 [2.79e-2, 2.84e-2] - 2.83e-2 [2.78e-2, 2.87e-2] * - 16 - 1.20e-2 - 1.21e-2 [1.18e-2, 1.24e-2] - 1.19e-2 [1.17e-2, 1.20e-2] - 1.19e-2 [1.16e-2, 1.22e-2] .. rubric:: MMSE: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 3.63 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.57 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.39e-1 [1.38e-1, 1.40e-1] - 1.38e-1 [1.38e-1, 1.39e-1] - 1.38e-1 [1.38e-1, 1.39e-1] * - 4 - 8.27e-2 [8.21e-2, 8.33e-2] - 8.40e-2 [8.36e-2, 8.44e-2] - 8.33e-2 [8.28e-2, 8.39e-2] * - 8 - 4.31e-2 [4.26e-2, 4.35e-2] - 4.29e-2 [4.26e-2, 4.32e-2] - 4.32e-2 [4.28e-2, 4.37e-2] * - 12 - 1.97e-2 [1.94e-2, 2.00e-2] - 1.99e-2 [1.97e-2, 2.01e-2] - 1.96e-2 [1.93e-2, 1.99e-2] * - 16 - 8.42e-3 [8.20e-3, 8.64e-3] - 8.33e-3 [8.19e-3, 8.47e-3] - 8.23e-3 [8.02e-3, 8.45e-3] .. rubric:: ML (exhaustive): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 2.53 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.09 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (320,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 0 - 1.25e-1 [1.23e-1, 1.26e-1] - 1.25e-1 [1.24e-1, 1.26e-1] - 1.25e-1 [1.23e-1, 1.27e-1] * - 4 - 4.03e-2 [3.90e-2, 4.16e-2] - 3.99e-2 [3.92e-2, 4.06e-2] - 4.05e-2 [3.92e-2, 4.17e-2] * - 8 - 4.12e-3 [3.68e-3, 4.56e-3] - 4.77e-3 [4.54e-3, 5.02e-3] - 4.47e-3 [4.02e-3, 4.93e-3] * - 12 - 2.34e-4 [1.24e-4, 3.45e-4] - 2.28e-4 [1.81e-4, 2.87e-4] - 1.59e-4 [7.61e-5, 2.43e-4] * - 16 - 0 [0, 9.38e-6] - 0 [0, 1.20e-5] - 1.25e-5 [0, 3.70e-5] .. rubric:: ML (sphere decoder): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 2.84 - yes - binomial .. dropdown:: BER and its 95 % interval, every point (1,600,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec * - 0 - 1.24e-1 [1.23e-1, 1.25e-1] - 1.25e-1 [1.24e-1, 1.25e-1] * - 4 - 3.97e-2 [3.92e-2, 4.03e-2] - 4.06e-2 [4.03e-2, 4.09e-2] * - 8 - 4.44e-3 [4.23e-3, 4.64e-3] - 4.62e-3 [4.51e-3, 4.73e-3] * - 12 - 1.91e-4 [1.49e-4, 2.32e-4] - 1.95e-4 [1.75e-4, 2.18e-4] * - 16 - 3.13e-6 [0, 6.37e-6] - 1.25e-5 [8.09e-6, 1.93e-5] .. rubric:: zero forcing: time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 2.99 - 2.87 - — - 1.00 - 0.0328 s * - comnum.run_spec - float64 - 2.51 - 0.451 - — - 0.839 - 0.0274 s * - Sionna - float32 (config.precision = 'single') - 5.49 - 2.24 - 0.162 - 1.84 - 0.0569 s .. rubric:: MMSE: time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 3.21 - 2.87 - — - 1.00 - 0.0306 s * - comnum.run_spec - float64 - 2.53 - 0.488 - — - 0.789 - 0.0362 s * - Sionna - float32 (config.precision = 'single') - 4.66 - 2.61 - 0.189 - 1.45 - 0.0545 s .. rubric:: ML (exhaustive): time Median of 5 runs, µs per vector; workload: 1000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 1315 - 1335 - — - 1.00 - 1.59 s * - comnum.run_spec - float64 - 1306 - 169 - — - 0.993 - 1.35 s * - Sionna - float32 (config.precision = 'single') - 19059 - 5567 - 71.3 - 14.5 - 19 s .. rubric:: ML (sphere decoder): time Median of 5 runs, µs per vector; workload: 10000 vectors at Eb/N0 = 12.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 3.27 - 3.36 - — - 1.00 - 0.0359 s * - comnum.run_spec - float64 - 3.37 - 0.641 - — - 1.03 - 0.0309 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib comnum --threads 1 --work runpod/_work --out runpod/_work/s3_mimo_comnum_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s3_mimo.py --lib comnum --threads all --work runpod/_work --out runpod/_work/s3_mimo_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib comnum-spec --threads 1 --work runpod/_work --out runpod/_work/s3_mimo_comnum-spec_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s3_mimo.py --lib comnum-spec --threads all --work runpod/_work --out runpod/_work/s3_mimo_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s3_mimo.py --lib sionna --threads 1 --work runpod/_work --out runpod/_work/s3_mimo_sionna_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s3_mimo.py --lib sionna --threads all --work runpod/_work --out runpod/_work/s3_mimo_sionna_all.json --timing-only OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s3_mimo.py --lib sionna --threads gpu --work runpod/_work --out runpod/_work/s3_mimo_sionna_gpu.json --timing-only Channel coding over BPSK and AWGN --------------------------------- Scenario 4: channel coding over BPSK and AWGN. Parameters: - convolutional (133, 171), soft Viterbi: K = 7, (133, 171), k = 1024 + 6 tail bits, soft Viterbi; Eb/N0 = 1, 2, 3, 4 dB - 5G NR LDPC (1024, 2048): 5G NR LDPC, k = 1024, n = 2048 (base graph 2, Z = 104), flooding sum-product, 50 iterations; Eb/N0 = 1, 1.5, 2, 2.5 dB - LDPC (1024, 512), same LLRs: PEG (1024, 512), flooding sum-product, 50 iterations, identical LLRs; Eb/N0 = 1.5, 2, 2.5, 3 dB - frames per point: 2000 (quick: 100) - timing: 200 frames (quick: 20) at Eb/N0 = 3 dB (conv) or 2 dB (LDPC); OptiCommPy without numba: 2 frames Script: `benchmarks/libraries/s4_coding.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s4_coding.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - comnum.run_spec - OptiCommPy (numba) - Sionna * - convolutional (133, 171), soft Viterbi - yes - yes - not available: no convolutional code - yes * - 5G NR LDPC (1024, 2048) - yes - not available: the specification format has no 5G NR LDPC code (NrLdpc: Python and Rust API) - not available: no 5G NR LDPC code (its decoder takes a parity-check matrix: see ldpc_same_llr) - yes * - LDPC (1024, 512), same LLRs - yes - not available: the runner draws its own channel (this case feeds the same LLRs to every decoder) - yes - yes .. rubric:: convolutional (133, 171), soft Viterbi: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - comnum.run_spec - float64 - comnum - 1.35 - 1.33 - yes - binomial * - Sionna - float32 (config.precision = 'single') - comnum - 1.41 - 2.53 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - union bound - comnum (functions) - comnum.run_spec - Sionna * - 1 - 3.05e-1 - 4.02e-2 [3.91e-2, 4.13e-2] - 4.02e-2 [3.99e-2, 4.05e-2] - 4.03e-2 [3.91e-2, 4.14e-2] * - 2 - 1.65e-2 - 4.97e-3 [4.64e-3, 5.30e-3] - 5.16e-3 [5.06e-3, 5.26e-3] - 5.32e-3 [4.97e-3, 5.67e-3] * - 3 - 6.27e-4 - 3.31e-4 [2.59e-4, 4.02e-4] - 2.98e-4 [2.76e-4, 3.23e-4] - 3.64e-4 [2.94e-4, 4.34e-4] * - 4 - 2.02e-5 - 2.54e-5 [8.46e-6, 4.23e-5] - 1.32e-5 [9.06e-6, 1.92e-5] - 3.08e-5 [9.04e-6, 5.25e-5] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - comnum.run_spec - Sionna * - 1 - 9.73e-1 [9.66e-1, 9.80e-1] - 9.77e-1 [9.69e-1, 9.83e-1] - 9.85e-1 [9.79e-1, 9.89e-1] * - 2 - 5.05e-1 [4.84e-1, 5.27e-1] - 4.84e-1 [4.63e-1, 5.06e-1] - 5.13e-1 [4.91e-1, 5.35e-1] * - 3 - 5.95e-2 [5.00e-2, 7.07e-2] - 6.05e-2 [5.09e-2, 7.18e-2] - 6.70e-2 [5.69e-2, 7.88e-2] * - 4 - 6.00e-3 [3.44e-3, 1.05e-2] - 5.00e-3 [2.72e-3, 9.18e-3] - 5.50e-3 [3.07e-3, 9.82e-3] .. rubric:: 5G NR LDPC (1024, 2048): error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - Sionna - float32 (config.precision = 'single') - comnum - 0.893 - 1.21 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - Sionna * - 1 - 8.79e-3 [7.46e-3, 1.01e-2] - 9.30e-3 [7.94e-3, 1.07e-2] * - 1.5 - 3.47e-5 [0, 1.03e-4] - 9.38e-5 [0, 2.04e-4] * - 2 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] * - 2.5 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - Sionna * - 1 - 9.40e-2 [8.20e-2, 1.08e-1] - 1.05e-1 [9.28e-2, 1.20e-1] * - 1.5 - 5.00e-4 [8.83e-5, 2.83e-3] - 1.50e-3 [5.10e-4, 4.40e-3] * - 2 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] * - 2.5 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] .. rubric:: LDPC (1024, 512), same LLRs: error rates .. list-table:: :header-rows: 1 * - Library - Precision - Against - max abs(z), BER - max abs(z), FER - Agrees - BER interval * - comnum (functions) - float64 - — - — - — - — - per-frame * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - comnum - 0.076 - 0 - yes - per-frame * - Sionna - float32 (config.precision = 'single') - comnum - 0.0846 - 0.0381 - yes - per-frame .. dropdown:: BER and its 95 % interval, every point (2,048,000 bits per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - OptiCommPy (numba) - Sionna * - 1.5 - 1.53e-2 [1.39e-2, 1.66e-2] - 1.53e-2 [1.40e-2, 1.67e-2] - 1.53e-2 [1.40e-2, 1.67e-2] * - 2 - 1.37e-3 [9.56e-4, 1.78e-3] - 1.37e-3 [9.55e-4, 1.78e-3] - 1.37e-3 [9.58e-4, 1.78e-3] * - 2.5 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] * - 3 - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] - 0 [0, 1.46e-6] .. dropdown:: Frame error rate and its 95 % interval (2,000 frames per point) .. list-table:: :header-rows: 1 * - Eb/N0 (dB) - comnum (functions) - OptiCommPy (numba) - Sionna * - 1.5 - 2.21e-1 [2.03e-1, 2.40e-1] - 2.21e-1 [2.03e-1, 2.40e-1] - 2.21e-1 [2.04e-1, 2.40e-1] * - 2 - 2.30e-2 [1.73e-2, 3.05e-2] - 2.30e-2 [1.73e-2, 3.05e-2] - 2.30e-2 [1.73e-2, 3.05e-2] * - 2.5 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] * - 3 - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] - 0 [0, 1.92e-3] .. rubric:: Same LLRs: the decisions frame by frame The decoders received identical LLRs. Over all the points: the frames where a library makes as many bit errors as comnum, and the frames it decodes (or fails to decode) as comnum does. .. list-table:: :header-rows: 1 * - Library - Points - Frames - Same bit errors - Same frame outcome * - OptiCommPy (numba) - 4 - 8000 - 7996 (99.95 %) - 8000 (100.00 %) * - Sionna - 4 - 8000 - 7984 (99.80 %) - 7999 (99.99 %) .. rubric:: convolutional (133, 171), soft Viterbi: decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 3.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 0.75 - 0.756 - — - 1.00 - 0.0021 s * - Sionna - float32 (config.precision = 'single') - 4.14 - 3.85 - 1.26 - 5.52 - 0.171 s .. rubric:: 5G NR LDPC (1024, 2048): decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 8.53 - 8.28 - — - 1.00 - 0.0173 s * - Sionna - float32 (config.precision = 'single') - 47.3 - 12.5 - 0.171 - 5.55 - 0.0652 s .. rubric:: LDPC (1024, 512), same LLRs: decoder time Median of 5 runs, ms per frame; workload: decoder only, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 1.54 - 1.62 - — - 1.00 - 0.0032 s * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - 7.45 - 5.00 - — - 4.83 - 5.59 s * - Sionna - float32 (config.precision = 'single') - 8.59 - 2.14 - 0.172 - 5.57 - 0.0275 s .. rubric:: convolutional (133, 171), soft Viterbi: whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 3.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 0.967 - 0.966 - — - 1.00 - 0.0021 s * - comnum.run_spec - float64 - 0.966 - 0.202 - — - 0.999 - 0.192 s * - Sionna - float32 (config.precision = 'single') - 4.65 - 4.34 - 2.05 - 4.81 - 0.171 s .. rubric:: 5G NR LDPC (1024, 2048): whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 8.72 - 8.69 - — - 1.00 - 0.0173 s * - Sionna - float32 (config.precision = 'single') - 47.8 - 12.6 - 0.181 - 5.47 - 0.0652 s .. rubric:: LDPC (1024, 512), same LLRs: whole-chain time Median of 5 runs, ms per frame; workload: whole chain, 200 frames at Eb/N0 = 2.0 dB. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions) - float64 - 1.64 - 1.68 - — - 1.00 - 0.0032 s * - OptiCommPy (numba) - float32 (decodeLDPC default prec = np.float32) - 7.08 - 4.91 - — - 4.32 - 5.59 s * - Sionna - float32 (config.precision = 'single') - 9.09 - 2.24 - 0.357 - 5.55 - 0.0275 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib comnum --threads 1 --work runpod/_work --out runpod/_work/s4_coding_comnum_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s4_coding.py --lib comnum --threads all --work runpod/_work --out runpod/_work/s4_coding_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib comnum-spec --threads 1 --work runpod/_work --out runpod/_work/s4_coding_comnum-spec_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s4_coding.py --lib comnum-spec --threads all --work runpod/_work --out runpod/_work/s4_coding_comnum-spec_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib opticommpy --threads 1 --work runpod/_work --out runpod/_work/s4_coding_opticommpy_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s4_coding.py --lib opticommpy --threads all --work runpod/_work --out runpod/_work/s4_coding_opticommpy_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s4_coding.py --lib sionna --threads 1 --work runpod/_work --out runpod/_work/s4_coding_sionna_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s4_coding.py --lib sionna --threads all --work runpod/_work --out runpod/_work/s4_coding_sionna_all.json --timing-only OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s4_coding.py --lib sionna --threads gpu --work runpod/_work --out runpod/_work/s4_coding_sionna_gpu.json --timing-only Split-step Fourier propagation, 16-QAM, 32 GBd, 10 x 80 km ---------------------------------------------------------- Scenario 5: single-polarisation split-step Fourier propagation of 32 GBd 16-QAM over 10 x 80 km of standard single-mode fibre, a fixed step of 1 km (80 steps per span), noiseless amplification at the end of every span (gain e^{alpha L}). Parameters: - fibre: alpha = 0.2 dB/km, D = 17 ps/nm/km, gamma = 1.3 /W/km, fc = 193.1 THz - link: 10 x 80 km, ideal (noiseless) amplification after every span - step: 1 km (800 steps); reference: comnum at 0.05 km (16000 steps) - signal: 16-QAM, 32 GBd, RRC roll-off 0.1, 4 samples per symbol, 4 dBm; 2^14 symbols (quick: 2^11) - timing: one propagation over the 10 spans Script: `benchmarks/libraries/s5_ssfm.py `_. .. dropdown:: The script .. literalinclude:: ../../benchmarks/libraries/s5_ssfm.py :language: python .. rubric:: Support .. list-table:: :header-rows: 1 * - Case - comnum (functions) - OptiCommPy (numba) - Sionna * - split-step Fourier - yes - yes - yes .. rubric:: Output fields NMSE of the output field, ``sum |a - b|^2 / sum |b|^2``, against comnum in float64 at the same step and against the step-converged reference (comnum, 0.05 km). The NMSE against the reference is the error of the 1 km step itself. .. list-table:: :header-rows: 1 * - Field - NMSE vs comnum (float64, 1 km) - NMSE vs reference * - comnum, float32 - 1.14e-8 - 7.45e-8 * - comnum, float64 - 0 - 3.69e-8 * - OptiCommPy, float64 - 2.24e-26 - 3.69e-8 * - Sionna, float64 - 3.55e-25 - 3.69e-8 * - Sionna, float64, its own convention (not conjugated) - 2.03e0 - 2.03e0 * - Sionna, float32 - 2.15e-8 - 2.45e-8 * - Sionna, float32, its own convention (not conjugated) - 2.03e0 - 2.03e0 .. rubric:: split-step Fourier: time Median of 5 runs, s per propagation; workload: 65536 samples, 10 x 80 km, 1 km steps. *Relative*: the 1-thread time over that of comnum (functions) on 1 thread. .. list-table:: :header-rows: 1 * - Library - Precision - 1 thread - all threads - GPU - Relative - First call * - comnum (functions), f64 - float64 - 1.29 - 1.31 - — - 1.00 - 1.31 s * - comnum (functions), f32 - float32 - 0.614 - 0.619 - — - 0.476 - 0.625 s * - OptiCommPy (numba), f64 - float64 (prec = np.complex128) - 4.54 - 4.51 - 0.144 - 3.52 - 5.28 s * - OptiCommPy (numba), f32 - float32 (prec = cp.complex64) - — - — - 0.144 - — - 0.155 s * - Sionna, single - float32 (precision = 'single', the default) - 5.36 - 1.51 - 0.277 - 4.16 - 5.39 s * - Sionna, double - float64 (precision = 'double') - 6.80 - 5.65 - 0.272 - 5.28 - 6.4 s .. dropdown:: The commands of every run .. code-block:: bash OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib comnum --threads 1 --work runpod/_work --out runpod/_work/s5_ssfm_comnum_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s5_ssfm.py --lib comnum --threads all --work runpod/_work --out runpod/_work/s5_ssfm_comnum_all.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib opticommpy --threads 1 --work runpod/_work --out runpod/_work/s5_ssfm_opticommpy_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s5_ssfm.py --lib opticommpy --threads all --work runpod/_work --out runpod/_work/s5_ssfm_opticommpy_all.json --timing-only OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s5_ssfm.py --lib opticommpy --threads gpu --work runpod/_work --out runpod/_work/s5_ssfm_opticommpy_gpu.json --timing-only OMP_NUM_THREADS=1 OPENBLAS_NUM_THREADS=1 MKL_NUM_THREADS=1 VECLIB_MAXIMUM_THREADS=1 NUMBA_NUM_THREADS=1 RAYON_NUM_THREADS=1 python s5_ssfm.py --lib sionna --threads 1 --work runpod/_work --out runpod/_work/s5_ssfm_sionna_1.json OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s5_ssfm.py --lib sionna --threads all --work runpod/_work --out runpod/_work/s5_ssfm_sionna_all.json --timing-only OMP_NUM_THREADS=11 OPENBLAS_NUM_THREADS=11 MKL_NUM_THREADS=11 VECLIB_MAXIMUM_THREADS=11 NUMBA_NUM_THREADS=11 RAYON_NUM_THREADS=11 python s5_ssfm.py --lib sionna --threads gpu --work runpod/_work --out runpod/_work/s5_ssfm_sionna_gpu.json --timing-only