.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "auto_tutorials/plot_04_specifications.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note :ref:`Go to the end ` to download the full example code. .. rst-class:: sphx-glr-example-title .. _sphx_glr_auto_tutorials_plot_04_specifications.py: Specifications and the command line =================================== A campaign is a document: the grid, the link as a list of blocks, the metrics, the stopping rule and the expected results, in a YAML file. The file runs from Python (:func:`comnum.run_spec`) and from the command line (``comnum run``), on the parallel Rust runner, and its report records the version, the seed and the specification, so that every number can be traced back to what produced it. This tutorial writes the chain of the first tutorial as a file, states what its results must be, searches the operating point of a metric, and shows the command line. .. GENERATED FROM PYTHON SOURCE LINES 13-24 .. code-block:: Python import tempfile from pathlib import Path import matplotlib.pyplot as plt import numpy as np import yaml import comnum from comnum import blocks as B .. GENERATED FROM PYTHON SOURCE LINES 25-27 **From a chain to a file.** ``to_spec`` returns the specification of a native chain as a ``dict``: the block names become labels, the variables ``$name``. .. GENERATED FROM PYTHON SOURCE LINES 27-42 .. code-block:: Python link = comnum.Chain( tx=B.Source(constellation={"kind": "qam", "m": comnum.var("m")}, n_symbols=4096), channel=B.AwgnEbn0(ebn0_db=comnum.var("ebn0_db")), ) spec = link.to_spec( name="qam_awgn", grid={"m": [4, 16, 64], "ebn0_db": {"from": 0, "to": 20, "step": 2}}, metrics=["ber", "ser"], stop={"min_errors": {"ber": 200}, "min_trials": 8, "max_trials": 4000}, seed=2024, ) spec["callbacks"] = {"skip_below": {"metric": "ber", "threshold": 1e-6, "along": "ebn0_db"}} text = yaml.safe_dump(spec, sort_keys=False) print(text) .. rst-class:: sphx-glr-script-out .. code-block:: none name: qam_awgn grid: m: - 4 - 16 - 64 ebn0_db: from: 0 to: 20 step: 2 metrics: report: - ber - ser stop: min_errors: ber: 200 min_trials: 8 max_trials: 4000 seed: 2024 link: - type: source label: tx constellation: kind: qam m: $m - type: awgn_ebn0 label: channel ebn0_db: $ebn0_db callbacks: skip_below: metric: ber threshold: 1.0e-06 along: ebn0_db .. GENERATED FROM PYTHON SOURCE LINES 43-47 **Running it.** :func:`comnum.run_spec` takes a path, a YAML text or a ``dict``, and returns one ``dict`` per condition (``condition``, ``status``, ``trials``, and every metric with its value and interval). The callback stops a sweep of ``ebn0_db`` once the BER falls below :math:`10^{-6}`: those conditions are reported as skipped. .. GENERATED FROM PYTHON SOURCE LINES 47-60 .. code-block:: Python folder = Path(tempfile.mkdtemp()) path = folder / "qam_awgn.yaml" path.write_text(text, encoding="utf-8") results = comnum.Results(comnum.run_spec(path), spec) print({r["status"] for r in results}, len(results), "conditions") fig, ax = plt.subplots() results.plot("ebn0_db", "ber", by="m", ax=ax, theory=lambda e, m: comnum.theory_ber(m, e, "qam")) # fmt: skip ax.set(xlabel="$E_b/N_0$ (dB)", ylabel="bit error rate", ylim=(1e-6, 0.5), title="Gray-mapped QAM over AWGN; dotted: closed form") # fmt: skip fig.tight_layout() .. image-sg:: /auto_tutorials/images/sphx_glr_plot_04_specifications_001.png :alt: Gray-mapped QAM over AWGN; dotted: closed form :srcset: /auto_tutorials/images/sphx_glr_plot_04_specifications_001.png :class: sphx-glr-single-img .. rst-class:: sphx-glr-script-out .. code-block:: none {'completed', 'skipped'} 33 conditions .. GENERATED FROM PYTHON SOURCE LINES 61-68 **Expected results.** An ``expect`` entry names a source and what the campaign must reproduce: a closed form (``theory``), a published curve (``curve``) or a value. The check passes when the expectation lies in the confidence interval of the metric (here the 99.9 % interval, plus a relative tolerance). :func:`comnum.verify_spec` runs the campaign and reports every check; the specifications of ``specs/validation/`` do this in CI for the error rates of Proakis & Salehi, the information rates and figures of the literature. .. GENERATED FROM PYTHON SOURCE LINES 68-83 .. code-block:: Python spec["expect"] = [ { "name": "SER of square QAM (exact)", "source": "Proakis & Salehi, Digital Communications, 5th ed., Sec. 4.3-2", "metric": "ser", "theory": {"function": "ser", "kind": "qam", "m": "$m", "ebn0_db": "$ebn0_db"}, "interval": 0.999, "rtol": 0.02, } ] report = comnum.verify_spec(spec) print("all checks passed:", report["passed"]) for c in report["checks"][:6]: print(f" {c['condition']}: {c['metric']} = {c['value']:.3e}, expected {c['expected']:.3e}") .. rst-class:: sphx-glr-script-out .. code-block:: none all checks passed: True m=4, ebn0_db=0: ser = 1.512e-01, expected 1.511e-01 m=4, ebn0_db=2: ser = 7.330e-02, expected 7.361e-02 m=4, ebn0_db=4: ser = 2.522e-02, expected 2.485e-02 m=4, ebn0_db=6: ser = 4.593e-03, expected 4.771e-03 m=4, ebn0_db=8: ser = 4.005e-04, expected 3.818e-04 m=4, ebn0_db=10: ser = 6.714e-06, expected 7.744e-06 .. GENERATED FROM PYTHON SOURCE LINES 84-89 **Operating points.** A ``search`` section replaces the sweep of a variable by the search of the value at which a metric reaches a target, for every combination of the other axes: here the :math:`E_b/N_0` required for a BER of :math:`10^{-3}`. The bracket is halved on the same random streams (common random numbers) until it is narrower than the tolerance. The result is compared with the inverted closed form. .. GENERATED FROM PYTHON SOURCE LINES 89-106 .. code-block:: Python search = dict(spec) search.pop("expect") search.pop("callbacks") search["grid"] = {"m": [4, 16, 64]} search["search"] = {"along": "ebn0_db", "range": [0, 20], "metric": "ber", "target": 1e-3, "tolerance": 0.05} # fmt: skip for r in comnum.search_spec(search): m = r["condition"]["m"] lo, hi = 0.0, 20.0 for _ in range(50): # the closed form, inverted by bisection mid = (lo + hi) / 2 lo, hi = (mid, hi) if comnum.theory_ber(m, mid, "qam") > 1e-3 else (lo, mid) print( f"{m:>2}-QAM: BER 1e-3 at Eb/N0 = {r['value']:.2f} dB " f"[{r['lo']:.2f}, {r['hi']:.2f}], closed form {lo:.2f} dB, {len(r['evaluations'])} evaluations" ) .. rst-class:: sphx-glr-script-out .. code-block:: none 4-QAM: BER 1e-3 at Eb/N0 = 6.86 dB [6.76, 6.97], closed form 6.79 dB, 11 evaluations 16-QAM: BER 1e-3 at Eb/N0 = 10.53 dB [10.46, 10.61], closed form 10.52 dB, 11 evaluations 64-QAM: BER 1e-3 at Eb/N0 = 14.79 dB [14.72, 14.86], closed form 14.77 dB, 11 evaluations .. GENERATED FROM PYTHON SOURCE LINES 107-122 **The command line.** The binary ``comnum`` (``cargo install comnum-spec``, or ``cargo build --release``) runs the same files without Python, writes the JSON and CSV reports to ``results/``, and documents the blocks: .. code-block:: bash comnum check qam_awgn.yaml # the link, the grid and the ports comnum conditions qam_awgn.yaml # the conditions of the grid comnum run qam_awgn.yaml --out results --threads 8 comnum verify qam_awgn.yaml # the expected results; exit code 1 on a failure comnum blocks awgn_ebn0 source # the parameters of a block, with units and defaults comnum schema # the JSON Schema of the blocks, for editors ``comnum run --fast-dev-run`` runs one trial per condition to check a file before a long campaign. The reports are read back with :meth:`comnum.Results.load`. .. GENERATED FROM PYTHON SOURCE LINES 124-127 **Reading a file back.** ``Chain.from_spec`` turns a file into a chain again, with its grid, metrics and stopping rule as ``settings``: a published campaign is rerun with one block changed in a line. .. GENERATED FROM PYTHON SOURCE LINES 127-131 .. code-block:: Python again = comnum.Chain.from_spec(path) print(again.names, again.settings["grid"]) harder = again.replace(channel=B.AwgnEbn0(ebn0_db=comnum.var("ebn0_db"))) print(np.array([r["status"] for r in harder.run(grid={"m": [16], "ebn0_db": [8, 10]})])) .. rst-class:: sphx-glr-script-out .. code-block:: none ['tx', 'channel'] {'m': [4, 16, 64], 'ebn0_db': {'from': 0, 'to': 20, 'step': 2}} ['completed' 'completed'] .. rst-class:: sphx-glr-timing **Total running time of the script:** (0 minutes 7.282 seconds) .. _sphx_glr_download_auto_tutorials_plot_04_specifications.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: plot_04_specifications.ipynb ` .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: plot_04_specifications.py ` .. container:: sphx-glr-download sphx-glr-download-zip :download:`Download zipped: plot_04_specifications.zip ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_