.. DO NOT EDIT. .. THIS FILE WAS AUTOMATICALLY GENERATED BY SPHINX-GALLERY. .. TO MAKE CHANGES, EDIT THE SOURCE PYTHON FILE: .. "gallery/patterns/example3_resumable.py" .. LINE NUMBERS ARE GIVEN BELOW. .. only:: html .. note:: :class: sphx-glr-download-link-note Click :ref:`here ` to download the full example code .. rst-class:: sphx-glr-example-title .. _sphx_glr_gallery_patterns_example3_resumable.py: Resumable ======================== How to implement a resumable experiment. .. GENERATED FROM PYTHON SOURCE LINES 9-17 Overview ------------------------- We show how you can use ``dman`` to create a resumable script. To do so we introduce the following components: * ``tui.stack`` used for resumable for loops (requires ``rich``). * ``dman.uninterrupted`` to allow for keyboard interrupt safe storing of files. .. GENERATED FROM PYTHON SOURCE LINES 20-24 Setting up ---------------------- To setup the example you will need the following imports: .. GENERATED FROM PYTHON SOURCE LINES 24-36 .. code-block:: default import dman from dman import tui from dman.numeric import barray import numpy as np import numpy.random as npr import time from typing import Tuple .. GENERATED FROM PYTHON SOURCE LINES 37-39 We will also be using the following ``modelclass`` to store our data. The field ``state`` will keep track of the current state of the script. .. GENERATED FROM PYTHON SOURCE LINES 39-47 .. code-block:: default @dman.modelclass class Experiment: data: barray = dman.recordfield(stem="data", default=None) state: Tuple[int] = None .. GENERATED FROM PYTHON SOURCE LINES 48-51 Running the experiment ------------------------- We load the experiment if it exists, otherwise we create a default one. .. GENERATED FROM PYTHON SOURCE LINES 51-56 .. code-block:: default shape = (30, 10) exp: Experiment = dman.load("experiment", default_factory=Experiment) if exp.data is None: exp.data = np.zeros(shape) .. GENERATED FROM PYTHON SOURCE LINES 57-60 We use a stack to iterate through the two nested for loops, populating the data array. You can alternatively use ``sg(range(shape[1]), ...)``, replacing ``range`` with any other iterable. .. GENERATED FROM PYTHON SOURCE LINES 60-77 .. code-block:: default rg = npr.default_rng(1024) with tui.stack(exp.state) as sg: it = sg.range(shape[0], log={'value': np.nan}) for i in it: for j in sg.range(shape[1]): # generate new data point time.sleep(0.01) exp.data[i, j] = rg.normal() # update descriptors of tasks it.update(value=exp.data[i, j]) # store the state and current result exp.state = sg.state with dman.uninterrupted(): dman.save("experiment", exp) .. rst-class:: sphx-glr-script-out .. code-block:: none Range [30/30] | value=-0.7685216292562801 | ... ━━━━━━━━━━━━━━━━━━━ 100% 0:00:00 .. GENERATED FROM PYTHON SOURCE LINES 78-87 We used ``dman.uninterrupted`` to make sure that no keyboard interrupts occur while saving to disk. Instead they are captured and raised after ``dman.save`` is completed. You can try running the script and seeing what happens when you press ``CTRL+C`` and resume. No matter how many times you quit the script. Eventually the full array should be computed: .. GENERATED FROM PYTHON SOURCE LINES 87-91 .. code-block:: default exp: Experiment = dman.load("experiment") with np.printoptions(linewidth=80, formatter={"float": lambda f: f"{f:+0.2f}"}): tui.pprint(exp.data) .. rst-class:: sphx-glr-script-out .. code-block:: none barray([[-0.45, -0.14, +2.42, +0.08, +1.49, +0.46, -0.21, -0.66, -0.36, +0.45], │ │ [-1.47, -0.37, +1.56, +0.76, +0.25, -0.34, -0.89, +2.04, +0.59, -1.03], │ │ [-0.84, -0.52, -0.46, +0.60, +1.16, -0.09, +1.10, -1.23, +0.52, +0.66], │ │ [+1.17, +1.00, +0.95, -1.98, -0.51, -1.62, -0.25, -1.94, +0.88, -0.39], │ │ [+0.95, -0.69, -0.12, +0.81, -0.92, -1.94, +2.43, -1.63, +0.37, +0.90], │ │ [+0.55, -1.77, +0.66, +0.36, +0.72, -2.18, -0.07, -0.68, -1.36, -0.12], │ │ [-1.51, -0.05, -0.90, -0.33, -0.23, -2.00, -0.07, +0.49, -0.68, -0.03], │ │ [+0.23, +0.27, -0.66, -0.08, -0.46, +0.19, +2.17, +1.06, +0.51, +0.47], │ │ [-0.12, +0.26, +0.52, +0.28, +2.12, -0.48, +0.65, -0.62, +0.84, -0.26], │ │ [-1.31, +0.52, -0.04, -1.84, +0.14, +0.28, +0.37, +0.17, -1.39, +0.94], │ │ [+0.05, -0.43, -0.31, +0.09, -0.28, +1.16, +0.25, -0.85, +0.23, -0.80], │ │ [-0.19, -0.19, -0.43, +0.29, +0.54, +0.30, +1.18, +1.56, +0.70, -1.45], │ │ [+0.26, -2.29, +0.04, +0.56, -0.94, -1.60, -0.84, -0.29, +1.21, -0.61], │ │ [+1.19, +0.04, +0.46, +1.93, -0.44, -0.39, +1.42, -0.92, +2.12, +0.58], │ │ [+0.50, +0.63, +0.79, -1.00, -1.46, -0.32, +0.97, -0.27, -0.36, -1.63], │ │ [+0.11, -1.04, +0.36, -1.84, +0.91, +0.61, -1.09, -1.19, -0.16, -1.71], │ │ [-0.79, -2.16, -0.61, +0.51, -1.76, +0.38, -0.37, -0.43, +0.14, +0.73], │ │ [-2.06, -2.30, -0.32, -0.65, -1.66, -0.47, +0.12, -0.20, +0.17, -0.35], │ │ [-0.79, -1.22, -0.72, -0.04, +0.15, -0.27, -1.15, +1.51, -0.20, -1.05], │ │ [+0.70, -0.89, -0.58, +1.32, -0.92, -1.03, +0.79, -1.05, -0.34, +0.00], │ │ [+1.32, +0.49, -0.14, +0.97, -0.06, +0.26, +0.43, +1.58, -0.15, +0.08], │ │ [+0.75, -1.41, +0.30, -0.05, -0.44, +0.72, -0.90, -1.26, -0.40, -0.42], │ │ [+1.32, +0.21, -0.30, +1.07, -1.47, +1.38, +0.04, -0.99, -0.67, -0.84], │ │ [+1.67, -1.85, -0.48, -0.82, +1.79, -0.87, +1.55, -1.05, -1.35, -1.97], │ │ [+0.66, +0.05, -1.12, -0.46, -1.14, +0.60, -0.57, -0.33, -0.32, -1.37], │ │ [-0.10, -0.51, -0.41, -0.15, +1.04, -0.19, +1.62, -0.32, -0.59, +2.41], │ │ [-1.18, -0.80, +2.57, -0.20, +0.47, +0.96, +0.84, -1.17, -1.10, -1.89], │ │ [+0.99, +1.39, -0.15, -1.29, +0.11, -0.26, -0.41, -0.93, -0.63, -0.69], │ │ [+1.15, -0.26, -0.88, +0.48, -1.88, -0.32, -0.02, +0.31, +0.29, -0.42], │ │ [+0.91, +0.15, +0.15, +0.41, -0.52, -0.23, -0.67, +0.95, -0.80, -0.77]]) .. rst-class:: sphx-glr-timing **Total running time of the script:** ( 0 minutes 4.371 seconds) .. _sphx_glr_download_gallery_patterns_example3_resumable.py: .. only:: html .. container:: sphx-glr-footer sphx-glr-footer-example .. container:: sphx-glr-download sphx-glr-download-python :download:`Download Python source code: example3_resumable.py ` .. container:: sphx-glr-download sphx-glr-download-jupyter :download:`Download Jupyter notebook: example3_resumable.ipynb ` .. only:: html .. rst-class:: sphx-glr-signature `Gallery generated by Sphinx-Gallery `_