solprop.output.store

The one save protocol both output handlers implement, and the field retention.

Output is the protocol solprop.output.memory.MemoryOutput and solprop.output.hdf5.HDF5Output both implement, so processing reads either through the same accessors; PlaneStore is the base class both are built on, and FieldRetention says which planes keep their field. A handler converts each plane’s field to the optics convention on the way to storage (docs/dev/conventions.md §2; this module and the handlers are boundary place 2).

The manual page is docs/guide/results/output.md.

Module Attributes

FieldsChoice

Which saved planes keep their field: none of them, the named ones, or all.

Functions

field_retention_refusal(fields, field_planes)	Say what is wrong with a field-retention request, or that nothing is.
stored_spectrum(field, kind[, out])	Return solprop.fourier.spectrum_to_optics() of a large field, in threads.

Classes

CheckpointOutput(*args, **kwargs)	Declare optional outcome and terminal-checkpoint persistence for a handler.
FieldRetention([fields, field_planes])	Which saved planes of a run keep their field (SEP 22 section 5.1).
Output(*args, **kwargs)	What every output handler does, and all that processing may assume of one.
PlaneLayout(field_shape, stat_names, ...[, ...])	What every plane of one run has in common, fixed by the first plane saved.
PlaneStore()	The shared half of an output handler: one save protocol for every backing store.
RetainingOutput(*args, **kwargs)	Declare that a handler accepts field-free planes, and is told the run's rule.

class solprop.output.store.CheckpointOutput(*args, **kwargs)

Bases: Protocol

Declare optional outcome and terminal-checkpoint persistence for a handler.

property outcome: RunOutcome | None

Recorded status, or None when no outcome was recorded.

record_outcome(outcome, checkpoint=None)

Persist execution status and an optional accepted terminal field.

Parameters:

outcomeRunOutcome

Cumulative work and completion status.

checkpointTerminalCheckpoint or None

Accepted field between regular saves. None retains a pending checkpoint during progress and clears it when the run completes.

property terminal_checkpoint: TerminalCheckpoint | None

Latest accepted terminal field outside regular observations.

class solprop.output.store.FieldRetention(fields='all', field_planes=())

Bases: object

Which saved planes of a run keep their field (SEP 22 section 5.1).

A run is homogeneous: it keeps the field on every plane, on the planes it names, or on none, and its statistics on every plane either way. A plane without its field stores no field and no noise_phase; what it saves is its position, its statistics and its step history.

Attributes:

fields{‘none’, ‘planes’, ‘all’}

Keep no field, the fields of field_planes, or every field (the default).

field_planestuple of int

Saved-plane indices for "planes", the entrance as 0 and negative from the end until resolved(); empty otherwise.

Raises:

OutputError

If the request is not valid (field_retention_refusal()).

Examples

>>> from solprop.output.store import FieldRetention
>>> kept = FieldRetention("planes", (0, -1)).resolved(5)
>>> kept.field_planes, kept.keeps(4), kept.keeps(2), kept.kept(5)
((0, 4), True, False, 2)

describe()

Return the rule in words, for a refusal naming it.

Returns:

str

For example "fields='planes' (saved planes 0, 4)".

field_planes: tuple[int, ...] = ()

fields: Literal['none', 'planes', 'all'] = 'all'

keeps(index)

Return whether the saved plane at index keeps its field.

Parameters:

indexint

The plane’s index, the entrance as 0; the rule must be resolved().

Returns:

bool

True for every plane of an "all" run, a named plane of a "planes" run, and no plane of a "none" run.

kept(n_planes)

Return how many of a run’s planes keep their field.

Parameters:

n_planesint

The planes the run saves; the rule must be resolved().

Returns:

int

The planes whose field is stored.

resolved(n_planes)

Return the rule with its indices counted from the entrance, sorted, unique.

Parameters:

n_planesint

The planes the run saves, the entrance included.

Returns:

FieldRetention

The same rule with non-negative indices.

Raises:

OutputError

If an index lies outside the run’s planes.

solprop.output.store.FieldsChoice

Which saved planes keep their field: none of them, the named ones, or all.

alias of Literal[‘none’, ‘planes’, ‘all’]

class solprop.output.store.Output(*args, **kwargs)

Bases: Protocol

What every output handler does, and all that processing may assume of one.

One protocol so that solprop.output.memory.MemoryOutput and solprop.output.hdf5.HDF5Output are interchangeable: a run is written to memory for a scan and to disk for a long propagation without the driver knowing which.

Everything a handler hands back through __getitem__ is as stored, so the field is in the optics convention; fields is the accessor that also offers the internal one, for direct comparison with a file stored in the FFT convention.

close()

Finish the run, releasing any resource held. Calling it twice is harmless.

fields(convention='optics')

Return every stored field, in the requested sign convention.

Parameters:

convention{‘optics’, ‘fft’}, optional

'optics' returns the arrays as stored; 'fft' converts them back to the internal convention a backend works in, for direct comparison with a file stored in the FFT convention.

Returns:

numpy.ndarray

Shape (n_z, n_modes, n_pol, n_omega), complex.

property metadata: RunMetadata | None

Record of the run, or None before it is opened.

property n_saved: int

How many planes have been stored.

open(metadata, *, force=False)

Begin a run, recording what produced it.

Parameters:

metadataRunMetadata

The record of the run about to start.

forcebool, optional

Whether to overwrite metadata already held for a different run.

save(plane)

Store one saved plane.

A run gives the handler the backend’s observation itself, not a copy: the plane’s arrays are read-only, nothing writes to them again, and a handler may keep them or reduce them and let them go, with no copy of its own.

Parameters:

planeSavedPlane

The position, field, statistics and step history to store. On a plane that keeps no field, plane.field and plane.noise_phase are None (RetainingOutput).

property stat_names: tuple[str, ...]

The names of the statistics stored at every plane.

The names, sorted; empty before the first plane is saved.

property step_stat_names: tuple[str, ...]

The statistics stored between planes, if any.

The names, sorted; empty when the run recorded no periodic subset.

class solprop.output.store.PlaneLayout(field_shape, stat_names, has_noise_phase, step_stat_names=(), retention=FieldRetention(fields='all', field_planes=()))

Bases: object

What every plane of one run has in common, fixed by the first plane saved.

Attributes:

field_shapetuple of int

(n_modes, n_pol, n_omega) of the stored field, or of the run’s state when it keeps no field; empty when neither is known.

stat_namestuple of str

The statistic names, sorted.

has_noise_phasebool

Whether a noise phase is stored beside each stored field.

step_stat_namestuple of str

The statistics the between-planes channel carries, sorted; empty for a run without a periodic subset, which is the default.

retentionFieldRetention

Which planes keep their field, resolved; every plane by default.

Examples

>>> from solprop.output.store import PlaneLayout
>>> layout = PlaneLayout(
...     field_shape=(1, 1, 8), stat_names=("energy",), has_noise_phase=False
... )
>>> layout.field_shape, layout.stat_names, layout.retention.fields
((1, 1, 8), ('energy',), 'all')

field_shape: tuple[int, ...]

has_noise_phase: bool

retention: FieldRetention = FieldRetention(fields='all', field_planes=())

stat_names: tuple[str, ...]

step_stat_names: tuple[str, ...] = ()

class solprop.output.store.PlaneStore

Bases: Displayed, ABC

The shared half of an output handler: one save protocol for every backing store.

solprop.output.memory.MemoryOutput and solprop.output.hdf5.HDF5Output differ only in where the arrays go. What they must not differ in is what goes there: the sign convention of the stored field, the metadata overwrite rule, the key names and the step-history layout. Those live here, once, which is what makes the two handlers store a run bitwise identically, with one overwrite rule and one growth strategy.

The base class also holds the run’s layout – the field’s shape, the statistic names and whether a noise phase is stored – fixed by the first plane saved and checked against every later one, so that a plane that does not fit is refused by name in either handler, rather than being dropped or raising deep inside the writer.

A subclass implements four hooks: _write_metadata, _append, _read and _release, and may implement a fifth, _write_annotation, to persist the post-run annotations (annotate()) a store held only in memory otherwise.

A driver that knows how many planes the run will save says so with reserve(), and each store sizes its first allocation for them; a run that saves more grows past it, so the number is a hint, never a limit.

A run’s session exports a kept field straight into the stored form (SEP 47 section 5.4): field_destination() gives the room to fill, a store’s own (MemoryOutput) or none, for a new array (HDF5Output), and save_exported() takes the plane, so that the field is converted once and, in memory, written once, where it is kept.

Examples

>>> from solprop.output import MemoryOutput
>>> from solprop.output.store import Output, PlaneStore
>>> handler = MemoryOutput()
>>> isinstance(handler, PlaneStore), isinstance(handler, Output)
(True, True)

annotate(name, value)

Record a post-run annotation, replacing any of the same name.

The run’s record is not touched: an annotation stays outside the configuration hash, so a file annotated after its run is still the same run to solprop.output.hdf5.check_cache() and to a scan. A store may be annotated after it is closed; a file is then reopened briefly to write it.

Parameters:

namestr

A non-empty name without /, such as PEAK_FIELDS_ANNOTATION.

valueobject

Plain JSON values: mappings with string keys, lists, strings, numbers, booleans, None; anything else is refused here, the store’s boundary.

Raises:

OutputError

If the store was never opened, the name is empty or holds /, or the value is not plain JSON.

Examples

>>> from solprop.output import MemoryOutput
>>> from solprop.output.metadata import _example_metadata
>>> handler = MemoryOutput()
>>> handler.open(_example_metadata())
>>> handler.annotate("note", {"peak": 1.5})
>>> handler.annotations["note"]
{'peak': 1.5}

property annotations: Mapping[str, JsonValue]

Post-run annotations the store holds, by name.

An annotation is something learned only once the run has closed – the material manifest’s peak-field report (PEAK_FIELDS_ANNOTATION) is one – so it is not part of the run’s record: the configuration hash, a resume and a scan’s reuse never see it. A store without any, a file written before annotations existed included, answers with an empty mapping.

Each annotation’s plain JSON values, by name; a copy.

Examples

>>> from solprop.output import MemoryOutput
>>> dict(MemoryOutput().annotations)
{}

close()

Finish the run, releasing whatever the store holds open.

Calling it a second time does nothing, so a driver may close in a finally and a caller may close again without checking.

property closed: bool

Whether the handler has been closed.

field_destination(plane, shape)

Return where the next plane’s field is to be exported, or None.

The run’s service asks before it exports a kept field (SEP 47 section 5.4). A store with room of its own for the field returns that room, a writeable, C-contiguous complex128 array of shape, and the export fills it once in the stored form, the true-power amplitude (or a spatial route’s field measure) in the optics convention; save_exported() then takes the plane. The room is the store’s: no other plane is given it and nothing writes to it again once the plane is stored. A store without such room returns None and is handed a new array, which it takes as its own.

Parameters:

planeSavedPlane

The plane about to be stored, without its field: its statistics fix the layout of a store’s first allocation.

shapetuple of int

(n_modes, n_pol, n_omega) of the field.

Returns:

numpy.ndarray or None

The room, or None for a new array. This base class has none.

Raises:

OutputError

If the handler is closed or was never opened, the run keeps no field at this plane (retain()), or shape is not the run’s field shape.

property field_shape: tuple[int, ...]

Shape of one plane’s field, stored or not.

(n_modes, n_pol, n_omega); empty before anything has said it.

fields(convention='optics')

Return every stored field, in the requested sign convention.

Parameters:

convention{‘optics’, ‘fft’}, optional

'optics' returns the arrays exactly as stored; 'fft' converts them back to the internal convention a backend works in, so that a file stored in the FFT convention can be compared with them bin by bin.

Returns:

numpy.ndarray

Shape (n_z, n_modes, n_pol, n_omega), complex.

Raises:

OutputError

If nothing has been stored yet, or the run did not keep the field on every plane (plane_field() reads a plane that kept one).

property metadata: RunMetadata | None

Record of the run, or None before it is opened.

property n_saved: int

How many planes have been stored.

open(metadata, *, force=False)

Begin a run, recording what produced it.

Parameters:

metadataRunMetadata

The record of the run about to start.

forcebool, optional

Whether to overwrite metadata already held for a different run. Without it a second, different record raises; with it the record is replaced and the replacement logged (resolve_metadata_write()).

Raises:

OutputError

If the handler is closed, or a different record is already held and force is false.

property outcome: RunOutcome | None

Execution outcome, or None when none was recorded.

The last recorded completion state and cumulative ledgers.

plane_field(index, convention='optics')

Return one saved plane’s field, in the requested sign convention.

The accessor that works whatever the run kept: a plane that kept its field returns it, and one that did not is refused by name.

Parameters:

indexint

The saved plane, the entrance as 0 and negative from the end.

convention{‘optics’, ‘fft’}, optional

As for fields().

Returns:

numpy.ndarray

Shape (n_modes, n_pol, n_omega), complex.

Raises:

OutputError

If the index is out of range, or that plane kept no field, naming the run’s fields.

Examples

>>> import numpy as np
>>> from solprop.output import MemoryOutput
>>> from solprop.output.metadata import _example_metadata
>>> from solprop.output.records import SavedPlane
>>> handler = MemoryOutput()
>>> handler.open(_example_metadata())
>>> n_omega = handler.metadata.grid.n_omega
>>> handler.save(SavedPlane(z=0.0, field=np.ones((1, 1, n_omega), complex)))
>>> handler.plane_field(-1).shape
(1, 1, 2560)

property recipe: RecipeRecord | None

The recipe the run was built from, or None for a run from a script.

record_outcome(outcome, checkpoint=None)

Persist status and optional terminal field without extending regular saves.

Parameters:

outcomeRunOutcome

Current status and cumulative call ledgers.

checkpointTerminalCheckpoint or None

Accepted field, with no newly requested atomic statistics. With a "running" outcome it is a running checkpoint (SEP 47 section 5.5): an accepted endpoint between saved planes, written so that a snapshot or a later failure has a state to resume from. Its attempts and periodic rows are not consumed: the next regular plane still carries them, and saving that plane supersedes the checkpoint. None keeps a pending checkpoint during progress and clears it otherwise; passing back the checkpoint held (terminal_checkpoint), or a copy, with a closing outcome keeps it as the state to resume from.

Raises:

OutputError

If the handler is closed or holds no metadata, or the checkpoint disagrees with the outcome or stored field layout.

record_recipe(record)

Record the recipe the run was built from, unless one is recorded already.

Written once, at the first run start, and never rewritten (SEP 41, S2): a later session of the same file, from a script or from an edited recipe, leaves the first session’s recipe in place and is recorded in its own execution record.

Parameters:

recordRecipeRecord

The authored and resolved texts and the data files.

reserve(n_planes)

Expect a run of n_planes saved planes in all, before the first is saved.

A store sizes its plane-indexed arrays for this many planes when the first plane fixes their shapes, rather than growing them plane by plane; a run that saves more grows them past it (by doubling), and one that saves fewer uses only what it saved, so the count is a hint and changes nothing that is stored. The run loop (solprop.driver.run.run_problem()) reserves the entrance plane and every save plane. A later call can only raise the count.

Parameters:

n_planesint

The planes the run will save, the entrance included; non-negative.

Raises:

OutputError

If n_planes is negative.

Examples

>>> from solprop.output import MemoryOutput
>>> handler = MemoryOutput()
>>> handler.reserve(201)
>>> handler.n_saved
0

retain(retention, field_shape)

Say which planes of the run keep their field, before the first is saved.

The run loop (solprop.driver.run.run_problem()) calls it with the run’s resolved rule. A store then refuses a plane that breaks the rule, sizes its field storage for the planes that keep one, and answers field reads on the others by naming the rule. A store continuing a run (a resumed file) already holds a rule and accepts only the same one.

Parameters:

retentionFieldRetention

The rule, resolved() against the run’s planes.

field_shapetuple of int

(n_modes, n_pol, n_omega) of the run’s state, which a run that keeps no field records in its place.

Raises:

OutputError

If the store already holds planes under another rule.

Examples

>>> from solprop.output import MemoryOutput
>>> from solprop.output.store import FieldRetention
>>> handler = MemoryOutput()
>>> handler.retain(FieldRetention("none"), (1, 1, 8))
>>> handler.retention.fields, handler.field_shape
('none', (1, 1, 8))

property retention: FieldRetention

Which planes of the run keep their field.

The rule the run declared (retain()), every plane by default.

save(plane)

Store one saved plane, converting its field to the optics convention.

Parameters:

planeSavedPlane

The position, field, statistics and step history to store. Its field is in the internal FFT convention and is written out as solprop.fourier.spectrum_to_optics() of it; a field-free plane stores none.

Raises:

OutputError

If the handler is closed or was never opened, or the plane does not fit the run’s layout: a field where the run keeps none, none where it keeps one (retain()), another shape or other statistics.

save_exported(plane, stored)

Store one saved plane whose field was exported already in its stored form.

The run’s service calls it instead of save() when the store takes_exports; what is stored is what save() stores for the same field, since the export’s conversion is the one save() makes.

Parameters:

planeSavedPlane

The position, statistics and step history to store, with no field of its own.

storedStoredField

Its field in the stored form, in the room field_destination() returned or in any other array, which is copied in where the store keeps fields in its own room; and its noise phase.

Raises:

OutputError

If plane carries a field, or for any reason save() refuses a plane.

property stat_names: tuple[str, ...]

The names of the statistics stored at every plane.

Read off the layout the first plane fixed, so that the two handlers answer alike, and a file continued or opened for reading answers from what it holds.

The names, sorted; empty before the first plane is saved.

property step_stat_names: tuple[str, ...]

The statistics the between-planes channel carries.

Empty for a run that asked for no periodic subset, which is how a reader tells whether the channel is there to prefer over the planes.

The names, sorted; empty when the run has no channel.

property takes_exports: bool

Whether a run hands this store its kept fields already stored.

A store that takes them is given each kept plane through save_exported(), its field exported by the run’s session straight into the stored form, and into field_destination() (SEP 47 section 5.4). A subclass whose save, _append or _stored_field comes from a class that does not vouch for the exported path – its own override, or a mixin’s ahead of the store in its method resolution order – does not: it sees every plane, and its field, through save as before. A class vouches with _takes_exports = True in its own body, and opts out with False.

Whether save_exported() is how the run stores kept fields here.

Examples

>>> from solprop.output import MemoryOutput
>>> MemoryOutput().takes_exports
True

property terminal_checkpoint: TerminalCheckpoint | None

Terminal checkpoint without treating it as a saved plane.

A field in the internal FFT convention and output amplitude units.

class solprop.output.store.RetainingOutput(*args, **kwargs)

Bases: Protocol

Declare that a handler accepts field-free planes, and is told the run’s rule.

Separate from Output, as CheckpointOutput is, because it is optional: a handler that stores every field need not implement it, and a run that keeps fewer than every field refuses a handler that does not (fields="planes" and fields="none", SimulationSettings). On a plane that keeps no field, Output.save() is given field=None and noise_phase=None.

Examples

>>> from solprop.output.store import RetainingOutput
>>> class Spectra:
...     def retain(self, retention, field_shape):
...         self.keeps = retention.field_planes
>>> isinstance(Spectra(), RetainingOutput)
True

retain(retention, field_shape)

Say which planes of the run keep their field, before the first is saved.

The run loop (solprop.driver.run.run_problem()) calls it once per run, after Output.open() and the handler’s reserve if it has one and before any Output.save(), and for fields="all" too.

Parameters:

retentionFieldRetention

The rule, already resolved() against the run’s planes (its indices count from the entrance, and none is negative).

field_shapetuple of int

(n_modes, n_pol, n_omega) of the run’s state, known here even when no plane will carry a field.

Raises:

OutputError

A handler that already holds planes under a different rule raises this, rather than mixing two rules in one record.

solprop.output.store.field_retention_refusal(fields, field_planes)

Say what is wrong with a field-retention request, or that nothing is.

The one validator behind solprop.driver.settings.SimulationSettings, solprop.driver.run.run_problem(), solprop.driver.scan.run_scan() and FieldRetention; each door raises its own layer’s error with the message.

Parameters:

fieldsobject

The request, one of "none", "planes" and "all".

field_planesobject

Saved-plane indices, the entrance as 0 and negative from the end; required exactly when fields is "planes".

Returns:

str or None

Why the request is refused, or None when it is valid.

Examples

>>> from solprop.output.store import field_retention_refusal
>>> field_retention_refusal("planes", [0, -1]) is None
True
>>> field_retention_refusal("planes", [])
"field_planes is required exactly when fields='planes'"

solprop.output.store.stored_spectrum(field, kind, out=None)

Return solprop.fourier.spectrum_to_optics() of a large field, in threads.

One thread’s conjugation and sign flip cost more than the state’s whole crossing from a device, and a state’s rows are independent. The bits are the same.

Parameters:

fieldnumpy.ndarray

A contiguous spectrum in the internal convention.

kind{‘field’, ‘envelope’}

The grid’s spectral layout.

outnumpy.ndarray or None, optional

Contiguous room to write it to; a new array by default.

Returns:

numpy.ndarray

out, holding the spectrum in the optics convention.