solprop.output.serialisation#

Serialisation of configuration objects into the JSON a run record stores.

serialise() turns any configuration object into JSON primitives, dropping nothing: what it cannot reduce it records by repr and marks irreproducible. A large numeric array becomes a reference to its bytes (RECORD_ARRAY_LIMIT), and a large array input is recorded by its ArrayDigest or by a StatedInput (recorded_input()). The private helpers here read a stored record back (_rebuild, _resolved) for the record types of solprop.output.records and solprop.output.metadata.

The manual page is docs/guide/results/provenance.md.

Module Attributes

`RECORD_ARRAY_LIMIT`	The most elements a numeric array of a run record holds as JSON (#309).
`INLINE_ARRAY_LIMIT`	The most elements an array input is recorded with inline (SEP 22, Q4).

Functions

`recorded_input`(value)	Return an input as the run record carries it: inline, or by its digest.
`serialise`(value)	Return any configuration object as a JSON-safe structure, dropping nothing.

Classes

`ArrayDigest`(sha256, dtype, shape)	An array recorded by the digest of its contents rather than inline.
`StatedInput`(identity, dtype, shape)	An array input recorded by the identity its caller states, its bytes unread.

class solprop.output.serialisation.ArrayDigest(sha256, dtype, shape)[source]#

Bases: object

An array recorded by the digest of its contents rather than inline.

What a run record holds for an array input above INLINE_ARRAY_LIMIT elements (SEP 22 section 5.2, Q4): the SHA-256 of its bytes in little-endian order, with its dtype and shape. Two inputs that differ in one sample record different digests, so their runs have different configuration hashes; the array itself is not recoverable from the record.

Attributes:

sha256str: Sixty-four lowercase hexadecimal characters.
dtypestr: The array’s dtype, as NumPy names it ("complex128").
shapetuple of int: The array’s shape.

Examples

>>> import numpy as np
>>> from solprop.output.serialisation import ArrayDigest
>>> digest = ArrayDigest.of(np.zeros((2, 3)))
>>> digest.dtype, digest.shape, len(digest.sha256)
('float64', (2, 3), 64)

dtype: str#

classmethod of(values)[source]#

Return the digest of an array’s contents.

Parameters:

valuesnumpy.ndarray: The array; read once, in C order and little-endian byte order, a block at a time unless it is held that way already, so that a broadcast input never becomes a whole volume.

Returns:

ArrayDigest: The digest, the dtype and the shape.

sha256: str#

shape: tuple[int, ...]#

solprop.output.serialisation.INLINE_ARRAY_LIMIT = 1048576#

The most elements an array input is recorded with inline (SEP 22, Q4).

A larger array – a gigabyte spatial launch – is recorded by an ArrayDigest of its bytes, dtype and shape instead, which costs seconds of hashing per input rather than tens of gigabytes of JSON, and still changes with any one sample. A caller may state its identity instead, with a StatedInput.

type solprop.output.serialisation.JsonValue = bool | int | float | str | list[JsonValue] | dict[str, JsonValue] | None#

solprop.output.serialisation.RECORD_ARRAY_LIMIT = 4096#

The most elements a numeric array of a run record holds as JSON (#309).

A larger one – a per-node weight, a flux plan’s coefficients – is recorded by a reference holding its digest, dtype and shape, and its bytes are stored beside the record, named by that digest (record_arrays()). So the hash takes the array’s bytes rather than its text, which costs milliseconds rather than seconds, and the array stays readable. The limit keeps a grid’s axes in the text, for a person reading it.

class solprop.output.serialisation.StatedInput(identity, dtype, shape)[source]#

Bases: object

An array input recorded by the identity its caller states, its bytes unread.

What a run record holds under input_identity (solprop.driver.simulation.Simulation.with_input()): for a caller that already identifies its input, such as by a scan’s delay and its own configuration, and has no use for a digest of it. The identity is taken on trust: unlike an ArrayDigest, two arrays that differ record the same thing if they are stated alike, so a resume and a scan’s cache accept one for the other.

Attributes:

identitystr: What the caller says the input is.
dtypestr: The array’s dtype, as NumPy names it ("complex128"), little-endian as an ArrayDigest records it.
shapetuple of int: The array’s shape.

Examples

>>> import numpy as np
>>> from solprop.output.serialisation import StatedInput
>>> StatedInput.of(np.zeros((2, 3), complex), "delay 12 fs")
StatedInput(identity='delay 12 fs', dtype='complex128', shape=(2, 3))

dtype: str#

identity: str#

classmethod of(values, identity)[source]#

Return the record of an array input under a stated identity.

Parameters:

valuesnumpy.ndarray: The array, of which only the dtype and the shape are read.
identitystr: What the caller says it is.

Returns:

StatedInput: The identity, the dtype and the shape.

shape: tuple[int, ...]#

solprop.output.serialisation.recorded_input(value)[source]#

Return an input as the run record carries it: inline, or by its digest.

Parameters:

valueobject: A pulse, a spatial pulse or an array input.

Returns:

object: value itself, unless it is an array of more than INLINE_ARRAY_LIMIT elements, which is recorded as its ArrayDigest.

Examples

>>> import numpy as np
>>> from solprop.output.serialisation import recorded_input
>>> recorded_input(np.zeros(4)).shape
(4,)
>>> type(recorded_input(np.zeros(2**20 + 1))).__name__
'ArrayDigest'

solprop.output.serialisation.serialise(value)[source]#

Return any configuration object as a JSON-safe structure, dropping nothing.

Dataclasses become tagged dictionaries of their constructor fields (less a live handle whose field metadata says record=False), enumerations their qualified name and value, NumPy arrays a nested list with their dtype and shape, or above RECORD_ARRAY_LIMIT elements a reference to their bytes, cubic splines their knots, coefficients, axis and extrapolation setting, complex numbers a real/imaginary pair, and mappings and sequences are recursed. Anything that cannot be reduced – a function, a lambda, an object with no structure – is recorded as its repr and marked reproducible: False, because a configuration that is silently incomplete is worse than one that says where it stops.

The function is idempotent: applying it to its own output returns that output, so a round-tripped RunMetadata hashes exactly as the original does.

Parameters:

valueobject: Anything at all.

Returns:

NoneType or bool or int or float or str or list or dict: A structure of JSON primitives, lists and string-keyed dictionaries.

Examples

>>> import numpy as np
>>> from solprop.output.serialisation import serialise
>>> serialise({"radius": 125e-6})
{'radius': 0.000125}
>>> serialise(np.array([1.0, 2.0]))["dtype"]
'float64'
>>> serialise(abs)["reproducible"]
False