emmi_data_management.checkpoint.iohash

Attributes

DEFAULT_CHUNK

Functions

atomic_write_and_hash(destination_path, chunks, *[, ...])

Write bytes chunks to a temp file in the same directory as destination_path.

Module Contents

emmi_data_management.checkpoint.iohash.DEFAULT_CHUNK = 1048576
emmi_data_management.checkpoint.iohash.atomic_write_and_hash(destination_path, chunks, *, compute_hash=True, progress=None)

Write bytes chunks to a temp file in the same directory as destination_path. Additionally, updates SHA-256 while writing , then atomically replaces destination_path.

Parameters:

  • destination_path (pathlib.Path) – Destination path.

  • chunks (collections.abc.Iterable[bytes]) – Iterable of bytes to write.

  • compute_hash (bool) – Compute hash of chunks.

  • progress (object | None) – Progress object or None for no progress. Expected to have .update(int) and .close() methods.

Returns:

A hex digest.

Return type:

  • str