flint.archive

Operations around preserving files and products from an flint run

Functions

cli(→ None)

copy_files_into(→ pathlib.Path)

Copy a set of specified files into an output directory, If a file happens to

copy_sbid_files_archive(→ pathlib.Path)

Copy files from an SBID processing folder into a final location. Uses the

create_sbid_tar_archive(→ pathlib.Path)

Create a tar file of key products in a SBID folder.

get_archive_options_from_yaml(→ dict[str, Any])

Load the archive options from a specified strategy file

get_parser(→ argparse.ArgumentParser)

resolve_glob_expressions(→ tuple[pathlib.Path, Ellipsis])

Collect a set of files given a base directory and a set of glob expressions. Unique

tar_files_into(→ pathlib.Path)

Create a tar file given a desired output path and list of files to tar.

verify_tarball(→ bool)

Verify that a tarball was created properly by examining its

Module Contents

flint.archive.cli() → None[source]
flint.archive.copy_files_into(copy_out_path: pathlib.Path, files_to_copy: Collection[pathlib.Path]) → pathlib.Path[source]

Copy a set of specified files into an output directory, If a file happens to be a folder then it will be copied over.

Parameters:

  • copy_out_path (Path) – Path to copy files into

  • files_to_copy (Collection[Path]) – Files that shall be copied

Returns:

The path files were copied into

Return type:

Path

flint.archive.copy_sbid_files_archive(copy_out_path: pathlib.Path, base_path: pathlib.Path, archive_options: flint.options.ArchiveOptions) → pathlib.Path[source]

Copy files from an SBID processing folder into a final location. Uses the copy_file_globs set of expressions to identify files to copy.

Parameters:

  • copy_out_path (Path) – The output location of the tarball to write

  • base_path (Path) – The base directory that contains files to archive

  • archive_options (ArchiveOptions) – Options relating to how files are found and archived

Returns:

Output tarball directory

Return type:

Path

flint.archive.create_sbid_tar_archive(tar_out_path: pathlib.Path, base_path: pathlib.Path, archive_options: flint.options.ArchiveOptions) → pathlib.Path[source]

Create a tar file of key products in a SBID folder.

Parameters:

  • tar_out_path (Path) – The output location of the tarball to write

  • base_path (Path) – The base directory that contains files to archive

  • archive_options (ArchiveOptions) – Options relating to how files are found and archived

Returns:

Output tarball directory

Return type:

Path

flint.archive.get_archive_options_from_yaml(strategy_yaml_path: pathlib.Path) → dict[str, Any][source]

Load the archive options from a specified strategy file

Parameters:

strategy_yaml_path (Path) – The path to the strategy yaml file containing archive options

Returns:

Loaded options for ArchiveOptions

Return type:

Dict[str, Any]

flint.archive.get_parser() → argparse.ArgumentParser[source]
flint.archive.resolve_glob_expressions(base_path: pathlib.Path, file_re_patterns: Collection[str]) → tuple[pathlib.Path, Ellipsis][source]

Collect a set of files given a base directory and a set of glob expressions. Unique paths are returned.

Parameters:

  • base_path (Path) – The base folder with files to consider

  • file_re_patterns (Collection[str]) – An iterable with a set of regular-expression patterns to evaluate

Returns:

Unique collection of paths

Return type:

Tuple[Path,…]

flint.archive.tar_files_into(tar_out_path: pathlib.Path, files_to_tar: Collection[pathlib.Path], verify: bool = True) → pathlib.Path[source]

Create a tar file given a desired output path and list of files to tar.

Parameters:

  • tar_out_path (Path) – The output path of the tarball. The parent directory will be created if necessary.

  • files_to_tar (Collection[Path]) – All the files to tarball up

  • verify (bool, optional) – Verify that the tarball was correctly formed. Defaults to True.

Raises:

FileExistsError – The path of the tarball created

Returns:

There exists a tarball of the same name

Return type:

Path

flint.archive.verify_tarball(tarball: pathlib.Path) → bool[source]

Verify that a tarball was created properly by examining its table. Internally this calls tar through a subprocess call. Hence, tar needs to be available on the system PATH.

Parameters:

tarball (Path) – The tarball to examine

Returns:

True if the tar exit code is 0, False otherwise

Return type:

bool