flint.options

Contains the core of the option class containers that are used to hold stateful properties throughout the flint codebase.

Attributes

DEFAULT_COPY_RE_PATTERNS
DEFAULT_TAR_RE_PATTERNS
T
U

Classes

AddModelSubtractFieldOptions	Options related to predicting a continuum model during the SubtractFieldOptions workflow.
ArchiveOptions	Container for options related to archiving products from flint workflows
BandpassOptions	Container that represents the flint related options that
BaseOptions	A base class that Options style flint classes can
FieldOptions	Container that represents the flint related options that
FitsCubeOptions	Container of opptions used to combine images into a single cube using the fitscube package.
MS	Helper to keep track of measurement set information
PolFieldOptions	Container that represents the flint related options that
SubtractFieldOptions	Container for options related to the

Functions

_create_argparse_options(→ tuple[str, dict[str, Any]])	Convert a pydantic Field into dict to splate into ArgumentParser.add_argument()
add_options_to_parser(→ argparse.ArgumentParser)	Given an established argument parser and a class derived
create_options_from_parser(→ U)	Given a BaseOptions derived class, extract the corresponding
dump_field_options_to_yaml(→ pathlib.Path)	Dump the supplied instance of FieldOptions to a yaml file
options_to_dict(→ dict)	Helper function to convert an Options type class to a dictionary.

Module Contents

class flint.options.AddModelSubtractFieldOptions(/, **data: Any)

Bases: BaseOptions

Options related to predicting a continuum model during the SubtractFieldOptions workflow. Specifically these options deal with identifying the wsclean produced source list model, which may be used by admodel to predict model visibilities. See utilities around the aocalibrate functions and routines.

addmodel_cluster_config: pathlib.Path | None = None

Specify a new cluster configuration file different to the preferred on. If None, drawn from preferred cluster config

calibrate_container: pathlib.Path | None = None

Path to the container with the calibrate software (including addmodel)

wsclean_pol_mode: list[str] = ['i']

The polarisation of the wsclean model that was generated

class flint.options.ArchiveOptions(/, **data: Any)

Bases: BaseOptions

Container for options related to archiving products from flint workflows

copy_file_re_patterns: tuple[str, Ellipsis] = ('.*linmos.*fits', '.*weight\\.fits', '.*png', '.*csv')

Regular-expressions used to identify files to copy into a final location (not tarred)

tar_file_re_patterns: tuple[str, Ellipsis] = ('.*MFS.*image\\.fits', '.*linmos.*', '.*weight\\.fits', '.*yaml', '.*\\.txt', '.*png',...

Regular-expressions to use to collect files that should be tarballed

class flint.options.BandpassOptions(/, **data: Any)

Bases: BaseOptions

Container that represents the flint related options that might be used throughout the processing of bandpass calibration data.

In its present form this BandpassOptions class is not intended to contain properties of the data that arebeing processed, rather how these data will be processed.

These settings are not meant to be adjustabled throughout a single bandpass pipeline run

calibrate_container: pathlib.Path | None = None

Path to the singularity calibrate container

expected_ms: int = 36

The expected number of measurement set files to find

flag_calibrate_rounds: int = 3

The number of times the bandpass will be calibrated, flagged, then recalibrated

flagger_container: pathlib.Path | None = None

Path to the singularity aoflagger container

minuv: float | None = None

The minimum baseline length, in meters, for data to be included in bandpass calibration stage

preflagger_ant_mean_tolerance: float = 0.2

Tolerance that the mean x/y antenna gain ratio test before the antenna is flagged

preflagger_jones_max_amplitude: float | None = None

Flag Jones matrix if any amplitudes with a Jones are above this value

preflagger_mesh_ant_flags: bool = False

Share channel flags from bandpass solutions between all antenna

smooth_polynomial_order: int = 4

The polynomial order used by the Savgol filter when smoothing the bandpass solutions

smooth_solutions: bool = False

Will activate the smoothing of the bandpass solutions

smooth_window_size: int = 16

The width of the smoothing window used to smooth the bandpass solutions

class flint.options.BaseOptions(/, **data: Any)

Bases: pydantic.BaseModel

A base class that Options style flint classes can inherit from. This is derived from pydantic.BaseModel, and can be used for validation of supplied values.

Class derived from BaseOptions are immutable by default, and have the docstrings of attributes extracted.

_asdict() → dict[str, Any]

with_options(/, **kwargs) → T

model_config

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

class flint.options.FieldOptions(/, **data: Any)

Bases: BaseOptions

Container that represents the flint related options that might be used throughout components related to the actual pipeline.

In its present form this FieldOptions class is not intended to contain properties of the data that are being processed, rather how those data will be processed.

These settings are not meant to be adjustable throughout rounds of self-calibration.

aegean_container: pathlib.Path | None = None

Path to the singularity aegean container

beam_cutoff: float = 150

Cutoff in arcseconds to use when calculating the common beam to convol to

calibrate_container: pathlib.Path | None = None

Path to the singularity calibrate container

casa_container: pathlib.Path | None = None

Path to the singularity CASA container

coadd_cubes: bool = False

Co-add cubes formed throughout imaging together. Cubes will be smoothed channel-wise to a common resolution. Only performed on final set of images

expected_ms: int = 36

The expected number of measurement set files to find

fixed_beam_shape: tuple[float, float, float] | None = None

Specify the final beamsize of linmos field images in (arcsec, arcsec, deg)

flagger_container: pathlib.Path | None = None

Path to the singularity aoflagger container

holofile: pathlib.Path | None = None

Path to the holography FITS cube that will be used when co-adding beams

imaging_strategy: pathlib.Path | None = None

Path to a FLINT imaging yaml file that contains settings to use throughout imaging

linmos_residuals: bool = False

Linmos the cleaning residuals together into a field image

no_imaging: bool = False

Whether to skip the imaging process (including self-calibration)

pb_cutoff: float = 0.1

Primary beam attenuation cutoff to use during linmos

potato_container: pathlib.Path | None = None

Path to the singularity potato peel container

reference_catalogue_directory: pathlib.Path | None = None

Path to the directory container the reference catalogues, used to generate validation plots

rename_ms: bool = False

Rename MSs throughout rounds of imaging and self-cal instead of creating copies. This will delete data-columns throughout.

rounds: int = 2

Number of required rouds of self-calibration and imaging to perform

run_aegean: bool = False

Whether to run the aegean source finding tool

sbid_archive_path: pathlib.Path | None = None

Path that SBID archive tarballs will be created under. If None no archive tarballs are created. See ArchiveOptions.

sbid_copy_path: pathlib.Path | None = None

Path that final processed products will be copied into. If None no copying of file products is performed. See ArchiveOptions.

skip_selfcal_on_rounds: list[int] | None = None

Do not perform the derive and apply self-calibration solutions on these rounds

stokes_v_imaging: bool = False

Specifies whether Stokes-V imaging will be carried out after the final round of imagine (whether or not self-calibration is enabled).

update_model_data_with_source_list: bool = False

Attempt to update a MSs MODEL_DATA column with a source list (e.g. source list output from wsclean)

use_beam_masks: bool = False

Construct beam masks from MFS images to use for the next round of imaging.

use_beam_masks_from: int = 1

If use_beam_masks is True, this sets the round where beam masks will be generated from

use_beam_masks_rounds: list[int] | None = None

If use_beam_masks is True, this sets which rounds should have a mask applied

use_jolly_tukey_tractor: bool = False

Use the jolly roger tukey tractor. See the TukeyTractorOptions and the jolly-roger package for more details.

use_preflagger: bool = False

Whether to apply (or search for solutions with) bandpass solutions that have gone through the preflagging operations

use_smoothed: bool = False

Whether to apply (or search for solutions with) a bandpass smoothing operation applied

wsclean_container: pathlib.Path | None = None

Path to the singularity wsclean container

yandasoft_container: pathlib.Path | None = None

Path to the singularity yandasoft container

zip_ms: bool = False

Whether to zip measurement sets once they are no longer required

class flint.options.FitsCubeOptions(/, **data: Any)

Bases: BaseOptions

Container of opptions used to combine images into a single cube using the fitscube package. This is particularly useful to manage the larger concatenations.

bounding_box: bool = False

Whether to attempt to trim images when combining

invalidate_zeros: bool = True

Set pixels whose values are exactly 0.0 to not-a-number (nan)

max_workers: int = 4

The number of concurrent workers (readers/writers) that are permitted at a time

class flint.options.MS

Bases: NamedTuple

Helper to keep track of measurement set information

This is the class that should be used when describing a measurement set that will be operated on.

classmethod cast(ms: MS | pathlib.Path) → MS

Create/return a MS instance given either a Path or MS.

If the input is neither a MS instance or Path, the object will be checked to see if it has a .ms attribute. If it does then this will be used.

Parameters:

ms (Union[MS, Path]) – The input type to consider

Raises:

MSError – Raised when the input ms can not be cast to an MS instance

Returns:

A normalised MS

Return type:

MS

with_options(**kwargs) → MS

Create a new MS instance with keywords updated

Returns:

New MS instance with updated attributes

Return type:

MS

beam: int | None = None

The beam ID of the MS within an ASKAP field

column: str | None = None

Column that should be operated against

field: str | None = None

The field name of the data

model_column: str | None = None

The column name of the most recently MODEL data

property ms: MS

path: pathlib.Path

Path to the measurement set that is being represented

spw: int | None = None

Intended to be used with ASKAP high-frequency resolution modes, where the MS is divided into SPWs

class flint.options.PolFieldOptions(/, **data: Any)

Bases: BaseOptions

Container that represents the flint related options that might be used throughout components related to the actual pipeline.

In its present form this PolFieldOptions class is not intended to contain properties of the data that are being processed, rather how those data will be processed.

These settings are not meant to be adjustable across different polarisations.

beam_cutoff: float = 150

Cutoff in arcseconds to use when calculating the common beam to convol to

casa_container: pathlib.Path | None = None

Path to the singularity CASA container

expected_ms: int = 36

The expected number of measurement set files to find

fixed_beam_shape: tuple[float, float, float] | None = None

Specify the final beamsize of linmos field images in (arcsec, arcsec, deg)

holofile: pathlib.Path | None = None

Path to the holography FITS cube that will be used when co-adding beams

imaging_strategy: pathlib.Path | None = None

Path to a FLINT imaging yaml file that contains settings to use throughout imaging

pb_cutoff: float = 0.1

Primary beam attenuation cutoff to use during linmos

sbid_copy_path: pathlib.Path | None = None

Path that final processed products will be copied into. If None no copying of file products is performed. See ArchiveOptions.

trim_linmos_fits: bool = False

Trim the linmos fits files to remove the padding that is added. If True, the output fits files will be smaller but might be different shapes

wsclean_container: pathlib.Path | None = None

Path to the singularity wsclean container

yandasoft_container: pathlib.Path | None = None

Path to the singularity yandasoft container

class flint.options.SubtractFieldOptions(/, **data: Any)

Bases: BaseOptions

Container for options related to the continuum-subtracted pipeline

attempt_subtract: bool = False

Attempt to subtract the model column from the nominated data column

beam_cutoff: float = 150

Cutoff in arcseconds to use when calculating the common beam to convol to

channelwise_image: bool = False

Perform channel-wise imaing of the residuals

data_column: str = 'CORRECTED_DATA'

Describe the column that should be imaed and, if requested, have model subtracted from

expected_ms: int = 36

The number of measurement sets that should exist

fitscube_remove_original_images: bool = False

Remove the images that go into forming the fitscube

holofile: pathlib.Path | None = None

Path to the holography FITS cube that will be used when co-adding beams

imaging_strategy: pathlib.Path | None = None

Path to a FLINT imaging yaml file that contains settings to use throughout imaging

linmos_residuals: bool = False

Linmos the cleaning residuals together into a field image

max_intervals: int = 500

The maximum number of scans/channels to consider

pb_cutoff: float = 0.1

Primary beam attenuation cutoff to use during linmos

predict_wsclean_model: bool = False

Search for the continuum model produced by wsclean and subtract

stagger_delay_seconds: float | None = None

The delay, in seconds, that should be used when submitting items in batches (e.g. looping over channels)

subtract_data_column: str = 'DATA'

Should the continuum model be subtracted, where to store the output. This will update the column to be imaged.

subtract_model_data: bool = False

Subtract the MODEL_DATA column from the nominated data column

subtract_only: bool = False

Only perform the continuum subtraction

timestep_image: bool = False

Perform timestep imaging after subtraction

use_addmodel: bool = False

Invoke the addmodel visibility prediction, including the search for the wsclean source list

use_crystalball: bool = False

Attempt to predict the model visibilities using crystalball

wsclean_container: pathlib.Path

Path to the container with wsclean

yandasoft_container: pathlib.Path

Path to the container with yandasoft

flint.options._create_argparse_options(name: str, field: pydantic.fields.FieldInfo) → tuple[str, dict[str, Any]]

Convert a pydantic Field into dict to splate into ArgumentParser.add_argument()

flint.options.add_options_to_parser(parser: argparse.ArgumentParser, options_class: type[BaseOptions], description: str | None = None) → argparse.ArgumentParser

Given an established argument parser and a class derived from a pydantic.BaseModel, populate the argument parser with the model properties.

Parameters:

• parser (ArgumentParser) – Parser that arguments will be added to

• options_class (type[BaseModel]) – A Options style class derived from BaseOptions

Returns:

Updated argument parser

Return type:

ArgumentParser

flint.options.create_options_from_parser(parser_namespace: argparse.Namespace, options_class: type[U]) → U

Given a BaseOptions derived class, extract the corresponding arguments from an argparse.nNamespace. These options correspond to ones generated by add_options_to_parser.

Parameters:

• parser_namespace (Namespace) – The argument parser corresponding to those in the BaseOptions class

• options_class (U) – A BaseOptions derived class

Returns:

An populated options class with arguments drawn from CLI argument parser

Return type:

U

flint.options.dump_field_options_to_yaml(output_path: pathlib.Path, field_options: FieldOptions | PolFieldOptions | SubtractFieldOptions, overwrite: bool = False) → pathlib.Path

Dump the supplied instance of FieldOptions to a yaml file for record keeping.

The parent directory of the output_path will be created if it does not already exist.

Parameters:

• output_path (Path) – Path of the output file.

• field_options (FieldOptions) – The FieldOptions class to write.

• overwrite (bool, optional) – Overwrite the file if it exists. Defaults to False.

Raises:

FileExistsError – Raise if output_path already exists and overwrite is False

Returns:

Output path written to.

Return type:

Path

flint.options.options_to_dict(input_options: Any) → dict

Helper function to convert an Options type class to a dictionary.

Most of flint Option and Result classes used typing.NamedTuples, which carry with it a _asdict method to convert them to a dictionary. Future roadmap plans to move over to pydantic type models. This is a place holder function to help transition to this.

Parameters:

input_options (Any) – Item to convert to a dictionary

Raises:

TypeError – Raised if the conversion to a dictionary was not successful

Returns:

The dictionary version of the input options

Return type:

Dict

flint.options.DEFAULT_COPY_RE_PATTERNS = ('.*linmos.*fits', '.*weight\\.fits', '.*png', '.*csv')

flint.options.DEFAULT_TAR_RE_PATTERNS = ('.*MFS.*image\\.fits', '.*linmos.*', '.*weight\\.fits', '.*yaml', '.*\\.txt', '.*png',...

flint.options.T

flint.options.U