flint.prefect.common.imaging¶

This contains common utilities to enable components of the prefect imaging flowws. The majority of the items here are the task decorated functions. Effort should be made to avoid putting in too many items that are not going to be directly used by prefect imaging flows.

Attributes¶

`FlagMS`
`P`
`R`
`task_bane_fits_image`
`task_combine_images_to_cube`
`task_convolve_image_set`
`task_convolve_images`
`task_convolve_images_or_blank`
`task_convolve_plane_to_beam`
`task_copy_and_preprocess_casda_askap_ms`
`task_create_apply_solutions_cmd`
`task_create_name_from_common_fields`
`task_dp3_applycal`
`task_dp3_ddecal`
`task_dp3_flag`
`task_dp3_gaincal`
`task_dp3_solve_apply`
`task_get_common_beam_from_image_set`
`task_get_common_beam_from_image_sets`
`task_get_common_beam_from_images`
`task_get_common_beam_from_results`
`task_get_common_bounding_box`
`task_image_set_from_result`
`task_merge_image_sets`
`task_merge_image_sets_from_results`
`task_preprocess_askap_ms`
`task_remove_files_folders`
`task_rename_column_in_ms`
`task_select_solution_for_ms`
`task_split_and_get_image_set`
`task_split_by_field`
`task_split_cube_into_planes`
`task_transpose_and_sort_channel_images`

Classes¶

`ConvolvedLinmosCubes`	The futures `create_convolve_linmos_cubes` hands back
`LinmosCubes`	The cubes `linmos_channel_groups_to_cubes` builds from one Stokes

Functions¶

`_resolution_suffix`(→ str)	The linmos suffix with the resolution of the product appended
`convolve_channel_groups_to_natural_resolution`(...)	Convolve per-channel beam images to one common beam per channel, which is
`convolve_image_set`(→ list[pathlib.Path])	Convolve images to a specified resolution
`convolve_mfs_beam_images_to_common_resolution`(...)	Convolve the per-beam MFS products to a single common beam.
`convolve_then_linmos`(→ flint.coadd.linmos.LinmosResult)	An internal function that launches the convolution to a common resolution
`create_convol_linmos_images`(...)	Derive the appropriate set of beam shapes and then produce corresponding
`create_convolve_linmos_cubes`(→ ConvolvedLinmosCubes)	Co-add the beam cubes channel by channel, each channel convolved to a common resolution first.
`get_common_beam_from_image_set`(→ flint.convol.BeamShape)	Compute a common beam size that all input images will be convoled to.
`get_common_beam_from_image_sets`(→ flint.convol.BeamShape)	Compute a common beam size that all input images will be convoled to.
`get_common_beam_from_images`(→ flint.convol.BeamShape)	Compute a common beam size that all input images will be convoled to.
`get_common_beam_from_results`(→ flint.convol.BeamShape)	Compute a common beam size that all input images will be convoled to.
`linmos_channel_groups_to_cubes`(→ LinmosCubes)	Co-add beam images one channel at a time, in parallel, then stack the
`task_bandpass_create_apply_solutions_cmd`(...)	Apply a ao-calibrate solutions file to the subject measurement set.
`task_common_beam_convolve_linmos`(...)
`task_convolve_image`(...)	Convolve images to a specified resolution
`task_convolve_linmos_to_fixed_shape`(...)	Smooth the linmos input image to a fixed resolution
`task_create_image_mask_model`(...)	Make a clean mask for an image with eye-patch, returning the names of the
`task_create_validation_plot`(→ pathlib.Path)	Create a multi-panel figure highlighting the RMS, flux scale and astrometry of a field
`task_create_validation_tables`(...)	Create a set of validation tables that can be used to assess the
`task_extract_solution_path`(→ pathlib.Path)	Extract the solution path from a calibrate command. This is often required when
`task_flag_ms_aoflagger`(→ FlagMS)
`task_flatten_mask`(→ eye_patch.naming.FITSMaskNames)	Flatten a per-scale mask into a plain clean mask, returning any other mask as is
`task_gaincal_applycal_ms`(→ flint.ms.MS)	Perform self-calibration using CASA gaincal and applycal.
`task_get_channel_images_from_paths`(→ list[pathlib.Path])
`task_get_mfs_image_from_paths`(→ pathlib.Path)	Get the single MFS image from a list of paths.
`task_linmos_images`(→ flint.coadd.linmos.LinmosResult)	Linmos together a set of input images.
`task_potato_peel`(→ flint.ms.MS)
`task_run_bane_and_aegean`(...)	Run BANE and Aegean against a FITS image.
`task_split_beam_cube_into_planes`(→ list[pathlib.Path])	Split a per-beam wsclean image cube into its per-channel planes, ready
`task_wsclean_imager`(→ flint.imager.wsclean.WSCleanResult)	Run the wsclean imager against an input measurement set
`task_zip_ms`(→ pathlib.Path)	Zip a measurement set
`total_beam_from_channel_groups`(...)	Solve the one beam that covers every channel of every Stokes, which is the
`validation_items`(→ None \| tuple[pathlib.Path, ...)	Construct the validation plot and validation table items for the imaged field.

Module Contents¶

class flint.prefect.common.imaging.ConvolvedLinmosCubes[source]¶

Bases: NamedTuple

The futures create_convolve_linmos_cubes hands back

cleanup: list[prefect.futures.PrefectFuture[list[pathlib.Path]]][source]¶: Removal of the planes the cubes were made from. These still have to be waited on, as dask forgets a task once nothing holds its future

cubes: list[prefect.futures.PrefectFuture[pathlib.Path]][source]¶: The co-added cubes

class flint.prefect.common.imaging.LinmosCubes[source]¶

Bases: NamedTuple

The cubes linmos_channel_groups_to_cubes builds from one Stokes

bkg: prefect.futures.PrefectFuture[pathlib.Path] | None = None[source]¶: Background cube, None unless BANE was asked for

property futures: list[prefect.futures.PrefectFuture[pathlib.Path]][source]¶: Every cube actually being made, for callers that only need to wait

image: prefect.futures.PrefectFuture[pathlib.Path][source]¶: The co-added image cube

rms: prefect.futures.PrefectFuture[pathlib.Path] | None = None[source]¶: RMS noise cube, None unless BANE was asked for

total_bkg: prefect.futures.PrefectFuture[pathlib.Path] | None = None[source]¶: Background cube measured on the total-resolution planes

total_image: prefect.futures.PrefectFuture[pathlib.Path] | None = None[source]¶: The image cube at one beam covering the band, None unless asked for

total_rms: prefect.futures.PrefectFuture[pathlib.Path] | None = None[source]¶: RMS noise cube measured on the total-resolution planes

weight: prefect.futures.PrefectFuture[pathlib.Path][source]¶: The linmos weight cube

flint.prefect.common.imaging._resolution_suffix(suffix_str: str | None, mode: flint.naming.ResolutionModes) → str[source]¶: The linmos suffix with the resolution of the product appended

flint.prefect.common.imaging.convolve_channel_groups_to_natural_resolution(stokes_channel_groups: dict[str, list[list[pathlib.Path]]], cutoff: float | None = None, fixed_beam_shape: collections.abc.Sequence[float] | None = None) → dict[str, list[list[pathlib.Path]]][source]¶

Convolve per-channel beam images to one common beam per channel, which is the ‘natural’ resolution mode of racs_tools: resolution follows frequency rather than every channel being dragged out to the coarsest in the band. Contrast the ‘total’ mode of convolve_cubes_to_common_resolution, which RM-synthesis needs of its own inputs.

The beam of a channel is solved over every beam image of every Stokes at that channel, so channel N of I, Q and U share one resolution and a per-channel polarisation product is meaningful. Solving per Stokes would leave them at differing resolutions in the same channel.

Parameters:

stokes_channel_groups (dict[str, list[list[Path]]]) – For each Stokes, the beam images of each channel
cutoff (float | None, optional) – Images whose major axis exceeds this, in arcsec, are blanked rather than convolved to. Defaults to no cutoff.
fixed_beam_shape (Sequence[float] | None, optional) – Convolve every channel to this (arcsec, arcsec, deg) rather than a solved beam. Defaults to solving each channel.

Returns:

The convolved beam images of each channel, for each Stokes

Return type:

dict[str, list[list[Path]]]

flint.prefect.common.imaging.convolve_image_set(image_set: flint.imageset.ImageSet, beam_shape: flint.convol.BeamShape, cutoff: float = 60, mode: str = 'image', filter_str: str | None = None, convol_suffix_str: str = 'conv', remove_original_images: bool = False) → list[pathlib.Path][source]¶

Convolve images to a specified resolution

Parameters:

wsclean_result (WSCleanResult) – Collection of output images from wsclean that will be convolved
beam_shape (BeamShape) – The shape images will be convolved to
cutoff (float, optional) – Maximum major beam axis an image is allowed to have before it will not be convolved. Defaults to 60.
filter_str (Optional[str], optional) – This string must be contained in the image path for it to be convolved. Defaults to None.
convol_suffix_str (str, optional) – The suffix added to the convolved images. Defaults to ‘conv’.
remove_original_images (bool, optional) – If True remove the original image after they have been convolved. Defaults to False.

Returns:

Path to the output images that have been convolved.

Return type:

Collection[Path]

flint.prefect.common.imaging.convolve_mfs_beam_images_to_common_resolution(mfs_beam_images: dict[str, dict[str, list[pathlib.Path]]], cutoff: float | None = None, fixed_beam_shape: collections.abc.Sequence[float] | None = None) → dict[str, dict[str, list[pathlib.Path]]][source]¶

Convolve the per-beam MFS products to a single common beam.

An MFS image has no frequency axis, so there is nothing for a natural, per-channel beam to vary over: one beam is solved over the MFS images of every beam and every Stokes. The beam is taken from the image products alone and applied to the model and residual products too, which is what a beam solved over ImageSet.image has always done here.

Parameters:

mfs_beam_images (dict[str, dict[str, list[Path]]]) – For each Stokes, the per-beam MFS image of each product type
cutoff (float | None, optional) – Images whose major axis exceeds this, in arcsec, are blanked rather than convolved to. Defaults to no cutoff.
fixed_beam_shape (Sequence[float] | None, optional) – Convolve to this (arcsec, arcsec, deg) rather than a solved beam. Defaults to solving it.

Returns:

The convolved MFS products, keyed as the input

Return type:

dict[str, dict[str, list[Path]]]

flint.prefect.common.imaging.convolve_then_linmos(wsclean_results: collections.abc.Collection[flint.imager.wsclean.WSCleanResult], beam_shape: flint.convol.BeamShape, field_options: flint.options.FieldOptions, linmos_suffix_str: str | None, field_summary: flint.summary.FieldSummary | None = None, convol_mode: str = 'image', convol_filter: str = '.MFS.', convol_suffix_str: str = 'conv', trim_linmos_fits: bool = True, remove_original_images: bool = False, cleanup_linmos: bool = False, holofile: pathlib.Path | None = None) → flint.coadd.linmos.LinmosResult[source]¶

An internal function that launches the convolution to a common resolution and subsequent linmos of the wsclean residual images.

Parameters:

wsclean_results (Collection[WSCleanResult]) – Collection of wsclean imaging results, with residual images described in the attached ImageSet
beam_shape (BeamShape) – The beam shape that residual images will be convolved to
field_options (FieldOptions) – Options related to the processing of the field
linmos_suffix_str (str) – The suffix string passed to the linmos parset name
field_summary (Optional[FieldSummary], optional) – The summary of the field, including (importantly) to orientation of the third-axis. Defaults to None.
convol_mode (str, optional) – The mode passed to the convol task to describe the images to extract. Support image or residual. Defaults to image.
convol_filter (str, optional) – A text file applied when assessing images to co-add. Defaults to ‘.MFS.’.
convol_suffix_str (str, optional) – The suffix added to the convolved images. Defaults to ‘conv’.
trim_linmos_fits (bool, optional) – Attempt to trim the output linmos files of as much empty space as possible. Defaults to True.
remove_original_images (bool, optional) – If True remove the original image after they have been convolved. Defaults to False.
cleanup_linmos (bool, optional) – Clean up items created throughout linmos, including the per-channel weight text files for each input image. Defaults to False.
holofile (Path | None, optional) – Path of a holography file. If provided will override the one presented by field_options. Defaults to None.

Returns:

Resulting linmos command parset

Return type:

LinmosResult

flint.prefect.common.imaging.create_convol_linmos_images(wsclean_results: collections.abc.Collection[flint.imager.wsclean.WSCleanResult], field_options: flint.options.FieldOptions, field_summary: flint.summary.FieldSummary | None = None, additional_linmos_suffix_str: str | None = None, holofile: pathlib.Path | None = None) → list[flint.coadd.linmos.LinmosResult][source]¶

Derive the appropriate set of beam shapes and then produce corresponding convolved and co-added images

Parameters:

wsclean_results (Collection[WSCleanResult]) – Set of wsclean commands that have been executed
field_options (FieldOptions) – Set of field imaging options, containing details of the beam/s
field_summary (Optional[FieldSummary], optional) – Summary of the MSs, importantly containing their third-axis rotation. Defaults to None.
additional_linmos_suffix_str (Optional[str], optional) – An additional string added to the end of the auto-generated linmos base name. Defaults to None.
holofile (Optional[Path], optional) – The path to the holofile. If provided will override one presented by field_options. Defaults to None.

Returns:

The collection of linmos commands executed.

Return type:

List[LinmosResult]

flint.prefect.common.imaging.create_convolve_linmos_cubes(wsclean_results: collections.abc.Collection[flint.imager.wsclean.WSCleanResult], field_options: flint.options.FieldOptions, fitscube_options: flint.options.FitsCubeOptions, current_round: int | None = None, additional_linmos_suffix_str: str | None = 'cube', holofile: pathlib.Path | None = None) → ConvolvedLinmosCubes[source]¶: Co-add the beam cubes channel by channel, each channel convolved to a common resolution first.

flint.prefect.common.imaging.get_common_beam_from_image_set(image_set: flint.imageset.ImageSet, cutoff: float = 25, filter_str: str | None = None, fixed_beam_shape: list[float] | None = None) → flint.convol.BeamShape[source]¶

Compute a common beam size that all input images will be convoled to.

Parameters:

wsclean_results (Collection[WSCleanResult]) – Input images whose restoring beam properties will be considered
cutoff (float, optional) – Major axis larger than this valur, in arcseconds, will be ignored. Defaults to 25.
filter_str (Optional[str], optional) – Only include images when considering beam shape if this string is in the file path. Defaults to None.
fixed_beam_shape (Optional[List[float]], optional) – Specify the final beamsize of linmos field images in (arcsec, arcsec, deg). If None it is deduced from images. Defaults to None;

Returns:

The final convolving beam size to be used

Return type:

BeamShape

flint.prefect.common.imaging.get_common_beam_from_image_sets(image_sets: list[flint.imageset.ImageSet], cutoff: float = 25, filter_str: str | None = None, fixed_beam_shape: list[float] | None = None) → flint.convol.BeamShape[source]¶

Compute a common beam size that all input images will be convoled to.

Parameters:

wsclean_results (Collection[WSCleanResult]) – Input images whose restoring beam properties will be considered
cutoff (float, optional) – Major axis larger than this valur, in arcseconds, will be ignored. Defaults to 25.
filter_str (Optional[str], optional) – Only include images when considering beam shape if this string is in the file path. Defaults to None.
fixed_beam_shape (Optional[List[float]], optional) – Specify the final beamsize of linmos field images in (arcsec, arcsec, deg). If None it is deduced from images. Defaults to None;

Returns:

The final convolving beam size to be used

Return type:

BeamShape

flint.prefect.common.imaging.get_common_beam_from_images(image_paths: list[pathlib.Path], cutoff: float = 25, filter_str: str | None = None, fixed_beam_shape: list[float] | None = None) → flint.convol.BeamShape[source]¶

Compute a common beam size that all input images will be convoled to.

Parameters:

image_paths (list[Path]) – Input images whose restoring beam properties will be considered
cutoff (float, optional) – Major axis larger than this valur, in arcseconds, will be ignored. Defaults to 25.
filter_str (Optional[str], optional) – Only include images when considering beam shape if this string is in the file path. Defaults to None.
fixed_beam_shape (Optional[List[float]], optional) – Specify the final beamsize of linmos field images in (arcsec, arcsec, deg). If None it is deduced from images. Defaults to None;

Returns:

The final convolving beam size to be used

Return type:

BeamShape

flint.prefect.common.imaging.get_common_beam_from_results(wsclean_results: list[flint.imager.wsclean.WSCleanResult], cutoff: float = 25, filter_str: str | None = None, fixed_beam_shape: list[float] | None = None) → flint.convol.BeamShape[source]¶

Compute a common beam size that all input images will be convoled to.

Parameters:

wsclean_results (Collection[WSCleanResult]) – Input images whose restoring beam properties will be considered
cutoff (float, optional) – Major axis larger than this valur, in arcseconds, will be ignored. Defaults to 25.
filter_str (Optional[str], optional) – Only include images when considering beam shape if this string is in the file path. Defaults to None.
fixed_beam_shape (Optional[List[float]], optional) – Specify the final beamsize of linmos field images in (arcsec, arcsec, deg). If None it is deduced from images. Defaults to None;

Returns:

The final convolving beam size to be used

Return type:

BeamShape

flint.prefect.common.imaging.linmos_channel_groups_to_cubes(channel_groups: collections.abc.Collection[collections.abc.Collection[pathlib.Path]], container: pathlib.Path, linmos_options: flint.coadd.linmos.LinmosOptions, fitscube_options: flint.options.FitsCubeOptions, stokesi_channel_groups: collections.abc.Collection[collections.abc.Collection[pathlib.Path]] | None = None, field_summary: flint.summary.FieldSummary | None = None, suffix_str: str | None = None, holofile: pathlib.Path | None = None, fft_bane_options: flint.bane.FFTBANEOptions | None = None, total_beam_shape: prefect.futures.PrefectFuture[flint.convol.BeamShape] | None = None, beam_cutoff: float | None = None) → LinmosCubes[source]¶

Co-add beam images one channel at a time, in parallel, then stack the resulting mosaics back into image and weight cubes.

Should fitscube_options.bounding_box be True, the common bounding box of valid pixels is computed once every channel has been co-added and is passed into the cube assembly so fitscube trims both the image and weight cubes to the same shared pixel grid while writing them.

Parameters:

channel_groups (Collection[Collection[Path]]) – For each channel, the beam images to co-add
container (Path) – Path to a yandasoft singularity container
linmos_options (LinmosOptions) – Options passed to each per-channel linmos
fitscube_options (FitsCubeOptions, optional) – Options for controlling the FITS cube creation. Defaults to None.
stokesi_channel_groups (Collection[Collection[Path]] | None, optional) – For each channel, the Stokes I beam images used for the leakage correction. Defaults to None.
field_summary (FieldSummary | None, optional) – Description of the field, used to get the pol_axis. Defaults to None.
suffix_str (str | None, optional) – Additional suffix added to the linmos and cube names. Defaults to None.
holofile (Path | None, optional) – Holography file overriding the one in linmos_options. Defaults to None.
fft_bane_options (FFTBANEOptions | None, optional) – When given, each co-added channel also gets a BANE background and RMS map, stacked into their own cubes. Defaults to None, i.e. no BANE.
total_beam_shape (PrefectFuture[BeamShape] | None, optional) – When given, each co-added plane is also convolved to this one beam and stacked into a second set of cubes. See total_beam_from_channel_groups. Defaults to None, i.e. the natural-resolution cubes alone.
beam_cutoff (float | None, optional) – Planes coarser than this, in arcsec, are blanked rather than convolved to total_beam_shape. Defaults to no cutoff.

Returns:

The cubes being created

Return type:

LinmosCubes

flint.prefect.common.imaging.task_bandpass_create_apply_solutions_cmd(ms: flint.ms.MS, calibrate_cmd: flint.calibrate.aocalibrate.CalibrateCommand, container: pathlib.Path) → flint.calibrate.aocalibrate.ApplySolutions[source]¶

Apply a ao-calibrate solutions file to the subject measurement set.

Parameters:

ms (MS) – The measurement set that will have the solutions file applied
calibrate_cmd (CalibrateCommand) – A resulting ao-calibrate command and meta-data item
container (Path) – The container that can apply the ao-calibrate style solutions file to the measurement set

Returns:

The resulting apply solutions command and meta-data

Return type:

ApplySolutions

flint.prefect.common.imaging.task_common_beam_convolve_linmos(wsclean_results: list[flint.imager.wsclean.WSCleanResult], field_options: flint.options.SubtractFieldOptions, convol_mode: str, convol_filter: str | None = None, convol_suffix_str: str | None = None, remove_original_images: bool = False, trim_linmos_fits: bool = False, cleanup_linmos: bool = False, linmos_suffix_str: str | None = None, field_summary: flint.summary.FieldSummary | None = None) → flint.coadd.linmos.LinmosResult[source]¶

flint.prefect.common.imaging.task_convolve_image(wsclean_result: flint.imager.wsclean.WSCleanResult, beam_shape: flint.convol.BeamShape, cutoff: float = 60, mode: str = 'image', filter_str: str | None = None, convol_suffix_str: str = 'conv', remove_original_images: bool = False) → collections.abc.Collection[pathlib.Path][source]¶

Convolve images to a specified resolution

Parameters:

wsclean_result (WSCleanResult) – Collection of output images from wsclean that will be convolved
beam_shape (BeamShape) – The shape images will be convolved to
cutoff (float, optional) – Maximum major beam axis an image is allowed to have before it will not be convolved. Defaults to 60.
filter_str (Optional[str], optional) – This string must be contained in the image path for it to be convolved. Defaults to None.
convol_suffix_str (str, optional) – The suffix added to the convolved images. Defaults to ‘conv’.
remove_original_images (bool, optional) – If True remove the original image after they have been convolved. Defaults to False.

Returns:

Path to the output images that have been convolved.

Return type:

Collection[Path]

flint.prefect.common.imaging.task_convolve_linmos_to_fixed_shape(linmos_result: flint.coadd.linmos.LinmosResult, field_options: flint.options.FieldOptions) → flint.coadd.linmos.LinmosResult[source]¶

Smooth the linmos input image to a fixed resolution

Parameters:

linmos_result (LinmosResult) – An existing linmos results set representing a created image
field_options (FieldOptions) – A field options instance with an set fixed beeam shape set

Returns:

A smoothed version of the input linmos image

Return type:

LinmosResult

flint.prefect.common.imaging.task_create_image_mask_model(image: flint.coadd.linmos.LinmosResult | flint.imageset.ImageSet | flint.imager.wsclean.WSCleanResult, image_products: flint.source_finding.aegean.AegeanOutputs | None = None, update_masking_options: dict[str, Any] | None = None) → eye_patch.naming.FITSMaskNames[source]¶: Make a clean mask for an image with eye-patch, returning the names of the mask files it wrote. RMS and background images are needed unless use_mac or convolve_first is set.

flint.prefect.common.imaging.task_create_validation_plot(field_summary: flint.summary.FieldSummary, aegean_outputs: flint.source_finding.aegean.AegeanOutputs, reference_catalogue_directory: pathlib.Path, upload_artifact: bool = True) → pathlib.Path[source]¶

Create a multi-panel figure highlighting the RMS, flux scale and astrometry of a field

Parameters:

aegean_outputs (AegeanOutputs) – Output aegean products
reference_catalogue_directory (Path) – Directory containing NVSS, SUMSS and ICRS reference catalogues. These catalogues are reconginised internally and have expected names.
upload_artifact (bool, optional) – If True the validation plot will be uploaded to the prefect service as an artifact. Defaults to True.

Returns:

Path to the output figure created

Return type:

Path

flint.prefect.common.imaging.task_create_validation_tables(field_summary: flint.summary.FieldSummary, aegean_outputs: flint.source_finding.aegean.AegeanOutputs, reference_catalogue_directory: pathlib.Path, upload_artifacts: bool = True) → flint.validation.ValidationTables[source]¶

Create a set of validation tables that can be used to assess the correctness of an image and associated source catalogue.

Parameters:

processed_ms_paths (List[Path]) – The processed MS files that were used to create the source catalogue
rms_image_path (Path) – The RMS fits image the source catalogue was constructed against.
source_catalogue_path (Path) – The source catalogue.
output_path (Path) – The output path of the figure to create
reference_catalogue_directory (Path) – The directory that contains the reference catalogues installed

Returns:

The tables that were created

Return type:

ValidationTables

flint.prefect.common.imaging.task_extract_solution_path(calibrate_cmd: flint.calibrate.aocalibrate.CalibrateCommand) → pathlib.Path[source]¶

Extract the solution path from a calibrate command. This is often required when interacting with a PrefectFuture wrapped CalibrateCommand result.

Parameters:: calibrate_cmd (CalibrateCommand) – The subject calibrate command the solution file will be extracted from
Returns:: Path to the solution file
Return type:: Path

flint.prefect.common.imaging.task_flag_ms_aoflagger(ms: FlagMS, container: pathlib.Path | None) → FlagMS[source]¶

flint.prefect.common.imaging.task_flatten_mask(fits_mask: eye_patch.naming.FITSMaskNames) → eye_patch.naming.FITSMaskNames[source]¶: Flatten a per-scale mask into a plain clean mask, returning any other mask as is

flint.prefect.common.imaging.task_gaincal_applycal_ms(ms: flint.ms.MS | flint.imager.wsclean.WSCleanResult, selfcal_round: int, casa_container: pathlib.Path, update_gain_cal_options: dict[str, Any] | None = None, archive_input_ms: bool = False, skip_selfcal: bool = False, rename_ms: bool = False, archive_cal_table: bool = False) → flint.ms.MS[source]¶

Perform self-calibration using CASA gaincal and applycal.

Parameters:

ms (Union[MS, WSCleanResult]) – A resulting wsclean output. This is used purely to extract the .ms attribute.
selfcal_round (int) – Counter indication which self-calibration round is being performed. A name is included based on this.
casa_container (Path) – A path to a singularity container with CASA tooling.
update_gain_cal_options (Optional[Dict[str, Any]], optional) – Options used to overwrite the default gaincal options. Defaults to None.
archive_input_ms (bool, optional) – If True the input measurement set is zipped. Defaults to False.
skip_selfcal (bool, optional) – Should this self-cal be skipped. If True, the a new MS is created but not calibrated the appropriate new name and returned.
rename_ms (bool, optional) – It True simply rename a MS and adjust columns appropriately (potentially deleting them) instead of copying the complete MS. If True archive_input_ms is ignored. Defaults to False.
archive_cal_table (bool, optional) – Archive the output calibration table in a tarball. Defaults to False.

Raises:

ValueError – Raised when a .ms attribute can not be obtained

Returns:

Self-calibrated measurement set

Return type:

flint.prefect.common.imaging.task_get_channel_images_from_paths(paths: list[pathlib.Path]) → list[pathlib.Path][source]¶

flint.prefect.common.imaging.task_get_mfs_image_from_paths(paths: list[pathlib.Path]) → pathlib.Path[source]¶

Get the single MFS image from a list of paths.

Inverse of task_get_channel_images_from_paths

flint.prefect.common.imaging.task_linmos_images(image_list: list[pathlib.Path], container: pathlib.Path, linmos_options: flint.coadd.linmos.LinmosOptions, field_summary: flint.summary.FieldSummary | None = None, suffix_str: str | None = None, parset_output_path: str | None = None, holofile: pathlib.Path | None = None) → flint.coadd.linmos.LinmosResult[source]¶

Linmos together a set of input images.

The linmos_options.image_output_name is updated to be a name based on the commonn unique fields of image_list.

Parameters:

image_list (list[Path]) – Collection of images to coadd
container (Path) – Path to a yandasoft singularity container
linmos_options (LinmosOptions) – Collection of linmos options
field_summary (FieldSummary | None, optional) – Description of the field, used to get the pol_axis of the field. Defaults to None.
suffix_str (str | None, optional) – Additional suffix str to add when generating the output file names. Defaults to None.
parset_output_path (str | None, optional) – The output parameter set that will be generated. Defaults to None.
holofile (Path | None, optional) – Path to a holofile that will overwrite the one specified in linmos options. Defaults to None.

Returns:

Collection of linmos items generated

Return type:

LinmosResult

flint.prefect.common.imaging.task_potato_peel(ms: flint.ms.MS, potato_container: pathlib.Path, update_potato_config_options: dict[str, Any] | None = None, update_potato_peel_options: dict[str, Any] | None = None, update_wsclean_options: dict[str, Any] | None = None) → flint.ms.MS[source]¶

flint.prefect.common.imaging.task_run_bane_and_aegean(image: flint.imager.wsclean.WSCleanResult | flint.coadd.linmos.LinmosResult, aegean_container: pathlib.Path, update_bane_options: dict[str, Any] | None = None, update_aegean_options: dict[str, Any] | None = None) → flint.source_finding.aegean.AegeanOutputs[source]¶

Run BANE and Aegean against a FITS image.

Notes

It has been noted that BANE can sometimes get caught in a interpolation error which haults execution. The timelimit_seconds will attempt to detect long runnings BANE processes and raise an error. The retry functionality of prefect should then restart the task. Since this task is pure (e.g. no last dataproducts, modification to data etc) simply restarting should be fine.

Parameters:

image (Union[WSCleanResult, LinmosResult]) – The image that will be searched
aegean_container (Path) – Path to a singularity container containing BANE and aegean
update_bane_options (dict[str, Any] | None, optional) – Over-ride any default options of BANEOptions. If None defaults are used. Defaults to None.
update_aegean_options (dict[str, Any] | None, optional) – Over-ride any default options of AegeanOptions. If None defaults are used. Defaults to None.

Raises:

ValueError – Raised when image is not a supported type

Returns:

Output BANE and aegean products, including the RMS and BKG images

Return type:

AegeanOutputs

flint.prefect.common.imaging.task_split_beam_cube_into_planes(wsclean_result: flint.imager.wsclean.WSCleanResult) → list[pathlib.Path][source]¶

Split a per-beam wsclean image cube into its per-channel planes, ready to be convolved individually rather than as a whole cube.

Parameters:: wsclean_result (WSCleanResult) – Output of wsclean whose image cube will be split
Returns:: The per-channel planes extracted from the image cube
Return type:: list[Path]

flint.prefect.common.imaging.task_wsclean_imager(in_ms: flint.calibrate.aocalibrate.ApplySolutions | flint.ms.MS | tuple[flint.ms.MS, ...], wsclean_container: pathlib.Path, update_wsclean_options: dict[str, Any] | None = None, update_fitscube_options: dict[str, Any] | None = None, fits_mask: eye_patch.naming.FITSMaskNames | None = None, channel_range: tuple[int, int] | None = None, scan_range: tuple[int, int] | None = None, make_cube_from_subbands: bool = True, extra_output_types: tuple[str, ...] | None = None) → flint.imager.wsclean.WSCleanResult[source]¶

Run the wsclean imager against an input measurement set

The channel_range and scan_range options over-write the update_wsclean_options, as some flow programmatically create these.

Parameters:

in_ms (Union[ApplySolutions, MS, tuple[MS, ...]]) – The measurement set that will be imaged
wsclean_container (Path) – Path to a singularity container with wsclean packages
update_wsclean_options (Optional[Dict[str, Any]], optional) – Options to update from the default wsclean options. Defaults to None.
fits_mask (Optional[FITSMaskNames], optional) – A path to a clean guard mask. Defaults to None.
channel_range (Optional[Tuple[int,int]], optional) – Add to the wsclean options the specific channel range to be imaged. Defaults to None.
scan_range (Optional[Tuple[int,int]], optional) – Add to the wsclean options the specific scan range to be imaged. Defaults to None.
extra_output_types (tuple[str, ...] | None, optional) – Additional output types (beyond image/residual) to search for and attach to the returned ImageSet. Defaults to None.

Returns:

A resulting wsclean command and resulting meta-data

Return type:

WSCleanResult

flint.prefect.common.imaging.task_zip_ms(in_item: flint.imager.wsclean.WSCleanResult) → pathlib.Path[source]¶

Zip a measurement set

Parameters:: in_item (WSCleanResult) – The inpute item with a .ms attribute of type MS.
Returns:: Output path of the zipped measurement set
Return type:: Path

flint.prefect.common.imaging.total_beam_from_channel_groups(stokes_channel_groups: dict[str, list[list[pathlib.Path]]], cutoff: float | None = None, fixed_beam_shape: collections.abc.Sequence[float] | None = None) → prefect.futures.PrefectFuture[flint.convol.BeamShape][source]¶

Solve the one beam that covers every channel of every Stokes, which is the ‘total’ resolution mode of racs_tools and what RM-synthesis needs of its inputs.

Solved over a single image per channel. convolve_channel_groups_to_natural_resolution has already brought every beam and every Stokes of a channel to one resolution, so the rest of a channel’s images carry nothing the first does not.

Parameters:

stokes_channel_groups (dict[str, list[list[Path]]]) – For each Stokes, the beam images of each channel, already at their natural resolution
cutoff (float | None, optional) – Images whose major axis exceeds this, in arcsec, are left out of the solve rather than dragging the beam out to them. Defaults to no cutoff.
fixed_beam_shape (Sequence[float] | None, optional) – Use this (arcsec, arcsec, deg) rather than solving one. Defaults to solving.

Returns:

The beam every channel is brought to

Return type:

PrefectFuture[BeamShape]

flint.prefect.common.imaging.validation_items(field_summary: flint.summary.FieldSummary, aegean_outputs: flint.source_finding.aegean.AegeanOutputs, reference_catalogue_directory: pathlib.Path | None = None) → None | tuple[pathlib.Path, pathlib.Path][source]¶

Construct the validation plot and validation table items for the imaged field.

Internally these are submitting the prefect task versions of: - task_create_validation_plot - task_create_validation_tables

If reference_catalogue_directory does not exist or unspecified than this function returns None. Otherwise a Path objects to output data products are returned.

Parameters:

field_summary (FieldSummary) – Container representing the SBID being imaged and its populated characteristics
aegean_outputs (AegeanOutputs) – Source finding results
reference_catalogue_directory (Path | None, optional) – Location of directory containing the reference known NVSS, SUMSS and ICRS catalogues. Defaults to None.
Returns
tuple[Path (None |)
exist (Path] -- None is returned if the reference_catalogue_directory does not)
returned. (other the paths to output data products are)

flint.prefect.common.imaging.FlagMS[source]¶

flint.prefect.common.imaging.P[source]¶

flint.prefect.common.imaging.R[source]¶

flint.prefect.common.imaging.task_bane_fits_image[source]¶

flint.prefect.common.imaging.task_combine_images_to_cube[source]¶

flint.prefect.common.imaging.task_convolve_image_set: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_convolve_images[source]¶

flint.prefect.common.imaging.task_convolve_images_or_blank[source]¶

flint.prefect.common.imaging.task_convolve_plane_to_beam[source]¶

flint.prefect.common.imaging.task_copy_and_preprocess_casda_askap_ms: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_create_apply_solutions_cmd: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_create_name_from_common_fields[source]¶

flint.prefect.common.imaging.task_dp3_applycal[source]¶

flint.prefect.common.imaging.task_dp3_ddecal[source]¶

flint.prefect.common.imaging.task_dp3_flag[source]¶

flint.prefect.common.imaging.task_dp3_gaincal[source]¶

flint.prefect.common.imaging.task_dp3_solve_apply[source]¶

flint.prefect.common.imaging.task_get_common_beam_from_image_set: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_get_common_beam_from_image_sets: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_get_common_beam_from_images: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_get_common_beam_from_results: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_get_common_bounding_box[source]¶

flint.prefect.common.imaging.task_image_set_from_result[source]¶

flint.prefect.common.imaging.task_merge_image_sets[source]¶

flint.prefect.common.imaging.task_merge_image_sets_from_results[source]¶

flint.prefect.common.imaging.task_preprocess_askap_ms: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_remove_files_folders[source]¶

flint.prefect.common.imaging.task_rename_column_in_ms: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_select_solution_for_ms: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_split_and_get_image_set[source]¶

flint.prefect.common.imaging.task_split_by_field: prefect.Task[P, R][source]¶

flint.prefect.common.imaging.task_split_cube_into_planes[source]¶

flint.prefect.common.imaging.task_transpose_and_sort_channel_images[source]¶