flint.peel.potato ================= .. py:module:: flint.peel.potato .. autoapi-nested-parse:: Utilities the connect to Stefan Duchesne potatopeel module, which is responsible for peeling sources in racs. The repository is available at: https://gitlab.com/Sunmish/potato/-/tree/main Potato stands for "Peel Out That Annoying Terrible Object". Although this is a python module, for the moment it is expected to be in a singularity container. There are several reasons for this, but the principal one is that the numba module used by potatopeel may be difficult to get working correctly alongside dask and flint. Keeping it simple at this point is the main aim. There is also the problem of casatasks + python-casacore not jiving in newer python versions. Classes ------- .. autoapisummary:: flint.peel.potato.NormalisedSources flint.peel.potato.PotatoConfigCommand flint.peel.potato.PotatoConfigOptions flint.peel.potato.PotatoPeelArguments flint.peel.potato.PotatoPeelCommand flint.peel.potato.PotatoPeelOptions Functions --------- .. autoapisummary:: flint.peel.potato._potato_config_command flint.peel.potato._potato_options_to_command flint.peel.potato._potato_peel_command flint.peel.potato._prepare_potato_options flint.peel.potato._print_ms_colnames flint.peel.potato.cli flint.peel.potato.create_run_potato_config flint.peel.potato.create_run_potato_peel flint.peel.potato.find_sources_to_peel flint.peel.potato.get_parser flint.peel.potato.get_source_props_from_table flint.peel.potato.load_known_peel_sources flint.peel.potato.potato_peel flint.peel.potato.prepare_ms_for_potato flint.peel.potato.source_within_image_fov Module Contents --------------- .. py:class:: NormalisedSources Bases: :py:obj:`NamedTuple` Temporary container to hold the normalised source properties that would be provided to potato .. py:attribute:: source_decs :type: tuple[float] The Decs in degrees .. py:attribute:: source_fovs :type: tuple[float] The size of each source to image in degrees .. py:attribute:: source_names :type: tuple[str] The name of each source .. py:attribute:: source_ras :type: tuple[float] The RAs in degrees .. py:class:: PotatoConfigCommand Bases: :py:obj:`NamedTuple` Container for potato configuration command results .. py:attribute:: command :type: str The command string that should be executed .. py:attribute:: config_path :type: pathlib.Path Path to the configuration file generated .. py:class:: PotatoConfigOptions Bases: :py:obj:`NamedTuple` Container class to hold options that go into the potatopy configuration creation software by Stefan Duchesne. See: https://gitlab.com/Sunmish/potato .. py:attribute:: image_briggs :type: float :value: -1.5 Briggs robust parameter for the in-field image .. py:attribute:: image_channels :type: int :value: 4 Number of output channels for the in-field image .. py:attribute:: image_minuvl :type: float :value: 700 "Minimum (u,v)- distance in wavelengths for data to be selected .. py:attribute:: image_scale :type: float :value: 0.0006944 The pixel scale of the in-field image in degrees .. py:attribute:: image_size :type: int :value: 6148 Size of an in-field image .. py:attribute:: peel_channels :type: int :value: 16 Number of output channels for the peel images .. py:attribute:: peel_minuvl :type: float :value: 700 "Minimum (u,v)- distance in wavelengths for data to be selected for the peel image .. py:attribute:: peel_multiscale :type: bool :value: True Whether multi-scale is to be used for the peel sources .. py:attribute:: peel_nmiter :type: int :value: 7 Number of major iterations allowed for the peel sources .. py:attribute:: peel_scale :type: float :value: 0.0006944 Pixel scale of the peel images in degrees .. py:attribute:: peel_size :type: int :value: 1000 Size of the peel image to make, in pixels .. py:class:: PotatoPeelArguments Bases: :py:obj:`NamedTuple` The mandatory arguments for potato peel. .. py:attribute:: decs :type: Collection[float] The source Dec in degrees to peel .. py:attribute:: image_fov :type: float The field-of-view in degrees of the main in-field image. If a sources is within this radius it is not peeled (because it would be imaged) .. py:attribute:: ms :type: pathlib.Path The measurement set that will be examined for peeling .. py:attribute:: n :type: Collection[str] Name of the source being peeled .. py:attribute:: peel_fovs :type: Collection[float] The field-of-views that should be created for the peel source in degrees .. py:attribute:: ras :type: Collection[float] The source RA in degrees to peel .. py:class:: PotatoPeelCommand Bases: :py:obj:`NamedTuple` Container to hold the items of the hot potato command .. py:attribute:: command :type: str The hot potato command that will be executed .. py:attribute:: ms :type: flint.options.MS The measurement set that potato has operated against .. py:class:: PotatoPeelOptions Bases: :py:obj:`NamedTuple` Container class to hold options that go to the potato peel software by Stefan Duchesne. FLINT uses the `hot_potato` version. ee: https://gitlab.com/Sunmish/potato .. py:method:: with_options(**kwargs) -> PotatoPeelOptions .. py:attribute:: c :type: pathlib.Path | None :value: None Path to the potatopeel configuration file .. py:attribute:: calmode :type: str :value: 'P' Self-calibration mode to use (see casatasks gaincal) .. py:attribute:: direct_subtract :type: bool :value: True Whether a direct model subtraction (without self-cal) should be used ift he source is faint .. py:attribute:: intermediate_peels :type: bool :value: True Creates an image after each calibration and subtraction loop to show iterative improvements of the subject peel source .. py:attribute:: minpeelflux :type: float :value: 0.5 Minimum flux, in Jy, for the peeling procedure (image->selfcal->image) .. py:attribute:: minuvimage :type: float | None :value: None The minimum uv distance in wavelengths to use for imaging .. py:attribute:: minuvpeel :type: float | None :value: None The minimum uv distance in wavelengths to use when attempting to self-calibrate .. py:attribute:: refant :type: int :value: 1 Reference antenna to use when solving for self-cal solutions .. py:attribute:: solint :type: float :value: 30 Solution interval to use when applying gaincal .. py:attribute:: tmp :type: str | pathlib.Path | None :value: None Where the temporary wsclean files will be written to .. py:function:: _potato_config_command(config_path: pathlib.Path, potato_config_options: PotatoConfigOptions) -> PotatoConfigCommand Create the peel_configuration.py command that will be called in the potato singularity image. This is the CLI version of the code (not calling the python function). :param config_path: Output location of the configuration file :type config_path: Path :param potato_config_options: Instance of all the options to use :type potato_config_options: PotatoConfigOptions :returns: The CLI command that will be executed to create a potato configuration file :rtype: PotatoconfigCommand .. py:function:: _potato_options_to_command(potato_options: PotatoPeelArguments | PotatoConfigOptions | PotatoPeelOptions, skip_keys: Collection[str] | None = None, check_double_keys: bool = False) -> str Construct the CLI options that would be provided to a potato peel CLI program :param potato_options: An instance of one of the option classes to draw from :type potato_options: Union[PotatoPeelArguments,PotatoConfigOptions, PotatoPeelOptions] :param skip_keys: A collections of keys to ignore when build the CLI. If None all keys in the provided options instance are used. Defaults to None. :type skip_keys: Optional[Collection[str]], optional :param check_double_leys: Some long form names in `hot_potato` are single dash while others are double dash. This is not the case in the config creation tool. This will check to see if the double should be used. Defaults to False. :type check_double_leys: bool, optional :raises TypeError: When an unrecognised data type is found in the provided options class :returns: A string of the CLI options and keys :rtype: str .. py:function:: _potato_peel_command(ms: flint.options.MS, potato_peel_arguments: PotatoPeelArguments, potato_peel_options: PotatoPeelOptions) -> PotatoPeelCommand Construct the CLI command for `hot_potato`, and appropriately handle the mandatory and optional arguments. :param ms: The measurement set that will be peeled :type ms: MS :param potato_peel_arguments: The mandatory arguments for `hot_potato` :type potato_peel_arguments: PotatoPeelArguments :param potato_peel_options: The `hot_potato` options to supply :type potato_peel_options: PotatoPeelOptions :returns: The `hot_potato` command that was constructed from the input `PotatoPeelOptions` :rtype: PotatoPeelCommand .. py:function:: _prepare_potato_options(ms: flint.options.MS, potato_container: pathlib.Path, update_potato_config_options: dict[str, Any] | None = None, update_potato_peel_options: dict[str, Any] | None = None) -> tuple[PotatoConfigOptions | None, PotatoPeelOptions] Create the appropriate set of potato option class instances. This function will assess if a configuration file has been specified, and if not it will generate one from the `potato` configuration generator program. This is a call out to singularity. :param ms: The measurement set to be peeled :type ms: MS :param potato_container: Path to the container that holds `potato` :type potato_container: Path :param update_potato_config_options: Options to override defaults of the PotatoConfigOptions. Defaults to None. :type update_potato_config_options: dict[str, Any] :param update_potato_peel_options: Options to override defaults of the PotatoPeelOptions. Defaults to None. :type update_potato_peel_options: dict[str, Any] :returns: The populated set of classes. If a user provide configuration was provided there None is returned in place of PotatoConfigOptions :rtype: tuple[PotatoConfigOptions | None, PotatoPeelOptions] .. py:function:: _print_ms_colnames(ms: flint.options.MS) -> flint.options.MS A dummy function to print colnames in a MS table .. py:function:: cli() .. py:function:: create_run_potato_config(potato_container: pathlib.Path, ms_path: pathlib.Path | flint.options.MS, potato_config_options: PotatoConfigOptions) -> PotatoConfigCommand Construct and run a CLI command into the `peel_configuration.py` script of the `potatopeel` package. :param potato_container: Container with the `potatopeel` package installed :type potato_container: Path :param ms_path: Path to the measurement set that will be peeled :type ms_path: Union[Path, MS] :param potato_config_options: Options to tweak the values in the peel configuration :type potato_config_options: PotatoConfigOptions :returns: Container of the path to the peel configuration file and the corresponding command that generated it :rtype: PotatoConfigCommand .. py:function:: create_run_potato_peel(potato_container: pathlib.Path, ms: flint.options.MS, potato_peel_arguments: PotatoPeelArguments, potato_peel_options: PotatoPeelOptions) -> PotatoPeelCommand Construct and run a `hot_potato` command to peel out sources from a measurement set. :param potato_container: Container with the potato peel software and appropriate tools (including wsclean) :type potato_container: Path :param ms: The measurement set that contains sources to peel :type ms: MS :param potato_peel_arguments: The mandatory arguments that go into `hot_potato`. :type potato_peel_arguments: PotatoPeelArguments :param potato_peel_options: Options that are supplied to `hot_potato`. :type potato_peel_options: PotatoPeelOptions :returns: The executed `hot_potato` command :rtype: PotatoPeelCommand .. py:function:: find_sources_to_peel(ms: flint.options.MS, image_options: flint.imager.wsclean.WSCleanOptions, field_idx: int = 0, maximum_offset: float = 30, minimum_apparent_brightness: float = 0.5, override_beam_position_with: astropy.coordinates.SkyCoord | None = None) -> astropy.table.Table | None Obtain a set of sources to peel from a reference candidate set. This will evaluate whether a source should be peels based on two criteria: - if it is below a nominal primary beam cut off level (10 percent, assuming a gaussian primary beam) - if the sources is within some separation of the imaging center :param ms: The measurement set that is being considered :type ms: MS :param image_options: The imaging parameters that will be used to compute a placeholder WCS :type image_options: WSCleanOptions :param field_idx: Which field in the MS to draw the position from. Defaults to 0. :type field_idx: int, optional :param maximum_offset: The largest separation, in degrees, before a source is ignored. Defaults to 30.0. :type maximum_offset: float, optional :param minimum_apparent_brightness: The minimum apparent brightnessm, in Jy, a source should be before attempting to peel. Defaults to 0.5. :type minimum_apparent_brightness: float, optional :param override_beam_position_with: Ignore the beam position of the input MS, instead use this. Do not rely on this option as it may be taken away. Defaults to None. :type override_beam_position_with: Optional[SkyCoord], optional :returns: Collection of sources to peel from the reference table. Column names are Name, RA, Dec, Aperture. This is the package table. If no sources need to be peeled None is returned. :rtype: Union[Table,None] .. py:function:: get_parser() -> argparse.ArgumentParser .. py:function:: get_source_props_from_table(table: astropy.table.Table) -> NormalisedSources Given the astropy table of known sources to peel, normalise their inputs for the potato peel CLI :param table: Table of sources to peel :type table: Table :returns: Collection of normalised sources properties that will be provided to `hot_potato` :rtype: NormalisedSources .. py:function:: load_known_peel_sources() -> astropy.table.Table Locate and load the packaged set of known sources to peel. These sources are drawn from Duchesne et. al. (2023), Table 3. :returns: The astropy Table of candidate sources to remove. :rtype: Table .. py:function:: potato_peel(ms: flint.options.MS, potato_container: pathlib.Path, update_potato_config_options: dict[str, Any] | None = None, update_potato_peel_options: dict[str, Any] | None = None, image_options: flint.imager.wsclean.WSCleanOptions | None = None) -> flint.options.MS Peel out sources from a measurement set using PotatoPeel. Candidate sources from a known list of sources (see Table 3 or RACS-Mid paper) are considered. :param ms: The measurement set to peel out known sources :type ms: MS :param potato_container: Location of container with potatopeel software installed :type potato_container: Path :param update_potato_config_options: A dictionary with values to use to update the default options within the `PotatoConfigOptions`. If None use the defaults. Defaults to None. :type update_potato_config_options: Optional[Dict[str, Any]], optional :param update_potato_peel_options: A dictionary with values to use to update the default options within the `PotatoPeelOptions`. If None use the defaults. Defaults to None. :type update_potato_peel_options: Optional[Dict[str, Any]], optional :param image_options: Any imaging options that should be used to determine if sources require peeling (e.g. image size, pixel size) :type image_options: Optional[WSCleanOptions], optional :returns: Updated measurement set :rtype: MS .. py:function:: prepare_ms_for_potato(ms: flint.options.MS) -> flint.options.MS The potatopeel software requires the data column being operated against to be called DATA. This is a requirement of CASA and its gaincal / applysolution task. If there is already a DATA column and this is not the nominated data column described by the MS then it is removed, and the nominated data column is renamed. If the DATA column already exists and it is the nominated column described by MS then this is returned. If there is a CORRECTED_DATA column in this situation it is removed. :param ms: The measurement set that will be edited :type ms: MS :raises ValueError: Raised when a column name is expected but can not be found. :returns: measurement set with the data column moved into the correct location :rtype: MS .. py:function:: source_within_image_fov(source_coord: astropy.coordinates.SkyCoord, beam_coord: astropy.coordinates.SkyCoord, image_size: int, pixel_scale: astropy.units.Quantity | str) -> bool Evaluate whether a source will be within the field of view of an image. :param source_coord: The source position to consider :type source_coord: SkyCoord :param beam_coord: The center beam position to consider :type beam_coord: SkyCoord :param image_size: The image size. This assumes a square image. :type image_size: int :param pixel_scale: The pixel size, assuming square pixels. :type pixel_scale: Union[u.Quantity, str] :returns: Whether the source is expected to be in the field of view :rtype: bool