flint.validation ================ .. py:module:: flint.validation .. autoapi-nested-parse:: Utility function to create a one figure multi panel validation plot for continuum imaging of RACS data Attributes ---------- .. autoapisummary:: flint.validation.F_HUGE flint.validation.F_LARGE flint.validation.F_MED flint.validation.F_SMALL flint.validation.MAX_SOURCES_TO_PLOT Classes ------- .. autoapisummary:: flint.validation.MatchResult flint.validation.PSFTableRow flint.validation.RMSImageInfo flint.validation.SourceCounts flint.validation.Tables flint.validation.ValidationCatalogues flint.validation.ValidationTables flint.validation.ValidatorLayout flint.validation.XMatchTables Functions --------- .. autoapisummary:: flint.validation._make_beam_psf_row flint.validation.calculate_area_correction_per_flux flint.validation.cli flint.validation.create_validation_plot flint.validation.create_validation_tables flint.validation.extract_inner_image_array_region flint.validation.get_known_catalogue_info flint.validation.get_parser flint.validation.get_rms_image_info flint.validation.get_source_counts flint.validation.load_catalogues flint.validation.load_known_catalogue flint.validation.make_field_stats_table flint.validation.make_psf_table flint.validation.make_validator_axes_layout flint.validation.make_xmatch_table flint.validation.match_nearest_neighbour flint.validation.plot_astrometry_comparison flint.validation.plot_field_info flint.validation.plot_flag_summary flint.validation.plot_flux_comparison flint.validation.plot_psf flint.validation.plot_rms_map flint.validation.plot_source_counts flint.validation.scale_flux_alpha Module Contents --------------- .. py:class:: MatchResult Bases: :py:obj:`NamedTuple` Simple container to hold results of a quick cross match .. py:attribute:: flux1 :type: numpy.ndarray | None :value: None Brightness in Jy of source in the first survey .. py:attribute:: flux2 :type: numpy.ndarray | None :value: None Brightness in Jy of source in the second survey .. py:attribute:: freq1 :type: float Frequency in Hz of the first survey .. py:attribute:: freq2 :type: float Frequency in Hz of the second survey .. py:attribute:: idx1 :type: numpy.ndarray The indices of the matched sources from the original input table 1 .. py:attribute:: idx2 :type: numpy.ndarray The indices of the matched sources from the original input table 2 .. py:attribute:: name1 :type: str Name of the first survey .. py:attribute:: name2 :type: str Name of the second survey .. py:attribute:: pos1 :type: astropy.coordinates.SkyCoord Sky-positions of sources in the first survey .. py:attribute:: pos2 :type: astropy.coordinates.SkyCoord Sky-positions of sources in the second survey .. py:class:: PSFTableRow Bases: :py:obj:`NamedTuple` Something that is only used to internally represent the PSF information of a beam image. Currently no reason to use this. .. py:attribute:: b :type: float .. py:attribute:: beam :type: int .. py:attribute:: bmaj :type: float .. py:attribute:: bmin :type: float .. py:attribute:: bpa :type: float .. py:attribute:: dec :type: float .. py:attribute:: image_name :type: str .. py:attribute:: l :type: float .. py:attribute:: ra :type: float .. py:attribute:: vis_flagged :type: int .. py:attribute:: vis_total :type: int .. py:class:: RMSImageInfo Bases: :py:obj:`NamedTuple` Class to hold basic RMS Image information, excluding the actual raw image data .. py:attribute:: area :type: float The area of valid sky, in degrees squared .. py:attribute:: centre :type: astropy.coordinates.SkyCoord The centre of the image .. py:attribute:: header :type: astropy.io.fits.Header Header from the FITS image .. py:attribute:: maximum :type: float The maximum value of the image .. py:attribute:: median :type: float The median value of the image .. py:attribute:: minimum :type: float The minimum value of the image .. py:attribute:: mode :type: float The mode of the image .. py:attribute:: no_valid_pixels :type: int Number of valid pixels in the image .. py:attribute:: path :type: pathlib.Path Path to the RMS fits image .. py:attribute:: shape :type: tuple[int, int] Dimension of the image .. py:attribute:: std :type: float The standard deviation of the image .. py:attribute:: wcs :type: astropy.wcs.WCS The WCS of the image .. py:class:: SourceCounts Bases: :py:obj:`NamedTuple` A small container to pass around source count information .. py:attribute:: area :type: float The area in square degrees that the sources cover, i.e. image footprint sky-area .. py:attribute:: area_fraction :type: numpy.ndarray | None :value: None The fraction of the image that was above a sigma level per flux bin. This may be used as a rough term to scale the Euclidean Normalised source counts. This is not intended to be a robust way of correcting the source counts - just quick .. py:attribute:: bin_center :type: numpy.ndarray The bin centers to use in Jy .. py:attribute:: bins :type: numpy.ndarray The bin edges of the source counts in Jy .. py:attribute:: counts_per_bin :type: numpy.ndarray The counts of sources per flux bin .. py:attribute:: counts_per_bin_err :type: numpy.ndarray The rough estimate on the errors per bin .. py:attribute:: euclid_counts :type: numpy.ndarray Euclidean normalised source counts .. py:attribute:: euclid_counts_err :type: numpy.ndarray Rough estimate of error on the euclidean normalised source counts .. py:class:: Tables Bases: :py:obj:`NamedTuple` Container for all the tables that are loaded in .. py:attribute:: askap :type: astropy.table.Table ASKAP catalogue .. py:attribute:: icrf :type: astropy.table.Table ICRF catalogue .. py:attribute:: nvss :type: astropy.table.Table NVSS catalogue .. py:attribute:: racs_high :type: astropy.table.Table | None :value: None RACS high catalogue .. py:attribute:: racs_mid :type: astropy.table.Table | None :value: None RACS mid catalogue .. py:attribute:: sumss :type: astropy.table.Table SUMSS catalogue .. py:attribute:: tgss :type: astropy.table.Table | None :value: None TGSS catalogue .. py:attribute:: vlass :type: astropy.table.Table | None :value: None VLASS catalogue .. py:class:: ValidationCatalogues Bases: :py:obj:`NamedTuple` Container for all the catalogues that are loaded in and used throughout validation processing .. py:attribute:: askap :type: flint.catalogue.Catalogue ASKAP catalogue .. py:attribute:: icrf :type: flint.catalogue.Catalogue ICRF catalogue .. py:attribute:: nvss :type: flint.catalogue.Catalogue NVSS catalogue .. py:attribute:: racs_high :type: flint.catalogue.Catalogue | None :value: None RACS high catalogue .. py:attribute:: racs_mid :type: flint.catalogue.Catalogue | None :value: None RACS mid catalogue .. py:attribute:: sumss :type: flint.catalogue.Catalogue SUMSS catalogue .. py:attribute:: tgss :type: flint.catalogue.Catalogue | None :value: None TGSS catalogue .. py:attribute:: vlass :type: flint.catalogue.Catalogue | None :value: None VLASS catalogue .. py:class:: ValidationTables Bases: :py:obj:`NamedTuple` Container for all the tables that are generated by the validation routine .. py:attribute:: psf_table_path :type: pathlib.Path Path to the PSF table .. py:attribute:: stats_table_path :type: pathlib.Path Path to the statistics table .. py:attribute:: xmatch_tables :type: XMatchTables Cross-matched tables .. py:class:: ValidatorLayout Bases: :py:obj:`NamedTuple` Simple container for all the matplotlib axes objects .. py:attribute:: ax_astrometry :type: matplotlib.axes.Axes Axes to compare astrometry of sources .. py:attribute:: ax_astrometry1 :type: matplotlib.axes.Axes Axes to compare astrometry of sources from the first catalogue .. py:attribute:: ax_astrometry2 :type: matplotlib.axes.Axes Axes to compare astromnetry of sources from the first catalogue .. py:attribute:: ax_brightness1 :type: matplotlib.axes.Axes Axes to compare brightness of sources from the first catalogue .. py:attribute:: ax_brightness2 :type: matplotlib.axes.Axes Axes to compare brightness of sources from the first catalogue .. py:attribute:: ax_counts :type: matplotlib.axes.Axes Axes for quick look source counts .. py:attribute:: ax_flag_summary :type: matplotlib.axes.Axes Axes to summarise the flagging of the MS .. py:attribute:: ax_legend :type: matplotlib.axes.Axes Container for basic SBID information .. py:attribute:: ax_psf :type: matplotlib.axes.Axes Axes for the PSF of the image .. py:attribute:: ax_rms :type: matplotlib.axes.Axes Axes for the RMS of the field .. py:class:: XMatchTables Bases: :py:obj:`NamedTuple` Container for all the cross matched tables .. py:attribute:: icrf :type: pathlib.Path ICRF catalogue .. py:attribute:: nvss :type: pathlib.Path NVSS catalogue .. py:attribute:: racs_high :type: pathlib.Path | None :value: None RACS high catalogue .. py:attribute:: racs_mid :type: pathlib.Path | None :value: None RACS mid catalogue .. py:attribute:: sumss :type: pathlib.Path SUMSS catalogue .. py:attribute:: tgss :type: pathlib.Path | None :value: None TGSS catalogue .. py:attribute:: vlass :type: pathlib.Path | None :value: None VLASS catalogue .. py:function:: _make_beam_psf_row(beam_summary: flint.summary.BeamSummary) -> PSFTableRow Collects the information required of a single beam image and measurement set for entry into the PSF table. Not intended for usage other than int he creation of the PSF table. :param beam_summary: The input set of collected properties :type beam_summary: BeamSummary :returns: Extracted information :rtype: PSFTableRow .. py:function:: calculate_area_correction_per_flux(rms_image_path: pathlib.Path, flux_bin_centre: numpy.ndarray, sigma: float = 5) -> numpy.ndarray This derives a rough correction to the area term when calculating the source counts. This is not intended to correct for completeness, although they are closely related. The RMS image is read in and a CDF is calculated. This is used to return a scaling term that indicates what fraction a flux-bin was `sigma` times the RMS, which could be used to scale the area. :param rms_image_path: RMS image that will be use to calculate the area correction :type rms_image_path: Path :param flux_bin_centre: Fluxes to calculate the area fraction at :type flux_bin_centre: np.ndarray :param sigma: Mutliplicate term indicating what sigma level source finding cropped at. Defaults to 5. :type sigma: float, optional :returns: Fraction of the image that was available for source finding at a flux density :rtype: np.ndarray .. py:function:: cli() -> None CLI entry point for validation plot creation .. py:function:: create_validation_plot(field_summary: flint.summary.FieldSummary, rms_image_path: pathlib.Path, source_catalogue_path: pathlib.Path, output_path: pathlib.Path, reference_catalogue_directory: pathlib.Path) -> pathlib.Path Create a simple multi-panel validation figure intended to asses the correctness of an image and associated source catalogue. The image described by `rms_image_path` should be a FITS file. The WCS of this file is used for plotting and rreading the synthesised beam information using the standard CRVAL/BMAJ/BMIN keywords. The source catalogue is read using astropy.table.Table. This routine also expects that some level of units are embedded in the catalogue. For Aegean produced catalogues this is the case. The reference_catalogue_directory sets the directory to look into when searching for the reference ICRF, NVSS and SUMSS cataloues. :param field_summary: A description of the key properties of the field :type field_summary: FieldSummary :param rms_image_path: The RMS fits image the source catalogue was constructed against. :type rms_image_path: Path :param source_catalogue_path: The source catalogue. :type source_catalogue_path: Path :param output_path: The output path of the figure to create :type output_path: Path :param reference_catalogue_directory: The directory that contains the reference ICRF, NVSS and SUMSS catalogues. :type reference_catalogue_directory: Path :returns: The output path of the figure :rtype: Path .. py:function:: create_validation_tables(field_summary: flint.summary.FieldSummary, rms_image_path: pathlib.Path, source_catalogue_path: pathlib.Path, output_path: pathlib.Path, reference_catalogue_directory: pathlib.Path) -> ValidationTables Create a set of validation tables that can be used to assess the correctness of an image and associated source catalogue. :param field_summary: A description of the key properties of the field :type field_summary: FieldSummary :param rms_image_path: The RMS fits image the source catalogue was constructed against. :type rms_image_path: Path :param source_catalogue_path: The source catalogue. :type source_catalogue_path: Path :param output_path: The output path of the figure to create :type output_path: Path :param reference_catalogue_directory: The directory that contains the reference catalogues installed :type reference_catalogue_directory: Path :returns: The tables that were created :rtype: ValidationTables .. py:function:: extract_inner_image_array_region(image: numpy.ndarray, fraction: float) -> numpy.ndarray Extract an inner region of an image array. The size of the extracted array is specified via the `fraction` parameter, which should be in the range of 0 to 1. :param image: The image to extract pixels from :type image: np.ndarray :param fraction: The size of the region to extraction. :type fraction: float :returns: The extracted sub-region :rtype: np.ndarray .. py:function:: get_known_catalogue_info(name: str) -> flint.catalogue.Catalogue Return the parameters of a recognised catalogue. These are currently hardcoded. The structure returneed outlines the name of columns of interest: - RA - Dec - Integrated flux - Major / Minor / PA :param name: The survey name of interest :type name: str :raises ValueError: Raised when an unrecongised catalogue is provided :returns: Information of the survey catalogue :rtype: Catalogue .. py:function:: get_parser() -> argparse.ArgumentParser Create the argument parser for the validation plot creation :returns: CLI entry point :rtype: ArgumentParser .. py:function:: get_rms_image_info(rms_path: pathlib.Path, extract_fraction: float = 0.5) -> RMSImageInfo Extract information about the RMS image and construct a representative structure When computing the pixel statistics an inner region of the RMS map is extracted to avoid the primary beam roll off. Other properties of the ``RMSImageInfo`` are computed on the whole image (e.g. area) :param rms_path: The RMS image that will be presented :type rms_path: Path :param extract_fraction: Extract the inner region of the base RMS image map. Defaults to 0.5. :type extract_fraction: float, optional :returns: Extracted RMS image information :rtype: RMSImageInfo .. py:function:: get_source_counts(fluxes: numpy.ndarray, area: float, minlogf: float = -4, maxlogf: float = 2, Nbins: int = 81, rms_image_path: pathlib.Path | None = None) -> SourceCounts Derive source counts for a set of fluxes and known area :param fluxes: The fluxes in Jy to count :type fluxes: np.ndarray :param area: Area over which the sources were collected :type area: float :param minlogf: The minimum bin edge, in Jy. Defaults to -4. :type minlogf: float, optional :param maxlogf: The maximum bin edgem, in Jy. Defaults to 2. :type maxlogf: float, optional :param Nbins: Number of bins to include in the source counts. Defaults to 81. :type Nbins: int, optional :param rms_image_path: Path to the RMS image. If not None, it is used to calculate a rough area correction. Defaults to None. :type rms_image_path: Optional[Path], optional :returns: Source counts and their properties :rtype: SourceCounts .. py:function:: load_catalogues(source_catalogue_path: pathlib.Path, reference_catalogue_directory: pathlib.Path, askap_survey_name: str, rms_info: RMSImageInfo) -> tuple[ValidationCatalogues, Tables] Load in all the catalogues that are required for the validation. :param source_catalogue_path: The source catalogue to load :type source_catalogue_path: Path :param reference_catalogue_directory: The directory that contains the reference ICRF, NVSS and SUMSS catalogues. :type reference_catalogue_directory: Path :param askap_survey_name: The name that will be given to the ASKAP field data :type askap_survey_name: str :param rms_info: The extracted information from the RMS image :type rms_info: RMSImageInfo :returns: The loaded catalogues and tables :rtype: Tuple[ValidationCatalogues, Tables] .. py:function:: load_known_catalogue(name: str, reference_catalogue_directory: pathlib.Path) -> tuple[astropy.table.Table, flint.catalogue.Catalogue] Load in a known catalogue table :param name: Name of the survey to load :type name: str :param reference_catalogue_directory: The directory location with the reference catalogues installed :type reference_catalogue_directory: Path :returns: The loaded table and Catalogue structure describing the columns :rtype: Tuple[Table,Catalogue] .. py:function:: make_field_stats_table(field_summary: flint.summary.FieldSummary, rms_info: RMSImageInfo, output_path: pathlib.Path) -> pathlib.Path .. py:function:: make_psf_table(field_summary: flint.summary.FieldSummary, output_path: pathlib.Path) -> pathlib.Path .. py:function:: make_validator_axes_layout(fig: matplotlib.figure.Figure, rms_path: pathlib.Path) -> ValidatorLayout Create the figure layout to use for the quick look validation plot. :param fig: The figure canvas to add the axes to :type fig: Figure :param rms_path: Path to the RMS image that will be presented. Loaded to access the WCS :type rms_path: Path :returns: Representation of all axes objects :rtype: ValidatorLayout .. py:function:: make_xmatch_table(table1: astropy.table.Table, table2: astropy.table.Table, catalogue1: flint.catalogue.Catalogue, catalogue2: flint.catalogue.Catalogue, match_result: MatchResult, output_path: pathlib.Path) -> tuple[astropy.table.Table, pathlib.Path] Create a simple cross match table between two catalogues :param table1: The catalogue table from survey one :type table1: Table :param table2: The catalogue table from survey two :type table2: Table :param catalogue1: Catalogue metadata for survey one :type catalogue1: Catalogue :param catalogue2: Catalogue metadata for survey two :type catalogue2: Catalogue :param match_result: A result set of the cross-match between two catalogues :type match_result: MatchResult :param output_path: Location to save the table to :type output_path: Path :returns: The table and the output path :rtype: Tuple[Table, Path] .. py:function:: match_nearest_neighbour(table1: astropy.table.Table, table2: astropy.table.Table, catalogue1: flint.catalogue.Catalogue, catalogue2: flint.catalogue.Catalogue, radius: float = 10) -> MatchResult Match two catalogues together, and construct common properties. :param table1: The catalogue table from survey one :type table1: Table :param table2: The catalogue table from survey two :type table2: Table :param catalogue1: Catalogue metadata for survey one :type catalogue1: Catalogue :param catalogue2: Catalogue metadata for survey two :type catalogue2: Catalogue :param radius: Maximum matching radius. Defaults to 10. :type radius: float, optional :returns: Object containing source matches and common properties :rtype: MatchResult .. py:function:: plot_astrometry_comparison(fig: matplotlib.figure.Figure, ax: matplotlib.axes.Axes, match_result: MatchResult) -> matplotlib.axes.Axes Plot the astrometry of cross matches from a match result set :param fig: The figure canvas plotting on :type fig: Figure :param ax: The Axes being plotted on :type ax: Axes :param match_result: The set of sources cross-matched and found in common :type match_result: MatchResult :returns: The Axes plotted on :rtype: Axes .. py:function:: plot_field_info(fig: matplotlib.figure.Figure, ax: matplotlib.axes.Axes, field_summary: flint.summary.FieldSummary, rms_info: RMSImageInfo, askap_table: astropy.table.Table) -> matplotlib.axes.Axes .. py:function:: plot_flag_summary(fig: matplotlib.figure.Figure, ax: matplotlib.axes.Axes, field_summary: flint.summary.FieldSummary) -> matplotlib.axes.Axes Plot the percentage of the spectrum that is flagged for each measurement set :param fig: The figure canvas that is being plotted to :type fig: Figure :param ax: The axes object that is being plotted to :type ax: Axes :param field_summary: An active field summary object with the collection of MSSummary structures :type field_summary: FieldSummary :returns: The axes object with the plotted RMS image :rtype: Axes .. py:function:: plot_flux_comparison(fig: matplotlib.figure.Figure, ax: matplotlib.axes.Axes, match_result: MatchResult) -> matplotlib.axes.Axes Create a flux comparison plot showing the flux densities from two catalogues compared to one another. :param fig: The figure canvas that the axes is on :type fig: Figure :param ax: The axes object that will be render the plot :type ax: Axes :param match_result: A result set of the cross-match between two catalogues :type match_result: MatchResult :returns: The aces object that was used for plotting :rtype: Axes .. py:function:: plot_psf(fig: matplotlib.figure.Figure, ax: matplotlib.axes.Axes, rms_info: RMSImageInfo) -> matplotlib.axes.Axes Create a plot highlighting the synthesised beam recorded in the RMS image header :param fig: Fogire canvas being used :type fig: Figure :param ax: The axes object that will be used for plotting :type ax: Axes :param rms_info: Extracted information from the RMS image :type rms_info: RMSImageInfo :returns: The aces object used for plotting :rtype: Axes .. py:function:: plot_rms_map(fig: matplotlib.figure.Figure, ax: matplotlib.axes.Axes, rms_path: pathlib.Path, source_positions: astropy.coordinates.SkyCoord | None = None) -> matplotlib.axes.Axes Add the RMS image to the figure :param fig: Figure that contains the axes object :type fig: Figure :param ax: The axes that will be plotted :type ax: Axes :param rms_path: Location of the RMS image :type rms_path: Path :param source_positions: Sources in the ASKAP catalogue to overlay onto the RMS image. Defaults to None. :type source_positions: Optional[SkyCoord], optionals :returns: The axes object with the plotted RMS image :rtype: Axes .. py:function:: plot_source_counts(catalogue: astropy.table.Table, rms_info: RMSImageInfo, ax: matplotlib.axes.Axes, freq: float | None = None, dezotti: astropy.table.Table | None = None, skads: astropy.table.Table | None = None) -> matplotlib.axes.Axes Create a figure of source counts from a astropy Table. If `freq` and either `dezotti` / `skads` are supplied then these precomputed source count tables are also included in the panel. When computing the source counts for `catalogue`, only a minimumal set of corrections are derived and applied. :param catalogue: The catalogue to derive source counts for :type catalogue: Table :param rms_info: Look up information from the RMS file that catalogue was constructed against :type rms_info: RMSImageInfo :param ax: The axes panel the counts will be plottedd on :type ax: Axes :param freq: Frequency that the source catalogue. Used to scale the Dezotti and SKADS tables. Defaults to None. :type freq: Optional[float], optional :param dezotti: Loaded reference table of Dezotti source counts. Defaults to None. :type dezotti: Optional[Table], optional :param skads: Loaded reference table of SKADS source counts. Defaults to None. :type skads: Optional[Table], optional :returns: The axes object used for plotting :rtype: Axes .. py:function:: scale_flux_alpha(flux: float | numpy.ndarray, freq: float | numpy.ndarray, ref_freq: float, alpha: float = -0.8) -> float | numpy.ndarray Scale a flux density to a reference frequency using a spectral index :param flux: The flux density to scale :type flux: float :param freq: The frequency of the flux density :type freq: float :param ref_freq: The reference frequency to scale to :type ref_freq: float :param alpha: The spectral index to use. Defaults to -0.8. :type alpha: float, optional :returns: The scaled flux density :rtype: float .. py:data:: F_HUGE :value: 20 .. py:data:: F_LARGE :value: 12 .. py:data:: F_MED :value: 8 .. py:data:: F_SMALL :value: 7 .. py:data:: MAX_SOURCES_TO_PLOT :value: 10000