Bandpass calibration

The ASKAP Observatory uses the source PKS B1934-638 as its primary flux reference and bandpass calibrator. A bandpass observation consists of pointing each formed beam on the reference sources for ~2min of integration. A single bandpass SBID contains the data for all beams as each beam is pointed on-source. We note that it is ultimately up to the user to ensure that the bandpass obtained matches the science field in both time and beam-forming weights. We also note that raw bandpass and target data is not typically provided by the observatory to science users.

Using Flint for calibration

This is a prefect flow that will run the following stages:

  • Split the on-source portions of the bandpass data

  • Apply appropriate corrections to visibilities to for use with flint tooling

  • Flag RFI from the bandpass data

  • Derive bandpass solutions from PKS B1934-638 using the Reynolds 1994 model

  • Apply the solutions to the bandpass data, flag RFI, and rederive solutions (repeated FLAG_CALIBRATE_ROUNDS times)

  • Apply a final set of flagging and smoothing the the bandpass solutions themselves

The above procedure attempts to identify faint sources of RFI that only become apparent after an initial calibration has been applied. When recalibrating with updatede flags the raw data are used, i.e. we are not calibrating off already calibratede data.

Outputs

The prefect workflow described above will output:

  • A measurement set for each beam centered on PKS B1934-638

  • The set of bandpass solutions appropriately named

  • Validation plots of the derived solutions.

At present the bandpass solver principally relied upon in flint is calibrate, which implements the MitchCal algorithm. The output set of solutions are a series of Jones matrices packed into a binary solutions file. When selecting a set of bandpass solutions to apply flint will examine the meta-data encoded in each of the PKS B1934-638 per beam measurement sets in order to ensure consistent frequency coverage, channelisation and beam. This information is not encoded in the output binary solutions file, so the measurement sets are necessary and should be preserved. In principal, appropriate solutions could be selected on name alone (since flint controls the output naiming scheme), but as a general principal it is believed that important meta-data should not be stored principally in a file path name.

In subsequent workflows that operate against a field and a bandpass is required, the path the folder that contains the collection of per-beam observations of PKS B1934-638 and the corresponding set of solutions should be supplied.

Accessing via the CLI

The primary entry point for bandpass calibration in flint is the flint_flow_bandpass_calibrate:

Perform bandpass calibration against an ASKAP SBID.

usage: flint_flow_bandpass_calibrate [-h] [--split-path SPLIT_PATH]
                                     [--cluster-config CLUSTER_CONFIG]
                                     [--flagger-container FLAGGER_CONTAINER]
                                     [--calibrate-container CALIBRATE_CONTAINER]
                                     [--expected-ms EXPECTED_MS]
                                     [--smooth-solutions]
                                     [--smooth-window-size SMOOTH_WINDOW_SIZE]
                                     [--smooth-polynomial-order SMOOTH_POLYNOMIAL_ORDER]
                                     [--flag-calibrate-rounds FLAG_CALIBRATE_ROUNDS]
                                     [--minuv MINUV]
                                     [--preflagger-ant-mean-tolerance PREFLAGGER_ANT_MEAN_TOLERANCE]
                                     [--preflagger-mesh-ant-flags]
                                     [--preflagger-jones-max-amplitude PREFLAGGER_JONES_MAX_AMPLITUDE]
                                     bandpass_path

Positional Arguments

bandpass_path

Path to the directory containing the uncalibrated bandpass measurement sets.

Named Arguments

--split-path

Location to write the field-split MSs. Will attempt to create a directory using the SBID of the bandpass observation.

Default: .

--cluster-config

Path to a cluster configuration file, or a known cluster name.

Default: 'petrichor'

Inputs for BandpassOptions

--flagger-container

Path to the singularity aoflagger container

--calibrate-container

Path to the singularity calibrate container

--expected-ms

The expected number of measurement set files to find

Default: 36

--smooth-solutions

Will activate the smoothing of the bandpass solutions

Default: False

--smooth-window-size

The width of the smoothing window used to smooth the bandpass solutions

Default: 16

--smooth-polynomial-order

The polynomial order used by the Savgol filter when smoothing the bandpass solutions

Default: 4

--flag-calibrate-rounds

The number of times the bandpass will be calibrated, flagged, then recalibrated

Default: 3

--minuv

The minimum baseline length, in meters, for data to be included in bandpass calibration stage

--preflagger-ant-mean-tolerance

Tolerance that the mean x/y antenna gain ratio test before the antenna is flagged

Default: 0.2

--preflagger-mesh-ant-flags

Share channel flags from bandpass solutions between all antenna

Default: False

--preflagger-jones-max-amplitude

Flag Jones matrix if any amplitudes with a Jones are above this value

The BandpassOptions class

Embedded below is the flint Options class used to drive the flint_flow_bandpass_calibrate workflow. Input values are validated by pydantic to ensure they are appropriately typed.

class BandpassOptions(BaseOptions):
    """Container that represents the flint related options that
    might be used throughout the processing of bandpass calibration
    data.

    In its present form this `BandpassOptions` class is not intended
    to contain properties of the data that arebeing processed, rather
    how these data will be processed.

    These settings are not meant to be adjustabled throughout
    a single bandpass pipeline run
    """

    flagger_container: Path | None = None
    """Path to the singularity aoflagger container"""
    calibrate_container: Path | None = None
    """Path to the singularity calibrate container"""
    expected_ms: int = 36
    """The expected number of measurement set files to find"""
    smooth_solutions: bool = False
    """Will activate the smoothing of the bandpass solutions"""
    smooth_window_size: int = 16
    """The width of the smoothing window used to smooth the bandpass solutions"""
    smooth_polynomial_order: int = 4
    """The polynomial order used by the Savgol filter when smoothing the bandpass solutions"""
    flag_calibrate_rounds: int = 3
    """The number of times the bandpass will be calibrated, flagged, then recalibrated"""
    minuv: float | None = None
    """The minimum baseline length, in meters, for data to be included in bandpass calibration stage"""
    preflagger_ant_mean_tolerance: float = 0.2
    """Tolerance that the mean x/y antenna gain ratio test before the antenna is flagged"""
    preflagger_mesh_ant_flags: bool = False
    """Share channel flags from bandpass solutions between all antenna"""
    preflagger_jones_max_amplitude: float | None = None
    """Flag Jones matrix if any amplitudes with a Jones are above this value"""