flint.prefect.clusters
======================

.. py:module:: flint.prefect.clusters

.. autoapi-nested-parse::

   Some utility functions around the creation of Prefect task funners.

   For this work we will be using Dask backed workers to perform the compute
   operations.



Functions
---------

.. autoapisummary::

   flint.prefect.clusters.get_cluster_spec
   flint.prefect.clusters.get_dask_runner


Module Contents
---------------

.. py:function:: get_cluster_spec(cluster: str | pathlib.Path) -> dict[Any, Any]

   Given a cluster name, obtain the appropriate SLURM configuration
   file appropriate for use with SLURMCluster.

   This cluster spec is expected to be consistent with the cluster_class
   and cluster_kwargs parameters that are used by dask_jobqueue based
   specifications.

   :param cluster: Name of cluster or path to a configuration to look up for processing
   :type cluster: Union[str,Path]

   :raises ValueError: Raised when cluster is not in KNOWN_CLUSTERS and has not corresponding YAML file.

   :returns: Dictionary of know options/parameters for dask_jobqueue.SLURMCluster
   :rtype: dict[Any, Any]


.. py:function:: get_dask_runner(cluster: str | pathlib.Path = 'galaxy_small', extra_cluster_kwargs: dict[str, Any] | None = None) -> prefect_dask.DaskTaskRunner

   Creates and returns a DaskTaskRunner configured to established a SLURMCluster instance
   to manage a set of dask-workers. The SLURMCluster is currently configured only for Galaxy.

   :keyword cluster: The cluster name that will be used to search for a cluster specification file.
                     This could be the name of a known cluster, or the name of a yaml file installed
                     among the `cluster_configs` directory of the aces module.
   :kwtype cluster: Union[str,Path]

   :returns: A dask task runner capable of being used as a task_runner for a prefect flow
   :rtype: DaskTaskRunner