User Guide
Using as a python library
Under Construction
See Examples:
Command Line Use
For large scale runs, it is recommended to make use of SERVAL's command line interface (CLI), especially for runs on HPC systems. The parameters for these commands can be specified with either TOML configuration files or command line arguments.
Currently supported CLI commands are:
and
These are described below. Help for both can be accessed with e.g.:
and configuration templates can be printed to stdout with e.g.:
Note
Parameters can be expressed across multiple config files as well as command line arguments. Parameters given in command line arguments take priority over those given in configuration files. Parameters given in more than one configuration file will take the value specified in the last configuration file listed.
There is an example of end-to-end usage of the these commands and how to generate their input files in this notebook.
These commands can parallelize over multiple tasks with a given memory target for each task. Some examples of submission scripts for large runs on HPCs will be provided in the future. To get a sense of how many resources a run will use, you can do a dry run that only produces a task distribution plan using the --plan-only flag.
Task Distribution
The --dist-mode option on the root serval command controls how tasks are distributed:
| Mode | Description |
|---|---|
auto (default) |
Auto-detects: uses MPI if running under mpirun/srun with >1 task, then tries SLURM array env vars, then falls back to serial. |
mpi |
Explicit MPI via mpi4py. Errors if mpi4py is not importable. |
array |
SLURM array job. Reads SLURM_ARRAY_TASK_ID (→ task ID) and SLURM_ARRAY_TASK_COUNT (→ task count). Errors if either env var is missing. Requires a contiguous 0-based array (--array=0-N). |
serial |
Single task. |
The chosen mode and resulting task ID/count are always printed to stdout at startup.
Example invocations:
# Run under MPI (auto-detected or explicit)
mpirun -n 8 serval gencache fixed-sky --config my_config.toml
mpirun -n 8 serval --dist-mode mpi gencache fixed-sky --config my_config.toml
# SLURM array job submission (#SBATCH --array=0-7)
serval --dist-mode array gencache fixed-sky --config my_config.toml
# Force serial (useful for debugging)
serval --dist-mode serial gencache fixed-sky --config my_config.toml
Generating an Integrator Cache using the CLI
SERVAL supports generating an integrator cache in the form of a cached set of PowerBeamVisProjector's from the command line.
This can be done through the command serval gencache fixed-sky whose usage is as follows:
The --sky-model argument expects a zarr store created from a SkyModel container. A SkyModel can be constructed directly from spherical harmonic coefficient arrays, from HEALPix maps via SkyModel.from_healpix_maps(), or from a point source catalogue via SkyModel.from_point_source_catalog(), and persisted with SkyModel.to_zarr_store(). The sky model frequencies must match the --frequencies argument passed to this command.
serval gencache fixed-sky --help
Usage: serval gencache fixed-sky [OPTIONS]
Generate a stored integrator cache to be used with power beams at a later
stage to generate visibilities. Fixes a telescope and sky model.
Options:
--config-template Output a template configuration file and
exit.
--plan-only Output the run plan only and exit.
--config PATH One or more optional TOML configuration
files to use for parameters. The last file
specified has the highest priority for
overlapping parameters.
--store-location PATH The location of the zarr store for the
generate cache. Must be specified here or in
a config file.
--sky-model PATH The location of the harmonic sky model zarr
store (a SkyModel saved via
SkyModel.to_zarr_store()). Frequencies must
match those specified for the cache
generation. Must be specified here or in a
config file.
--latitude TEXT Telescope TIRS latitude, must be astropy
convertable or floats in degrees. Must be
specified here or in a config file.
--longitude TEXT Telescope TIRS longitude, must be astropy
convertable or floats in degrees. Must be
specified here or in a config file.
-b, --baselines_enu <TEXT TEXT TEXT>...
Telescope TIRS ENU Baselines, must be tree-
tuples of astropy convertable distances or
floats in meters. Must be specified here or
in a config file.
--power-beam-lmax INTEGER Maximum harmonic degree for the power beams.
Must be specified here or in a config file.
-f, --frequencies TEXT Telescope frequency channel centres, must be
astropy convertable or floats in MHz. Must
be specified here or in a config file.
--memory-per-task TEXT Target maxium memory usage per task, not
a hard limit. Astropy convertable or float
in GiB. [default: 4 GiB]
--max-absms-per-chunk INTEGER Maximum number of sky ms to compute per
task. If unbounded integration cache size
can blow up. [default: 128]
--start-chunk INTEGER Chunk index to start at if e.g. restarting
run. [default: 0]
--baseline-tag-ndigits INTEGER Number of decimal places in baseline group
tag strings. [default: 3]
--sky-pol TEXT Sky polarisations to cache. 'full' uses all
Stokes components present in the sky model.
Specify a subset string like 'I', 'IQU', or
'IQUV'. [default: full]
--optimized-cache / --no-optimized-cache
Whether to use a Gaunt Cache optimized for
this operation. [default: optimized-cache]
--pointing-contract / --no-pointing-contract
Whether to contract the cache to a specific
pointing. [default: no-pointing-contract]
--pointing-altitude TEXT If contracting to a pointing, the altitude
of the telescope pointing. Must be astropy
convertable or floats in degrees. [default:
90 deg]
--pointing-azimuth TEXT If contracting to a pointing, the azimuth of
the telescope pointing. Must be astropy
convertable or floats in degrees. [default:
180 deg]
--pointing-boresight TEXT If contracting to a pointing, the boresight
angle of the telescope pointing. Must be
astropy convertable or floats in degrees.
[default: 0 deg]
--pointed-beam-mmax INTEGER If contracting to a pointing, the harmonic
order maximum of the pointing relative power
beams. [default: 40]
--help Show this message and exit.
an example configuration file can be produced with serval gencache fixed-sky --config-template > example_gencache.toml which will produce a file like:
# Example serval gencache fixed-sky TOML configuration file
# Required:
# ---------
# The location of the zarr store for the generate cache
store_location = "/path/to/my/store.zarr"
# The location of the harmonic sky model, frequencies must match those specified for
# the cache generation
sky_model = "/path/to/my/sky_model.zarr"
# Telescope TIRS Coordinates, must be astropy convertable or floats in degrees
latitude = '-30d41m47.0s'
longitude = '21d34m20.0s'
# Telescope TIRS ENU Baselines, must be astropy convertable or floats in meters
baselines_enu = [
['0m', '8.5m', '0m'],
['6.5m', '0m', '0m'],
['6.5m', '8.5m', '0m']
]
# Maximum harmonic degree for the power beams.
power_beam_lmax = 100
# Telescope frequency channel centres, must be astropy convertable or floats in MHz
frequencies = [
'400MHz',
'500MHz',
]
# Optional:
# ---------
# Target maximum memory usage per task, not a hard limit. Astropy convertable or float in GiB.
memory_per_task = '4 GiB'
# Maximum number of sky ms to compute per task. If unbounded integration cache size can blow up
max_absms_per_chunk = 128
# Chunk index to start at if e.g. restarting run.
start_chunk = 0
# Sky polarisations to cache. 'full' uses all Stokes components present in the sky model.
# Specify a subset string like 'I', 'IQU', or 'IQUV'. [default: full]
sky_pol = 'full'
# Number of decimal places in baseline group tag strings. [default: 3]
baseline_tag_ndigits = 3
# Whether to use a Gaunt Cache optimized for this operation
optimized_cache = true
# Whether to contract the cache to a specific pointing
pointing_contract = false
# If true, the harmonic order maximum of the pointing relative power beams
pointed_beam_mmax = 40
# And the pointing coordinates, must be astropy convertable or floats in degrees
pointing_altitude = '90 deg'
pointing_azimuth = '180 deg'
pointing_boresight = '0 deg'
Generating Visibilities from an Integrator Cache and Power Beams
SERVAL additionally supports generating visibilities from an integrator cache produced using serval gencache fixedsky.
This can be done through the command serval genvis from-power-beams whose usage is as follows:
The --power-beams argument expects a zarr store where each group path contains a TIRSPowerBeam container. A TIRSPowerBeam is typically constructed from a pair of TIRSVoltageBeam instances via TIRSPowerBeam.from_voltage_beams(), and persisted with TIRSPowerBeam.to_zarr_store(). The group paths within the store correspond to the beam identifiers used in --pol-baseline-mapping.
Usage: serval genvis from-power-beams [OPTIONS]
Generate visibilities from a stored integrator cache with given power beams.
Options:
--config-template Output a template configuration file and
exit.
--plan-only Output the run plan only and exit.
--config PATH One or more optional TOML configuration
files to use for parameters. The last file
specified has the highest priority for
overlapping parameters.
--store-location PATH The location of the output zarr store for
the visibilities (and mmodes). Must be
specified here or in a config file.
--integrator-cache PATH The location of the input zarr store for the
integrator cache. Must be specified here or
in a config file.
--power-beams PATH The location of the power beam zarr store (a
TIRSPowerBeam saved via
TIRSPowerBeam.to_zarr_store()). Must have
matching frequencies of the integrator cache.
Must be specified here or in a config file.
-pbm, --pol-baseline-mapping <TEXT TEXT>...
Mapping between pol/baseline groups in the
integrator cache and power beam groups in
the power beam store. The bl_id format is
'pol/baseline_tag', e.g.
'I/bl_E_6p5m_N_0p0m_U_0p0m'. Accepts glob
patterns for the beam term; only beams
whose sky_pol matches the pol prefix are
included.
--memory-per-task TEXT Target maxium memory usage per task, not
a hard limit. Astropy convertable or float
in GiB. [default: 4 GiB]
--start-chunk INTEGER Chunk index to start at if e.g. restarting
run. [default: 0]
--help Show this message and exit.
an example configuration file can be produced with serval genvis from-power-beams --config-template > example_gencache.toml which will produce a file like:
# Example serval genvis from-power-beams TOML configuration file
# Required:
# ---------
# The location of the output zarr store for the visibilities (and mmodes)
store_location = "/path/to/my/visibility/store.zarr"
# The location of the input zarr store for the integrator cache
integrator_cache = "/path/to/my/integrator_cache/store.zarr"
# The location of the input power beam store, must have matching frequencies of
# integrator cache
power_beams = "/path/to/my/power_beams.zarr"
# Mapping between pol/baseline groups in the integrator cache and power beam groups in
# the power beam store. The bl_id format is 'pol/baseline_tag'. Accepts glob patterns
# for the beam term; only beams whose sky_pol matches the pol prefix are included.
pol_baseline_mapping = [
['I/bl_E_6p5m_N_0p0m_U_0p0m', 'dish0_X_dish1_X'],
['I/bl_E_6p5m_N_0p0m_U_0p0m', 'dish0_Y_dish1_Y'],
]
# Optional:
# ---------
# Target maximum memory usage per task, not a hard limit. Astropy convertable or float in GiB.
memory_per_task = '4 GiB'
# Chunk index to start at if e.g. restarting run.
start_chunk = 0
Generating Visibilities from an Integrator Cache and Voltage Beams
SERVAL additionally supports generating visibilities from an integrator cache using voltage beams directly, without requiring pre-computed power beams.
This can be done through the command serval genvis from-voltage-beams. In contrast to serval genvis from-power-beams, this command accepts a zarr store of TIRSVoltageBeam instances and computes TIRSPowerBeams on-the-fly from pairs of voltage beams. The --voltage-beams argument expects a zarr store where each group path contains a TIRSVoltageBeam container, constructed via TIRSVoltageBeam.to_zarr_store(). The polarisation product (e.g. XX, YY, XY, YX) is derived automatically from the polarisation attributes of the two specified voltage beams.
serval genvis from-voltage-beams --help
Usage: serval genvis from-voltage-beams [OPTIONS]
Generate visibilities from a stored integrator cache with given voltage
beams. Power beams are computed on-the-fly from voltage beam pairs. The
voltage beam zarr store can be constructed using the TIRSVoltageBeam
container class.
Options:
--config-template Output a template configuration file and
exit.
--plan-only Output the run plan only and exit.
--config PATH One or more optional TOML configuration
files to use for parameters. The last file
specified has the highest priority for
overlapping parameters.
--store-location PATH The location of the output zarr store for
the visibilities (and mmodes). Must be
specified here or in a config file.
--integrator-cache PATH The location of the input zarr store for the
integrator cache. Must be specified here or
in a config file.
--voltage-beams PATH The location of the voltage beam zarr store
(a TIRSVoltageBeam saved via
TIRSVoltageBeam.to_zarr_store()). Must have
matching frequencies of the integrator
cache. Must be specified here or in a config
file.
-pbm, --pol-baseline-mapping <TEXT TEXT TEXT>...
Mapping between pol/baseline groups in the
integrator cache and voltage beam group
pairs. Each entry is (bl_id, vbeam_i_group,
vbeam_j_group), e.g.
'I/bl_E_6p5m_N_0p0m_U_0p0m dish0_X dish1_X'.
--memory-per-task TEXT Target maxium memory usage per task, not a
hard limit. Astropy convertable or float in
GiB. [default: 4 GiB]
--start-chunk INTEGER Chunk index to start at if e.g. restarting
run. [default: 0]
--help Show this message and exit.
an example configuration file can be produced with serval genvis from-voltage-beams --config-template > example_genvis_vb.toml which will produce a file like:
# Example serval genvis from-voltage-beams TOML configuration file
# Required:
# ---------
# The location of the output zarr store for the visibilities (and mmodes)
store_location = "/path/to/my/visibility/store.zarr"
# The location of the input zarr store for the integrator cache
integrator_cache = "/path/to/my/integrator_cache/store.zarr"
# The location of the input voltage beam store (TIRSVoltageBeam saved via to_zarr_store())
voltage_beams = "/path/to/my/voltage_beams.zarr"
# Mapping between pol/baseline groups in the integrator cache and voltage beam group pairs.
# Each entry is [bl_id, vbeam_i_group, vbeam_j_group]. The pol_product is derived from
# the polarisation attributes of the two voltage beams.
pol_baseline_mapping = [
['I/bl_E_6p5m_N_0p0m_U_0p0m', 'dish0_X', 'dish1_X'],
['I/bl_E_6p5m_N_0p0m_U_0p0m', 'dish0_Y', 'dish1_Y'],
]
# Optional:
# ---------
# Target maximum memory usage per task, not a hard limit. Astropy convertable or float in GiB.
memory_per_task = '4 GiB'
# Chunk index to start at if e.g. restarting run.
start_chunk = 0