ufig.psf_estimation package

Submodules

ufig.psf_estimation.cnn_predictions module

class ufig.psf_estimation.cnn_predictions.CNNPredictorPSF(config)

Bases: object

Handles CNN-based PSF parameter prediction.

This class manages: - CNN model loading and configuration - Batch prediction on star stamp images - Post-processing and quality assessment of predictions - Application of systematic corrections

predict_psf_parameters(cat_gaia, cube_gaia, filepath_cnn)

Predict PSF parameters for a catalog of stars using a CNN model.

Parameters:

• cat_gaia – Input catalog of stars with GAIA matching.

• cube_gaia – Cube of star stamp images.

• filepath_cnn – Path to the CNN model file.

Returns:

Dictionary with predicted PSF parameters and quality flags.

ufig.psf_estimation.cnn_util module

Created on May 25, 2018 author: Joerg Herbel Original Source: ethz_des_mccl.psf_estimation.cnn_psf.cnn_util

class ufig.psf_estimation.cnn_util.CNNPredictor(path_trained_cnn)

Bases: object

ufig.psf_estimation.cnn_util.create_cnn(input, filter_sizes, n_filters_start, n_resnet_layers, resnet_layers_kernel_size, n_fc, dropout_rate, n_out, activation_function='relu', apply_dropout=True, downsampling_method='max_pool', padding='same')

ufig.psf_estimation.cnn_util.get_path_output_config(path_cnn)

ufig.psf_estimation.cnn_util.normalize_stamps(stamps)

ufig.psf_estimation.cnn_util.res_layer(tensor_in, kernel_shape, activation)

Standard resnet layer (from tiny image net)

ufig.psf_estimation.core module

Core PSF estimation pipeline functionality.

This module contains the main PSF estimation pipeline that orchestrates the complete process from image loading to model creation.

class ufig.psf_estimation.core.PSFEstimationPipeline(**kwargs)

Bases: object

Main PSF estimation pipeline orchestrator.

This class coordinates the complete PSF estimation process: 1. Data loading and preparation 2. Star selection and quality cuts 3. CNN-based PSF parameter prediction 4. Polynomial interpolation model fitting 5. Model validation and output generation

create_psf_model(filepath_image, filepath_sexcat, filepath_sysmaps, filepath_gaia, filepath_cnn, filepath_out_model, filepath_out_cat=None)

Estimates the PSF of the image and saves all necessary files for a later image simulation.

Parameters:

• filepath_image – Path to the input image file to estimate the PSF from.

• filepath_sexcat – Path to the SExtractor catalog file of the image.

• filepath_sysmaps – Path to the systematics maps file.

• filepath_gaia – Path to the Gaia catalog file.

• filepath_cnn – Path to the pretrained CNN model

• filepath_out_model – Path to save the output PSF model file.

• filepath_cat_out – Path to save the enriched sextractor catalog, if None, the catalog at filepath_sexcat will be enriched

ufig.psf_estimation.correct_brighter_fatter module

Created on 05 May, 2018 @author: Tomasz Kacprzak

ufig.psf_estimation.correct_brighter_fatter.brighter_fatter_add(col_mag, col_fwhm, col_e1, col_e2, dict_corr)

ufig.psf_estimation.correct_brighter_fatter.brighter_fatter_remove(col_mag, col_fwhm, col_e1, col_e2, dict_corr)

ufig.psf_estimation.cutouts_utils module

Created on Feb 23, 2021 @author: Tomasz Kacprzak

ufig.psf_estimation.cutouts_utils.find_max_flux_pos(x, y, delta_x, delta_y, image)

Find the position of the maximum flux in a 2D image around a given position (x, y).

Parameters:

• x – x-coordinate of the center position

• y – y-coordinate of the center position

• delta_x – array of x offsets to check

• delta_y – array of y offsets to check

• image – 2D numpy array representing the image

Returns:

(xi_max, yi_max) - coordinates of the pixel with maximum flux

ufig.psf_estimation.cutouts_utils.get_cutouts(x_peak, y_peak, image, pointings_maps, stamp_shape)

Get cube of coutouts around a list of coordinates.

Parameters:

• x_peak – list of x positions

• y_peak – list of y positions

• image – 2D image

• pointings_maps – an array with binary pointings indicators

• stamp_shape – (nx, ny) shape of stamp, fixed for all objects

ufig.psf_estimation.data_preparation module

class ufig.psf_estimation.data_preparation.PSFDataPreparator(config)

Bases: object

Handles data preparation for PSF estimation.

This class is responsible for: - Loading and preprocessing image data - GAIA catalog matching for stellar sample selection - Initial quality cuts and flag management - Preparation of data structures for CNN processing

prepare_data(filepath_image, filepath_sexcat, filepath_sysmaps, filepath_gaia)

Prepare the data for the PSF estimation. This includes - loading the image and the SExtractor catalog, - matching with GAIA catalog, - applying initial quality cuts.

Parameters:

• filepath_image – Path to the input image file to estimate the PSF from.

• filepath_sexcat – Path to the SExtractor catalog file of the image.

• filepath_sysmaps – Path to the systematics maps file.

• filepath_gaia – Path to the GAIA catalog file.

• config – Configuration dictionary with parameters for data preparation.

ufig.psf_estimation.polynomial_fitting module

class ufig.psf_estimation.polynomial_fitting.PolynomialPSFModel(config)

Bases: object

Handles polynomial interpolation fitting for PSF spatial variation.

This class manages: - Polynomial model configuration and fitting - Iterative outlier removal during fitting - Cross-validation for regularization parameter selection - Model validation and quality assessment

fit_model(cat, position_weights)

Fit polynomial interpolation model to CNN predictions.

ufig.psf_estimation.psf_predictions module

ufig.psf_estimation.psf_predictions.colnames_derivative(cols, ax)

Create column names for derivatives with respect to an axis.

Parameters:

• cols – List of column names to derive from

• ax – Axis for the derivative (‘x’ or ‘y’)

Returns:

List of new column names for derivatives

ufig.psf_estimation.psf_predictions.get_model_derivatives(cat, filepath_psfmodel, cols, psfmodel_corr_brighter_fatter, delta=0.01)

Calculate spatial derivatives of PSF model parameters.

Parameters:

• cat – Input catalog with X_IMAGE, Y_IMAGE columns

• filepath_psfmodel – Path to the PSF model file

• cols – List of column names to calculate derivatives for

• psfmodel_corr_brighter_fatter – Parameters for brighter-fatter correction

• delta – Step size for finite difference calculation

Returns:

Catalog with additional columns for derivatives

ufig.psf_estimation.psf_predictions.predict_psf(position_xy, position_weights, regressor, settings, n_per_chunk=1000)

Predict PSF parameters at given positions using a fitted regressor.

Parameters:

• position_xy – Array of (x,y) positions, shape (n, 2)

• position_weights – Weights for each position, shape (n, m)

• regressor – Fitted regressor object

• settings – Dictionary with model settings, including scale factors

• n_per_chunk – Number of samples to process in each batch

Returns:

Tuple of predicted parameters and a mask for positions with no coverage

ufig.psf_estimation.psf_predictions.predict_psf_for_catalogue(cat, filepath_psfmodel, id_pointing='all', psfmodel_corr_brighter_fatter=None)

Predict PSF parameters for a given catalog.

Parameters:

• cat – Input catalog with X_IMAGE, Y_IMAGE (and MAG_AUTO if applicable) columns

• filepath_psfmodel – Path to the PSF model file

• id_pointing – ID of the pointing to use, or ‘all’

• psfmodel_corr_brighter_fatter – Parameters for brighter-fatter correction

Returns:

Generator yielding predicted parameters and number of exposures

ufig.psf_estimation.psf_predictions.predict_psf_for_catalogue_storing(cat_in, filepath_psf_model, psfmodel_corr_brighter_fatter)

Predict PSF parameters for a catalog and format for storage.

Parameters:

• cat_in – Input catalog with X_IMAGE, Y_IMAGE columns

• filepath_psf_model – Path to the PSF model file

• psfmodel_corr_brighter_fatter – Parameters for brighter-fatter correction

Returns:

Tuple of output catalog with predicted parameters and number of exposures

ufig.psf_estimation.psf_predictions.predict_psf_with_file(position_xy, filepath_psfmodel, id_pointing='all')

Predict PSF parameters at given positions using a saved PSF model file.

Parameters:

• position_xy – Array of (x,y) positions, shape (n, 2)

• filepath_psfmodel – Path to the PSF model file

• id_pointing – ID of the pointing to use, or ‘all’

Returns:

Generator yielding predicted parameters and number of exposures

ufig.psf_estimation.psf_utils module

exception ufig.psf_estimation.psf_utils.PSFEstError

Bases: ValueError

Raised when too few stars for PSF estimation were found.

ufig.psf_estimation.psf_utils.adjust_psf_measurements(cat, psf_measurement_adjustment=None)

Adjust PSF measurements based on provided adjustment parameters.

Parameters:

• cat – Input catalog with PSF parameters.

• psf_measurement_adjustment – Dictionary with adjustment parameters for each PSF measurement.

ufig.psf_estimation.psf_utils.apply_brighter_fatter_correction(cat_cnn, psfmodel_corr_brighter_fatter)

Apply Brighter-Fatter correction to the PSF parameters in the catalog.

ufig.psf_estimation.psf_utils.get_position_weights(x, y, pointings_maps)

Get the position weights for each star based on the number of exposures at each pixel location.

Parameters:

• x – x-coordinates of stars

• y – y-coordinates of stars

• pointings_maps – bitmaps indicating exposure coverage

Returns:

position weights for each star

ufig.psf_estimation.psf_utils.get_star_cube_filename(filepath_cat_out)

ufig.psf_estimation.psf_utils.position_weights_to_nexp(position_weights)

Transform position weights to number of exposures.

ufig.psf_estimation.psf_utils.postprocess_catalog(cat)

Post-process the catalog after PSF prediction. This function ensures that the PSF flux ratio is within valid bounds.

Parameters:

cat – Input catalog with PSF parameters.

Returns:

None, modifies the catalog in place.

ufig.psf_estimation.psf_utils.select_validation_stars(n_stars, fraction_validation)

Select validation stars for PSF model testing.

Parameters:

• n_stars – Total number of stars in the catalog.

• fraction_validation – Fraction of stars to select for validation.

Returns:

Array of indices for validation stars.

ufig.psf_estimation.psf_utils.transform_forward(vec, scale)

ufig.psf_estimation.psf_utils.transform_inverse(vec_transformed, scale)

ufig.psf_estimation.psf_utils.write_empty_file(path)

ufig.psf_estimation.psf_utils.write_empty_output(filepath_out, filepath_cat_out=None, save_star_cube=False)

Creates empty output files when PSF estimation fails.

This function creates placeholder files to ensure that even when PSF estimation fails, the expected output files exist, preventing downstream processes from failing due to missing files.

Parameters

filepath_outstr

Path to the main PSF model output file

filepath_cat_outstr, optional

Path to the catalog output file

save_star_cubebool, optional

Whether a star cube file was expected

ufig.psf_estimation.psf_utils.write_star_cube(star_cube, cat, filepath_cat_out)

ufig.psf_estimation.save_model module

class ufig.psf_estimation.save_model.PSFSave(config)

Bases: object

Saves PSF models and predictions to HDF5 format.

This class manages:

• Saving PSF models to HDF5 format

• Writing output catalogs with PSF predictions

• Storing diagnostic data and star cubes

• Creating grid predictions for visualization

save_psf_model(filepath_out, processed_data, cnn_results, model_results, filepath_sysmaps=None, filepath_cat_out=None)

Save the PSF model and predictions to an HDF5 file.

ufig.psf_estimation.star_sample_selection_cnn module

Created on Aug 03, 2017 @author: Joerg Herbel

ufig.psf_estimation.star_sample_selection_cnn.beta_cut(beta, beta_lim=(1.5, 10))

Select stars based on their beta parameter.

Parameters:

• beta – Beta parameter for each star.

• beta_lim – Tuple defining the limits for beta selection.

Returns:

Boolean array indicating which stars have beta within the specified limits.

ufig.psf_estimation.star_sample_selection_cnn.cut_boundaries(cat, image, pointings_maps, star_stamp_shape)

Checks the position of the star stamps within the image and returns boolean arrays indicating whether the stars are within the image boundaries, coadd boundaries and whether the cutout is centered on the maximum flux pixel.

Parameters:

• cat – Catalog with SExtractor measurements.

• image – Image data for cutout extraction.

• pointings_maps – Pointing maps for exposure coverage.

• star_stamp_shape – Shape of the cutout stamps for stars.

ufig.psf_estimation.star_sample_selection_cnn.cut_magnitude(cat, mag_min, mag_max)

Select stars based on their magnitude.

Parameters:

• cat – Catalog with SExtractor measurements.

• mag_min – Minimum magnitude for selection.

• mag_max – Maximum magnitude for selection.

Returns:

Boolean array indicating which stars are within the magnitude range.

ufig.psf_estimation.star_sample_selection_cnn.cut_moments(cat, moments_lim)

ufig.psf_estimation.star_sample_selection_cnn.cut_n_exp(weights, n_exp_min)

Select stars based on the number of exposures they are covered by.

Parameters:

• weights – Position weights for each star.

• n_exp_min – Minimum number of exposures required for selection.

Returns:

Boolean array indicating which stars have enough exposures.

ufig.psf_estimation.star_sample_selection_cnn.cut_nearby_bright_star(cat)

ufig.psf_estimation.star_sample_selection_cnn.cut_position_weights(weights)

Select only stars with finite weights

Parameters:

weights – Position weights for each star.

Returns:

Boolean array indicating which stars have valid position weights.

ufig.psf_estimation.star_sample_selection_cnn.cut_sextractor_flag(cat, flags=None)

Select stars based on SExtractor flags.

Parameters:

• cat – Catalog with SExtractor measurements.

• flags – list of flags to consider, if None, all flags are considered.

Returns:

Boolean array indicating which stars have acceptable SExtractor flags.

ufig.psf_estimation.star_sample_selection_cnn.cut_sysmaps_delta_weight(cat)

ufig.psf_estimation.star_sample_selection_cnn.cut_sysmaps_survey_mask(cat)

ufig.psf_estimation.star_sample_selection_cnn.ellipticity_cut(e1, e2, ellipticity_lim=(-0.3, 0.3))

Select stars based on their ellipticity parameters.

Parameters:

• e1 – First ellipticity component for each star.

• e2 – Second ellipticity component for each star.

• ellipticity_lim – Tuple defining the limits for ellipticity selection.

Returns:

Boolean array indicating which stars have ellipticity within the specified limits.

ufig.psf_estimation.star_sample_selection_cnn.flexion_cut(f1, f2, g1, g2, flexion_lim=(-0.3, 0.3))

Select stars based on their flexion parameters.

Parameters:

• f1 – First flexion component for each star.

• f2 – Second flexion component for each star.

• g1 – Third flexion component for each star.

• g2 – Fourth flexion component for each star.

• flexion_lim – Tuple defining the limits for flexion selection.

Returns:

Boolean array indicating which stars have flexion within the specified limits.

ufig.psf_estimation.star_sample_selection_cnn.fwhm_cut(fwhm, fwhm_lim=(1, 10))

Select stars based on their FWHM parameter.

Parameters:

• fwhm – FWHM parameter for each star.

• fwhm_lim – Tuple defining the limits for FWHM selection.

Returns:

Boolean array indicating which stars have FWHM within the specified limits.

ufig.psf_estimation.star_sample_selection_cnn.get_gaia_image_coords(filepath_image, cat)

Convert GAIA coordinates (RA/Dec) to image pixel coordinates.

Parameters:

• filepath_image – Path to the image file for coordinate conversion.

• cat – Catalog containing GAIA coordinates.

Returns:

Catalog with additional columns for GAIA pixel coordinates.

ufig.psf_estimation.star_sample_selection_cnn.get_gaia_match(cat_in, filepath_gaia, max_dist_arcsec)

Match the objects from the SExtractor catalog with the GAIA catalog based on their sky coordinates with a maximum distance defined by max_dist_arcsec. The function returns a catalog with additional columns to flag the matches and the matched RA/Dec coordinates from GAIA.

Parameters:

• cat_in – SExtractor catalog data.

• filepath_gaia – Path to the GAIA catalog file.

• max_dist_arcsec – Maximum distance in arcseconds for matching.

Returns:

Catalog enriched with GAIA matching information

ufig.psf_estimation.star_sample_selection_cnn.get_match_vector(cat, cat_gaia, max_dist_arcsec)

Match the SExtractor catalog with the GAIA catalog based on RA/Dec coordinates. The matching is done using a nearest neighbor search with a maximum distance defined by max_dist_arcsec.

Parameters:

• cat – SExtractor catalog data.

• cat_gaia – GAIA catalog data.

• max_dist_arcsec – Maximum distance in arcseconds for matching.

Returns:

Tuple of boolean selection array and matched RA/Dec coordinates from GAIA.

ufig.psf_estimation.star_sample_selection_cnn.get_stars_for_cnn(cat, image, star_stamp_shape, pointings_maps, position_weights, star_mag_range=(18, 22), min_n_exposures=1, sextractor_flags=None, flag_coadd_boundaries=False, moments_lim=(-99, 99))

Select the stars for the CNN model based on various quality cuts.

Parameters:

• cat – Catalog with SExtractor measurements.

• image – Image data for cutout extraction.

• star_stamp_shape – Shape of the cutout stamps for stars.

• pointings_maps – Pointing maps for exposure coverage.

• position_weights – Position weights for each star.

• star_mag_range – Tuple defining the magnitude range for star selection.

• min_n_exposures – Minimum number of exposures required for a star to be selected.

• sextractor_flags – SExtractor flags to consider for quality cuts.

Returns:

flags, cube: Flags indicating the quality of each star and the cutout cube of selected stars.

ufig.psf_estimation.star_sample_selection_cnn.inlims(x, lims)

Check if values in x are within the specified limits.

Parameters:

• x – Array of values to check.

• lims – Tuple of (min, max) limits.

Returns:

Boolean array indicating whether each value is within the limits.

ufig.psf_estimation.star_sample_selection_cnn.kurtosis_cut(kurtosis, kurtosis_lim=(-1, 1))

Select stars based on their kurtosis parameter.

Parameters:

• kurtosis – Kurtosis parameter for each star.

• kurtosis_lim – Tuple defining the limits for kurtosis selection.

Returns:

Boolean array indicating which stars have kurtosis within the specified limits.

ufig.psf_estimation.star_sample_selection_cnn.remove_outliers(x, y, n_sigma, list_cols_use='all')

Remove outliers based on the difference between predicted and actual values.

Parameters:

• x – Predicted values (e.g., CNN predictions).

• y – Actual values (e.g., SExtractor measurements).

• n_sigma – Number of standard deviations for clipping.

• list_cols_use – List of columns to use for outlier detection, or ‘all’ to use all columns.

Returns:

Boolean array indicating which samples are not outliers.

ufig.psf_estimation.star_sample_selection_cnn.select_cnn_predictions(flags, pred, beta_lim=(1.5, 10), fwhm_lim=(1, 10), ellipticity_lim=(-0.3, 0.3), flexion_lim=(-0.3, 0.3), kurtosis_lim=(-1, 1))

Apply various cuts to the CNN predictions to select stars based on their PSF parameters.

Parameters:

• flags – Flags array indicating the quality of each star.

• pred – Predictions from the CNN model containing PSF parameters.

Returns:

Updated flags after applying the cuts.

ufig.psf_estimation.star_sample_selection_cnn.set_flag_bit_and_log(select, flags, flagbit, msg)

ufig.psf_estimation.tiled_regressor module

class ufig.psf_estimation.tiled_regressor.TiledRobustPolynomialRegressor(poly_order=3, ridge_alpha=0, n_input_dim=2, polynomial_type='standard', poly_coefficients=None, set_unseen_to_mean=False, unseen_pointings=None, raise_underdetermined=False)

Bases: BaseEstimator, RegressorMixin

fit(X, y, var_y=None, method='ridge')

n_free_params(n_pointings)

predict(X, batch_size=1000)

set_fit_request(*, method: bool | None | str = '$UNCHANGED$', var_y: bool | None | str = '$UNCHANGED$') → TiledRobustPolynomialRegressor

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to fit.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

methodstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for method parameter in fit.

var_ystr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for var_y parameter in fit.

selfobject

The updated object.

set_predict_request(*, batch_size: bool | None | str = '$UNCHANGED$') → TiledRobustPolynomialRegressor

Configure whether metadata should be requested to be passed to the predict method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to predict.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

batch_sizestr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for batch_size parameter in predict.

selfobject

The updated object.

set_score_request(*, sample_weight: bool | None | str = '$UNCHANGED$') → TiledRobustPolynomialRegressor

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config()). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

• True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.

• False: metadata is not requested and the meta-estimator will not pass it to score.

• None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.

• str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED

Metadata routing for sample_weight parameter in score.

selfobject

The updated object.

exception ufig.psf_estimation.tiled_regressor.UnderdeterminedError

Bases: ValueError

Raised when trying to fit an underdetermined model.

ufig.psf_estimation.tiled_regressor.get_poly_weight_basis(position_xy_transformed, position_weights, poly_order, polynomial_type)

ufig.psf_estimation.tiled_regressor.var_to_weight(v)

Module contents