The logger module allows to create convenient and nice looking logs for your python code. Just place the following two lines at the top of your python scripts:
[2]:
from ekit import logger
LOGGER = logger.init_logger(__name__, logging_level=4)
The verbosity level goes from 0 to 4. 0 will only log critical errors, 1 adds all remaining errors, 2 adds warnings, 3 adds info messages and 4 debug messages. The colorized option allows for some prettier output. To log something in you code you can use:
[4]:
LOGGER.critical("Really bad error")
LOGGER.error("Normal error")
LOGGER.warning("You did something you might not want to do")
LOGGER.info("Some standard blabla")
LOGGER.debug("Some prints to check that code does what you want it to do")
2022-03-18 14:55:48,184 - 4016984765.py: 1 - CRITICAL - Really bad error
2022-03-18 14:55:48,185 - 4016984765.py: 2 - ERROR - Normal error
2022-03-18 14:55:48,186 - 4016984765.py: 3 - WARNING - You did something you might not want to do
2022-03-18 14:55:48,186 - 4016984765.py: 4 - INFO - Some standard blabla
2022-03-18 14:55:48,187 - 4016984765.py: 5 - DEBUG - Some prints to check that code does what you want it to do
The context module can be used to pass configuration parameters around. You can specify which parameters are allowed and what data types they must have. You can also specify default values for them.
[2]:
from ekit import context
This creates the context using just the default values
[3]:
allowed_parameters = ['val1', 'val2']
parameter_types = ['int', 'str']
parameter_defaults = [0, 'bla']
# Initialize with defaults
ctx = context.setup_context(allowed=allowed_parameters,
types=parameter_types,
defaults=parameter_defaults,
verbosity=4)
print(ctx)
2022-03-18 14:59:56,151 - context.py: 42 - DEBUG - Setting parameter val1 to default 0
2022-03-18 14:59:56,153 - context.py: 42 - DEBUG - Setting parameter val2 to default bla
2022-03-18 14:59:56,155 - context.py: 87 - DEBUG - Set context to:
2022-03-18 14:59:56,156 - context.py: 88 - DEBUG - ######################################
2022-03-18 14:59:56,157 - context.py: 90 - DEBUG - val1 : 0
2022-03-18 14:59:56,158 - context.py: 90 - DEBUG - val2 : bla
2022-03-18 14:59:56,159 - context.py: 91 - DEBUG - ######################################
{'val1': 0, 'val2': 'bla'}
You can also pass a directory with values that should overwrite the defaults.
[5]:
# Initialize using dictionary
input = {'val1': 2, 'val2': 'something'}
ctx = context.setup_context(ctx_in=input,
allowed=allowed_parameters,
types=parameter_types,
defaults=parameter_defaults,
verbosity=4)
print(ctx)
2022-03-18 15:00:50,065 - context.py: 42 - DEBUG - Setting parameter val1 to default 0
2022-03-18 15:00:50,067 - context.py: 42 - DEBUG - Setting parameter val2 to default bla
2022-03-18 15:00:50,069 - context.py: 75 - DEBUG - Setting parameter val1 -> 2
2022-03-18 15:00:50,070 - context.py: 75 - DEBUG - Setting parameter val2 -> something
2022-03-18 15:00:50,070 - context.py: 87 - DEBUG - Set context to:
2022-03-18 15:00:50,071 - context.py: 88 - DEBUG - ######################################
2022-03-18 15:00:50,071 - context.py: 90 - DEBUG - val1 : 2
2022-03-18 15:00:50,072 - context.py: 90 - DEBUG - val2 : something
2022-03-18 15:00:50,072 - context.py: 91 - DEBUG - ######################################
{'val1': 2, 'val2': 'something'}
Or have the values read from a YAML file.
[8]:
# Initialize using YAML file
ctx = context.setup_context(ctx_in='data.yml',
allowed=allowed_parameters,
types=parameter_types,
defaults=parameter_defaults,
verbosity=4)
print(ctx)
2022-03-18 15:01:51,325 - context.py: 42 - DEBUG - Setting parameter val1 to default 0
2022-03-18 15:01:51,326 - context.py: 42 - DEBUG - Setting parameter val2 to default bla
2022-03-18 15:01:51,327 - context.py: 48 - INFO - Setting context using configuration file data.yml.
2022-03-18 15:01:51,329 - context.py: 56 - DEBUG - Setting parameter val1 -> 3
2022-03-18 15:01:51,330 - context.py: 56 - DEBUG - Setting parameter val2 -> other
2022-03-18 15:01:51,331 - context.py: 87 - DEBUG - Set context to:
2022-03-18 15:01:51,331 - context.py: 88 - DEBUG - ######################################
2022-03-18 15:01:51,332 - context.py: 90 - DEBUG - val1 : 3
2022-03-18 15:01:51,332 - context.py: 90 - DEBUG - val2 : other
2022-03-18 15:01:51,333 - context.py: 91 - DEBUG - ######################################
{'val1': 3, 'val2': 'other'}
The paths module allows to pack meta data into paths and to retrieve it again from the paths. It diffferentiates between named (passed via a directory) and unnamed parameters (passed via a list).
Note that your parameter names MUST NOT contain = or _ signs as these are used to parse the metadata from the path.
[10]:
from ekit import paths
[12]:
# create a path
id = 'DATA'
out_folder = '/home/User/Files'
suffix = '.fits'
defined_parameters = {'key': 'bla', 'key2': True, 'random': 45.65, 'number': 20}
undefined_parameters = [3, 5.4, 'hey']
path = paths.create_path(identity=id,
out_folder=out_folder,
defined_parameters=defined_parameters,
undefined_parameters=undefined_parameters,
suffix=suffix, verbosity=4)
print(path)
2022-03-18 15:03:16,047 - paths.py: 44 - DEBUG - Adding defined key key with value bla
2022-03-18 15:03:16,048 - paths.py: 44 - DEBUG - Adding defined key key2 with value True
2022-03-18 15:03:16,049 - paths.py: 44 - DEBUG - Adding defined key number with value 20
2022-03-18 15:03:16,050 - paths.py: 44 - DEBUG - Adding defined key random with value 45.65
2022-03-18 15:03:16,050 - paths.py: 50 - DEBUG - Adding undefined parameter with value [3, 5.4, 'hey']
2022-03-18 15:03:16,051 - paths.py: 50 - DEBUG - Adding undefined parameter with value [3, 5.4, 'hey']
2022-03-18 15:03:16,051 - paths.py: 50 - DEBUG - Adding undefined parameter with value [3, 5.4, 'hey']
/home/User/Files/DATA_key=bla_key2=True_number=20_random=45.65_3_5.4_hey.fits
Now lets retrieve the parameters again. By default ekit will try the following formats in this order: boolean, integer, float, string
[15]:
defs, udefs = paths.get_parameters_from_path(path)
print(defs)
print(udefs)
{'key': array(['bla'], dtype=object), 'key2': array([True], dtype=object), 'number': array([20], dtype=object), 'random': array([45.65], dtype=object)}
[[3 5.4 'hey']]
You can also tell ekit what format to use for which paramter, if you are not happy with the deaults.
[17]:
defs, udefs = paths.get_parameters_from_path(path, fmt="%s_%b_%f_%f_%f_%s_%s")
print(defs)
print(udefs)
{'key': array(['bla'], dtype=object), 'key2': array([True], dtype=object), 'number': array([20.0], dtype=object), 'random': array([45.65], dtype=object)}
[[3.0 '5.4' 'hey']]