remote Package

BashSlurmLrms Module

Created on Feb 12, 2014

@author: jakeret

class cpod.remote.BashSlurmLrms.BashSlurmLrms(name, architecture, max_cores, max_cores_per_job, max_memory_per_core, max_walltime, auth, frontend, transport, **extra_args)[source]

Bases: gc3libs.backends.slurm.SlurmLrms

Subclass of the standard gc3pie SlurmLrms which ensures to load a bash shell on the remote system when submitting jobs. Also upload recursively the input directories. Necessary to run jobs on monch.cscs.ch

submit_job(*args, **kwargs)[source]

This method will create a remote directory to store job’s sandbox, and will copy the sandbox in there.

cpod_workflow Module

Created on Feb 17, 2014

author: jakeret

class cpod.remote.cpod_workflow.CPodScript[source]

Bases: gc3libs.cmdline.SessionBasedScript

A session based script for the CPOD simulation using gc3pie

description = u'cpod_workflow'
new_tasks(extra_args)[source]

Iterate over jobs that should be added to the current session. Each item yielded must have the form (jobname, cls, args, kwargs), where:

  • jobname is a string uniquely identifying the job in the session; if a job with the same name already exists, this item will be ignored.
  • cls is a callable that returns an instance of gc3libs.Application when called as cls(*args, **kwargs).
  • args is a tuple of arguments for calling cls.
  • kwargs is a dictionary used to provide keyword arguments when calling cls.

This method is called by the default process_args(), passing self.extra as the extra parameter.

The default implementation of this method scans the arguments on the command-line for files matching the glob pattern self.input_filename_pattern, and for each matching file returns a job name formed by the base name of the file (sans extension), the class given by self.application, and the full path to the input file as sole argument.

If self.instances_per_file and self.instances_per_job are set to a value other than 1, for each matching file N jobs are generated, where N is the quotient of self.instances_per_file by self.instances_per_job.

See also: process_args()

pre_run()[source]

Perform parsing of standard command-line options and call into parse_args() to do non-optional argument processing.

setup_options()[source]

Override this method to add command-line options.

version = u'1.1'
class cpod.remote.cpod_workflow.CPodWorkflow(cpodConfPath, deltaConfig, seed, **extra_args)[source]

Bases: gc3libs.workflow.StagedTaskCollection

A stage based workflow ensuring to call the CPOD apps in correct order

Parameters:
  • cpodConfPath – path to the cpod ini
  • deltaConfig – path to a delta config
  • seed – the random seed to use
  • exta_args – a kwarg passed to gc3pie
stage0()[source]
stage1()[source]
class cpod.remote.cpod_workflow.ParallelSimCreationCollection(cpodConfPath, deltaConfig, simCount, **extra_args)[source]

Bases: gc3libs.workflow.ParallelTaskCollection

Parameters:
  • cpodConfPath –
  • deltaConfig –
  • simCount –

remote_cpod Module

Created on Feb 12, 2014

author: jakeret

class cpod.remote.remote_cpod.ArgumetFactory[source]

Bases: object

static createGadgetArguments(remoteSimulationName)[source]
static createRockstarArguments(remoteSimulationName)[source]
static createSlurmCoreArguments()[source]
class cpod.remote.remote_cpod.RemoteGadget2App(simManager, local_simulation_path, **extra_args)[source]

Bases: gc3libs.Application

App implementation calling MUSIC and launching a mpirun for GADGET2

class cpod.remote.remote_cpod.RemoteRockstarApp(simManager, local_simulation_path, **extra_args)[source]

Bases: gc3libs.Application

App implementation calling ROCKSTAR

terminated()[source]

Called when the job state transitions to TERMINATED, i.e., the job has finished execution (with whatever exit status, see returncode) and the final output has been retrieved.

The location where the final output has been stored is available in attribute self.output_dir.

The default implementation does nothing, override in derived classes to implement additional behavior.

rockstar_setup Module

Created on Mar 3, 2014

author: jakeret

cpod.remote.rockstar_setup.setup(output_path)[source]

simulation_facade Module

Created on Feb 19, 2014

author: jakeret

class cpod.remote.simulation_facade.SimulationFacade[source]

Bases: object

A facade hiding the loading process of remotely stored CPOD simulations and launching the generation of new simulations on a cluster

load_simulation_by_config_file(deltaConfigPath, minSimulations=1, maxSimulations=None)[source]

Loads CPOD simulations which match the given delta configuration

Parameters:
  • deltaConfigPath – the path to the delta config to use
  • minSimulations – default: 1, min number of simulations
  • maxSimulations – (optional) max number of simulations. If not specified, all matching simulations will be loaded
load_simulation_by_config_parser(deltaConfigParser, minSimulations=1, maxSimulations=None)[source]

Loads CPOD simulations which match the given delta configuration parser

Parameters:
  • deltaConfigParser – the parser to use
  • minSimulations – default: 1, min number of simulations
  • maxSimulations – (optional) max number of simulations. If not specified, all matching simulations will be loaded

utils Module

Created on Feb 17, 2014

author: jakeret

cpod.remote.utils.patch_gc3_pie_resource(core, cfg, name, lrms_cls)[source]

Patches the lrms resource of gc3pie

Parameters:
  • core – the core instance to patch
  • cfg – the config to use
  • name – name of the resource to patch
  • lrms_cls – the class of the lrms to use for patching