Main interface class to access workflow functions. More...
#include <gridmd.h>
Public Member Functions | |
| gmManager (int mode=gmMODE_LOCAL) | |
| Constructor. | |
| ~gmManager () | |
| Destructor. | |
| int | init (int argc, char **argv, const string &name_="gmexp") |
| Numerical experiment initialization. | |
| int | add_resource (const gmResourceDescr &rdescr, const string &name="") |
| Adds the compute resource into the remote resource description list Default resource name is the same as hostname in the description. | |
| int | load_resources (const char *filename) |
| Adds compute resources found in the given XML file to the list of resources. | |
| int | save_resources (const char *filename) |
| Saves current list of compute resources into XML file. | |
| int | write_graph () |
| Writes the graph to corresponding file(s) (graphwiz, xml, etc). | |
| int | set_link_files (int file_flags=0) |
| Bit flags for temporary files handling modes (see gmTMP_FILES). | |
| int | set_graphviz_flags (int flags=gmGV_ALL) |
| Sets graphviz output flags (combination of gmGV_FIELDS). | |
| gmGraph * | get_graph () |
| Returns the graph object (for use in skeleton extensions only). | |
| int | set_execution (int flags=gmEXE_LOCAL) |
| Set execution mode (one of gmEXE_MODES). | |
| int | get_mode () const |
| Returns current operation mode (see gmMODES). | |
| int | get_recursion () const |
| Returns current gridmd_main recurison level between 0 and MAX_RECURSION. | |
| int | begin_distributed () |
| Here begins the distributed portion of gridmd_main(...) that ends with end_distributed(). | |
| int | end_distributed () |
| Here ends the distributed part of the application that begins with begin_distributed(). | |
| int | mark_distributed () |
| Same as begin_distributed(). | |
| int | execute () |
| Same as end_distributed(). | |
| template<class link_type > | |
| int | mark_node (const string &name, gmSelector input=gmNODE_PREV, const link_type &t=gmHardLink(), int srcport=-1) |
| Adds a node to the graph (construction mode), triggers processing of this node (worker mode). | |
| int | mark_node (const string &name, gmSelector input=gmNODE_PREV) |
| Same as above for hard links. | |
| int | select_nodes (vector< pair< int, int > > &nodes, gmSelector selector, bool input=true, int def_port=-1, int closest_to=-1) |
| Selects nodes/ports matching given selector into a pair vector {nodeid,portid}. | |
| template<class data_t > | |
| int | link (gmSelector from, gmSelector to, int srcport=-1) |
| Links the nodes indicated by selectors from and to with data or hard link. | |
| int | process_cur_node () |
| Returns 1 only in worker mode and only when current node needs to be processed. | |
| template<class data_t > | |
| gmlink_traits< data_t > ::value_type & | node_output (int portid=0) |
| Get output associated with given data port (default is 0) for the current node. | |
| template<class data_t > | |
| const gmlink_traits< data_t > ::value_type & | node_input (int portid=0) |
| Get input associated with given data port (default is 0) for the current node. | |
| gmNodeID | get_curnode () const |
| Get current node ID. | |
| int | check_distrnode () const |
| check the distr_node before advancing nodeid also checks the error status of the last graph, otherwise returns 0 | |
| int | check_subgraph (gmNodeID node) |
| checks that the node with given ID belongs to the subgraph being executed throws exception when the subgraph is finished | |
| bool | in_construction () const |
| Returns TRUE if the manager is in construction mode and in the current distributed section of the code, FALSE otherwise. | |
Protected Member Functions | |
| bool | queue_jobs () |
| Puts the jobs from threads into execution queue. | |
| bool | check_jobs () |
| Returns true if job list is empty or if some of the jobs are finished. | |
| void | dump_data () |
| Dumps all repository data. | |
| template<class data_t > | |
| int | _get_data (data_t **data, const string &data_name, int update=1) |
| 1= OK -1 = no file -2 = wrong type | |
Protected Attributes | |
| string | name |
| <name of the current experiment | |
| gmNodeID | max_node |
| maximal allowed nodeid (for working mode) | |
| int | distr_node |
| start of the distributed node generation | |
| gmScheduler * | sched |
| Scheduler for managing resources. | |
| int | rec_level |
| Level of recursion in gridmd_main() calls. | |
| int | graph_error |
| Indicates that graph error (deadlock or such) happened during the execution. | |
| int | max_node_restarts |
| Maximum number of restarts after job fails allowed for a node. | |
| int | graph_update |
| Graph write frequency: 0= never, 1=once, 2=after each change. | |
| int | construct_only |
| Flag to iondicate that we want only to construct a graph. | |
| int | wr_flags |
| Flags to iondicate what information is needed in the graph picture. | |
| int | start_as_worker |
| Flag to indicate that girdmd_main is a standalone worker. | |
| int | full_restart |
| Flag to indicate that we have to repeat all local nodes when restarting (-R command line option). | |
Main interface class to access workflow functions.
Currently only one instance of gmManager is possible, which is called gmExperiment. Many of the useful member calls to gmExperiment object may be performed the same way in the global scope (namespace gridmd), for example gmExperiment().mark_node("A") is equivalent to gridmd::mark_node("A") and to gridmd::node("A")
| gmManager::gmManager | ( | int | mode = gmMODE_LOCAL |
) |
Constructor.
The optional mode parameter specifies the mode in which the gmManager is created (one of gmMODES). Normally the default (gmMODE_LOCAL) should be used. The operation modes are then selectd automatically. Most of gmManager functions work dependent on the execution mode. They also should be called from inside of gridmd_main(...) function.
| int gmManager::add_resource | ( | const gmResourceDescr & | rdescr, | |
| const string & | name = "" | |||
| ) |
Adds the compute resource into the remote resource description list Default resource name is the same as hostname in the description.
Operation modes: local
| int gmManager::begin_distributed | ( | ) | [inline] |
Here begins the distributed portion of gridmd_main(...) that ends with end_distributed().
There can be several distributed portions, but nesting of them is not allowed. This function adds the 'start' node to the graph, so processing of the start node (nodeid=0) should be performed if this function returns 1.
Operation modes: local, construction, worker
The course of execution after calling begin_distributed() depends on the execution mode (see set_execution()). In the serial and construction_only execution mode the application proceeds normally. In the remote and local execution modes the main function gridmd_main() is called (recursed) from this function and proceeds in construction or local mode.
| bool gmManager::check_jobs | ( | ) | [protected] |
Returns true if job list is empty or if some of the jobs are finished.
Removes the finished jobs from threads and marks finished nodes as processed (PROC).
| int gmManager::end_distributed | ( | ) | [inline] |
Here ends the distributed part of the application that begins with begin_distributed().
This function adds the 'finish' node to the graph and connects all nodes without outputs constructed so far to the 'finish' node with hard links. The processing of the 'finish' node should be performed if this function returns 1. Note that this node is always local, so it (and all nodes connected with it by incoming hard links) is never transferred to a remote host for execution. Moreover, it is guranteed that the 'finish' node is performed by the same gridmd_main() instance that called begin_distributed() for the first time (the first gridmd_main() instance in fact). So local variables that existed before begin_distributed() call have valid values and can be used when processing the 'finish' node and its hard-linked parents. Note that this is not true for other nodes including the 'start' node.
Operation modes: local, construction, worker
| int gmManager::get_recursion | ( | ) | const [inline] |
Returns current gridmd_main recurison level between 0 and MAX_RECURSION.
This is to take into account when handling global variables.
| bool gmManager::in_construction | ( | ) | const [inline] |
Returns TRUE if the manager is in construction mode and in the current distributed section of the code, FALSE otherwise.
Used to program workflow skeletons for detecting the construction mode.
| int gmManager::init | ( | int | argc, | |
| char ** | argv, | |||
| const string & | name_ = "gmexp" | |||
| ) | [inline] |
Numerical experiment initialization.
Local mode: initializes the experiment passing the command line parameters. For proper worker mode operation parameters argc and argv must be redirected from main() command line arguments. GridMD will interpret the following comand line parameters:
-w<node_list>, where <node_list> is a sequence of numbers delimited by ':' -- a switch indicating worker mode with corresponding nodes execution request;
-r [xmlfile] -- restart flag (manager mode only), xmlfile will be read as saved experiment configuration, 'expname.xml' is assumed when the file is not specified, where expname is the experiment name supplied to gmManager::init().
| name_ | name for the current experiment which will be used to construct job names, and file names for saving configurations/checkpoints. Default is 'gmexp'. Other modes: does nothing |
| int gmManager::link< gmHardLink > | ( | gmSelector | from, | |
| gmSelector | to, | |||
| int | srcport = -1 | |||
| ) | [inline] |
Links the nodes indicated by selectors from and to with data or hard link.
Each of the selectors may have the following values (gmNODES):
>=0 connect with the node having nodeID input,
gmNODE_NONE no links needed
gmNODE_ALL connect with all availbale nodes
gmNODE_PREV connect with the previously constructed node
gmNODE_NOOUTPUTS connect with nodes having no outputs so far
The link type is deduced from the template argument type, use link<gmHardLink> or link<void> for hard links, and link<data_t> for data links envolving transfer of data_t. For the data link a source port of a link may be indicated by a parameter srcport. The corresponding destination port (input port) is added automatically on the destination nodes. Operation modes: construction
| int gmManager::load_resources | ( | const char * | filename | ) |
Adds compute resources found in the given XML file to the list of resources.
Searches for <scheduler> entries at top level of the file and loads resource descriptions from each of the <resource> child found. Operation modes: local
| int gmManager::mark_node | ( | const string & | name, | |
| gmSelector | input = gmNODE_PREV, |
|||
| const link_type & | t = gmHardLink(), |
|||
| int | srcport = -1 | |||
| ) | [inline] |
Adds a node to the graph (construction mode), triggers processing of this node (worker mode).
Should be called inside distributed gridmd_main() section (between begin_distributed() and end_distributed()). The parameters are arbitrary node name and the selector (gmNODES) telling how to conect this node with the previously constructed ones. The connection type is specified by a template parameter link_type, which can be either
gmHardLink to indicate a hard link, or
gmDataLink<data_t> to indicate a data link envolving data of type data_t.
For the data link a source port of a link may be indicated by a parameter srcport. The corresponding destination port (input port) is added automatically on the node being constructed. Selector input may have the following values:
>=0 connect with the node having nodeID input,
gmNODE_NONE no links needed
gmNODE_ALL connect with all availbale nodes
gmNODE_PREV connect with the previously constructed node
gmNODE_NOINPUTS connect with nodes having no inputs so far
gmNODE_NOOUTPUTS connect with nodes having no outputs so far
gmNODE_AUTO same as gmNODE_PREV Operation modes: construction, worker
| const int & gmManager::node_input< void > | ( | int | portid = 0 |
) | [inline] |
Get input associated with given data port (default is 0) for the current node.
The data type must mutch the one indicated by link creation, otherwise the warning is given (at worker mode runtime) and temporary reference (whith garbage data) is returned. Operation modes: worker
The transfer of data corresponding to the input ports is performed by GridMD. This function performs data reading from a file transferred to a destination node and returns the reference to a variable which can be used as input in course of current node processing.
| int & gmManager::node_output< void > | ( | int | portid = 0 |
) | [inline] |
Get output associated with given data port (default is 0) for the current node.
The data type must mutch the one indicated by link creation, otherwise the warning is given (at worker mode runtime) and temporary reference (not transferred anywhere by GridMD) is returned. Operation modes: worker
The transfer of data corresponding to the output ports is performed by GridMD. This function returns a reference to the variable which can be used as output in course of current node processing. When node processing is finished, the corresponding data file is written and transfered to the next destination node.
| int gmManager::process_cur_node | ( | ) | [inline] |
Returns 1 only in worker mode and only when current node needs to be processed.
Otherwise returns 0. This function may be used alternatively to checking the return code of mark_node function when there are additional links to be added to the node before processing.
| bool gmManager::queue_jobs | ( | ) | [protected] |
Puts the jobs from threads into execution queue.
| int gmManager::save_resources | ( | const char * | filename | ) |
Saves current list of compute resources into XML file.
Operation modes: local
| int gmManager::set_execution | ( | int | flags = gmEXE_LOCAL |
) | [inline] |
Set execution mode (one of gmEXE_MODES).
Operation modes: local
Do not mix with operation mode! This parameters determines the way the experiment will be executed:
gmEXE_LOCAL: locally, recursively using the same application
gmEXE_REMOTE: on remote resource(s) which have to be configured by add_resource()
gmEXE_MIXED: mixing remote and local execution based on performance measurements (not implemented yet)
gmEXE_CONSTRUCT_ONLY: locally, skipping the node execution (for testing the graph integrity)
gmEXE_SERIAL: executes nodes locally just as they appear in the graph (in construction time), no recursive calls performed. This mode is not guaranteed to work properly because some links may be created after the node is added to the graph. In this case broken link warning is given at node execution.
| int gmManager::set_graphviz_flags | ( | int | flags = gmGV_ALL |
) | [inline] |
Sets graphviz output flags (combination of gmGV_FIELDS).
Operation modes: local
| int gmManager::set_link_files | ( | int | file_flags = 0 |
) | [inline] |
Bit flags for temporary files handling modes (see gmTMP_FILES).
gmFILES_LOCAL:
Tells to create all link files even in the local mode. Note that restart (-R and -r command line options) in local mode will not produce correct results if this flag is not set.
gmFILES_CLEANUP:
Tells to cleanup all link files that are not in use (after connected nodes are processed). Note that full restart (-R command line option) may not be possible if this flag is on.
Operation modes: local
| int gmManager::write_graph | ( | ) |
Writes the graph to corresponding file(s) (graphwiz, xml, etc).
Operation modes: local, construction
1.7.1