Skip to content

Data Model

Jump to bottom
andre-merzky edited this page Dec 29, 2014 · 1 revision

RADICAL-Pilot could use Redis as central persistent data storage, accessed via the SAGA Advert Package. The data model is as follows:

State for all entities is kept in a name space hierarchy, which separates entities for individual users and application sessions:

/sinon/v1/users/<username>/<sid>/

UnitManager and PilotManager are subdirs of that session root, under their respective ID:

/sinon/v1/users/<username>/<sid>/pm.1/
/sinon/v1/users/<username>/<sid>/pm.2/
/sinon/v1/users/<username>/<sid>/pm.3/
/sinon/v1/users/<username>/<sid>/um.1/
/sinon/v1/users/<username>/<sid>/um.2/

Submitting (unscheduled) Units creates entries in the UnitManager directory:

/sinon/v1/users/<username>/<sid>/pm.1/
/sinon/v1/users/<username>/<sid>/pm.2/ 
/sinon/v1/users/<username>/<sid>/pm.3/
/sinon/v1/users/<username>/<sid>/um.1/u.1
/sinon/v1/users/<username>/<sid>/um.2/

Pilot creation adds a pilot dir to the PilotManager sub-tree:

/sinon/v1/users/<username>/<sid>/pm.1/
/sinon/v1/users/<username>/<sid>/pm.2/
/sinon/v1/users/<username>/<sid>/pm.2/p.1/
/sinon/v1/users/<username>/<sid>/pm.3/
/sinon/v1/users/<username>/<sid>/um.1/
/sinon/v1/users/<username>/<sid>/um.1/u.1
/sinon/v1/users/<username>/<sid>/um.2/

When a Pilot is added to a UnitManager, the UnitManager dir gets a Pilot specific subdir, and the UnitManager ID is added in theUnitManager's attribute of the Pilot dir:

/sinon/v1/users/<username>/<sid>/pm.1/
/sinon/v1/users/<username>/<sid>/pm.2/ 
/sinon/v1/users/<username>/<sid>/pm.2/p.1/  {UnitManagers: [um.1]}
/sinon/v1/users/<username>/<sid>/pm.3/
/sinon/v1/users/<username>/<sid>/um.1/
/sinon/v1/users/<username>/<sid>/um.1/u.1
/sinon/v1/users/<username>/<sid>/um.1/p.1/
/sinon/v1/users/<username>/<sid>/um.2/

Scheduling moves these Units into the Pilot directories:

/sinon/v1/users/<username>/<sid>/pm.1/
/sinon/v1/users/<username>/<sid>/pm.2/ 
/sinon/v1/users/<username>/<sid>/pm.2/p.1/  {UnitManagers: [um.1]}
/sinon/v1/users/<username>/<sid>/pm.3/
/sinon/v1/users/<username>/<sid>/um.1/
/sinon/v1/users/<username>/<sid>/um.1/p.1/u.1
/sinon/v1/users/<username>/<sid>/um.2/

A Pilot can be added to multiple UnitManagers, and serve Units from all of them:

/sinon/v1/users/<username>/<sid>/pm.1/
/sinon/v1/users/<username>/<sid>/pm.2/ 
/sinon/v1/users/<username>/<sid>/pm.2/p.1/  {UnitManagers: [um.1, um.2]}
/sinon/v1/users/<username>/<sid>/pm.3/
/sinon/v1/users/<username>/<sid>/um.1/
/sinon/v1/users/<username>/<sid>/um.1/p.1/u.1
/sinon/v1/users/<username>/<sid>/um.2/
/sinon/v1/users/<username>/<sid>/um.2/p.1/u.2

Pilot agents can subscribe to events on a specific Pilot, so they are aware when Pilots get added to new UnitManagers, and when new Units get assigned to them. The UnitManager instances in Sinon can subscribe to events from their Pilots and Units, too, so that they get notified of state changes etc.

The Scheduler lives in its own thread, within the UnitManager instance, and gets information about new Unit submissions via a queue. It is then tasked to move the Units around in the UnitManager subtree, according to some clever scheme.

Move operations are considered atomic. Unit Entries are owned by the parent entity (UM or Pilot), and only that entity is allowed to update state, and to move the unit.

Clone this wiki locally