Topology

This repository contains the registry of OSG projects, site resources, and Virtual Organizations, as well as scripts for managing and displaying this information. The information was sourced from the former OIM site that was hosted at oim.grid.iu.edu. This README contains the following sections:

• Structure of the Registry
• How to Register
  • How to Register Downtime
• Viewing the Registry
• Getting Help

Structure of the Registry

The data is organized as files in YAML format as follows:

• projects/ contains information about research projects that run jobs on OSG. Each project has its own file, named <PROJECT>.yaml

• virtual-organizations/ contains information about Virtual Organizations (VOs). Each VO has its own file, named <VO>.yaml. Additionally, each VO has a file that contains information about WLCG metric reporting groups called REPORTING_GROUPS.yaml.

• topology/ contains information about the topology of the resources that OSG sites provide. Resources such as HTCondor-CE, GridFTP, XRootD, or Squid are collected into "resource groups." Each resource group has its own file, which are further organized by facility, and then by site. Resource group files are named <FACILITY>/<SITE>/<RESOURCEGROUP>.yaml

  For example, the OSG resources in the Center for High Throughput Computing can be found in the following file: topology/University of Wisconsin/CHTC/CHTC.yaml

How to Register

The formatting of the YAML files for the different types of registration data are described in the following template files:

The following data...	Is defined by template file...	And should be copied to location, relative to the Git root directory...
Project	template-project.yaml	projects/<PROJECT NAME>.yaml
Resource Downtime	template-downtime.yaml	topology/<FACILITY>/<SITE>/<RESOURCE GROUP NAME>_downtime.yaml
Resource Topology	template-resourcegroup.yaml	topology/<FACILITY>/<SITE>/<RESOURCE GROUP NAME>.yaml
Virtual Organization	template-virtual-organization.yaml	virtual-organizations/<VO NAME>.yaml

The comments in the template files explain the structure and the meaning of the data.

Note: File and directory names must match the name of your project, VO, facility, site, or resource group, as appropriate. This includes case and spaces.

New registrations

To create a new resource group, project, or VO, please create the YAML file according to the table above, and use the corresponding template file to fill in the appropriate information. If you do not feel comfortable creating the new file yourself, send an email to [email protected] with details about your resource group, project, or VO.

Updating existing registrations

To update the data for your site resources, project, or VO, make and submit your changes using one of the following methods:

• Modify the corresponding YAML file and submit your changes as a GitHub pull request.
• Send an email to [email protected] requesting your desired changes.

For definitions for the various fields, consult the corresponding template file for the type of data you are updating.

How to Register Downtime

Downtime is a period of time for which one or more services you provide are unavailable. You should register downtime if one of these is true:

• your site is part of the WLCG
• your CE is one of the services affected

1. Find the file that should contain downtime information about resources you own. It is named topology/<FACILITY>/<SITE>/<RESOURCE GROUP NAME>_downtime.yaml.

   For example, topology/University of Wisconsin/GLOW/GLOW.yaml has the corresponding downtime file topology/University of Wisconsin/GLOW/GLOW_downtime.yaml.

   If the downtime file does not exist, create it.

   To find out what resource group a host is in, search for the FQDN of the host with something like:

   grep -F "<FQDN>" topology/*/*/*.yaml | grep -Fv _downtime.yaml | grep -Fv SITE.yaml

   If the above command returns nothing, then the host is not registered in the topology data and you don't need to register downtime for it.

2. Add the contents of template-downtime.yaml to the end of the downtime file in the path above.

3. Follow the instructions in the comments to fill out the necessary fields.

   Note: Make sure the info you add matches the formatting and indentation of the template or the other downtime entries in the file. In particular, make sure no additional indentation gets added when pasting in the new data.

4. Submit your changes as a GitHub pull request.

Alternatively, send an email to [email protected] requesting your desired changes.

Viewing the Registry

Registry data can be accessed in two formats: the YAML files can be accessed directly through the GitHub interface or from a clone of the GitHub repository. Alternatively, the registry data can be accessed in XML format at the following URLs:

The following data...	Can be accessed in XML format via URL...
Project	https://my.opensciencegrid.org/miscproject/xml
Resource Downtime	https://my.opensciencegrid.org/rgdowntime/xml
Resource Topology	https://my.opensciencegrid.org/rgsummary/xml
Virtual Organization	https://my.opensciencegrid.org/vosummary/xml

These XML pages are compatible with the XML format once provided by myosg.grid.iu.edu.

Getting Help

If you have any questions or encounter any issues with the registry, please contact open a ticket at https://support.opensciencegrid.org or email us at [email protected].