ESGFNode|InstallationProcedure

1. P2P Data/Compute Node From Scratch Installation

This page contains information extending that documentation.

The installation script is written in BASH and is targeted to run on

• NIX operating systems. In particular the target platform is RH or CentOS,

because of minor command idiosyncrasies, however, the script has been used successfully executed on other *NIX systems. The install script pulls down installation artifacts from our distribution server. All artifacts and even the scripts themselves are verified against posted checksums. If verification fails the script will abort. There is a single master script, however, there is actually a series of scripts that are used to install the ESGF Node stack. The script is able to run at start up via chkconfig, supporting [start|stop|restart|update] directives. The script is also self- updating and will alert you if it has been tampered with. There are many flags that may be used with the script (see usage by using --help|-h options), as it is the central mechanism for controlling the maintenance life cycle of the data node.

The script is the preferred way to install the ESGF Node stack. It is recommended to create an "esg-user" account that is able to issue the command "sudo -s" to become root.

1. 1. Procedure

Unix User

    esg-user

Prerequisites

Make sure epel repository is enabled and rpmforge is disabled, then run

    yum install autoconf automake bison file flex gcc gcc-c++ gettext-devel libtool uuid-devel libuuid-devel libxml2 libxml2-devel libxslt libxslt-devel lsof make openssl-devel pam-devel pax readline-devel tk-devel wget zlib-devel perl-Archive-Tar perl-XML-Parser libX11-devel libtool-ltdl-devel e2fsprogs-devel.x86_64 gcc-gfortran libicu-devel.x86_64 libgtextutils-devel.x86_64 *ExtUtils*

Download the installation script

    cd /usr/local/bin
    wget -O esg-bootstrap  http://198.128.245.140/dist/esgf-installer/esg-bootstrap
    diff <(md5sum esg-bootstrap | tr -s " " | cut -d " " -f 1) <(curl -s  http://198.128.245.140/dist/esgf-installer/esg-bootstrap.md5 | tr -s " " | cut -d " " -f 1) 
    chmod 555 esg-bootstrap
    esg-bootstrap

Start the installation process

    esg-node --version
    [VERIFIED]
    Version: v1.6.2-bushwick_myrtle-release-master
    Release: bushwick_myrtle
    Earth Systems Grid Federation (http://esgf.org)
    ESGF Node Installation Script

    script -a -c "esg-node --type data compute --install" /tmp/esgf_install.log

Use script instead of tee to log the installation. tee can cause configuration files corruption.

Interactive questionnaire #1

    Are you ready to begin the installation? [Y/n] Y

    Configured host IP address does not match available IPs...
    Detected multiple IP addresses bound to this host...
    Please select the IP address to use for this installation
    -------------------------------------------
    [0] : 134.157.179.48
    [1] : 172.20.3.43
    -------------------------------------------    
    select [] > 0

    -------------------------------------------------------
    Welcome to the ESGF Node installation program! :-)

What is the fully qualified domain name of this node? [vesgint-data.ipsl.jussieu.fr]: What is the admin password to use for this installation? (alpha-numeric only) []: Please re-enter password: What is the name of your organization? [jussieu]: Please give this node a "short" name: []: VESGINT-DATA Please give this node a more descriptive "long" name []: vesgint-data.ipsl.jussieu.fr What is the namespace to use for this node? (set to your reverse fqdn - Ex: "gov.llnl") [fr.jussieu.ipsl]: What peer group(s) will this node participate in? (if not sure, use default) [esgf-test]: esgf-prod What is the default peer to this node? [esgf-node1.llnl.gov]: esgf-node.ipsl.fr What is the hostname of the node do you plan to publish to? [esgf-node1.llnl.gov]: esgf-node.ipsl.fr What email address should notifications be sent as? []: [email protected] Is the database external to this node? [y/N]: Please enter the database connection string...

 (form: postgresql://[username]@[host]:[port]/esgcet)

What is the database connection string? postgresql://dbsuper@localhost:5432/esgcet: postgresql:// entered: postgresql://dbsuper@localhost:5432/esgcet What is the (low priv) db account for publisher? [esgcet]: What is the db password for publisher user (esgcet)? []: }}}

Questionnaire interactif 1

Questionnaire interactif 2

Questionnaire interactif 3

Questionnaire interactif 4

Questionnaire interactif 5

Questionnaire interactif 6

Questionnaire interactif 7

Questionnaire interactif 8

Questionnaire interactif 9

Questionnaire interactif 10

Questionnaire interactif 11

Questionnaire interactif 12

Redémarrage du noeud

Ajout de l'IDP à la whitelist. Ajouter cette ligne au fichier esg/config/esgf_idp_static.xm

Configuration du thredds_dataset_roots dans le fichier /esg/config/esgcet/esg.ini

Configuration de la section initialize dans le fichier /esg/config/esgcet/esg.ini

Ajout des lignes suivantes dans le fichier /esg/config/esgcet/esgcet_models_table.txt:

Configuration de l'URL las dans le fichier esg.ini

Redémarrage du noeud

Vérification des web front ends

En cas de non fonctionnement de l'interface web las, redémarrer le noeud autant de fois que nécéessaire:

Génération d'une Certificate Signing Request et Installation du certificat

Préparation

Installation du fichier attaché à cette page localopenssl.cnf dans /etc/grid-security/workdir

Génération de la CSR

Vérifier la CSR puis l'envoyer à l'administrateur de l'IDP: IPSL

Installer le certificat machine signé renvoyé par l'IPSL sur le noeud

Installer le certificat CA envoyé par l'IPSL sur le noeud

Installer la paire de clés

Initialisation

Initialisation esg

1. 1. Directory structure

    esg_root_dir = /esg
    workdir = /usr/local/src/esgf

 Location 

 Description 

${esg_root_dir}

Top level directory location of the ESG configuration files and logs (default /esg).

${esg_root_dir}/backups

Application stack and database data archive location

${esg_root_dir}/config

ESGF configuration files

${esg_root_dir}/content

Thredds catalogs & LAS data files

${esg_root_dir}/data

Top level directory for data (.nc) files

${esg_root_dir}/data.replica

Top level directory for all replicated data from other nodes

${esg_root_dir}/data-index-*

Search index directories

${esg_root_dir}/gridftp_root

Chroot directory for gridftp access to data

${esg_root_dir}/log

ESGF log files

${esg_root_dir}/tools

ESGF tools (currently; esg_usage_parser)

${esg_root_dir}/config/esgcet/esg.ini or ~/.esgcet/esg.ini

esg publisher setup file. (system vs personal install)

%{esg_root_dir}/etc

Ancillary scripts and files

${esg_root_dir}/esgf-install-manifest

Log of all installed components of the application stack (date, name, location, version)

/etc/esg.env

Environment variables required by the script and used in node operation

${workdir}

Installation "scratch" directory for installation-time artifacts; source, helper scripts, et. al.

${workdir}/globus

globus sources

${workdir}/esg

publisher,thredds and other sources

• * *

1. 1. 1. Starting/Stopping etc. the Node

The esg-node installation script is also the boot script

To stop/start or restart or check the status of the node...

    %> esg-node stop
    %> esg-node start
    %> esg-node status
    %> esg-node restart