ParaView

You are here: Home > PIConGPU User Documentation > ParaView

---

This is a short introduction for the post processing of our data.

Simulation

Let us assume you created our output with libSplash using the --hdf5.period <N> output flags.

Afterwards, run our small script that analyses your outputs meta data and creates an xdmf file for it:

python pic2xdmf.py -t <PATH-to-your-run>/simOutput/h5

Prepare Tunnels

Assuming your cluster looks more or less like this:

  [YOUR-COMPUTER]
       |               ~
*big-bad-internet*     ~ [ClusterNode001] [ClusterNode002]
       |               ~ [ClusterNode003] [ClusterNode004]
       V               ~ [ClusterNode005] [ClusterNodeXYZ]
  [LOGIN-NODE]         ~~~~~~~ some more firewall'n ~~~~~~~~
       |                       ^
~~~~FIREWALL~~~~              /
       |                  *batch system*
       \_->  [HEAD-NODE] ---/

In case your working at HZDR, replace the words like this:

• LOGIN-NODE -> uts.fz-rossendorf.de
• HEAD-NODE -> hypnos2

Open a Tunnel to the HEAD-NODE:

ssh -f -L 44333:<HEAD-NODE>:22 <user>@<LOGIN-NODE> -N
# this makes logins easier:
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 44333 <user>@localhost

Prepare Start Scripts

Log into the HEAD-NODE

ssh -p 44333 <user>@localhost

and prepare a job script startPV.sh like this one (assuming a CPU only queue):

#PBS -q laser
#PBS -l walltime=01:00:00
#PBS -N pvserver
#PBS -l nodes=8:ppn=32
#PBS -d .

#PBS -o stdout
#PBS -e stderr
echo 'Running reverse pvserver...'

cd .

export MODULES_NO_OUTPUT=1
source ~/picongpu.profile
unset MODULES_NO_OUTPUT
module load tools/mesa/7.8
module load analysis/paraview/4.1.0.laser

#set user rights to u=rwx;g=r-x;o=---
umask 0027
sleep 2

mpiexec --prefix $MPIHOME -x LIBRARY_PATH -x LD_LIBRARY_PATH -npernode 32 -n 256 \
    pvserver --use-offscreen-rendering -rc -ch=hypnos2
# some interesting flags one can use:
#   --mca mpi_yield_when_idle 1
#       reduces load while idle (no busy loop)
#       http://www.open-mpi.org/faq/?category=running#force-aggressive-degraded
#   -am $HOME/openib.conf
#       in case you send HUGE data chunks over infiniband

Start and Connect to the Server

You might have spotted it already above: we are starting parallel data/render servers on the cluster nodes which normally can not be reached from the head node. (E.g. you can not connect via ssh to them.) But there is a trick: the cluster nodes itself can connect reversely to the head node (-rc -ch=<HEAD-NODE>).

Submit the jobs:

  ssh -p 44333 <user>@localhost "/opt/torque/bin/qsub ~/startPV.sh"

Now forward your local listening 11111 to the HEAD-NODE: it can now wait for connections from running pvserver's at the cluster-nodes. Finally, we bind it to the so called wildcard adress for extern connections.

  # to <HEAD-NODE> loopback:
  ssh -p 44333 -f -R 11112:localhost:11111 <user>@localhost -N
  # make public on <HEAD-NODE>
  ssh -p 44333 <user>@localhost  "ssh -g -f -L 11111:localhost:11112 localhost -N"