ParaView
You are here: Home > PIConGPU User Documentation > ParaView
This is a short introduction for the post processing of our data.
Contents
On your local computer, download the ParaView 3.98 binaries and install
Install ssh-askpass, e.g. with
sudo apt-get install ssh-askpassStart the downloaded (and unzipped) paraview binary in bin/ and go to
Connect -> Fetch Servers -> Edit Sources.
Add the lines
# Selected PIConGPU ParaView Servers
pvsc https://raw.githubusercontent.com/ComputationalRadiationPhysics/picongpu/dev/src/tools/share/paraview.pvsc PIConGPU Rendering Queues
and Save. Select the new servers that show up and hit Import Selected.
Let us assume you created our output with libSplash using the --hdf5.period <N> output flags.
Afterwards, run our small script that analyses your outputs meta data and creates an xdmf file for it:
python pic2xdmf.py -t <PATH-to-your-run>/simOutput/h5Note: <SPLASH_INSTALL>/bin/ needs to be in your PYTHONPATH in order for pic2xdmf.py to find the required libSplash python module.
This is the extended technical documentation of the reverse connected remote rendering. If you are not a maintainer, this is not for you.
Assuming your cluster looks more or less like this:
[YOUR-COMPUTER]
| ~
*big-bad-internet* ~ [ClusterNode001] [ClusterNode002]
| ~ [ClusterNode003] [ClusterNode004]
V ~ [ClusterNode005] [ClusterNodeXYZ]
[LOGIN-NODE] ~~~~~~~ some more firewall'n ~~~~~~~~
| ^
~~~~FIREWALL~~~~ /
| *batch system*
\_-> [HEAD-NODE] ---/
In case your working at HZDR, replace the words like this:
-
LOGIN-NODE->uts.fz-rossendorf.de -
HEAD-NODE->hypnos2
Open a Tunnel to the HEAD-NODE:
ssh -f -L 44333:<HEAD-NODE>:22 <user>@<LOGIN-NODE> -N
# this makes logins easier:
ssh-copy-id -i ~/.ssh/id_rsa.pub -p 44333 <user>@localhostLog into the HEAD-NODE
ssh -p 44333 <user>@localhostand prepare a job script startPV.sh like this one (assuming a CPU only queue):
#PBS -q laser
#PBS -l walltime=01:00:00
#PBS -N pvserver
#PBS -l nodes=8:ppn=32
#PBS -d .
#PBS -o stdout
#PBS -e stderr
echo 'Running reverse pvserver...'
cd .
export MODULES_NO_OUTPUT=1
source ~/picongpu.profile
unset MODULES_NO_OUTPUT
module load tools/mesa/7.8
module load analysis/paraview/4.0.1.laser
#set user rights to u=rwx;g=r-x;o=---
umask 0027
sleep 2
mpiexec --prefix $MPIHOME -x LIBRARY_PATH -x LD_LIBRARY_PATH -npernode 32 -n 256 \
pvserver --use-offscreen-rendering -rc -ch=hypnos2
# some interesting flags one can use:
# --mca mpi_yield_when_idle 1
# reduces load while idle (no busy loop)
# http://www.open-mpi.org/faq/?category=running#force-aggressive-degraded
# -am $HOME/openib.conf
# in case you send HUGE data chunks over infiniband
You might have spotted it already above: we are starting parallel data/render servers on the cluster nodes which normally can not be reached from the head node. (E.g. you can not connect via ssh to them.) But there is a trick: the cluster nodes itself can connect reversely to the head node (-rc -ch=<HEAD-NODE>).
Now start ParaView and hit Connect
We are going to Add Server to configure a new reverse connection:
Now go to Configure and add the start-up mode Command:
ssh -p 44333 <user>@localhost "/opt/torque/bin/qsub ~/startPV.sh"
Configure wait for 5,0 seconds below and save.
Al right! Now go to Connect again, select your Server and start with Connect
Now run locally:
# to <HEAD-NODE> loopback:
ssh -p 44333 -f -R 11112:localhost:11111 <user>@localhost -N
# make public on <HEAD-NODE>
ssh -p 44333 <user>@localhost "ssh -g -f -L 11111:localhost:11112 localhost -N"This will forward your local port 11111 which is listening the HEAD-NODE: it can now wait for connections from running pvserver's at the cluster-nodes.
Finally, we bind it to the so called wildcard adress for extern connections.
The last commands can be shortened up with GatewayPorts clientspecified (1) in the HEAD-NODE's /etc/ssh/sshd_config (2) to:
# forward local 11111 port to <HEAD-NODE> and make public for all interfaces
ssh -p 44333 -R :11111:localhost:11111 <user>@localhost -N