4.7. Select Debian 8 as resource

Compute nodes are currently under operating system upgrade process from Debian 7.9 to Debian 8.6; all nodes are in the same cluster and compute queues of the Open Grid Scheduler. 
 
Main differences and news:
  • - Last stable release of Debian 8 Release 6 (September 17th, 2016).
  • - Open Grid Scheduler integration.
  • - Same user homes for all nodes. 
  • - New software repository compiled on demand for HPC cluster.
    • - Python 3.5.3 (SimpleITK, tensorflow, theano, scipy, lmdb, pandas, ...)
    • - GCC compiler 6.2
    • - CUDA toolkit 8.0.4
    • - OpenMPI 2.0.1
    • - lmod 5.1.5
    • - 3D-Caffe
    • [If you need more software, just request it by CAU]
 
By default this systems cannot be selected, unless otherwise is stated; this guideline provide you the instructions to connect and compute in Debian 8.
 
Debian 8 is a resource of the cluster named as 'debian8' or shortcut 'd8', to use that it must be specified ondemand. 
 
Some common resources of the cluster:
- cpu
​- slots
- gpu
- mem_used and mem_free
- h_cpu and h_vmem
- debian8

Recomendation: First of all you should keep in mind if you have compiled software in your home, compiled software in Debian 8 might not work in older releases. Our recomendation is execute in the system that you compiled your software, if not, you must recompile.


* Interactive session:

To request an interactive session, first of all you must be logged in login node. Then execute next command with 'd8' resource:
<your_user>@login:~$ qrsh -l d8

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.
Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Wed Jan 18 09:08:25 2017 from 192.168.7.6
<your_user>@node11:~$
Grid Scheduler avaluate your needs and select a node with this resource (interactive tasks can be checked as usual):
<your_user>@node11:~$ qstat

job-ID  prior   name       user         state submit/start at     queue             slots ja-task-ID 
----------------------------------------------------------------------------------------------
3832099 0.44872 QRLOGIN    <your_user>      r     01/18/2017 09:28:10 inter.q@node11               1        

* Batch session:

An example of file script to request (for example) gpu, debian8 and default.q queue:
 
#!/bin/bash
#$ -N CudaSample
#$ -l gpu=1
#$ -l debian8
#$ -q default.q
#$ -e $HOME/logs/$JOB_NAME-$JOB_ID.err
#$ -o $HOME/logs/$JOB_NAME-$JOB_ID.out

module load cuda/8.0
/soft/cuda/NVIDIA_CUDA-8.0_Samples/0_Simple/vectorAdd/vectorAdd
hostname
 
Grid Scheduler evaluate your needs and select a node to compute your batch job:
<your_user>@login:~/Scripts$ qstat

job-ID  prior   name       user         state submit/start at     queue            slots ja-task-ID 
---------------------------------------------------------------------------------------------------
3832177 0.30833 CudaSample <your_user>      r     01/18/2017 09:35:25 default.q@node11            1       
Result of the job executed:
<your_user>@login:~/Scripts$ cat ../logs/CudaSample-3832177.*

[Vector addition of 50000 elements]
Copy input data from the host memory to the CUDA device
CUDA kernel launch with 196 blocks of 256 threads
Copy output data from the CUDA device to the host memory
Test PASSED
Done
node11