ID=103

• Dataset: 6n4o
• Software: PMEMD.cuda.MPI (amber/20.9-20.15-gomklc-2020a-avx512)
• Resource: 1 tasks, 1 cores, nodes, 1 GPUs, with NVLink
• CPU: Xeon Silver 4216 (Cascade Lake), 2.1 GHz
• GPU: Tesla-V100-SXM2-32GB, 5 cores/GPU
• Simulation speed: 41.4786 ns/day
• Efficiency: 100.0 %
• Site: Cedar
• Date: Jan. 14, 2022, 3 p.m.
• Submission script:

  #!/bin/bash
  #SBATCH -c1 --gpus=v100l:1 
  #SBATCH --mem-per-cpu=2000 --time=1:0:0
  INPFILE=pmemd_prod.in
  LOGFILE=md_${SLURM_JOBID}.log 
  module --force purge
  ml StdEnv/2020  gcc/9.3.0 cuda/11.0 openmpi/4.0.3 amber/20.9-20.15 
  pmemd.cuda -O -i $INPFILE -o $LOGFILE -p prmtop.parm7 -c restart.rst7

• Notes:

  Benchmark submission:

  #!/bin/bash
  #SBATCH -c1 --gpus=v100l:1
  #SBATCH --mem-per-cpu=2000 --time=1:0:0
  # Usage: sbatch submit.cuda.sh
  INPFILE=pmemd_prod.in
  STEPS=10000
  # End of user input
  TMPFILE=tf_${SLURM_JOBID}
  LOGFILE=md_${SLURM_JOBID}.log
  module --force purge
  ml StdEnv/2020 gcc/9.3.0 cuda/11.0 openmpi/4.0.3 amber/20.9-20.15
  # Print resource info
  echo ${SLURM_NODELIST} running on ${SLURM_CPUS_PER_TASK} cores
  cat /proc/cpuinfo | grep "model name" | uniq
  nvidia-smi --query-gpu=gpu_name --format=csv
  pmemd.cuda -O -i $INPFILE -o $LOGFILE -p prmtop.parm7 -c restart.rst7
  grep "NonSetup CPU time" $LOGFILE > $TMPFILE
  # Print average of three runs.
  echo -n "ns/day:"
  awk -v steps=$STEPS '{total += $5; count++ } END { print count*3.6*2.4*steps*0.01/total}' $TMPFILE
  rm $TMPFILE $LOGFILE mdinfo mdcrd restrt

• Simulation input file:

  Benchmark
   &cntrl
    imin=0,irest=1,ntx=5,
    nstlim=20000,dt=0.001,
    ntc=2,ntf=2,
    cut=8.0, 
    ntpr=100, ntwx=1000,
    ntb=2, ntp=1, taup=2.0,
    ntt=3, gamma_ln=2.0, temp0=300.0,
   /
   &ewald
    nfft1=128,
    nfft2=128,
    nfft3=128,
   /