CUDA-C GPU port of VASP: Difference between revisions

Latest revision as of 08:27, 20 October 2023

Warning: As of VASP.6.3.0, the CUDA-C GPU port of VASP is deprecated. Switch to the OpenACC GPU port of VASP.

Several core algorithms of VASP have been ported to run on GPU-accelerated hardware (as of VASP.5.4.1.05Feb16).

Explicity ported to run on GPU-accelerated hardware

Electronic minimisation: the blocked-Davidson and RMM-DIIS algorithms (ALGO= Normal, Fast, and VeryFast).
Hybrid functionals: the action of the Fock-exchange potential on the wave functions (LHFCALC=.TRUE.).

Unsuported (for now)

LREAL=.FALSE. is currently unsuported . The GPU port of VASP requires the use of real-space-projection operators (i.e., LREAL= Auto | .TRUE.).
LCALCEPS=.TRUE.
NCORE ≠ 1 (or equivalently: NPAR ≠ #of-MPI-ranks / KPAR) is not supported at the moment. The GPU port of VASP requires NCORE=1 (default).
Using scaLAPACK for the orthonormalization of the wave functions is not supported by the GPU port of VASP. Actually, this particular operation has been ported to the GPU (just not by means of scaLAPACK). If you have compiled your code with -DscaLAPACK you have to set:

LSCAAWARE = .FALSE.

in your INCAR to avoid the use of scaLAPACK for the orthonormalization of the wave functions.

The gamma-only version of VASP has not been ported to GPU (yet).

N.B.: The GPU port of VASP is freely available to VASP5-licensees.

Hardware/Software requirements

Hardware requirements

Required GPU Architecture is Kepler or newer:

Tesla K40 or Tesla K80, with 12 and 24 GB memory respectively, are strongly recommended
Tesla K20 and Tesla K20X, with 5 GB and 6 GB respectively, may run out of memory on larger problems

Recommended software stack

CUDA toolkit, newer is better but anything >=6.5 should work

Usage

Building

See Installing VASP and do not forget the patch(es).

Running

People

The GPU port of VASP only exist because of the excellent work of the following people:

Maxwell Hutchinson (University of Chicago) and Mike Widom (Carnegie Mellon)
Xavier Rozanska and Paul Fleurat-Lessard (ENS-Lyon)
Mohamed Hacene, Ani Anciaux-Sedrakian, Diego Klahr, and Thomas Guignon (IFPEN)
Jeroen Bedorf, Przemyslaw Tredak, Dusan Stosic, Arash Ashari, Paul Springer, Darko Stosic, Christoph Angerer, and Sarah Tariq (NVIDIA)

Publications

Accelerating VASP Electronic Structure Calculations Using Graphic Processing Units, M. Hacene et al., J. Comput. Chem. 33, 2581 (2012).
VASP on a GPU: Application to exact-exchange calculations of the stability of elemental boron, M. Hutchinson and W. Widom, Comput. Phys. Comm. 183, 1422 (2012).
Electronic Structure Calculations on Graphics Processing Units: From Quantum Chemistry to Condensed Matter Physics, Ross Walker and Andreas Goetz (Editors), John Wiley & Sons, Inc., UK.
Speeding up plane-wave electronic-structure calculations using graphics-processing units, S. Mainz et al., Comput. Phys. Comm. 182, 1421 (2011).

Additional information

Related Tags and Sections

ALGO, LHFCALC, LREAL, LCALCEPS, Installing VASP

@@ Line 1: / Line 1: @@
+{{NB|warning|As of VASP.6.3.0, the CUDA-C GPU port of VASP is deprecated. Switch to [[OpenACC GPU port of VASP|the OpenACC GPU port of VASP]].}}
 Several core algorithms of VASP have been ported to run on GPU-accelerated hardware (as of VASP.5.4.1.05Feb16).
 ;Explicity ported to run on GPU-accelerated hardware:
-* {{TAG|ALGO}}= Normal, Fast, and VeryFast have been ported to GPU.
+* Electronic minimisation: the blocked-Davidson and RMM-DIIS algorithms ({{TAG|ALGO}}= Normal, Fast, and VeryFast).
-* {{TAG|LHFCALC}}=.TRUE.
+* Hybrid functionals: the action of the Fock-exchange potential on the wave functions ({{TAG|LHFCALC}}=.TRUE.).
 ;Unsuported (for now)
-* {{TAG|LREAL}}=.TRUE. is currently unsuported . The GPU port of VASP requires the use of real-space-projection operators (i.e., {{TAG|LREAL}}= Auto {{!}} .TRUE.).
+* {{TAG|LREAL}}=.FALSE. is currently unsuported . The GPU port of VASP requires the use of real-space-projection operators (i.e., {{TAG|LREAL}}= Auto {{!}} .TRUE.).
 * {{TAG|LCALCEPS}}=.TRUE.
+* {{TAG|NCORE}} &ne; 1 (or equivalently: {{TAG|NPAR}} &ne; ''#of-MPI-ranks'' / {{TAG|KPAR}}) is not supported at the moment. The GPU port of VASP requires {{TAG|NCORE}}=1 (default).
+* Using scaLAPACK for the orthonormalization of the wave functions is not supported by the GPU port of VASP. Actually, this particular operation has been ported to the GPU (just not by means of scaLAPACK). If you have compiled your code with <code>-DscaLAPACK</code> you have to set:
+ LSCAAWARE = .FALSE.
+:in your {{FILE|INCAR}} to avoid the use of scaLAPACK for the orthonormalization of the wave functions.
 * The gamma-only version of VASP has not been ported to GPU (yet).
 '''N.B.''': The GPU port of VASP is freely available to VASP5-licensees.
+== Hardware/Software requirements ==
+=== Hardware requirements ===
+Required GPU Architecture is Kepler or newer:
+* Tesla K40 or Tesla K80, with 12 and 24 GB memory respectively, are strongly recommended
+* Tesla K20 and Tesla K20X, with 5 GB and 6 GB respectively, may run out of memory on larger problems
+=== Recommended software stack ===
+* CUDA toolkit, newer is better but anything >=6.5 should work
+== Usage ==
+=== Building ===
+* See [[Installing VASP]] and do not forget the [[Installing VASP#For vasp.5.4.1.05Feb16 (with GPU support)|patch(es)]].
+=== Running ===
 == People ==
@@ Line 19: / Line 42: @@
 *Xavier Rozanska and Paul Fleurat-Lessard (ENS-Lyon)
 *Mohamed Hacene, Ani Anciaux-Sedrakian, Diego Klahr, and Thomas Guignon (IFPEN)
-*Jeroen Bedorf, Przemyslaw Tredak, Dusan Stosic, Arash Ashari, Paul Springer Darko Stosic, Christoph Angerer, and Sarah Tariq (NVIDIA)
+*Jeroen Bedorf, Przemyslaw Tredak, Dusan Stosic, Arash Ashari, Paul Springer, Darko Stosic, Christoph Angerer, and Sarah Tariq (NVIDIA)
 == Publications ==
 * ''Accelerating VASP Electronic Structure Calculations Using Graphic Processing Units'', M. Hacene ''et al.'', [http://dx.doi.org/10.1002/jcc.23096 J. Comput. Chem. 33, 2581 (2012)].
 * ''VASP on a GPU: Application to exact-exchange calculations of the stability of elemental boron'', M. Hutchinson and W. Widom, [http://dx.doi.org/10.1016/j.cpc.2012.02.017 Comput. Phys. Comm. 183, 1422 (2012)].
+* [http://www.wiley.com/WileyCDA/WileyTitle/productCd-1118661788.html ''Electronic Structure Calculations on Graphics Processing Units: From Quantum Chemistry to Condensed Matter Physics''], Ross Walker and Andreas Goetz (Editors), John Wiley & Sons, Inc., UK.
 * ''Speeding up plane-wave electronic-structure calculations using graphics-processing units'', S. Mainz ''et al.'', [http://dx.doi.org/10.1016/j.cpc.2011.03.010 Comput. Phys. Comm. 182, 1421 (2011)].
 == Additional information ==
-* [http://images.nvidia.com/events/sc15/SC5120-vasp-gpus.html The presentation by GPU developer, Max Hutchinson, at SC15.]
+* [http://images.nvidia.com/events/sc15/SC5120-vasp-gpus.html The presentation at SC15 by GPU developer Max Hutchinson.]
 * [http://on-demand.gputechconf.com/gtc/2014/video/S4692-vasp-accelerating-plane-wave-dft-codes.mp4  GTC 2013 audio & video presentation on the development of GPU-accelerated VASP.]
 * [https://www.nsc.liu.se/~pla/ Dr. Peter Larsson's blog (National Supercomputer Centre at Linköping University, Sweden).]
+== Related Tags and Sections ==
+{{TAG|ALGO}},
+{{TAG|LHFCALC}},
+{{TAG|LREAL}},
+{{TAG|LCALCEPS}},
+[[Installing_VASP|Installing VASP]]
 ----
-[[The_VASP_Manual|Contents]]
-[[Category:VASP]][[Category:Installation]]
+[[Category:VASP]][[Category:Performance]][[Category:GPU]][[Category:Installation]]