Whenever we are trying to build any neural network model, the model training phase is the most resource-consuming job. Each iteration of model training comprises thousands (or even more) of matrix multiplication operations taking place. If there are less than around 1 lakh parameters in a neural network model, then it would not take more than a few minutes (or few hours at most) to train.  But when we have millions of parameters, that is when our sizable computers would probably give up. This is where GPUs come into the picture. GPUs (Graphics Processing Units) are nothing but CPUs but with more ALUs (Arithmetic logic units) than our normal CPUs which are specifically meant for this kind of heavy mathematical computation.