Question - Explain the k-Fold cross-validation technique with the algorithm.
Answer -
k-Fold CV is a method that minimizes the disadvantages of the hold-out method. k-Fold introduces a new way of splitting the dataset which helps to overcome the “test only once bottleneck”. The algorithm of the k-Fold technique:
- Firstly, select a number of folds – k. Usually, k is 5 or 10 but you can select any number which is less than the dataset’s length.
- Secondly, divide the dataset into k equal (if possible) parts (they are called folds)
- Then, select k – 1 folds which will be the training set. The remaining fold will be the test set
- Fourthly, train the model on the training set. On each iteration of cross-validation, you must train a new model independently of the model trained on the previous iteration
- Then, validate on the test set After that, save the result of the validation
- Now, repeat steps 3 – 6 k times.
- Every time use the remaining fold as the test set.
In the end, you should have validated the model on every fold that you have. Lastly, for having the final score average the results that you got on step 6.