What is a common technique to prevent overfitting in machine learning?

Regularization methods, such as Lasso and Ridge regression, are common techniques designed to prevent overfitting in machine learning models. Overfitting occurs when a model learns to capture noise in the training data instead of the underlying pattern, leading to poor generalization to new, unseen data.

Regularization works by adding a penalty term to the loss function that the model tries to minimize. In Lasso regression, this penalty is the absolute value of the coefficients, which can lead to some coefficients being exactly zero, effectively performing feature selection. Ridge regression, on the other hand, adds a penalty based on the squared values of the coefficients, which helps to shrink the coefficient values towards zero, ultimately reducing model complexity. Both approaches help to simplify the model, promoting better generalization and reducing the likelihood of overfitting.

In contrast, while cross-validation techniques are valuable for assessing model performance and ensuring that a model is not overfitting during training, they do not inherently prevent overfitting on their own. Adding more features to the model can sometimes lead to overfitting by providing the model with more complexity without improving its predictive power. Using more training data may help to mitigate overfitting, but it is not a direct method of regularization and