What are hyperparameters in machine learning?

Hyperparameters are crucial components in machine learning as they define the configurations that control the learning process of an algorithm. Unlike parameters that the model learns during training, hyperparameters are set before the training begins and dictate aspects such as the learning rate, the number of hidden layers in a neural network, the batch size, and the number of trees in a random forest, among others. Adjusting these hyperparameters can significantly influence the performance and effectiveness of the model.

Understanding hyperparameters helps practitioners optimize the model's learning process and performance, as tuning them can lead to better generalization on unseen data. Hyperparameter tuning is typically performed using techniques like grid search or random search to find the best combination for a given dataset and task, indicating their foundational role in shaping how a model learns.

The other options do not correctly capture the essence of hyperparameters. For instance, the idea of user-defined values that never change doesn’t reflect their dynamic nature in training configurations. Output variables that report performance refer to metrics derived after the model has been trained, while parameters that are automatically optimized by the model usually pertain to the weights and biases learned during training, rather than the hyperparameters set beforehand.