Do Gradient Boosted Decision Trees begin with high or low bias?

Gradient Boosted Decision Trees (GBDT) typically begin with high bias and low variance. This stems from the nature of the boosting algorithm, which constructs a model in a sequential fashion. Initially, the first decision tree is built on the data, and it is relatively simple. As a result, this initial tree may not capture the underlying patterns in the data very well, leading to high bias.

Bias refers to the error introduced by approximating a real-world problem, which may have complex relationships, with a simplified model. Since the first tree is a rudimentary model, it tends to underfit the data, thus exhibiting high bias. Variance, on the other hand, measures how sensitive a model is to fluctuations in the training data. With only one tree to begin with, the variance is low because it does not change significantly with different datasets—there's little room for overfitting.

As the GBDT model trains further, more trees are added to reduce bias and capture more complex patterns in the data. This leads to a gradual decline in bias and an increase in variance as the model becomes more intricate. However, at the very start of the training process, the effect is characterized by high bias and low variance, making the provided answer