Technical Question
How can you tune the hyper parameters of XGboost algorithm?

Question

Technical Question

How can you tune the hyper parameters of XGboost algorithm?

Accepted Answer

The overall parameters have been divided into 3 categories by XGBoost authors:

General Parameters: Guide the overall functioning
Booster Parameters: Guide the individual booster (tree/regression) at each step
Learning Task Parameters: Guide the optimization performed

The various steps to be performed for Parameter Tuning are:

1) Choose a relatively high learning rate. Generally a learning rate of 0.1 works but somewhere between 0.05 to 0.3 should work for different problems. Determine the optimum number of trees for this learning rate. XGBoost has a very useful function called as “cv” which performs cross-validation at each boosting iteration and thus returns the optimum number of trees required.

2) Tune tree-specific parameters ( max_depth, min_child_weight, gamma, subsample, colsample_bytree) for decided learning rate and number of trees. Note that we can choose different parameters to define a tree and I’ll take up an example here.

3) Tune regularization parameters (lambda, alpha) for xgboost which can help reduce model complexity and enhance performance.

4) Lower the learning rate and decide the optimal parameters .

Accepted Answer

we can change the max depth of the tree according to the model complexity and, choose the number of the decision tree.

we can use grid search cv to obtain the best hyperparameters according to our given dataset because datasets also play a significant role in obtaining the best hyperparameters, because if we looking to run the same model for different datasets then we do just tune the hyperparameters don't need to train a model from scratch to save time and redundant work.

There are several hyperparameters which we can tune to , like cross-validation ,early stoping ,

Top Walmart Data Scientist interview questions & answers

Popular interview questions of Data Scientist

Top HR questions asked in Walmart Data Scientist