Skip to content

Preparing for Hyperparameter Tuning - Hyperparameters & Objectives

Once you have identified the best performing model you will need to identify the set of hyperparameters that you would like to tune. You will also need to identify an objective metric to use as the basis for evaluating the relative performance of different hyperparameter settings. Lastly, you will want to exclude models we are no longer considering from the pipeline by skipping all cells for those models.

To perform hyperparameter tuning all parameters must first be introduced as variables and not as static inputs as parameters in the model function call. Secondly for Kale and Katib to consider the hyperparameters for tuning they must be in their own cell and tagged accordingly.

Follow Along

Please follow along in your own copy of our notebook as we complete the steps below.

1. Create Hyperparamter Cell

Create a new cell directly below the first Imports cell.

new-cell-under-imports

Create three new hyperparameters with the respective default values.

hyperparam-cell

NUM_ESTIMATORS = 500
MAX_DEPTH = 3
LEARNING_RATE = 0.01

Click the edit icon and tag the cell with Pipeline Parameters.

hyperparam-cell-tagged

2. Parametize XGB Model

Scroll down to the cell where the xgb model is created and edit the cell to parametize the XGB model creation.

xgb1 = XGBRegressor()
parameters = {'n_estimators': [NUM_ESTIMATORS], 'max_depth': [MAX_DEPTH], 'learning_rate': [LEARNING_RATE]}
xgb_grid = GridSearchCV(xgb1,parameters,cv = 2)

xgb_grid.fit(x,y)

parametized xgboost

3. Set Pipeline Metrics for Optimization

In order to perform an experiment within Kale you will need to add print(eval_metric) statements into the final cell of the notebook. You will also need to tag this cell with Pipeline Metrics.

Create a new cell at the bottom of the Notebook and enter

print(xgb_r2_score)

creating pipeline metrics

You will complete this cell as part of the next lab.

4. Skip Unnecessary Cells

You will need to omit all unnecessary steps from your pipeline given that at this point you know the model you want to work with. Skipping these cells will reduce the size of the KubeFlow pipeline that is created for experimentation and subsequently speed up the hyperparameter tuning process.

Within the Build Models section scroll down to the train_lgbm cell. Edit this cell and change the Kale tag to skip.

skip train lgbm

Still in the Build Models section scroll down to the eval_lgbm cell. Edit this cell and change the Kale tag to skip.

skip eval lgbm

You will complete skipping cells are part of the next lab.