Btukfyl

I am an actuarial student preparing for an upcoming predictive analytics exam in December. Part of an exercise is to build a model using boosting with caret and xgbTree. See the code below, the caravan dataset is from the ISLR package:

library(caret)

library(ggplot2)

set.seed(1000)

data.Caravan <- read.csv(file = "Caravan.csv")





data.Caravan$Purchase <- factor(data.Caravan$Purchase)

levels(data.Caravan$Purchase) <- c("No", "Yes")





data.Caravan.train <- data.Caravan[1:1000, ]

data.Caravan.test <- data.Caravan[1001:nrow(data.Caravan), ]

grid <- expand.grid(max_depth = c(1:7),

                    nrounds = 500,

                    eta =  c(.01, .05, .01),

                    colsample_bytree = c(.5, .8),

                    gamma = 0,

                    min_child_weight = 1,

                    subsample = .6)



control <- trainControl(method = "cv", 

                        number = 4,

                        classProbs = TRUE,

                        sampling = c("up", "down"))



caravan.boost <- train(formula = Purchase ~ .,

                       data =  data.Caravan.train, 

                       method = "xgbTree", 

                       metric = "Accuracy",

                       trControl = control, 

                       tuneGrid = grid)

The definitions in expand.grid and trainControl were specified by the problem, but I keep getting an error:

Error: sampling methods are only implemented for classification problems

If I remove the sampling method from trainControl, I get a new error that states "Metric Accuracy not applicable for regression models". If I remove the Accuracy metric, I get an error stating

cannnot compute class probabilities for regression" and "Error in names(res$trainingData) %in% as.character(form[[2]]) : argument "form" is missing, with no default"

Ultimately the problem is that caret is defining the problem as regression, not classification, even though the target variable is set as a factor variable and classProbs is set to TRUE. Can someone explain how to tell caret to run classification and not regression?

caret::train does not have a formula argument, but rather a form argument in which you specify the formula. So for instance this works:

caravan.boost <- train(form = Purchase ~ .,

                       data =  data.Caravan.train, 

                       method = "xgbTree", 

                       metric = "Accuracy",

                       trControl = control, 

                       tuneGrid = grid)



#output:

eXtreme Gradient Boosting 



1000 samples

  85 predictor

   2 classes: 'No', 'Yes' 



No pre-processing

Resampling: Cross-Validated (4 fold) 

Summary of sample sizes: 751, 749, 750, 750 

Addtional sampling using up-sampling



Resampling results across tuning parameters:



  eta   max_depth  colsample_bytree  Accuracy   Kappa     

  0.01  1          0.5               0.7020495  0.10170007

  0.01  1          0.8               0.7100335  0.09732773

  0.01  2          0.5               0.7730581  0.12361444

  0.01  2          0.8               0.7690620  0.11293561

  0.01  3          0.5               0.8330506  0.14461709

  0.01  3          0.8               0.8290146  0.06908344

  0.01  4          0.5               0.8659949  0.07396586

  0.01  4          0.8               0.8749790  0.07451637

  0.01  5          0.5               0.8949792  0.07599005

  0.01  5          0.8               0.8949792  0.07525191

  0.01  6          0.5               0.9079873  0.09766492

  0.01  6          0.8               0.9099793  0.10420720

  0.01  7          0.5               0.9169833  0.11769151

  0.01  7          0.8               0.9119753  0.10873268

  0.05  1          0.5               0.7640699  0.08281792

  0.05  1          0.8               0.7700580  0.09201503

  0.05  2          0.5               0.8709909  0.09034807

  0.05  2          0.8               0.8739990  0.10440898

  0.05  3          0.5               0.9039792  0.12166348

  0.05  3          0.8               0.9089832  0.11850402

  0.05  4          0.5               0.9149793  0.11602447

  0.05  4          0.8               0.9119713  0.11207786

  0.05  5          0.5               0.9139633  0.11853793

  0.05  5          0.8               0.9159754  0.11968085

  0.05  6          0.5               0.9219794  0.11744643

  0.05  6          0.8               0.9199794  0.12803204

  0.05  7          0.5               0.9179873  0.08701058

  0.05  7          0.8               0.9179793  0.10702619



Tuning parameter 'nrounds' was held constant at a value of 500

Tuning parameter 'gamma' was held constant

 at a value of 0

Tuning parameter 'min_child_weight' was held constant at a value of 1

Tuning

 parameter 'subsample' was held constant at a value of 0.6

Accuracy was used to select the optimal model using the largest value.

The final values used for the model were nrounds = 500, max_depth = 6, eta = 0.05, gamma =

 0, colsample_bytree = 0.5, min_child_weight = 1 and subsample = 0.6.

You can also use the non formula interface in which you specify the x and y separately:

caravan.boost <- train(x = data.Caravan.train[,-ncol(data.Caravan.train)],

                       y =  data.Caravan.train$Purchase, 

                       method = "xgbTree", 

                       metric = "Accuracy",

                       trControl = control, 

                       tuneGrid = grid)

do note that these two ways of specification do not always produce the same result when there are factor variables in x since the formula interface calls model.matrix for most algorithms.

To get the data:

library(ISLR)

data(Caravan)