ROC curve of the testing dataset-CodePudding

I am using caret packages to compare different models.

After training a model, how to plot the ROC curve and find the area.

# Split data 
a<- createDataPartition(data$target, p = .8, list = FALSE)
train <- data[ a,]
test <- data[-a,]

myControl = trainControl(
    method = "cv",
    summaryFunction = twoClassSummary,
    classProbs = TRUE,
    verboseIter = FALSE,
)

model_knn = train(
    target ~ .,
    train,
    method = "knn",
    metric = "ROC",
    tuneLength = 10,
    trControl = myControl)

For example, this is one of the models built. If I do the following, I can get the ROC curve of my training set. But to get the ROC of my testing data set?

model_knn
plot(model)

CodePudding user response：

As you have not provided any data, I am using Sonar data. You can use the following code to make ROC plot for test data

library(caret)
library(MLeval)

data(Sonar)

# Split data 
a <- createDataPartition(Sonar$Class, p=0.8, list=FALSE)
train <- Sonar[ a, ]
test <- Sonar[ -a, ]

myControl = trainControl(
  method = "cv",
  summaryFunction = twoClassSummary,
  classProbs = TRUE,
  verboseIter = FALSE,
)

model_knn = train(
  Class ~ .,
  train,
  method = "knn",
  metric = "ROC",
  tuneLength = 10,
  trControl = myControl)

pred <- predict(model_knn, newdata=test, type="prob")
ROC <- evalm(data.frame(pred, test$Class, Group = "KNN"))