Home > Enterprise >  How to more completely validate CSV file when uploading to Shiny App?
How to more completely validate CSV file when uploading to Shiny App?

Time:11-21

When running the below code, the user can upload and download input data. The user can download and save inputs, and later retrieve those inputs via upload. I'm trying to improve the upload validation because in reality it will be very easy for the user to select an incorrect file and I'd rather flag with a warning rather than have the App crash like it does now.

All downloads are saved as a 2-column matrix with headers of X and Y. That (and the fact that it's a CSV) are my key upload validations per the below code. The App correctly downloads and uploads CSV data as shown in image 1 (download) and image 2 (upload) below, but crashes when it tries uploading incorrectly formatted CSV data per image 3 below.

So my questions are:

  1. How do I specify which columns in the csv file to look for the "X" and "Y" headers? Currently it reads everywhere for X and Y headers. I tried read.csv(...colClasses=c(NA, NA))) as shown below, I also tried read.csv(...)[ , 1:2] and neither one works.
  2. More generally, is there a way to abort the upload if it would otherwise cause an error or crash? Sort of like if(iserror(...)) in Excel
  3. OK now I'm pushing it, please feel free to ignore this if this is too much. Any way to move the upload warning into modalDialog? I can always move this to another post if I can't figure it out, after resolving the above.

MWE code:

library(dplyr)
library(shiny)
library(shinyMatrix)

interpol <- function(a, b) { # a = periods, b = matrix inputs
  c <- rep(NA, a)
  c[1] <- b[1]
  c[a] <- b[2]
  c <- approx(seq_along(c)[!is.na(c)], c[!is.na(c)], seq_along(c))$y 
  return(c)
}

ui <- fluidPage(
  sidebarLayout(
    sidebarPanel(
      fileInput("file1", "Optionally choose input file (csv)", accept = ".csv"),
      sliderInput('periods', 'Periods to interpolate over:', min=1, max=10, value=10),
      matrixInput("matrix1", 
                  value = matrix(c(1,5), 
                          ncol = 2, 
                          dimnames = list("Interpolate",c("X","Y"))
                  ),
                  cols =  list(names = TRUE),
                  class = "numeric"
      ),
      downloadButton("download")
    ),
    mainPanel(
      tableOutput("contents"),
      plotOutput("plot")
    )
  )
)

server <- function(input, output, session) {
  
  input_file <- reactive({
    file <- input$file1
    ext <- tools::file_ext(file$datapath)
    req(file)
    
    if(is.null(file))
      return(NULL)
    
    file_contents <- read.csv(file$datapath,header=TRUE,colClasses=c(NA, NA))
    required_columns <- c('X','Y')
    column_names <- colnames(file_contents)
    
    shiny::validate(
      need(ext == "csv", "Incorrect file type"),
      need(required_columns %in% column_names, "Incorrect file type")
    )
    
    file_contents
    
  })
  
  output$contents <- renderTable({
    input_file()
  })
  
  data <- function(){
    tibble(
      X = seq_len(input$periods),
      Y = interpol(input$periods,matrix(c(input$matrix1[1,1],input$matrix1[1,2])))
    )
  }  
  
  output$plot<-renderPlot({plot(data(),type="l",xlab="Periods (X)", ylab="Interpolated Y values")})
  
  observeEvent(input$file1,{
    updateMatrixInput(session, 
                      inputId = "matrix1", 
                      value = matrix(as.matrix(input_file()),
                                     ncol=2,
                                     dimnames = list("Interpolate",c("X","Y"))
                              )
    )
                      
  })
  
  output$download <- downloadHandler(
    filename = function() {
      paste("Inputs","csv",sep=".")
    },
    content = function(file) {
      write.csv(input$matrix1, file,row.names=FALSE)
    }
  )
}

shinyApp(ui, server)

enter image description here

enter image description here

enter image description here

enter image description here

CodePudding user response:

The following should address your need

shiny::validate(
  need(ext == "csv", "Incorrect file type"),
  need(required_columns %in% column_names, "Incorrect file type"),
  need(sum(!column_names %in% required_columns)==0, "Incorrect columns in file")
)

UPDATE: if you are ok to modify your checks, you can do the following.

shiny::validate(
  need(ext == "csv", "Incorrect file type"),
  need(sum(required_columns %in% column_names)==2 & sum(!column_names %in% required_columns)==0, "Incorrect file type")
)

CodePudding user response:

I've created a minimal version of your app (without interpolation or downloads) that I think addresses (1) and (2) and your desire for an existing matrix and plot to be preserved in the event that an invalid upload occurs. You should be able to rebuild your app by modifying this skeleton, but, before doing that, you should try to understand how this app works.

Note that I've added a dependency on package shinyFeedback, which places warning messages near the appropriate input panels. Let me know if that's a problem...

library("shiny")
library("shinyFeedback")
library("shinyMatrix")

## Your variable names
nms <- c("X", "Y")

ui <- fluidPage(
  useShinyFeedback(),
  sidebarLayout(
    sidebarPanel(
      fileInput("file", label = "CSV file", accept = ".csv"),
      matrixInput("mat", label = "Matrix", value = matrix(rnorm(12L), 6L, 2L, dimnames = list(NULL, nms)), class = "numeric", rows = list(names = FALSE))
    ),
    mainPanel(
      plotOutput("plot"),
      verbatimTextOutput("verb")
    )
  )
)

server <- function(input, output, session) {
  rawdata <- reactive({
    req(input$file)
    try(read.csv(input$file$datapath, header = TRUE))
  })

  observeEvent(rawdata(), {
    ## If 'rawdata()' is a data frame with numeric variables named 'nms'
    if (is.data.frame(rawdata()) && all(nms %in% names(rawdata())) && all(vapply(rawdata()[nms], is.numeric, NA))) {
      ## Then update matrix by extracting those variables, ignoring the rest (if any)
      updateMatrixInput(session, "mat", as.matrix(rawdata()[nms]))
      ## And suppress warning if visible
      hideFeedback("file")
    } else {
      ## Otherwise show warning
      showFeedbackWarning("file", "Invalid upload.")
    }
  })
  
  ## Plots matrix rows as points
  output$plot <- renderPlot(plot(input$mat))
  ## Prints "try-error" if 'read.csv' threw error, "data.frame" otherwise
  output$verb <- renderPrint(class(rawdata()))  
}

shinyApp(ui, server)

Here is code that you can use to create test files. Each one tests a different behaviour of the app.

## OK
cat("X,Y,Z\na,1,3,5\nb,2,4,6\n", file = "test1.csv")
## OK: file contents matter, not file extension
cat("X,Y,Z\na,1,3,5\nb,2,4,6\n", file = "test2.txt")
## Missing 'X'
cat("W,Y,Z\na,1,3,5\nb,2,4,6\n", file = "test3.csv")
## 'X' is not numeric
cat("X,Y,Z\na,hello,3,5\nb,world,4,6\n", file = "test4.csv")
## Not a valid CSV file
cat("read.csv\nwill,not,like,this,file\n", file = "test5.csv")
  • Related