Combining CSV data frames-CodePudding

I used this code to combine several csv files into one file at the beginning of my project around 5 months ago. I have now returned to re-sort my data in a different way. However, it now refuses to work simply returning an empty data set.

I am running R-Studio 4.2.0. Has there been an update to the console that has rendered my code useless?

my code:


# Initilise Libraries
library("magrittr")
library("dplyr")                                                  
library("readr")
library("tidyverse")
library("reticulate")
library("purrr")
library("data.table")
library("jsonlite")


# combine all data sets (CSV Files). This will Store all files in list

data_all <- list.files(path = "C:\\Users\\Surface\\Documents\\FinalProject\\Data\\Sky_Data\\csvData",    
                       pattern = "*.csv", full.names = TRUE) %>%
  
  lapply(read_csv) %>%
  bind_rows

I then tried reading in one of the CSV files independently:

df <- read.csv("C:\\Users\\Surface\\Documents\\FinalProject\\Data\\Sky_Data\\csvData\\bookmaker-data-10-01.csv")

This now reads in the data from one file fine showing the files in question are in the specified folder.

CodePudding user response：

I ended up getting around this error using the following code

path <- "C:\\Users\\Surface\\Documents\\FinalProject\\Data\\Sky_Data\\csvData"

multmerge = function(path){
  filenames=list.files(path=path, full.names=TRUE)
  rbindlist(lapply(filenames, fread))
}

alldata <- multmerge(path)

However I am still not sure on what the initial problem was

CodePudding user response：

According to the comments, the problem seemed to be the (common!) mistake that *.csv selects all csv-files in a folder. it does not..

The dot . (and the * for that matter) are special regex-operators, where the . means "any character". So a file like thisisnotacsv.txt will match the regex *.csv (acsv will match the regex). If you want a literal dot, you have to escape it.. if only you want files with the .csv-extention, add an end-character.. like: .*\\.csv$