4

I want to use a loop to read in multiple csv files and append a list in R.

path = "~/path/to/csv/" file.names <- dir(path, pattern =".csv") mylist=c() for(i in 1:length(file.names)){ datatmp <- read.csv(file.names[i],header=TRUE, sep=";", stringsAsFactors=FALSE) listtmp = datatmp[ ,6] finallist <- append(mylist, listtmp) } finallist 

For each csv file, the desired column has a different length. In the end, I want to get the full appended list with all values in that certain column from all csv files.

3
  • and what error do you get? Commented Mar 31, 2019 at 21:38
  • See this answer Commented Mar 31, 2019 at 21:42
  • 1
    The final list only consists of the column values from the very last csv file Commented Mar 31, 2019 at 21:49

1 Answer 1

9

There are four errors in your approach.

First, file.names <- dir(path, pattern =".csv") will extract just file names, without path. So, when you try to import then, read.csv() doesn't find.

Building the path

You can build the right path including paste0():

path = "~/path/to/csv/" file.names <- paste0(path, dir(path, pattern =".csv")) 

Or file.path(), which add slashes automaticaly.

path = "~/path/to/csv" file.names <- file.path(path, dir(path, pattern =".csv")) 

And another way to create the path, for me more efficient, is that suggested in the answer commented by Tung.

file.names <- list.files(path = "~/path/to/csv", recursive = TRUE, pattern = "\\.csv$", full.names = TRUE) 

This is better because in addition to being all in one step, you can use within a directory containing multiple files of various formats. The code above will match all .csv files in the folder.

Importing, selecting and creating the list

The second error is in mylist <- c(). You want a list, but this creates a vector. So, the correct is:

mylist <- list() 

And the last error is inside the loop. Instead of create other list when appending, use the same object created before the loop:

for(i in 1:length(file.names)){ datatmp <- read.csv(file.names[i], sep=";", stringsAsFactors=FALSE) listtmp = datatmp[, 6] mylist <- append(mylist, list(listtmp)) } mylist 

Another approach, easier and cleaner, is looping with lapply(). Just this:

mylist <- lapply(file.names, function(x) { df <- read.csv(x, sep = ";", stringsAsFactors = FALSE) df[, 6] }) 

Hope it helps!

Sign up to request clarification or add additional context in comments.

1 Comment

Alternatively, one could also use the full.names = T option in the dir function.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.