Is there a way to combine a potentially unlimited set of lists?-CodePudding

The following script pulls some specific country data from an API from the UN. I created a for loop to help with cases where the user would like to pull multiple countries from the database. The problem I am struggling with is after the for loop where I combine the lists from the "data" element within the "response" vector (response$data). I am currently using rbind() to group the multiple lists but would like to find a solution to account for a potentially unlimited amount of lists without having to manually write out each one.

library(jsonlite)
library(httr)

base_url <- "https://population.un.org/dataportalapi/api/v1/data"

locationlist <- list("528","40","620") # example list of country codes

target <- list()
response <- list()

for (i in locationlist) {
  
  target[[i]] <- paste0(base_url, "/indicators/",65,"/locations/",i,"/start/",2000,"/end/",2019) # url
  response[[i]] <- fromJSON(target[[i]]) # call API
  
}

# Here's the main issue :
df <- rbind(response[[1]]$data,response[[2]]$data,response[[3]]$data) # combine lists

I have tried incorporating the rbind() within the for loop instead without much success.

CodePudding user response：

Each response element is a list and we need to only extract the data component. So, loop over the response (which is a list), use either [[ or $ to extract the 'data' part, which returns a list of data.frames, that can be rbinded with do.call

out <- do.call(rbind, lapply(response, `[[`, "data"))

-output

> dim(out)
[1] 60 32

Or use tidyverse

library(purrr) # version 1.0.0
library(dplyr)
out <- map(response, pluck, "data") %>%
   list_rbind
#or may also use
out <- map_dfr(response, pluck, "data")

Instead of doing this in a for loop, it can be done with lapply or map

library(stringr)
urls <- str_c(base_url, "/indicators/",65,"/locations/",
     locationlist,"/start/",2000,"/end/",2019)
out <- map_dfr(urls, ~ fromJSON(.x) %>% 
            pluck("data"))
> dim(out)
[1] 60 32

out <- map(