How to download data from un comtrade for all reporters and all partners (comtradr)-CodePudding

The task is to build a social network for trading certain goods. I have decided to use the un comtrade database. But this database has a limit on the number of countries to retrieve data. I need to get data about absolutely all countries. Who, how many and where imported. My code to download the data and the error I get when I try to download it:

codes <- ct_commodity_lookup("crude", return_code = T, return_char = T)

data <  ct_search(reporters = "All",partners = "All", trade_direction = "imports", start_date = "2020", end_date = "2020", commod_codes = codes[42])

Error:

Error: API request failed. Err msg from Comtrade:
  Both 'all' reporters and 'all' partners may not be selected. Select a different reporter or partner.

I understand this error, but for the task I need to get for all countries, and I don't really want to manually download for each of the five countries. Who has encountered this and how has this problem been solved?

CodePudding user response：

Maybe the following can get you started. Loop through the countries names in English, taken from package countrycode, trap possible errors with tryCatch and after the loop separate the errors from the good.

library(comtradr)
library(countrycode)
library(jsonlite)

codes <- ct_commodity_lookup("crude", return_code = TRUE, return_char = TRUE)

data <- lapply(codelist$country.name.en[1:3], \(x) {
  Sys.sleep(1)
  tryCatch(
    ct_search(
      reporters = x, 
      partners = "All", 
      trade_direction = "imports", 
      start_date = "2020", 
      end_date = "2020", 
      commod_codes = codes[42]
    ),
    error = function(e) e
  )
})
names(data) <- codelist$country.name.en[1:3]
ok <- !sapply(data, inherits, "error")
err <- data[!ok]
data <- data[ok]
data

CodePudding user response：

I tried to do as you wrote, but the result is that I only get information for two countries and no values, just the names of the table columns from un comtrade:

dput(data):

list(Afghanistan = structure(list(classification = logical(0), 
    year = logical(0), period = logical(0), period_desc = logical(0), 
    aggregate_level = logical(0), is_leaf_code = logical(0), 
    trade_flow_code = logical(0), trade_flow = logical(0), reporter_code = logical(0), 
    reporter = logical(0), reporter_iso = logical(0), partner_code = logical(0), 
    partner = logical(0), partner_iso = logical(0), second_partner_code = logical(0), 
    second_partner = logical(0), second_partner_iso = logical(0), 
    customs_proc_code = logical(0), customs = logical(0), mode_of_transport_code = logical(0), 
    mode_of_transport = logical(0), commodity_code = logical(0), 
    commodity = logical(0), qty_unit_code = logical(0), qty_unit = logical(0), 
    alt_qty_unit_code = logical(0), alt_qty_unit = logical(0), 
    qty = logical(0), alt_qty = logical(0), netweight_kg = logical(0), 
    gross_weight_kg = logical(0), trade_value_usd = logical(0), 
    cif_trade_value_usd = logical(0), fob_trade_value_usd = logical(0), 
    flag = logical(0)), class = "data.frame", row.names = integer(0), url = "https://comtrade.un.org/api/get?max=50000&type=C&freq=A&px=HS&ps=2020&r=4&p=all&rg=1&cc=270900&head=H&fmt=json", time_stamp = structure(1653157698.10372, class = c("POSIXct", 
"POSIXt")), req_duration = 0.129484176635742), Albania = structure(list(
    classification = logical(0), year = logical(0), period = logical(0), 
    period_desc = logical(0), aggregate_level = logical(0), is_leaf_code = logical(0), 
    trade_flow_code = logical(0), trade_flow = logical(0), reporter_code = logical(0), 
    reporter = logical(0), reporter_iso = logical(0), partner_code = logical(0), 
    partner = logical(0), partner_iso = logical(0), second_partner_code = logical(0), 
    second_partner = logical(0), second_partner_iso = logical(0), 
    customs_proc_code = logical(0), customs = logical(0), mode_of_transport_code = logical(0), 
    mode_of_transport = logical(0), commodity_code = logical(0), 
    commodity = logical(0), qty_unit_code = logical(0), qty_unit = logical(0), 
    alt_qty_unit_code = logical(0), alt_qty_unit = logical(0), 
    qty = logical(0), alt_qty = logical(0), netweight_kg = logical(0), 
    gross_weight_kg = logical(0), trade_value_usd = logical(0), 
    cif_trade_value_usd = logical(0), fob_trade_value_usd = logical(0), 
    flag = logical(0)), class = "data.frame", row.names = integer(0), url = "https://comtrade.un.org/api/get?max=50000&type=C&freq=A&px=HS&ps=2020&r=8&p=all&rg=1&cc=270900&head=H&fmt=json", time_stamp = structure(1653157702.31531, class = c("POSIXct", 
"POSIXt")), req_duration = 0.125498056411743))