Is there any faster way than rowwise() to apply a function row-wise to one column of a tibble?-CodePudding

I have a tibble df in which each row contains a list (beta) that is a posterior distribution (4000 samples). I would like to compute Bayesfactor using bayestestR::bayesfactor_parameters, but the way I did using rowwise() is pretty slow (taking 20 minutes for around 3000 rows). Do you know any faster ways to apply this function to each row of the tibble? Thanks a lot.

df <- tibble(idx = seq(1, 3000), beta = list(rnorm(4000, 0.5, 3)))
df <- df %>% 
  slice(1:10) %>% 
  rowwise() %>% 
  mutate(ioi = bayestestR::
           bayesfactor_parameters(posterior = unlist(beta), prior = rnorm(1e4, 0, 10), 
                                  direction = "two-sided", 
                                  null = c(-1, 1))$log_BF) %>% 
  ungroup()

CodePudding user response：

Yes! Apply in parallel using multidplyr

cluster <- new_cluster(parallel::detectCores() - 2)
cluster_library(cluster, c('tidyverse', 'furrr'))
cluster_copy([...])
df %>% 
    rowwise() %>% 
    partition(cluster) %>% 
    mutate([...]) %>% 
    collect()

CodePudding user response：

You might try the following:

library(data.table)
setDT(dt)

library(foreach)
doParallel::registerDoParallel()

result = foreach(i=1:nrow(df),.inorder = F,.combine = rbind,.packages = c("data.table", "bayestestR")) %dopar% {
    data.frame(idx=i, log_bf= bayesfactor_parameters(
      posterior = df[i, unlist(beta)],
      prior = rnorm(1e4, 0, 10),
      direction = "two-sided",
      null= c(-1, 1))$log_BF)
}

Output (first 10 rows)

   idx    log_bf
1    1 -1.438289
2    2 -1.443515
3    3 -1.446068
4    4 -1.449608
5    5 -1.440932
6    6 -1.446644
7    7 -1.444527
8    8 -1.434655
9    9 -1.457718
10  10 -1.403027