igraph in R: efficiently count number of edges between multiple sets of vertices-CodePudding

I want to calculate a matrix of edge counts for a given graph and partition of this graph into groups. The solution I have at the moment does not scale for large graphs and I wonder if it is possible to speed up the computation.

I want to use the igraph R-package for this, so for a graph G and two sets of vertices set1, set2 I currently calculate the number of edges from one set to the other by using igraph's %->% operator.

el <- E(G)[set1 %->% set2]
length(el)

I wonder if there is a faster way to do this either natively in igraph or by homebrewing some solution (maybe using Rcpp)?

Example code

library(igraph)

# Set up toy graph and partition into two blocks
G <- make_full_graph(20, directed=TRUE)
m <- c(rep(1,10), rep(2,10))

block_edge_counts <- function(G, m){
  # Get list of vertices per group.
  c <- make_clusters(G, m, modularity = FALSE)
  # Calculate matrix of edge counts between blocks.
  E <- sapply(seq_along(c), function(r){
    sapply(seq_along(c), function(s){
      # Iterate over all block pairs
      el <- E(G)[c[[r]] %->% c[[s]]] # list of edges from block r to block s
      length(el) # get number of edges
    })
  })
}

block_edge_counts(G, m)
#>      [,1] [,2]
#> [1,]   90  100
#> [2,]  100   90

Example benchmark

#> install.packages("bench")

results <- bench::press(
  Nsize = c(10,100,1000),
  {
    G <- make_full_graph(Nsize, directed=TRUE)
    m <- c(rep(1,.5*Nsize), rep(2,.5*Nsize))
    bench::mark(block_edge_counts(G,m))
  }
)
#> Running with:
#>   Nsize
#> 1    10
#> 2   100
#> 3  1000
#> Warning: Some expressions had a GC in every iteration; so filtering is disabled.
results
#> # A tibble: 3 × 7
#>   expression              Nsize      min   median `itr/sec` mem_alloc `gc/sec`
#>   <bch:expr>              <dbl> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl>
#> 1 block_edge_counts(G, m)    10   1.24ms   1.31ms    752.     39.97KB     12.6
#> 2 block_edge_counts(G, m)   100   3.24ms   3.37ms    284.      4.11MB     32.1
#> 3 block_edge_counts(G, m)  1000 262.73ms 323.61ms      3.29  408.35MB     34.2

CodePudding user response：

You can get significantly better results by extracting the graph's adjacency matrix and working with it directly.

block_edge_counts_adj <- function(G, m) {
  # Get list of vertices per group.
  c <- make_clusters(G, m, modularity = FALSE)
  am <- as_adjacency_matrix(G, sparse=F)
  # Calculate matrix of edge counts between blocks.
  sapply(seq_along(c), function(r){
    sapply(seq_along(c), function(s){
      # Iterate over all block pairs
      sum(am[c[[r]], c[[s]]]) # number of edges from block r to block s
    })
  })
}

The sparse=T argument to as_adjacency_matrix is essential here because without it the function returns a sparse matrix whose computation takes much longer. Maybe for a sparse graph this would be beneficial in terms of memory usage but on a full graph like the one in your example it leads to much longer computation.

block_edge_counts_adj2 <- function(G, m) {  # using sparse matrix
  # Get list of vertices per group.
  c <- make_clusters(G, m, modularity = FALSE)
  am <- as_adjacency_matrix(G)
  # Calculate matrix of edge counts between blocks.
  sapply(seq_along(c), function(r){
    sapply(seq_along(c), function(s){
      # Iterate over all block pairs
      sum(am[c[[r]], c[[s]]]) # number of edges from block r to block s
    })
  })
}

results <- bench::press(
  Nsize = c(10, 100, 1000, 3000),
  {
    G <- make_full_graph(Nsize, directed=TRUE)
    m <- c(rep(1, .5*Nsize), rep(2, .5*Nsize))
    bench::mark(block_edge_counts(G, m),
                block_edge_counts_adj(G, m),
                block_edge_counts_adj2(G, m),
                min_iterations=5)
  }
)

results
# A tibble: 12 x 14
#    expression                   Nsize      min   median `itr/sec` mem_alloc `gc/sec` n_itr
#    <bch:expr>                   <dbl> <bch:tm> <bch:tm>     <dbl> <bch:byt>    <dbl> <int>
#  1 block_edge_counts(G, m)         10   2.44ms   2.88ms   301.      39.97KB    2.06    146
#  2 block_edge_counts_adj(G, m)     10   1.43ms   1.61ms   560.        1.8KB    2.05    273
#  3 block_edge_counts_adj2(G, m)    10   3.41ms   3.94ms   233.      14.46KB    2.05    114
#  4 block_edge_counts(G, m)        100   6.26ms    7.2ms   135.       4.11MB    0        68
#  5 block_edge_counts_adj(G, m)    100   2.26ms   2.46ms   376.     208.59KB    2.05    183
#  6 block_edge_counts_adj2(G, m)   100   4.82ms   5.32ms   181.       1.34MB    0        91
#  7 block_edge_counts(G, m)       1000 380.86ms 412.24ms     2.43   408.35MB    3.64      2
#  8 block_edge_counts_adj(G, m)   1000  25.85ms  27.46ms    36.0     15.71MB    2.25     16
#  9 block_edge_counts_adj2(G, m)  1000 114.91ms 133.71ms     7.71   130.09MB    1.93      4
# 10 block_edge_counts(G, m)       3000    3.78s    3.88s     0.260    3.59GB    1.92      5
# 11 block_edge_counts_adj(G, m)   3000 197.01ms 218.48ms     4.47   138.75MB    0.894     5
# 12 block_edge_counts_adj2(G, m)  3000    1.19s    1.25s     0.809    1.14GB    2.10      5

CodePudding user response：

One way is to contract the two sets into single vertices, then count edges between them.

Contract the two sets, obtaining vertex id 1 for the first and 2 for the second:

CG <- contract(G, m)

Now count 1 -> 2 and 2 -> 1 edges:

> count_multiple(CG, get.edge.ids(CG, c(1,2, 2,1)))
[1] 100 100

If you have a full partitioning of the graph, then you can count the fraction of intra-partition edges using

modularity(G, m, resolution = 0)

You example has a full partitioning into two sets, so you can get the number of edges between these two as

> ecount(G)*(1 - modularity(G, m, resolution = 0))
[1] 200