Efficiently replace string in multiple columns based on lookup table-CodePudding

I have a lookup table with 255 rows - value | replacement. It's a table for html codes (e.g.   corresponds to a space character ' '). Table name: lookup

I have 6 columns (out of 50) in my table that have HTML characters that need replacement for legibility, it currently has 600 rows but likely to grow. Table name: exp

The code I came up with is based on dplyr and a for loop: it goes row by row in the lookup table and checks for matches in the target variables.

len <- nrow(lookup)
for (i in 1:len){
  exp <- exp %>% 
         mutate_at(vars(c(var1, var2, var3, var4, var6, var8)), 
                   funs(gsub(pattern = lookup[i,1], replacement = lookup[i,2], x = .)))
}

It takes quite some time to run, I was wondering if there is a more efficient way to run the replacement?

Adding data example for future reference

Lookup:

Pattern Replacement
&cent;  ¢
&amp;   &
&reg;   ®
&trade  ™
&copy;  ©
&current;   ¤
&gt;    >
&lt;    <
&nbsp;  
&euro;  €
&quot;  “
&apos;  ‘

Exp:

> example
# A tibble: 3 x 4
  `Example 1`                               `Example 2`                             `Example 3`                        `Example 4`                        
  <chr>                                     <chr>                                   <chr>                              <chr>                              
1 &cent; It denotes Cent Sign of currency   &tradeTrade Mark                        &gt;It denotes greater than sign   &euro;It defines the British Euro ~
2 &amp;It denotes frequently used Ampersan~ &copy;Gives Copy-right Symbol           &lt;It denotes lesser than sign    &quot;Gives double quotes in a giv~
3 &reg;Gives Registered Symbol              &current; It defines a Generic currenc~ &nbsp;It defines for Non-Breaking~ &apos;Includes Apostrophe in a sen~

CodePudding user response：

With stri_replace_all_fixed from stringi, you can replace many patterns at once. The syntax is a bit confusing, but when you set vectorise_all = FALSE it replaces all instances of all patterns with corresponding replacements.

First, let's create some example data as you did not provide any:

library(tidyverse)
set.seed(1)
exp <- data.frame(matrix(sample(LETTERS, 1000, replace = TRUE), ncol = 100))

lookup <- tribble(
  ~pattern, ~replacement,
  "A",     ":",
  "F",     " ",
  "Y",     "Test"
)

Use mutate across which is the new version of mutate_at in this case (mutate_at is slowly phased out):

exp %>% 
  mutate(across(c(X1, X3), ~ stringi::stri_replace_all_fixed(
    str = .x,
    pattern = lookup[["pattern"]],
    replacement = lookup[["replacement"]],
    vectorise_all = FALSE
  ))) %>% 
  as_tibble()
#> # A tibble: 10 × 100
#>    X1    X2    X3    X4    X5    X6    X7    X8    X9    X10   X11   X12   X13  
#>    <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>
#>  1 Test  A     U     L     T     Y     N     H     M     V     W     B     U    
#>  2 D     U     E     O     T     W     B     F     H     L     S     J     L    
#>  3 G     U     I     A     Z     X     M     W     Y     P     V     A     G    
#>  4 :     J     Test  T     L     F     R     L     P     A     R     K     X    
#>  5 B     V     N     C     Y     Z     V     F     Y     M     Z     Z     U    
#>  6 W     N     E     F     W     G     N     H     W     U     P     O     V    
#>  7 K     J     E     J     F     S     F     G     N     F     K     Z     H    
#>  8 N     G     B     J     Y     J     A     K     T     Q     J     X     A    
#>  9 R     I     J     F     H     F     S     Q     G     I     G     J     S    
#> 10 S     O     Test  O     L     X     S     D     M     G     S     P     Z    
#> # … with 87 more variables: X14 <chr>, X15 <chr>, X16 <chr>, X17 <chr>,
#> #   X18 <chr>, X19 <chr>, X20 <chr>, X21 <chr>, X22 <chr>, X23 <chr>,
#> #   X24 <chr>, X25 <chr>, X26 <chr>, X27 <chr>, X28 <chr>, X29 <chr>,
#> #   X30 <chr>, X31 <chr>, X32 <chr>, X33 <chr>, X34 <chr>, X35 <chr>,
#> #   X36 <chr>, X37 <chr>, X38 <chr>, X39 <chr>, X40 <chr>, X41 <chr>,
#> #   X42 <chr>, X43 <chr>, X44 <chr>, X45 <chr>, X46 <chr>, X47 <chr>,
#> #   X48 <chr>, X49 <chr>, X50 <chr>, X51 <chr>, X52 <chr>, X53 <chr>, …

^{Created on 2022-02-16 by the reprex package (v2.0.1)}

This is as fast as it gets I believe.