R - how to count all words in a df row and add output to a new column? Ideally with tidyverse or tid-CodePudding

I'm trying to find the location of words in a text, and also the total wordcount of the same text.

# library(tidyverse)
# library(tidytext)
txt<-tibble(text=c("we're meeting here today to talk about our earnings. we will also discuss global_warming.", "hi all, global_warming and the on-going strike is at the top of our agenda, because unionizing threatens our revenue goals.", "we will discuss global_warming tomorrow, today the focus is our Q3 earnings"))
dict <- tibble(words=c("global_warming"))
x<-txt %>% unnest_tokens(output = "words",
                          input = "text",
                          drop = FALSE) %>%
  group_by(text) %>%
  mutate(word_loc = row_number()) %>%
  ungroup() %>%
  inner_join(dict)

This gives me the following output:

# A tibble: 3 x 3
  text                                                                                        words        word_loc
  <chr>                                                                                       <chr>           <int>
1 we're meeting here today to talk about our earnings. we will also discuss global_warming.   global_warm…       14
2 hi all, global_warming and the on-going strike is at the top of our agenda, because unioni… global_warm…        3
3 we will discuss global_warming tomorrow, today the focus is our Q3 earnings                 global_warm…        4

How can I add one column, that gives me the total word count for each row?

CodePudding user response：

We can use str_count to get the total number of words for each string, where \\S counts all sequences on non-space characters.

library(tidyverse)

x %>%
  mutate(count = str_count(text, "\\S "))

Or another option using base R:

x$count <- lengths(gregexpr("\\S ", x$text))

Output

  text                                           words word_loc count
  <chr>                                          <chr>    <int> <int>
1 we're meeting here today to talk about our ea… glob…       14    14
2 hi all, global_warming and the on-going strik… glob…        3    20
3 we will discuss global_warming tomorrow, toda… glob…        4    12

Or if you want to count contractions, words with hypens, etc. then you can use \\w instead:

x %>%
  mutate(count = str_count(text, "\\w "))

  text                                           words word_loc count
  <chr>                                          <chr>    <int> <int>
1 we're meeting here today to talk about our ea… glob…       14    15
2 hi all, global_warming and the on-going strik… glob…        3    21
3 we will discuss global_warming tomorrow, toda… glob…        4    12