What format of table is at Lineups.com and how to scrape it in R-CodePudding

I am new to scraping and have successfully scraped tables from these websites:-

https://www.numberfire.com/nba/daily-fantasy/daily-basketball-projections/guards
https://www.dailyfantasyfuel.com/nba/projections/draftkings/
https://www.sportsline.com/nba/expert-projections/simulation/

But this website:-

https://www.lineups.com/nba/nba-fantasy-basketball-projections

seems very tricky.

1. I have tried to read it as JSON

r <- read_html('https://www.lineups.com/nba/nba-fantasy-basketball-projections/') %>% html_element('script#__NEXT_DATA__') %>% html_text() %>% jsonlite::parse_json()

2. from rvest methods

    data <- "https://www.lineups.com/nba/nba-fantasy-basketball-projections" %>%
      read_html %>%
      html_nodes('script') %>%
      html_text()

3. As well as RSelenium but with no success.

Could you kindly tell me how to deal with these kinds of Tables found at "www.lineups.com" ?

Thanks

CodePudding user response：

Using RSelenium

library(RSelenium)
library(rvest)
library(dplyr)
driver = rsDriver(browser = c("firefox"))

remDr <- driver[["client"]]

url <- 'https://www.lineups.com/nba/nba-fantasy-basketball-projections'
remDr$navigate(url)

#get all the tables from webapage 
df = remDr$getPageSource()[[1]] %>% 
    read_html() %>% html_table()
[[2]]
# A tibble: 51 x 31
   Player   Player Player Player Player Player Player Player ``    ``    ``    ``    ``    ``    Game  Game  Game  Game  Game  `Projected Game~ `Projected Game~
   <chr>    <chr>  <chr>  <chr>  <chr>  <chr>  <chr>  <chr>  <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr> <chr>            <chr>           
 1 "Name"   Team   Pos    Proje~ Salary Pts/$~ FPPM   USG%   Pos   Proj~ Sala~ Pts/~ FPPM  USG%  Opp   DvP   Spre~ Total O/U   MINS             PTS             
 2 "Nikola~ DEN    C      58.37  $11,0~ 5.3    1.8    31.3%  HOU   30    13.5  126   238   33    27.7  7.7   13.7  1.1   0.9   5                18.8            
 3 "Gianni~ MIL    PF     57.84  $11,3~ 5.1    1.8    35.1%  CHI   27    -5    122.~ 240   33    30.5  5.9   11.7  1     1.4   7.8              19.1            
 4 "Joel E~ PHI    C      53.35  $10,8~ 4.9    1.7    37.4%  CLE   5     7     112   216.5 32    29.4  4.2   11    0.9   1.4   8.9              19