Brtdku

I have the below functions that will scrape the Title, Rating, and Review of apps off the Amazon app store. I was wondering if someone could help with two questions?

1. The date function is not working properly, I have selected the correct HTML nodes, but for some reason, it is just supplying an HTML code string.




2. My second question deals with joining all three function lists together. When I run the cBind function it attaches the lists, however, the columns are not attached to the correct records. For example, an extremely negative review will be attached to a 5-star rating. Is there a way to properly join the the lists, so that when attached, everything is attached to the corresponding record it is associated with?

Any help on these two questions would be greatly appreciated!

    library(rvest)

    library(dplyr)

    library(pander)

    library(stringr)





r.date <- lapply(paste0('https://www.amazon.com/NYTimes-Breaking-Local-National-World/product-reviews/B00BOW41P8/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews', 0:500),

                   function(url){

                     url %>% read_html() %>% 

                       html_nodes(".review-date") %>% 

                       html_attrs() %>%

                       gsub('[rnt]', '', .)



                   })





r.title <- lapply(paste0('https://www.amazon.com/NYTimes-Breaking-Local-National-World/product-reviews/B00BOW41P8/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews', 0:500),

                function(url){

                  url %>% read_html() %>% 

                    html_nodes(".review-title") %>% 

                    html_text() %>%

                    gsub('[rnt]', '', .)



                })





r.rating <- lapply(paste0('https://www.amazon.com/NBCUniversal-Media-LLC-NBC/product-reviews/B018IOV40E/ref=cm_cr_arp_d_paging_btm_2?ie=UTF8&reviewerType=all_reviews&pageNumber=', 0:500),

                function(url){

                  url %>% read_html() %>% 

                    html_nodes(".a-icon-alt") %>% 

                    html_text() %>%

                    gsub('[rnt]', '', .)



                })





r.review <- lapply(paste0('https://www.amazon.com/NYTimes-Breaking-Local-National-World/product-reviews/B00BOW41P8/ref=cm_cr_dp_d_show_all_btm?ie=UTF8&reviewerType=all_reviews', 0:500, 0:9)

               function(url){

                 url %>% read_html() %>% 

                   html_nodes(".review-text") %>% 

                   html_text() %>%

                   gsub('[rnt]', '', .)



               })



Date <- unlist(r.date)

Title <- unlist(r.title)

Stars <- unlist(r.rating)

Review <- unlist(r.review)



DF <- as.data.frame(Cbind(Date,Stars,Title,Review))