Brtdku

I have extracted Tweets by feeding in 44 different keywords, and the output is in a file which consists of 400k tweets in total. The output file has tweets that contain the relevant keywords. How could I create a separate ID column which contains the keyword present in that tweet?

Eg: The tweet is:

Andhra Pradesh is the highest state with crimes against women

the keyword here is "crimes against women"

I would like to create a column that assigns the keyword "crimes against women" to the tweet, a sort of ID column to be precise.

#input column 1

Tweet<-("Andhra Pradesh is the highest state with crimes against women")



#expected output column 2 beside the Tweet column

Keyword<-("crimes against women")

Edit: I do not want to extract any part of the tweet, I just want to be able to assign to the tweet, in a new column, the keyword it contains so it will help me segregate the tweets based on this keyword.

You can perform this analysis with the stringr package, however, I don't think you need to use sapply.

Consider the following keyword list and table with tweets:

keyword_list <- c("crimes against women", "downloading tweets", "r analysis")



tweets <- data.frame(

  tweet = c("Andhra Pradesh is the highest state with crimes against women",

            "I am downloading tweets",

            "I love r analysis",

            "downloading tweets helps with my r analysis")

)

First, you want to combine your keywords into one regular expression that searches for any of the strings.

keyword_pattern <- paste0(

  "(",

  paste0(keyword_list, collapse = "|"),

  ")"

)



keyword_pattern

#> [1] "(crimes against women|downloading tweets|r analysis)"

Finally, we can add a column to the data frame that extracts the keyword from the tweet.

tweets$keyword <- str_extract(tweets$tweet, keyword_pattern)



> tweets

#>                                                           tweet              keyword

#> 1 Andhra Pradesh is the highest state with crimes against women crimes against women

#> 2                                       I am downloading tweets   downloading tweets

#> 3                                             I love r analysis           r analysis

#> 4                   downloading tweets helps with my r analysis   downloading tweets

As the final example illustrates, you need to think about what you want to do when a tweet contains multiple keywords. In this case, the keyword returned is simply the first one found in the tweet. However, you can also use str_extract_all to return ALL keywords found in the tweet.

We can use stringr which is very handy for string operations and simply use str_extract, i.e.

str_extract(Tweet, Keyword)

#[1] "crimes against women"

For multiple keywords and multiple strings you need to apply, i.e.

Keyword <- c("crimes against women", "something")

Tweet <- c("Andhra Pradesh is the highest state with crimes against women", 

           "another string with something else")



sapply(Tweet, function(i)str_extract(i, paste(Keyword, collapse = '|')))



#    Andhra Pradesh is the highest state with crimes against women                            another string with something else 

#                                           "crimes against women"                                                   "something"