Social Media Sentiment Analysis

Sentiment analysis, sometimes called opinion mining, is essentially a natural language processing task that aims to detect sentiment, emotion and opinion in free-form textual data. The primary aim is to measure the sentiment expressed in text computationally using unsupervised or supervised approaches.

Organizations in most industries rely on data to extract insights for decision making, business analytics, CRM and marketing campaigns. Text data, being unstructured, is hard to manually inspect, and comes in many forms, including text documents, PDFs, Excel sheets, blog posts, forum discussions, microblog/Twitter posts, emails, SMSs, chat/Whatsapp messages, surveys, email threads, and the list goes on. This is especially true for social media data, such as Facebook, LinkedIn and Twitter posts.

Why? Because users do not follow formal spelling and grammar, and adopt their own internet slang. They use their own customized acronyms and shortened words, emojis, neologisms, foreign languages, emphatic lengthening, etc. This leaves us with informal, noisy text that is hard to parse by NLP programs that follow conventional spelling and grammatical rules. Modern sentiment analysis systems such as Vader are designed to handle this informal text. This allows these organizations to attempt to better understand these noisy and big data sets, and in turn extract useful insights from them for decision-making, with minimal manual inspection.

Vader

Vader, or the Valence Aware Dictionary and sEntiment Reasoner, is an unsupervised, lexicon-based sentiment analysis system that was built to handle informal, social media style text streams. It also contains hand crafted rules to capture context of words, and to handle social media type of syntax such as emphatic lengthening, all-caps, acronyms, and slang. The creators prove its robustness across multiple domains, for both formal (e.g. news articles) and informal (e.g. Twitter posts) text, giving it an advantage over conventional systems built to parse only formal text.

How it works

Vader uses the polarity_scores() function on a string to compute the neg, neu, pos and compound scores. The neg, neu and pos scores all must sum up to 1, and indicate the portion of text that is negative, neutral and positive respectively. The compound score is the final sentiment polarity strength of the text, after adding all the scores of each word in the text, and adjusting according to the built-in rules. This is in the range of -1 and +1, where a strength of -1 indicates maximum negativitiy and a strength of +1 indicates maximum positivity. They mention that, in order to classify sentences as positive or negative, the following thresholds should be used:

positive sentiment: compound score >= 0.05
neutral sentiment: (compound score > -0.05) and (compound score< 0.05)
negative sentiment: compound score <= -0.05

Examples

The code below uses Vader to classify the examples provided on their Github page.

from vaderSentiment.vaderSentiment import SentimentIntensityAnalyzer
sentiment = SentimentIntensityAnalyzer()
examples = ["VADER is smart, handsome, and funny.",  # positive sentence example
             "VADER is smart, handsome, and funny!",  # punctuation emphasis handled correctly (sentiment intensity adjusted)
             "VADER is very smart, handsome, and funny.", # booster words handled correctly (sentiment intensity adjusted)
             "VADER is VERY SMART, handsome, and FUNNY.",  # emphasis for ALLCAPS handled
             "VADER is VERY SMART, handsome, and FUNNY!!!", # combination of signals - VADER appropriately adjusts intensity
             "VADER is VERY SMART, uber handsome, and FRIGGIN FUNNY!!!", # booster words & punctuation make this close to ceiling for score
             "VADER is not smart, handsome, nor funny.",  # negation sentence example
             "The book was good.",  # positive sentence
             "At least it isn't a horrible book.",  # negated negative sentence with contraction
             "The book was only kind of good.", # qualified positive sentence is handled correctly (intensity adjusted)
             "The plot was good, but the characters are uncompelling and the dialog is not great.", # mixed negation sentence
             "Today SUX!",  # negative slang with capitalization emphasis
             "Today only kinda sux! But I'll get by, lol", # mixed sentiment example with slang and constrastive conjunction "but"
             "Make sure you :) or :D today!",  # emoticons handled
             "Catch utf-8 emoji such as such as 💘 and 💋 and 😁", 
             "Not bad at all"  # Capitalized negation
             ]
for example in examples:
    vs = sentiment.polarity_scores(example)
    print(example)
    print(str(vs))

As shown in the output below, it seems to handle the differences in these sentences well. For example, the compound score is increased from 0.8316 to 0.8439 after adding an exclamation mark to emphasize a stronger positive sentiment.

VADER is smart, handsome, and funny.
{'neg': 0.0, 'neu': 0.254, 'pos': 0.746, 'compound': 0.8316}
VADER is smart, handsome, and funny!
{'neg': 0.0, 'neu': 0.248, 'pos': 0.752, 'compound': 0.8439}
VADER is very smart, handsome, and funny.
{'neg': 0.0, 'neu': 0.299, 'pos': 0.701, 'compound': 0.8545}
VADER is VERY SMART, handsome, and FUNNY.
{'neg': 0.0, 'neu': 0.246, 'pos': 0.754, 'compound': 0.9227}
VADER is VERY SMART, handsome, and FUNNY!!!
{'neg': 0.0, 'neu': 0.233, 'pos': 0.767, 'compound': 0.9342}
VADER is VERY SMART, uber handsome, and FRIGGIN FUNNY!!!
{'neg': 0.0, 'neu': 0.294, 'pos': 0.706, 'compound': 0.9469}
VADER is not smart, handsome, nor funny.
{'neg': 0.646, 'neu': 0.354, 'pos': 0.0, 'compound': -0.7424}
The book was good.
{'neg': 0.0, 'neu': 0.508, 'pos': 0.492, 'compound': 0.4404}
At least it isn't a horrible book.
{'neg': 0.0, 'neu': 0.678, 'pos': 0.322, 'compound': 0.431}
The book was only kind of good.
{'neg': 0.0, 'neu': 0.697, 'pos': 0.303, 'compound': 0.3832}
The plot was good, but the characters are uncompelling and the dialog is not great.
{'neg': 0.327, 'neu': 0.579, 'pos': 0.094, 'compound': -0.7042}
Today SUX!
{'neg': 0.779, 'neu': 0.221, 'pos': 0.0, 'compound': -0.5461}
Today only kinda sux! But I'll get by, lol
{'neg': 0.127, 'neu': 0.556, 'pos': 0.317, 'compound': 0.5249}
Make sure you :) or :D today!
{'neg': 0.0, 'neu': 0.294, 'pos': 0.706, 'compound': 0.8633}
Catch utf-8 emoji such as such as 💘 and 💋 and 😁
{'neg': 0.0, 'neu': 0.746, 'pos': 0.254, 'compound': 0.7003}
Not bad at all
{'neg': 0.0, 'neu': 0.513, 'pos': 0.487, 'compound': 0.431}

Full code

The code below classifies the first 200,000 positive tweets and the first 200,000 negative tweets from the Sentiment140 data set. It then computes the accuracy, and prints it in a bar chart.

The output is too large to be displayed here, so the output for only the final three tweets is shown. The final accuracy for positive is about 62.8%, and for negative it is about 41.9%. This model can certainly perform better if fine-tuned according to the data set used. It took about 596 seconds (~10 min) to process 400,000 tweets using my Intel Core i7 CPU laptop. The bar chart showing the accuracy is also shown below.

tweet 199998 
 @damienfranco Its so common for it to crash now   I find I have to delete the process then its ok again for a while its eating memory
{'neg': 0.09, 'neu': 0.836, 'pos': 0.074, 'compound': -0.128}
Classified as Negative
~~~~~~~~~~~~~~~~~~~~~~~
tweet 199999 
 my baby boy is wearing big boy underwear 
{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}
Classified as Neutral
~~~~~~~~~~~~~~~~~~~~~~~
tweet 200000 
 Fml! I forgot my phone charger @home! 
{'neg': 0.0, 'neu': 1.0, 'pos': 0.0, 'compound': 0.0}
Classified as Neutral
~~~~~~~~~~~~~~~~~~~~~~~
Positive accuracy is 62.844% via 200000 tweets
Negative accuracy is 41.974000000000004% via 200000 tweets
~~~~~-Execution time: 596.1883578300476 Seconds~~~~~

Final words

Here we used Vader to classify social media posts (tweets) as positive or negative, and then compute the overall accuracy. The model could certainly perform better if modifying the settings and checking what helps to increase accuracy. Regardless of the outcome, the aim here is to learn how it practically works. Drop your thoughts in the comments below! 🙂