Top 10 Established Datasets for Sentiment Analysis in 2024

Sentiment analysis is the technique used for understanding people’s emotions and feelings, with the help of machine learning, regarding a particular product or service. Sentiment analysis models require a high volume of a specific dataset. 

One of the most challenging aspects of creating and training a model is acquiring the right volume and type of sentiment analysis dataset. At upGrad, we have compiled a list of ten accessible datasets that can help you get started with your project on sentiment analysis.

Sentiment analysis denotes how customers feel about a company. It determines whether the customers are related to investments, sales, or agreements. Many factors contribute to guaranteeing a trustworthy sentiment analysis. One of its key elements is the dataset utilized to train the models. But it is challenging to find a suitable dataset.

The data’s quantity is vital, but its quality also matters to guarantee reliable outcomes. For example, suppose a retail firm uses a dataset with financial terminologies to train a customer sentiment analysis model. The corresponding algorithm might not offer trustworthy results in this aspect based sentiment analysis. This is because the words that the algorithm would assess will be derived from a financial perspective.  Hence, using the correct training dataset is critical in assessing the reviews. It helps you to develop novel strategies with the insights being gathered.

 Sentiment analysis is widely used for brand monitoring, social media monitoring, market research, customer service, and the voice of the customer (VoC). The sentiment analysis in R uses algorithms and NLP methods that are hybrid, rule-based, or depend on machine learning methods to acquire data from datasets.

The data required in the sentiment analysis must be specialized and are needed in huge quantities. The greatest difficulty of the sentiment analysis training procedure is not finding the data in huge amounts; rather, it is finding the pertinent datasets. Such datasets should encompass a broad range of sentiment analysis use cases and applications.

Source 

Sentiment Analysis Datasets

1. Stanford Sentiment Treebank

The first dataset for sentiment analysis we would like to share is the Stanford Sentiment Treebank. The dataset contains user sentiment from Rotten Tomatoes, a great movie review website. 

It contains over 10,000 pieces of data from HTML files of the website containing user reviews. The sentiments are rated on a linear scale between 1 to 25. One is the most negative, whereas 25 is the most positive sentiment. The dataset is free to download, and you can find it on the Stanford website.

It builds a sentiment analysis system that automatically finds out the user opinions of the Stanford Sentiment Treebank in the context of three sentiments. These sentiments are negative, positive, and neutral.

Firstly, sentiment sentences are POS tagged and analyzed to the dependency structures. The Treebank’s all nodes, and their polarities are automatically accessed. Two Support Vector Machines models are trained in this dataset. One is for a node-level classification and the second is for a sentence level. Different types of features like POS tags, sentiment lexicons, word lexicons, sibling relations, and head-modifier relations are used in this dataset.

This dataset is a corpus with fully labeled parse trees. These trees facilitate a comprehensive analysis of the compositional effects of sentiment in language. Moreover, this corpus in sentiment analysis in R depends on the dataset presented by Pang and Lee in 2005. It includes 11,855 single sentences that are retrieved from movie reviews. Furthermore, it was analyzed with the Stanford parser and contains 215,154 unique phrases from these parse trees. Each of them is marked by 3 human judges.

2. IMDB Movie Reviews Dataset

The second dataset on our list is the IMDB Movie Reviews dataset. It has 25,000 user reviews from IMDB. The dataset is classified binary and also contains additional unlabelled data that can be used for training and testing purposes. 

The dataset is available to download from Kaggle or Stanford website, labeled ‘Large Movie Review Dataset. If you’re looking for an IMDB user reviews dataset for sentiment analysis, there are plenty of options available. You can choose one according to your purpose and use.

It features a huge movie dataset that includes a collection of approx. 50,000 movie reviews from IMDB.  Only highly polarized reviews are considered in this dataset. The negative and positive reviews are even in number. But the negative review records a score of ≤ 4 from 10. The positive review records a score of ≥ 7 from 10.

For example, movie reviews that feature positive sentiment like “It is a good movie” obtain a label of 1 for the last value. The negative sentiment reviews obtain a label of 0.  For the practical use case of sentiment analysis in R, you can allow longer reviews, for example, a word length of up to 70. It helps you to obtain more test and training reviews.

Read: Best Datasets for Machine Learning Projects

3. Paper Reviews Data Set

The Paper Reviews dataset contains reviews mostly in Spanish and English from a conference on computing. It has a total of 405 instances (N), which is evaluated with a 5-point scale. The evaluation done is as follows:

• -2: very negative
• -1: negative
• 0: neutral
• 1: positive
• 2: very positive

The sentiment score expresses the user’s opinion about the paper. The dataset can be useful in predicting the opinion of academic paper reviews. The dataset is available for download from the University of California website.

Learn Artificial Intelligence Course from the World’s top Universities. Earn Masters, Executive PGP, or Advanced Certificate Programs to fast-track your career.

The distribution of marks is consistent. There is a distinction between the review written by the reviewer and how the article is rated.

4. Twitter US Airline Sentiment

The Twitter US Airline Sentiment dataset, as the name suggests, contains tweets of user experience related to significant US airlines. The dataset includes tweets since February 2015 and is classified as positive, negative, or neutral. 

The dataset contains information such as the Twitter user ID, airline name, date and time of the tweet, and the airlines’ negative experiences. The dataset is available for download from Kaggle.

FYI: Free Deep Learning Course!

It covers the sentiment confidence score, Twitter user IDs, positive and negative reasons, tweet text, time, date, location, and retweet counts.

This sentiment analysis dataset includes negative and positive tagged reviews for plenty of Amazon products. The corresponding reviews feature ratings from 1 to 5 stars. The same can be transformed to binary if needed.

Different companies utilize social media to collect information from customers. The customers share their feedback and opinions about the services and products. Social Media turned out to be advantageous for companies because it helps them to analyze the feedback to enhance their products and services.

This dataset classifies twitter data into feedback through NLP and machine learning. It uses Python programming language. This sentiment analysis python categorizes tweets into positive, negative, and neutral sentiments. Various ML algorithms are used for the same. Using pre-trained word embedding and Keras, you can build a recurrent neural network (RNN) to categorize the sentiment of tweets about the airlines.

Exploratory Data Analysis

Firstly, this dataset looks at the counts for every sentiment label. Suppose the negative tweets are more than both neutral and positive combined. It means that there are more chances for people to be vocal on social media platforms like Twitter when something abnormal happens with their flight in contrast to when nothing wrong happens.

You can develop a word cloud for every category to better comprehend the dissimilarity in sentiments for negative, positive, and neutral tweets. Before creating this word count, make sure you clean the tweets by discarding special characters, Twitter handles, punctuation, numbers, and small words (less than or equal to 3 characters). It allows you to check words that are illustrative for every sentiment in contrast to words that are likely seen over each label.

This sentiment analysis python does not clean the actual tweets. Also, it decides not to discard special characters, Twitter handles, and punctuation.  The reason is that RNNs will automatically learn these patterns and variations.

5. Sentiment140

The Sentiment140 dataset for sentiment analysis is used to analyze user responses to different products, brands, or topics through user tweets on the social media platform Twitter. The dataset was collected using the Twitter API and contained around 1,60,000 tweets. The data is sorted into six fields;

• The polarity of the tweet (0 = negative, 2 = neutral, 4 = positive)
• The ID of the tweet
• The date of the tweet
• The query
• The Twitter user
• The textual data contained in the tweet

The dataset can be downloaded from the Sentiment140’s or Stanford’s website. The dataset is useful for brand management, polling, and purchase planning purposes.

Read: Top 4 Types of Sentiment Analysis & Where to Use

6. Opin-Rank Review Dataset

The Opin-Rank review dataset for sentiment analysis contains user reviews, around 3,00,000, about cars and hotels. The dataset comprises user reviews collected from websites such as Edmunds (cars), and TripAdvisor (hotels). 

The majority of the dataset contains full reviews from TripAdvisor, approx 2,59,000. Edmunds user reviews stand at approx 42,230. There are comprehensive reviews of hotels in 10 different cities from across the globe, such as Dubai, Chicago, Las Vegas, and Delhi, to name a few. The data fields include the date, review title, and the full review.

Similarly, there are car reviews from Edmund of car models from the year 2007 – 2009. The review data includes the date, author names, favorites, and the full report. The dataset is available to download from the GitHub website.

Three folders (2007, 2008, 2009) depict three model years in this sentiment analysis python example. Every file in these three folders would include all reviews for a specific car. The filename shows the car’s name. You need to set reviews in the following format in each car file.

<DOC>

<DATE>07/20/2009</DATE>

<AUTHOR>The author</AUTHOR>

<TEXT>Here is the review</TEXT>

<FEATURES>My favorite features about this car</ FEATURES >

</DOC>

Every review is surrounded by an <DOC> element. All the extracted items exist in this element.

7. Amazon Product Data

The Amazon product data is a subset of a much larger dataset for sentiment analysis of amazon products. The superset contains a 142.8 million Amazon review dataset. This subset was made available by Stanford professor Julian McAuley. 

It provides user reviews from May 1996 to July 2014 for products listed across various categories on Amazon. There is an updated version (2018 edition) available for download. It contains 233.1 million user reviews from May 1996 to Oct 2018. 

The old dataset can be downloaded from the University of San Diego website, whereas the new dataset can be found on GitHub. Both datasets contain data points such as ratings, price, product description, and helpful votes, to name a few. The new dataset contains additional data such as technical details and similar product tables. The reviews are categorized depending on their negative, positive, and neutral emotional tone in this aspect based sentiment analysis.

Check out: Sentiment Analysis Using Python: A Hands-on Guide

8. WordStat Sentiment Dictionary

The WordStat Sentiment Dictionary dataset for sentiment analysis was designed by integrating positive and negative words from the Harvard IV dictionary, the Regressive Imagery Dictionary, and the Linguistic and Word Count dictionary. It contains about 15,000 words of data combined. 

The dataset takes into account negations to classify user sentiment either as positive or negative. The dataset is available for the public for download. However, you cannot use it for commercial purposes without authorization. You can download the latest version of the dataset from Provalisresearch’s website.

Also Read: Top ML Dataset Project Ideas

9. Sentiment Lexicons For 81 Languages

Source

As the name suggests, the Sentiment Lexicon for 81 languages contains contextual data from Afrikaans to English to Yiddish, for a total of 81 words. The data includes positive as well as negative lexicons for the number mentioned above of languages. The dataset is useful for analysts and data scientists working on Natural Language Processing projects such as chatbots. 

Read: How to make chatbot in Python?

10. Bag of Words Meets Bag of Popcorns

The last but not least dataset for sentiment analysis is ‘bag of words meets the bag of popcorns.’ As you may have guessed, this dataset is also related to user sentiment of movies. It consists of 50,000 IMDB reviews. The dataset uses the binary classification for user sentiment. If the IMDB rating is less than 5 for a particular movie, the sentiment score is 0. Similarly, if the rating is greater than or equal to 7, the sentiment score is 1. You can download the dataset from Kaggle.

You need to transform the sentences to some type of numeric depiction for machine learning. A common approach you can use is a Bag of Words. It learns a vocabulary from all of the documents. Subsequently, it models every document by counting the number of times every word occurs.

We have a huge number of reviews in the IMDB data. It provides you with a huge vocabulary. You must select the maximum vocabulary size to restrict the feature vectors’ size. You can use 5000 most frequent words in this aspect based sentiment analysis. Make sure the stop words are already removed. You get 25,000 rows and a total of 5,000 features.

Conclusion

We hope this blog covering ten diverse datasets for sentiment analysis helped you. If you’re further interested in learning about sentiment analysis and the technologies associated, such as artificial intelligence and machine learning, you can check our Executive PG Programme in Machine Learning & AI course.