Twitter-analysis

About Project

Collected, cleansed, analyzed, and classified tweets related to the Lifetime television show Dance Moms. Dance Moms is a successful and frequently tweeted reality based show. It currently airs every Tuesday night. It is in its fifth season, and has a viewership of several million people. The official Dance Moms twitter account has about 500,000 followers, and individual show character accounts have up to about 450,000 followers.

The idea of the project was to answer the below question: Question: When Twitters users Tweet about Dance Moms, do they tweet more about the dancing element, or more about the dramatic interactions between the characters?

Data Collection

I used python programming to create a listener to collect live tweets using Twitter REST APIs. TwitterStreaming.py and mylistener.py python code are used to collect data based on a keyword list and twitter handles of the characters, who are associated with the show.

Data Cleaning

Wrote cleaningScript.py to clean the data and kept the relevant tweets/retweets for answering the question. Some of the rules that I followed are given below - Tweets having hashtag reference to the show and its characters Tweets having references to fighting, dancing in relation to the reality show Dance Moms

Data Analysis

I followed a code book to write the python code to identify the sentiment in the tweets. The goal of coding this dataset is to classify tweets as either relating to dancing, relating to fighting, or as unrelated to dancing or fighting. Below are a set of rules used in the code to determine which category a tweet belongs in.

• Ignore URLs included in a tweet.
• Ignore tweets that are written in foreign language.
• Ignore an emoji included in a tweet.
• If there is “fighting” language not related to the show itself, ignore it.
• If there are strong feeling about the show, but it is not related to a specific component of the show, mark it not applicable.
• Spelling errors/typos, try and make a reasonable guess as to the intention of the tweeter

Results

I performed two analyses from our original cleaned data set. One dataset which included all retweets, and one which did not include any. The data set which included retweets (dataset#1) had the following breakdown (Table 1): 11% dancing, 13% fighting, 76% not applicable. The dataset where retweets were removed (dataset #2) was as follows: 13% dancing, 17% fighting, and 70% not applicable.

Table 1. Classification of Dataset #1 and #2 through a combination of human coding and machine learning algorithm coding.

Data Visualizations

Performed exploratory data analysis and created some visualizations, which are shown below and are also available at Visualizations

Conclusion

Reality television is often fueled by conflict, but almost never by conflict alone. There is almost always an underlying story or event that provides context and depth to the conflict the viewer enjoys watching. In the case of Dance Moms, the conflict is contrasted with dancing. What brings viewers to watch reality television is often hard to identify. This remains true when it comes to what compels viewers to tweet as well. It is expected that an analysis of tweets would reveal a mixture of responses from the actual storyline, and the conflict that surrounds the storyline. In fact, this is what we found with Dance Moms.

The results give us insight into the original question: what do people tweet about when they tweet about Dance Moms dancing or fighting? When retweets are counted throughout the week, the numbers are close to even, although fighting still comes out on top. Fighting had approximately 2% more tweets relative to the total body of tweets, than its dancing counterpart.

The results indicate that when only original content is considered and retweets are ignored (dataset #2), tweets are more skewed towards fighting over the actual dancing itself. In fact, in this instance there were approximately 4% more tweets related to fighting than there were about dancing.

Based on the analysis, Twitter users tweet about Dance Moms tweet more frequently about fighting than they do about dancing. This difference becomes more pronounced when retweets are excluded from the analysis.