Chapter 13, Data Mining Twitter 1
Data Mining Twitter
13.1 Introduction
13.1 Q1: Which of the following statements is false?
a. Data mining is the process of searching through large collections of data, often
big data, to find insights that can be valuable to individuals and organizations.
b. You connect to Twitter via web services.
c. You can use Twitter’s Streaming API to tap into the enormous base of past
tweets and its Search API to sample the flood of new tweets as they happen.
d. Because of powerful libraries, you’ll often perform significant tasks with just a
few lines of code.
13.1 Q2: A term commonly used to mean the hundreds of millions of users who
have anything to do with sending, receiving and analyzing tweets is the ________.
a. Twitterverse
b. Twitterorb
c. Twitterglobe
d. Twitterearth
13.1 Q3: Which of the following statements a) b) or c) is false?
a. Hundreds of billions of tweets are sent every day with many millions sent per
second.
b. Tweeters with the largest followings are typically entertainers and politicians.
c. Developers can tap into the live stream of tweets as they’re happening. This has
been likened to “drinking from a fire hose,” because the tweets come at you so
quickly.
d. All of the above statements are true.
13.1 Q4: Which of the following statements a) b) or c) is false?
a. Twitter has become a favorite big data source for researchers and business peo-
ple worldwide. Through special arrangements with Twitter, some third-party
2 Chapter 13, Data Mining Twitter
businesses (and Twitter itself) offer paid access to much larger portions the all–
time tweets database.
b. You can’t always trust everything you read on the Internet, and tweets are no
exception. For example, people might use false information to try to manipulate
financial markets or influence political elections. That’s one of the challenges of
building business-critical and mission-critical systems based on social media con-
tent.
c. We can program with the same reliability as desktop apps when using web ser-
vices.
d. Each of the above statements is true.
13.2 Overview of the Twitter APIs
13.2 Q1: Which of the following statements is false?
a. Twitter’s APIs are cloud-based web services, so an Internet connection is re-
quired. Web services are methods that you call in the cloud.
b. Each API method has a web service endpoint, which is represented by a URL
that’s used to invoke that method over the Internet.
c. Most API methods have rate limits that restrict the number of times you can use
them in 15-minute intervals.
d. All of the above statements are true.
13.2 Q2. Which of the following Tweepy library APIs can you use before you’ve
passed your Twitter credentials to Twitter?
a. Authentication API.
b. Accounts and Users API.
c. Tweets API.
d. Trends API.
13.2 Q3: Which of the following statements a) b) or c) is false?
a. Each Twitter API method has a rate limit, which is the maximum number of
requests (that is, calls) you can make during a 15-minute window. Some methods
list both user rate limits and app rate limits.
b. Twitter may block you from using its APIs if you continue to call a given API
method after that method’s rate limit has been reached.
Chapter 13, Data Mining Twitter 3
c. You can configure Tweepy to wait when it encounters rate limits. This helps
prevent you from exceeding the rate-limit restrictions.
d. All of the above statements are true.
13.3 Creating a Twitter Account
No questions.
13.4 Getting Twitter Credentials—Creating an App
13.4 Q1: Which of the following statements a) b) or c) is false?
a. Once you have a Twitter developer account, you must obtain credentials for
interacting with the Twitter APIs.
b. To use the Twitter APIs, you first create an app.
c. Once you create an app and get a set of credentials, you reuse those credentials
for all apps you create.
d. Each of the above statements is true.
13.5 What’s in a Tweet?
13.5 Q1: Which of the following statements is false?
a. The Twitter API methods return JSON (JavaScript Object Notation) objects.
JSON is a text-based data-interchange format used to represent objects as collec-
tions of name–value pairs.
b. JSON is both a human-readable and computer-readable format that makes data
easy to send and receive across the Internet. It’s commonly used when invoking
web services.
c. JSON objects are similar to Python dictionaries. Each JSON object contains a list
of property names and values, in the following curly braced format:
{propertyName1: value1, propertyName2: value2}
d. JSON arrays are comma-separated values in parentheses:
(value1, value2, value3)
Chapter 13, Data Mining Twitter 5
a. After authenticating with Twitter, you can use the Tweepy API object’s
get_user method to get a tweepy.models.User object containing information
about a user’s Twitter account. The following code gets a User object for NASA’s
@nasa Twitter account:
nasa = api.get_user(‘nasa’)
b. Each Twitter method you call through Tweepy has a rate limit.
c. The tweepy.models classes each correspond to the JSON that Twitter returns.
For example, the User class corresponds to a Twitter user object. Each
tweepy.models class has a method that reads the JSON and turns it into an object
of the corresponding Tweepy class.
d. All of the above statements are true.
13.8 Q2: Which of the following statements about User object properties is true?
a. The id property is the account ID number you create for yourself when you join
Twitter.
b. The name property is the name associated with the user’s Twitter handle (such
as @nasa).
c. The name and screen_name must be real names.
d. All of the above statements are false.
13.8 Q3: Which of the following statements a), b) or c) is false?
a. The User object’s status property returns a tweepy.models.Status object,
which corresponds to a Twitter tweet object. The Status object’s text property
contains the text of the account’s most recent tweet.
b. The text property was originally for tweets up to 140 characters. When Twit-
ter increased the limit to 280 characters, they added an extended_tweet prop-
erty for accessing the text and other information from tweets between 141 and
280 characters.
c. Retweeting sends the characters exactly as they are received in the original
tweet.
d. All of the above statements are true.
Chapter 13, Data Mining Twitter 7
c. A Status’s text property contains the tweet’s text. The following code dis-
plays the screen_name and text for three tweets from @nasa:
nasa_tweets = api.user_timeline(screen_name=‘nasa’,
count=3)
for tweet in nasa_tweets:
print(f‘{tweet.user.screen_name}: {tweet.text}\n’)
d. If you wish to get more than the maximum number of tweets per call (200),
then you should use a Status object to call user_timeline.
13.10 Searching Recent Tweets
13.10 Q1: Which of the following statements is false?
a. The Tweepy API method search returns tweets that match a query string.
b. Twitter maintains its search index only for the previous seven days’ tweets.
c. A search is guaranteed to return all matching tweets.
d. Method search calls Twitter’s search/tweets method, which returns 15
tweets at a time by default, but can return up to 100.
13.10 Q2: Which of the following statements about the Twitter search operators
is false?
a. You can use various Twitter search operators in your query strings to refine
your search results. Multiple operators can be combined to construct more com-
plex queries.
b. The Twitter search expression flu? finds tweets that ask questions about flu.
c. The Twitter search expression tools –hammer finds tweets that contain tools
but not hammer.
d. The following code uses three Twitter search operators.
tweets = api.search(q=‘from:nasa since:2018–09–01′,
count=3)
10 Chapter 13, Data Mining Twitter
© Copyright 2020 by Pearson Education, Inc. All Rights Reserved.
Answer: a. Actually, Twitter’s free Streaming API sends to your app ran-
domly selected tweets dynamically as they occur—up to a maximum of one
percent of the tweets throughout day.
13.13.1 Creating a Subclass of StreamListener
13.13 Q2: Which of the following statements is false?
a. The Streaming API returns tweets that match your search criteria as they hap-
pen.
b. Rather than connecting to Twitter on each method call, a stream uses a persis-
tent connection to push (that is, send) tweets to your app.
c. The rate at which tweets arrive varies tremendously, based on your search cri-
teria. The more popular a topic is, the more likely it is that the tweets will arrive
quickly.
d. You create a subclass of Tweepy’s TweetListener class to process a tweet
stream. An object of this class is the listener that’s notified when each new tweet
arrives. Each message Twitter sends results in a call to a TweetListener
method.
13.13.2 Initiating Stream Processing
No questions.
13.14 Tweet Sentiment Analysis
13.14 Q1: Which of the following statements about applications of Twitter senti-
ment analysis are true?
a. Political researchers might check tweet sentiment during elections season to
understand how people feel about specific politicians and issues.
b. Companies might check tweet sentiment to see what people are saying about
their products and competitors’ products.
c. Both (a) and (b) are true.
d. Neither (a) nor (b) are true.
12 Chapter 13, Data Mining Twitter
df = pd.DataFrame(tweets)
df = df.dropna()
d. All of the above statements are true.
13.15.2 Class LocationListener
No questions.
13.16 Ways to Store Tweets
13.16 Q1: Which of the following statements is false? For analysis, you’ll com-
monly store tweets in:
a. CSV files.
b. pandas DataFrames in memory—CSV files can be loaded easily into Data-
Frames for cleaning and manipulation.
c. SQL databases—Such as MySQL, a free and open source relational database
management system (RDBMS).
d. NoSQL databases—Twitter returns tweets as XML documents, so the natural
way to store them is in a NoSQL XML document database, such as MongoDB.
Tweepy generally hides the XML from the developer, but you can manipulate it
directly, if you wish, using the PyMongo library.
13.17 Twitter and Time Series
13.17 Q1: Which of the following statements is false?
a. A time series is a sequence of values with timestamps. Some time series exam-
ples are daily closing stock prices, daily high temperatures at a given location,
monthly U.S. job-creation numbers, quarterly earnings for a given company and
more.
b. Tweets could be a natural for time-series analysis if they were time stamped.
In its next major release, Twitter is consider adding time stamps to tweets.
c. A popular machine learning technique called simple linear regression is used
to make predictions with time series.
d. A popular deep learning technique called recurrent neural networks also is
used to make predictions with time series.
Chapter 13, Data Mining Twitter 13
© Copyright 2020 by Pearson Education, Inc. All Rights Reserved.
Answer: b. Actually, Tweets are natural for time-series analysis specifically
because they already are time stamped.