Chapter 12: Natural Language Processing 1
Natural Language Processing (NLP)
12.1 Introduction
12.1 Q1: In the field of NLP, a text collection is generally known as a ________.
a. corpus
b. compilation
c. book
d. volume
12.1 Q2: Which of the following statements a), b) or c) is false?
a. Natural language lacks mathematical precision.
b. Nuances of meaning make natural language understanding difficult.
c. A text’s meaning can be influenced by its context and the reader’s “world view.”
d. All of the above statements are true.
12.2 TextBlob
12.2 Q1: Splitting text into meaningful units, such as words and numbers is called
________.
a. inflectionization
b. tokenization.
c. lemmatization.
d. parts-of–speech tagging
12.2 Q2: Which of the following is not a TextBlob capability?
a. Parts-of–speech (POS) tagging.
b. Language detection.
c. Sentiment analysis.
d. Similarity detection.
12.2 Q3: ________ are sets of consecutive words in a corpus for use in identifying
words that frequently appear adjacent to one another.
a. blobs
2 Chapter 12: Natural Language Processing
b. n-grams
c. stems
d. inflections
12.2.1 Create a TextBlob
12.2 Q4: Which of the following statements is false?
a. TextBlob is the fundamental class for NLP with the textblob module.
b. The following code creates a TextBlob containing two sentences:
from textblob import TextBlob
text = ‘Today is a beautiful day. Tomorrow looks like bad
weather.‘
blob = TextBlob(text)
c. TextBlobs, Sentences and Words cannot be compared with strings.
d. Sentences, Words and TextBlobs inherit from BaseBlob, so they have many
common methods and properties.
12.2.2 Tokenizing Text into Sentences and Words
12.2 Q5: Assuming you have a TextBlob named blob containing ‘Today is a
beautiful day. Tomorrow looks like bad weather.’, what property should
replace the ? in the following snippet to get the output shown below?
In [8]: blob.?
Out[8]: WordList([‘Today’, ‘is’, ‘a’, ‘beautiful’, ‘day’,
‘Tomorrow’, ‘looks’, ‘like’, ‘bad’, ‘weather’])
a. word
b. wordlist
c. words
d. None of the above
12.2.3 Parts-of-Speech Tagging
12.2 Q6: Which of the following statements a), b) or c) is false?
Chapter 12: Natural Language Processing 3
a. Parts-of-speech (POS) tagging is the process of evaluating words based on their
context to determine each word’s part of speech.
b. There are eight primary English parts of speech—nouns, pronouns, verbs, ad-
jectives, adverbs, prepositions, conjunctions and interjections (words that ex-
press emotion and that are typically followed by punctuation, like “Yes!” or “Ha!”).
c. An important use of POS tagging is determining a word’s meaning among its
possibly many meanings—this is important for helping computers “understand”
natural language.
d. All of the above statements are true.
12.2.4 Extracting Noun Phrases
No questions.
12.2.5 Sentiment Analysis with TextBlob’s Default Sentiment Analyzer
12.2 Q7: Which of the following statements a), b) or c) is false?
a. One of the most common and valuable NLP tasks is sentiment analysis, which
determines whether text is positive, neutral or negative.
b. Companies might use sentiment analysis to determine whether people are
speaking positively or negatively online about their products.
c. A sentence that contains the word “good” has positive sentiment.
d. All of the above statements are true.
12.2 Q8: Consider the following code:
In [18]: blob
Out[18]: TextBlob(“Today is a beautiful day. Tomorrow looks
like bad weather.”)
4 Chapter 12: Natural Language Processing
In [19]: blob.sentiment
Out[19]: Sentiment(polarity=0.07500000000000007,
subjectivity=0.8333333333333333)
Which of the following statements is false?
a. A TextBlob’s sentiment property returns a Sentiment object indicating
whether the text is positive or negative and whether it’s objective or subjective.
b. The polarity indicates sentiment with a value from –1.0 (negative) to 1.0
(positive) with 0.0 being neutral.
c. The subjectivity is a value from 0.0 (objective) to 1.0 (subjective).
d. Based on the values for this TextBlob, the overall sentiment is close to neutral,
and the text is mostly objective.
12.2.6 Sentiment Analysis with the NaiveBayesAnalyzer
No questions.
12.2.7 Language Detection and Translation
12.2 Q9: Which of the following statements is false?
a. The TextBlob library uses Google Translate to detect a text’s language and
translate TextBlobs, Sentences and Words into other languages.
b. The following code uses TextBlob’s translate method to translate a Text-
Blob’s text to Spanish (‘es’) then detect the language on the result. The to key-
word argument specifies the target language.
In [38]: spanish = blob.translate(to=‘es’)
In [39]: spanish
Out[39]: TextBlob(“Hoy es un hermoso dia. Mañana parece mal tiempo.”)
In [40]: spanish.detect_language()
Out[40]: ‘es’
c. Calling TextBlob’s translate method without arguments translates from the
detected source language to English.
d. All of the above statements are true.
6 Chapter 12: Natural Language Processing
© Copyright 2020 by Pearson Education, Inc. All Rights Reserved.
Answer: d. Actually, the word with the highest confidence value might not
be the correct word for the given context.
12.2 Q12: Which of the following statements a), b) or c) is false?
a. TextBlobs, Sentences and Words all have a correct method that you can call
to correct spelling.
b. Assuming word is a Word object containing ‘theyr’, calling correct on word
returns the correctly spelled word that has the highest confidence value (as re-
turned by spellcheck).
c. Calling correct on a TextBlob or Sentence checks the spelling of each word.
For each incorrect word, correct replaces it with the correctly spelled one that
has the highest confidence value:
In [6]: from textblob import Word
In [7]: sentence = TextBlob(‘This sentense has missplled wrds.’)
In [8]: sentence.correct()
Out[8]: TextBlob(“This sentence has misspelled words.”)
d. All of the above statements are true.
12.2.10 Normalization: Stemming and Lemmatization
12.2 Q13: Which of the following statements is false?
a. Stemming removes a prefix or suffix from a word leaving only a stem, which
may or may not be a real word.
b. Lemmatization is similar to stemming, but factors in the word’s part of speech
and meaning and results in a real word.
c. Stemming and lemmatization are normalization operations, in which you pre-
pare words for analysis.
d. Words support stemming and lemmatization via the methods stem and lem-
matize. The following code correctly stems and lemmatizes a Word:
In [1]: from textblob import Word
In [2]: word = Word(‘varieties‘)
In [3]: word.stem()
Out[3]: ‘variety’
Chapter 12: Natural Language Processing 7
In [4]: word.lemmatize()
Out[4]: ‘varieti’
12.2.11 Word Frequencies
No questions.
12.2.12 Getting Definitions, Synonyms and Antonyms from WordNet
12.2 Q14: Given the following Word object:
In [1]: from textblob import Word
In [2]: happy = Word(‘happy’)
which of the following statements a), b) or c) is false?
a. The TextBlob library uses the NLTK library’s WordNet interface, enabling you
to look up word definitions, and get synonyms and antonyms.
b. The Word class’s definitions property returns a list of all the word’s defini-
tions in the WordNet database:
In [3]: happy.definitions
Out[3]:
[‘enjoying or showing or marked by joy or pleasure’,
‘marked by good fortune’,
‘eagerly disposed to act or to be of service’,
‘well expressed and to the point’]
c. The Word class’s define method enables you to pass a part of speech as an
argument so you can get definitions matching only that part of speech.
8 Chapter 12: Natural Language Processing
d. All of the above statements are true.
12.2 Q15: Which of the following statements a), b) or c) is false?
a. You can get a Word’s synsets—that is, its sets of synonyms—via the synsets
property. The result of applying this property to a Word is a list of Synset objects:
In [4]: happy.synsets
Out[4]:
[Synset(‘happy.a.01’),
Synset(‘felicitous.s.02’),
Synset(‘glad.s.02’),
Synset(‘happy.s.04’)]
b. Each Synset represents a group of synonyms. In the code in part b) above, the
notation happy.a.01:
• happy is the original Word’s lemmatized form (in this case, it’s the same).
• a is the part of speech, which can be a for adjective, n for noun, v for verb,
r for adverb or s for adjective satellite.
• 01 is a 0-based index number. Many words have multiple meanings, and
this is the index number of the corresponding meaning in the WordNet
database.
c. There’s also a get_synsets method that enables you to pass a part of speech
as an argument so you can get Synsets matching only that part of speech.
d. All of the above statements are true.
12.2.13 Deleting Stop Words
12.2 Q16: Which of the following statements is false?
a. Stop words are common words like “a,” “of,” “is,” “it,” and the like that are often
removed from text before analyzing it because they typically do not provide use-
ful information. Before using NLTK’s stop-words lists, you must download them,
which you do with the nltk module’s download function:
In [1]: import nltk
In [2]: nltk.download(‘stopwords’)
b. The following code loads the ‘english’ stop words list:
Chapter 12: Natural Language Processing 9
In [3]: from nltk.corpus import stopwords
In [4]: stops = stopwords.words(‘english‘)
c. The following code creates a TextBlob from which we can remove stop words:
In [5]: from textblob import TextBlob
In [6]: blob = TextBlob(‘Today is a beautiful day.’)
d. To remove the stop words, we use the TextBlob’s words function in a list com-
prehension that adds each word to the resulting list only if the word is not in
stops:
In [7]: [word for word in blob.words]
Out[7]: [‘Today’, ‘beautiful’, ‘day’]
12.2.14 n-grams
12.2 Q17: Which of the following statements is false?
a. An n-gram is a sequence of n text items, such as letters in words or words in a
sentence. In natural language processing, n-grams can be used to identify letters
or words that frequently appear adjacent to one another.
b. For text-based user input, n-grams can help predict the next letter or word a
user will type—such as when completing items in IPython with tab-completion
or when entering a message to a friend in your favorite smartphone messaging
app. For speech-to-text, n-grams might be used to improve the quality of the tran-
scription.
c. You can pass the keyword argument n to TextBlob’s ngrams method to pro-
duce n-grams of any desired length.
d. The following code uses TextBlob’s ngrams method to create all the trigrams
from a sentence. The code is actually incorrect—it should have used the keyword
argument n=3 to TextBlob’s ngrams method:
In [1]: from textblob import TextBlob
10 Chapter 12: Natural Language Processing
In [2]: text = ‘Today is a beautiful day. Tomorrow looks like
bad weather.’
In [3]: blob = TextBlob(text)
In [4]: blob.ngrams()
Out[4]:
[WordList([‘Today’, ‘is’, ‘a’]),
WordList([‘is’, ‘a’, ‘beautiful’]),
WordList([‘a’, ‘beautiful’, ‘day’]),
WordList([‘beautiful’, ‘day’, ‘Tomorrow’]),
WordList([‘day‘, ‘Tomorrow’, ‘looks’]),
WordList([‘Tomorrow’, ‘looks’, ‘like’]),
WordList([‘looks’, ‘like’, ‘bad’]),
WordList([‘like’, ‘bad’, ‘weather‘])]
12.3 Visualizing Word Frequencies with Bar Charts and
Word Clouds
No questions.
12.3.1 Visualizing Word Frequencies with Pandas
No questions.
12.3.2 Visualizing Word Frequencies with Word Clouds
12.3 Q1: Which of the following statements a), b) or c) is false?
a. You can use the open source wordcloud module’s WordCloud class to generate
word clouds with just a few lines of code. By default, wordcloud creates rectan-
gular word clouds, but the library can create word clouds with arbitrary shapes.
b. To create a word cloud of a given shape, you can initialize a WordCloud object
with an image known as a mask. The WordCloud fills non-white areas of the mask
image with text.
c. The following code loads a mask image by using the imread function from the
imageio module that comes with Anaconda:
import imageio
Chapter 12: Natural Language Processing 11
mask_image = imageio.imread(‘mask_heart.png’)
This function returns the image as a NumPy array, which is required by
WordCloud.
d. All of the above statements are true.
12.3 Q2: The following code creates and configures a WordCloud object:
from wordcloud import WordCloud
wordcloud = WordCloud(colormap=‘prism‘, mask=mask_image,
background_color=‘white’)
Which of the following statements is false?
a. The default WordCloud width and height in pixels is 400×200, unless you spec-
ify width and height keyword arguments or a mask image. The mask keyword
argument specifies the mask image to use.
b. For a mask image, the WordCloud size is the image’s size.
c. WordCloud uses Matplotlib under the hood. WordCloud assigns random colors
from a color map. You can supply the colormap keyword argument and use one
of Matplotlib’s named color maps.
d. By default, the word is drawn on a white background.
12.3 Q3: Which of the following statements about WordCloud is false?
a. WordCloud’s generate method receives the text to use in the word cloud as
an argument and creates the word cloud, which it returns as a WordCloud object.
b. Before creating a word cloud, you should explicitly remove stop words from
the text to get the best word cloud.
c. Method generate calculates the word frequencies for the remaining words af-
ter stop words are removed.
d. Method generate uses a maximum of 200 words in the word cloud by default,
but you can customize this with the max_words keyword argument.
