Programming Languages Chapter 15 Machine Learning Classification Regression And Clustering Machine Learning Classification Regression And

Chapter 15, Machine Learning: Classification, Regression and Clustering 1

Machine Learning: Classification,

Regression and Clustering

15.1 Introduction to Machine Learning

15.1 Q1: Which of the following statements a), b) or c) is false?

a. We can make machines learn.

b. The “secret sauce” of machine learning is data—and lots of it.

c. With machine learning, rather than programming expertise into our applica-

tions, we program them to learn from data.

d. All of the above statements are true.

15.1.1 Scikit-Learn

15.1 Q2: Which of the following statements a), b) or c) is false?

a. Scikit-learn conveniently packages the most effective machine-learning algo-

rithms as evaluators.

b. Each scikit-learn algorithm is encapsulated, so you don’t see its intricate details,

including any heavy mathematics.

c. With scikit-learn and a small amount of Python code, you can create powerful

models quickly for analyzing data, extracting insights from the data and making

predictions.

d. All of the above statements are true.

15.1 Q3: Which of the following statements is false?

a. With scikit-learn, you train each model on a subset of your data, then test each

model on the rest to see how well your model works.

b. Once your models are trained, you put them to work making predictions based

on data they have not seen.

c. With machine learning, your computer will take on characteristics of intelli-

gence.

d. Although you can specify parameters to customize scikit-learn models and pos-

sibly improve their performance, if you use the models’ default parameters for

simplicity, you’ll generally obtain mediocre results.

Chapter 15, Machine Learning: Classification, Regression and Clustering 3

c. In supervised machine learning, each sample has an associated label called a

target (like “spam” or “not spam” for classifying e-mails). This is the value you’re

trying to predict for new data that you present to your models.

d. All of the above statements are true.

15.1 Q7: Which of the following statements a), b) or c) is false?

a. “Toy” datasets, generally have a small number of samples with a limited num-

ber of features. In the world of big data, datasets commonly have millions and

billions of samples, or even more.

b. There’s an enormous number of free and open datasets available for data sci-

ence studies. Libraries like scikit-learn bundle popular datasets for you to exper-

iment with and provide mechanisms for loading datasets from various reposito-

ries (such as openml.org).

c. Governments, businesses and other organizations worldwide offer datasets on

a vast range of subjects.

d. All of the above statements are true.

15.1 Q8: Which of the following statements is false?

a. Even though k-nearest neighbors is one of the most complex classification al-

gorithms, because of its superior prediction accuracy we use it to analyze the Dig-

its dataset bundled with scikit-learn.

b. Classification algorithms predict the discrete classes (categories) to which sam-

ples belong.

c. Binary classification uses two classes, such as “spam” or “not spam” in an e-mail

classification application. Multi-classification uses more than two classes, such as

the 10 classes, 0 through 9, in the Digits dataset.

d. A classification scheme looking at movie descriptions might try to classify them

as “action,” “adventure,” “fantasy,” “romance,” “history” and the like.

15.1 Q9: Which of the following statements is false?

a. Regression models predict a continuous output, such as the predicted temper-

ature output in a weather time-series analysis.

b. The LinearRegression estimator can perform simple linear regression.

c. The LinearRegression estimator also can perform multiple linear regression.

4 Chapter 15, Machine Learning: Classification, Regression and Clustering

d. The LinearRegression estimator, by default, uses all the nonnumerical fea-

tures in a dataset to make more sophisticated predictions than you can with a

single-feature simple linear regression.

15.1 Q10: Unsupervised machine learning uses ________ algorithms.

a. classification

b. clustering

c. regression

d. None of the above

15.1 Q11: Which of the following are related to compressing a dataset’s large

number of features down to two for visualization purposes.

a. dimensionality reduction

b. TSNE estimator

c. PCA estimator

d. All of the above.

15.1 Q12: Which of the following statements a), b) or c) is false?

a. The simplest supervised machine-learning algorithm we use is k-means clus-

tering.

b. In k-means clustering, each cluster’s centroid is the cluster’s center point.

c. You’ll often run multiple clustering estimators to compare their ability to divide

a dataset’s samples effectively into clusters.

d. All of the above statements are true.

15.1 Q13: Which of the following statements is false?

a. K-means clustering works through the data attempting to divide it into that

many clusters.

b. As with many machine learning algorithms, k-means clustering is recursive and

gradually zeros in on the clusters to match the number you specify.

Chapter 15, Machine Learning: Classification, Regression and Clustering 5

c. K-means clustering can find similarities in unlabeled data. This can ultimately

help with assigning labels to that data so that supervised learning estimators can

then process it.

d. Given that it’s tedious and error-prone for humans to have to assign labels to

unlabeled data, and given that the vast majority of the world’s data is unlabeled,

unsupervised machine learning is an important tool.

15.1 Q14: Which of the following statements a), b) or c) is false?

a. The amount of data that’s available today is already enormous and continues

to grow exponentially—the data produced in the world in the last few years alone

equals the amount produced up to that point since the dawn of civilization.

b. People used to say “I’m drowning in data and I don’t know what to do with it.

With machine learning, we now say, “Flood me with big data so I can use machine–

learning technology to extract insights and make predictions from it.”

c. The big data phenomenon is occurring at a time when computing power is ex-

ploding and computer memory and secondary storage are exploding in capacity

while costs dramatically decline. This enables us to think differently about solu-

tion approaches.

d. All of the above statements are true.

15.1.3 Datasets Bundled with Scikit-Learn

No questions.

15.1.4 Steps in a Typical Data Science Study

15.1 Q15: Which of the following are not steps in a typical machine-learning case

study?

a. loading the dataset and exploring the data with pandas and visualizations

b. transforming your data (converting non-numeric data to numeric data because

scikit-learn requires numeric data) and splitting the data for training and testing

c. creating, training and testing the model; tuning the model, evaluating its accu-

racy and making predictions on live data that the model hasn’t seen before.

d. All of the above are steps in a typical machine-learning case study.

8 Chapter 15, Machine Learning: Classification, Regression and Clustering

b. Generally, you should train your model with the smallest amount of data that

makes the model perform well.

c. It’s important to set aside a portion of your data for testing, so you can evaluate

a model’s performance using data that the model has not yet seen. Once you’re

confident that the model is performing well, you can use it to make predictions

using new data.

d. All of the above statements are true.

15.2 Q7: Which of the following statements a), b) or c) is false?

a. You should first break your data into a training set and a testing set to prepare

to train and test a model.

b. The function train_test_split from the sklearn.model_selection mod-

ule simply splits in order the dataset’s samples and target values into training and

testing sets. This helps ensure that the training and testing sets have similar char-

acteristics.

c. Function train_test_split provides the keyword argument random_state

for reproducibility. When you run the code in the future with the same seed value,

train_test_split will select the same data for the training set and the same

data for the testing set. In machine-learning studies, this helps others confirm

your results by working with the same randomly selected data.

d. All of the above statements are true.

15.2 Q8: Which of the following statements is false?

a. By default, train_test_split reserves 75% of the data for training and 25%

for testing.

b. To specify different splits, you can set the sizes of the testing and training sets

with the train_test_split function’s keyword arguments test_size and

train_size. Use floating-point values from 0.0 through 100.0 to specify the per-

centages of the data to use for each.

c. You can use integer values to set the precise numbers of samples.

d. If you specify one of the keyword arguments test_size and train_size, the

other is inferred—for example, the statement

X_train, X_test, y_train, y_test = train_test_split(

digits.data, digits.target, random_state=11, test_size=0.20)

specifies that 20% of the data is for testing, so train_size is inferred to be 0.80.

Chapter 15, Machine Learning: Classification, Regression and Clustering 9

Answer: b. Actually, to specify different splits, you can set the sizes of the

testing and training sets with the train_test_split function’s keyword ar-

guments test_size and train_size—use floating-point values from 0.0

through 1.0 to specify the percentages of the data to use for each.

15.2.5 Creating the Model

15.2 Q9: Which of the following statements a), b) or c) is false?

a. The KNeighborsClassifier estimator (module sklearn.neighbors) im-

plements the k-nearest neighbors algorithm.

b. The following code creates a KNeighborsClassifier estimator object:

from sklearn.neighbors import KNeighborsClassifier

knn = KNeighborsClassifier()

c. The internal details of how a KNeighborsClassifier object implements the

k-nearest neighbors algorithm are hidden in the object. You simply call its meth-

ods.

d. All of the above statements are true.

15.2.6 Training the Model

15.2 Q10: Which of the following statements a), b) or c) is false?

a. The following call to the KNeighborsClassifier object’s fit method loads

the training set’s samples (X_train) and targets (y_train) into the estimator:

knn.fit(X=X_train, y=y_train)

b. After the KNeighborsClassifier’s fit method loads the data into the esti-

mator, it uses that data to perform complex calculations behind the scenes that

learn from the data and train the model.

c. The KNeighborsClassifier estimator is said to be lazy because its work is

performed only when you use it to make predictions.

d. All of the above statements are true.

15.2 Q11: Which of the following statements a), b) or c) is false?

a. In real-world machine-learning applications, it can often take minutes, hours,

days or even months to train your models—special-purpose, high-performance

hardware called GPUs and TPUs can significantly reduce model training time.

b. The fit method returns the estimator object.

10 Chapter 15, Machine Learning: Classification, Regression and Clustering

c. For simplicity, we generally use the default estimator settings—by default, a

KNeighborsClassifier looks at the three nearest neighbors to make its pre-

dictions.

d. All of the above statements are true.

15.2.7 Predicting Digit Classes

15.2 Q12: Which of the following statements a), b) or c) is false?

a. Once we’ve loaded our data into s KNeighborsClassifier, we can use it with

the test samples to make predictions. Calling the estimator’s predict method

with the test samples (X_test) as an argument returns an array containing the

predicted class of each sample:

predicted = knn.predict(X=X_test)

b. If predicted and expected are arrays containing the predictions and ex-

pected target values, respectively, evaluating the following code snippets in IPy-

thon interactive mode displays the predicted and expected target values for

the first 20 test samples:

predicted[:20]

expected[:20]

c. If predicted and expected are arrays containing the predictions and ex-

pected target values, respectively, the following list comprehension locates all the

incorrect predictions for the entire test set—that is, the cases in which the pre-

dicted and expected values do not match:

wrong = [(p, e) for (p, e) in zip(predicted, expected) if p != e]

d. All of the above statements are true.

15.3 Case Study: Classification with k-Nearest

Neighbors and the Digits Dataset, Part 2

No questions.

15.3.1 Metrics for Model Accuracy

15.3 Q1: Which of the following statements a), b) or c) is false?

Chapter 15, Machine Learning: Classification, Regression and Clustering 11

a. Each estimator has a score method that returns an indication of how well the

estimator performs for the test data you pass as arguments.

b. For classification estimators, the score method returns the prediction accu-

racy for the test data.

c. You can perform hyperparameter tuning to try to determine the optimal value

for k.

d. All of the above statements are true.

15.3 Q2: Which of the following statements is false?

a. Another way to check a classification estimator’s accuracy is via a confusion

matrix, which shows only the incorrect predicted values (also known as the

misses) for a given class.

b. To create a confusion matrix imply call the function confusion_matrix from

the sklearn.metrics module, passing the expected classes and the pre-

dicted classes as arguments, as in:

from sklearn.metrics import confusion_matrix

confusion = confusion_matrix(y_true=expected, y_pred=predicted)

c. The y_true keyword argument in Part (b) specifies the test samples’ actual

classes.

d. The y_pred keyword argument in Part (b) specifies the predicted classes for

the test samples.

15.3 Q3: Consider the confusion matrix for the Digits dataset’s predictions:

array([[45, 0, 0, 0, 0, 0, 0, 0, 0, 0],

[ 0, 45, 0, 0, 0, 0, 0, 0, 0, 0],

[ 0, 0, 54, 0, 0, 0, 0, 0, 0, 0],

[ 0, 0, 0, 42, 0, 1, 0, 1, 0, 0],

[ 0, 0, 0, 0, 49, 0, 0, 1, 0, 0],

[ 0, 0, 0, 0, 0, 38, 0, 0, 0, 0],

[ 0, 0, 0, 0, 0, 0, 42, 0, 0, 0],

[ 0, 0, 0, 0, 0, 0, 0, 45, 0, 0],

[ 0, 1, 1, 2, 0, 0, 0, 0, 39, 1],

[ 0, 0, 0, 0, 1, 0, 0, 0, 1, 41]])

Which of the following statements is false?

a. The correct predictions are shown on the diagonal from top-left to bottom-

right—this is called the principal diagonal.

12 Chapter 15, Machine Learning: Classification, Regression and Clustering

b. The nonzero values that are not on the principal diagonal indicate incorrect

predictions (that is, misses).

c. Each row represents one distinct class—that is, one of the digits 0–9.

d. The columns within a row specify how many of the test samples were classified

incorrectly into each distinct class 0–9.

15.3 Q4: The sklearn.metrics module’s classification_report function

produces a table of classification metrics based on the expected and predicted

values, as in:

from sklearn.metrics import classification_report

names = [str(digit) for digit in digits.target_names]

print(classification_report(expected, predicted,

target_names=names))

precision recall f1-score support

0 1.00 1.00 1.00 45

1 0.98 1.00 0.99 45

2 0.98 1.00 0.99 54

3 0.95 0.95 0.95 44

4 0.98 0.98 0.98 50

5 0.97 1.00 0.99 38

6 1.00 1.00 1.00 42

7 0.96 1.00 0.98 45

8 0.97 0.89 0.93 44

9 0.98 0.95 0.96 43

micro avg 0.98 0.98 0.98 450

macro avg 0.98 0.98 0.98 450

weighted avg 0.98 0.98 0.98 450

Which of the following statements a), b) or c)about the report is false?

a. The precision column shows the total number of correct predictions for a

given digit divided by the total number of predictions for that digit. You can con-

firm the precision by looking at each column in the confusion matrix.

b. The recall column is the total number of correct predictions for a given digit

divided by the total number of samples that should have been predicted as that

digit. You can confirm the recall by looking at each row in the confusion matrix.

Chapter 15, Machine Learning: Classification, Regression and Clustering 13

c. The f1–score column is the average of the precision. The recall and the

support column is the number of samples with a given expected value—for ex-

ample, 50 samples were labeled as 4s, and 38 samples were labeled as 5s.

d. All of the above are true.

15.3.2 K-Fold Cross-Validation

15.3 Q5: Which of the following statements is false?

a. With K-fold cross-validation, you use all of your data at once for training your

model.

b. K-fold cross-validation splits the dataset into k equal-size folds.

c. You then repeatedly train your model with k – 1 folds and test the model with

the remaining fold.

d. Consider using k = 10 with folds numbered 1 through 10. With 10 folds, we’d

do 10 successive training and testing cycles:

• First, we’d train with folds 1–9, then test with fold 10.

• Next, we’d train with folds 1–8 and 10, then test with fold 9.

• Next, we’d train with folds 1–7 and 9–10, then test with fold 8.

This training and testing cycle continues until each fold has been used to test the

model.

15.3 Q6: Which of the following statements is false?

a. Scikit-learn provides the KFold class and the cross_val_score function

(both in the module sklearn.model_selection) to help you perform the train-

ing and testing cycles.

b. The following code creates a KFold object:

from sklearn.model_selection import KFold

kfold = KFold(n_folds=10, random_state=11, shuffle=True)

c. The keyword argument random_state=11 seeds the random number genera-

tor for reproducibility.

d. The keyword argument shuffle=True causes the KFold object to randomize

the data by shuffling it before splitting it into folds. This is particularly important

if the samples might be ordered or grouped.

15.3 Q7: Which of the following statements a), b) or c) is false?

a. The following code uses function cross_val_score to train and test a model:

from sklearn.model_selection import cross_val_score

scores = cross_val_score(estimator=knn, X=digits.data,

y=digits.target, cv=kfold)

b. The keyword arguments in Part (a) are:

• estimator=knn, which specifies the estimator you’d like to validate.

• X=digits.data, which specifies the samples to use for training and test-

ing.

• y=digits.target, which specifies the targets for the samples.

• cv=kfold, which specifies the cross-validation generator that defines

how to split the samples and targets for training and testing.

c. Function cross_val_score returns a single overall accuracy score for the

model.

d. All of the above statements are true.

15.3.3 Running Multiple Models to Find the Best One

15.3 Q8: Which of the following statements a), b) or c) is false?

a. It’s difficult to know in advance which machine learning model(s) will perform

best for a given dataset, especially when they hide the details of how they operate

from their users.

b. Even though the KNeighborsClassifier predicts digit images with a high

degree of accuracy, it’s possible that other scikit-learn estimators are even more

accurate.

c. Scikit-learn provides many models with which you can quickly train and test

your data. This encourages you to run multiple models to determine which is the

best for a particular machine learning study.

d. All of the above statements are true.

15.3.4 Hyperparameter Tuning

15.3 Q9: Which of the following statements is false?