A Guide to Unix Using Linux, Fourth Edition 5-1
Chapter 5
Advanced File Processing
At a Glance
Instructor’s Manual Table of Contents
• Overview
• Objectives
• Teaching Tips
• Quick Quizzes
A Guide to Unix Using Linux, Fourth Edition 5-2
Lecture Notes
Overview
In Chapter 5, students learn many new file-processing selection, manipulation, and
transformation commands to put in their expanding toolbox. The pipe operator (|) is
introduced. Students learn how to use selection commands (grep, uniq, comm, diff, and
wc) and manipulation/transformation commands (sed, tr, and pr). Finally, students learn
how to design a file-processing application in a step-by-step process by creating,
testing, and running shell scripts.
Chapter Objectives
• Use the pipe operator to redirect the output of one command to another command
• Use the grep command to search for a specified pattern in a file
Teaching Tips
Advancing Your File-Processing Techniques
2. Use Tables 5-1 and 5-2 to briefly describe the most useful selection and
manipulation/transformation commands.
Using the Selection Commands
1. Mention that UNIX/Linux has another redirection operator: the pipe (|) operator.
2. The following selection commands will be described in this section: grep, diff, uniq,
comm, and wc.
Using the Pipe Operator
1. Use several examples to show how the pipe operator (|) can be used to redirect the
output of one command to the input of another command. Figure 5-1 shows how three
A Guide to Unix Using Linux, Fourth Edition 5-3
Using the grep Command
1. Describe the syntax of the grep command.
Teaching
Three typical meanings are associated with grep: Global Regular Expression
3. Use one or more examples to show how grep can be used.
Teaching
searches for fixed or text strings only and not expressions. zgrep is used to
Many UNIX/Linux systems offer a combination of four grep-type commands:
grep, egrep, fgrep, and zgrep. egrep (also executed as grep -E) is used for
Using the uniq Command
1. Explain that uniq removes duplicate lines from a file. Stress that this command
compares only consecutive lines and requires sorted input.
3. Use the examples provided in the book (see Figures 5-2 and 5-3) to show students how
this command works.
A Guide to Unix Using Linux, Fourth Edition 5-4
Using the comm Command
1. Emphasize the following important points regarding the use of the comm command:
2. Describe the syntax of the comm command.
Using the diff Command
2. Note that diff is commonly used to determine the minimal set of changes needed to
convert file1 to file2.
4. Use one or more examples to show how to use this command and how to interpret its
output.
Using the wc Command
1. Describe the syntax of the wc command.
3. Stress that if you enter wc without options, you see counts of lines, words, and
characters in that order (see Figure 5-4).
Quick Quiz 1
1. The ____________________ redirects the output of one command to the input of
another command.
|
2. You use the ____________________ command to search for a specified pattern in a
file, such as a particular word or phrase.
3. The ____________________ command locates identical lines within two identically
sorted files.
4. The ____________________ command shows lines that differ between two files and is
commonly used to determine the minimal set of changes needed to convert file1 to
file2.
Using Manipulation and Transformation Commands
1. Several commands can be used to edit and transform data’s appearance before you
display or print it. This section studies the following commands: sed, tr, and pr.
Introducing the sed Command
2. Describe the syntax of the sed command.
Translating Characters Using the tr Command
1. Describe the syntax of the tr command.
3. Provide a couple of examples to show students how to use this command.
Using the pr Command to Format Your Output
2. Note that if no file is specified or if “–” is specified as the file, pr reads the standard
input.
3. By default, pr formats output into single-column pages of 66 lines, and each page has a
A Guide to Unix Using Linux, Fourth Edition 5-6
Quick Quiz 2
1. ____________________ is a stream editor that can be used on one or more files, and is
particularly useful for making global changes on large files.
2. To append new lines in sed, you must use the ____________________ command.
3. The ____________________ or translate command (also called the translate characters
command) copies data from the standard input to the standard output, substituting or
deleting characters specified by options and patterns.
4. The ____________________ command prints the specified files on the standard output
in paginated form.
Designing a New File Processing Application
2. Stress that how you set up records in a file can influence what you can do with an
application.
3. Explain the importance of carefully selecting the fields to include in a record.
Emphasize that an ID can be a key field for sorting.
Designing Records
2. Provide some tips that students can use as a guide when designing records. For
example:
a. You should design file records to store only those fields relevant to each
record’s primary purpose.
A Guide to Unix Using Linux, Fourth Edition 5-7
The programmer number field in the programmer file record should be numeric.
Linking Files with Keys
1. Explain that one or more files can be joined with a key (see Figure 5-5). A key is a
common field shared by each of the linked files.
2. Stress that it is recommended to plan the way files will be linked during the design
phase.
Creating the Programmer and Project Files
1. Describe the reasoning behind the files and process described in Figure 5-6.
As you lecture through these sections, you should ask students to complete each
Formatting Output
2. Explain that the printf function within awk is used to format output. Describe the syntax
of printf.
3. Use the example provided in the book (Figure 5-7) to show students how to use printf.
Using a Shell Script to Implement the Application
1. Note that it is important to test and debug each command before you place it in your
script file.
2. Let students know that shell scripts can be created in any text editor, such as vi or
3. Use an example to show how to use comments in shell scripts. Stress that commenting
shell scripts is important for the programmer of the script and for other programmers.
Running a Shell Script
2. Use a couple of examples to show how to run shell scripts in Bash. For example: sh
testscript and ./testscript.
3. Note that it is recommended to specify the shell that a script is intended to use. For
example: #!/bin/bash.
Putting It All Together to Produce the Report
1. Describe the advantages of combining small scripts into a larger file (versus creating
one large shell script from scratch).
Quick Quiz 3
1. Files, records, and fields are called ____________________ structures.
2. The first task in the record design phase is to define the ____________________ in the
records.
3. A(n) ____________________ is a common field shared by each of the linked files.
4. When you create a shell script to run in Bash, you can immediately run the script by
typing ____________________ (to call the Bash shell interpreter) and then the name of
the script.
A Guide to Unix Using Linux, Fourth Edition 5-9
Class Discussion Topics
1. Some students may have previous experience using the grep command. Ask them to
talk about situations in which they have previously used grep.
Additional Projects
2. printf supports several other format modifiers besides the ones explained in the chapter.
Ask students to do some research on the different format modifiers supported by printf.
They should summarize the role of each modifier in a table.
Additional Resources
Key Terms
➢ logical structure — The organization of information in files, records, and fields, each
of which represents a logical entity, such as a payroll file, an employee’s pay record, or
A Guide to Unix Using Linux, Fourth Edition 5-10
➢ selection commands — The file-processing commands that are used to extract
information.
Technical Notes for Hands-On Projects
Students complete these projects from the command line, such as from a terminal window, and
log in using their own accounts and home directory. The vi editor is used in several projects.
Additional requirements are listed below.
Hands-On Project 5-3: In this project, students work with the /etc/termcap file. Knoppix (or any
other Debian-based distribution) does not have the /etc/termcap file used in this project. For
students using Debian-based distributions, there are two alternatives: