How to Use Pipes and Redirects to Streamline Command Outputs in Unix

One of Unix’s greatest strengths is its ability to combine commands to process data efficiently. This is made possible by pipes and redirects, which allow you to connect commands and control where their input and output go.

In this blog, we’ll cover what pipes and redirects are, how they work, and practical examples to help you master them.

---

What Are Pipes and Redirects?

Pipes (|)

A pipe takes the output of one command and uses it as the input for another. Think of it as connecting two tools so they work together.

Redirects (>, >>, <)

Redirects control where a command’s input comes from or where its output goes. For example, you can send the output to a file instead of the screen.

---

Using Pipes (|)

Basic Pipe Example

Combine two commands with a pipe to pass data from one to the other.

Example:

$ ls -l | grep "txt"

• ls -l lists files.
• grep "txt" filters the output to show only lines containing “txt.”

---

Common Pipe Chains

1. Count Lines, Words, or Characters:
   Use wc (word count) with a pipe.

   $ cat myfile.txt | wc -l

This counts the number of lines in myfile.txt.

2. Sort and Remove Duplicates:
   Combine sort and uniq to process data.

   $ cat data.txt | sort | uniq

• sort: Sorts the lines alphabetically.
• uniq: Removes duplicate lines.

1. Paginate Long Output:
   Use less to scroll through large outputs.

   $ ls -l | less

---

Using Redirects

Redirect Output to a File: >

The > operator sends the output of a command to a file, replacing its content if it already exists.

Example:

$ echo "Hello, Unix!" > greeting.txt

This writes “Hello, Unix!” to greeting.txt.

---

Append Output to a File: >>

The >> operator adds the output to the end of a file without overwriting it.

Example:

$ echo "Adding another line." >> greeting.txt

---

Redirect Input from a File: <

The < operator uses a file as input for a command.

Example:

$ wc -l < greeting.txt

This counts the lines in greeting.txt.

---

Redirect Both Output and Errors: 2>

Sometimes, commands produce error messages. You can redirect these separately:

• Redirect errors to a file:

  $ ls nonexistentfile 2> errors.log

• Redirect both output and errors to the same file:

  $ ls / > output.log 2>&1

---

Combining Pipes and Redirects

You can combine pipes and redirects for even more flexibility.

Example:

$ ls /var | grep "log" > logs_list.txt

• ls /var: Lists files in /var.
• grep "log": Filters the list to show only items with “log.”
• >: Saves the output to logs_list.txt.

---

Practice Time!

1. List all files in your home directory and filter for .txt files:

   $ ls ~ | grep "txt"

2. Count the number of .log files in /var/log:

   $ ls /var/log | grep "log" | wc -l

3. Save the list of .log files to a file:

   $ ls /var/log | grep "log" > log_files.txt

---

Summary

• Use pipes (|) to connect commands and process data.
• Use redirects (>, >>, <, 2>) to control where output and input go.
• Combining these tools makes Unix commands incredibly powerful and flexible.