How to Search and Edit Text on Linux with grep, sed and awk

Logs, config files, command output — nearly everything on a server is text, and three tools handle almost all of it. grep finds lines, sed changes them, awk picks columns out. You don't need to master any of them; ten forms cover 95% of real use.

grep "PasswordAuthentication" /etc/ssh/sshd_config
grep -i "error" /var/log/nginx/error.log # -i ignores case
grep -ri "timeout" /etc/nginx/ # -r recurses, prints filename
grep -n "server_name" /etc/nginx/sites-enabled/* # -n shows line numbers
grep -v "^#" /etc/ssh/sshd_config | grep -v "^$" # hide comments and blank lines
grep -A3 -B3 "panic" /var/log/syslog # 3 lines of context either side
grep -c "Failed password" /var/log/auth.log # just the count
grep -E "error|warn" file.log # either word (extended regex)

Piping into grep is the everyday use: docker ps | grep nginx, ps aux | grep python, journalctl -u ssh | grep Failed.

sed 's/old/new/' file # first occurrence per line, prints result
sed 's/old/new/g' file # every occurrence
sed -i 's/old/new/g' file # -i edits the file in place
sed -i.bak 's/old/new/g' file # and keeps file.bak

Real examples:

sudo sed -i 's/^#PasswordAuthentication yes/PasswordAuthentication no/' /etc/ssh/sshd_config
sudo sed -i 's/^bind-address.*/bind-address = 0.0.0.0/' /etc/mysql/mariadb.conf.d/50-server.cnf
sed -i '/^#/d' config.txt # delete comment lines
sed -n '10,20p' file.log # print only lines 10 to 20
sed 's#/old/path#/new/path#g' file # any delimiter works — useful when the text has slashes

Always test without -i first, or use -i.bak. sed -i on the wrong file has no undo.

awk splits each line on whitespace into $1, $2, and so on; $0 is the whole line.

awk '{print $1}' /var/log/nginx/access.log # the IP address column
awk '{print $1}' access.log | sort | uniq -c | sort -rn | head # top visitors
df -h | awk '$5+0 > 80 {print $6, $5}' # mounts over 80% full
ps aux | awk '{s+=$6} END {print s/1024 " MB"}' # total memory used
awk -F: '{print $1}' /etc/passwd # -F sets the separator
awk '/error/ {n++} END {print n}' file.log # count matching lines
awk '{print $NF}' file # last column

sort | uniq -c | sort -rn is the classic trio for "what's most common" — pair it with awk or grep -o and you have a log analyser.

# Which IPs failed SSH login most, this boot
journalctl -u ssh -b | grep "Failed password" | awk '{print $(NF-3)}' | sort | uniq -c | sort -rn | head

# Space used by each Docker container's logs
sudo du -sh /var/lib/docker/containers/*/*-json.log | sort -h | tail

# Every port Nginx listens on
grep -rh "listen" /etc/nginx/sites-enabled/ | awk '{print $2}' | tr -d ';' | sort -u

^ start of line, $ end, . any character, .* anything, [0-9] a digit, . a literal dot. grep "^server" finds lines starting with "server"; grep ".php$" finds lines ending in .php. That's enough for config files and logs.

grep says "Binary file matches". Add -a to treat it as text, or you've grepped a database — don't.

sed changed nothing. The pattern didn't match — often because of ^ or trailing spaces. Test with grep first.

awk prints the wrong column. Count columns on a real line with awk '{print NF}', and remember fields split on any whitespace, so a date like Oct 8 is two fields.

Special characters in the replacement. & in sed's replacement means "the matched text"; escape it as \&. Slashes in paths: use # as the delimiter.

grep -ri to find, sed -i 's///g' to change, awk '{print $N}' to extract, and sort | uniq -c | sort -rn to count. Four patterns, and most questions about a log or config file take one line to answer.