Linux SysAdmin Toolbox

imageWorking with Linux over the years has really exposed me to many useful and unique tools. I’ll show you a few of the tools that help me get through my day a bit quicker and more efficiently. 

Moving/Grabbing FIles

Getting files to your Linux system can be easily achieved using the tools below.

wget

The wget command allows you to retrieve files from an FTP/HTTP or HTTPs source. Typically use this command to get new packages or new tarballs to your server.

Usage:
# wget <source>
# wget http://us2.php.net/get/php-5.2.6.tar.gz/from/this/mirror

scp

SCP allows you to transfer files over SSH (TCP Port 22). You can even transfer files between two servers over SSH.

Usage:

# scp <source> [email protected]:/path/on/remote/host

Looking for FIles

Lose a file? Need to list all files modified after a certain date? Want to remove all files with "foobar" in the name? Check out the commands below.

find

The find command allows you to search for a specific file by name, size or modified time. The criteria mentioned before is just part of what you can do with find. To see full functionality type "man find" to view the manual page.

# find / -name foo.atxt (this command finds all files named "foo.txt" starting in the root directory)

Another usage of find which is quite handy is the -exec flag.

# find / -name foo.txt -exec ls -lth {} \;

This command will list all files named foo.txt and run the command "ls -lth" on each file. This can be useful to apply a command to all files that match a certain string.

locate

Use the locate command to find a specific file quickly. This is more primitive than the before mentioned find command. It relies on a database of files which it performs its lookups. To update this database run a command, appropriately named, "updatedb".

Usage:
# locate foo.txt

Sorting Results

Since a Linux Administrator typically needs to sort through massive log files, show certain fields and manipulate output; an admin needs to know about the tools below.

sort

The sort command allows you to sort the results of your output. To sort the results by the first regular character (not a symbol). Here is an example of how to use sort with the locate command:

# locate bin | sort

Piping the output of locate to sort will show all results in order.

uniq

Uniq is short for, you guessed it, unique. Uniq accepts input from STDIN (Standard Input) and shows only unique lines of text. This command basically strips duplicates.

# cat foo.txt | uniq

awk

Awk is a pattern scanning engine. The most common usage is to separate rows of text into chunks. The chunks, by default, are separated by spaces. For instance this sentence: "The quick brown fox" when piped to this command: "awk {‘print $2’}" will print simply "quick".

# echo "The quick brown fox" | awk {‘ print $3 ‘}
brown

This command can be very useful when combining it with long field-driven lists.

wc

wc (Word Count) will count the number of words, newlines and bytes in a file. Typically this command is used with the "-l" flag to count the lines if output. This command accepts input from STDIN (Standard Input).

# cat foo.txt | wc -l
300

This means that the file "foo.txt" contains 300 lines.

Running Commands (that may take a while)

Since the command line interface (CLI) usually offers no progress bar (except for certain apps like wget), you’ll sometimes need to run a program for a long time. Searching through millions of files? Removing a directory containing large quantities of data/files? What if you get disconnected while running one of these commands in the foreground? Use one of the solutions below to run your commands.

Starting Commands in the Background

It is generally useful to start a command, which takes a long time, in the background. Think of these programs as "service" programs which run without a terminal (ssh session, or physical terminal) attached. To start a program in the background simply add a space and the ampersand (&) after the command.

# sleep 1000 &

The above command will be executed in the background. The command waits 1000 seconds to end. Check the progress if your command by issuing "ps aux | grep sleep" to view the PID and status of your command.

To force a command currently running in the foreground to run in the background press "CTRL + Z". You will then be presented the PID. Disregard the "Stopped" message. This indicates the foreground process has stopped and is now running in the background.

screen

Using the program called "screen" allows you to run multiple virtual terminals behind the scenes. This can be useful for programs (and commands) that can’t be ran in the background. Instead of running multiple SSH sessions you can simply type "screen" to initiate a new screen. Press "CTRL + AD" to detach the current screen. Type "screen -r" to reattach to the screen(s) currently active. To install screen on CentOS/RHEL issue the following command:

# yum -y install screen

Leave a Reply

Your email address will not be published / Required fields are marked *