Using SFDISK to backup your partition table

Many times we perform full backups of the root partition (/), including all mount points under root. How often do we back up our partition tables? What if we have total disk failure and our only restore option is a file-level restore? Will we know what our partition sizes were? Enter sfdisk.

 

Using SFDISK

The most common usage of sfdisk is to dump the partition sizes and count to a file for later import. This partition table dump can be included in your gzipped tarball.

Use this command to dump /dev/sda’s partition table:

# sfdisk -d /dev/sda > /backup/sda.part

 

The dump file will look like this:
# partition table of /dev/sda
unit: sectors

/dev/sda1 : start=       63, size=479990007, Id=83, bootable
/dev/sda2 : start=479990070, size=  8385930, Id=82
/dev/sda3 : start=        0, size=        0, Id= 0
/dev/sda4 : start=        0, size=        0, Id= 0

Restoring Partitions from SFDISK Dump

To restore your partition table from a dump file use this command:

# sfdisk /dev/sda < sda.part

Linux: The Sticky, SUID and SGID Bits

I’m sure anyone who has used Linux has heard of the Sticky, SUID or SGID bits. The most common (and easiest to explain) is the infamous "Sticky Bit".

 

 

The Sticky Bit

Back when systems had kilobytes of RAM (instead of gigabytes), this bit was used to mark a file (program) to run, and remain, primarily in memory. This was a great benefit back in "the day". Now the most common use for the sticky bit is to maintain the integrity of publicly accessible directories.

Setting the Sticky Bit

To set the sticky bit use this command:
# chmod +t <file/directory>

 

Looking for the Sticky Bit

To identify the sticky bit use the standard "ls" command to show all files. Look for a (t) in the listing.

For example:
-rw-r–r-T  1 root root 0 Jul 14 21:14 foo

 

The SUID Bit

SUID stands for "Set User ID". The SUID makes the program run as the user who owns the program (instead of the current user). I have an application called "test" which is owned by "dale" and the user "al" runs "test" the program will still run as "dale" if the SUID bit is set.

 

Setting the SUID Bit

To set the SUID bit use this command:
# chmod +s <file/directory>

 

Looking for the SUID Bit

To identify the SUID bit use the standard "ls" command to show all files. Look for an (S) in the listing.

For example:
-rwSr-Sr–  1 root root 0 Jul 14 21:14 foo

 

The SGID Bit

The SGID bit is much like the SUID bit but runs a program only as the set group ID group.

 

 

Examples

Setting SUID for user and not group
# chmod u+s <file/directory>

Setting sticky bit for group only
# chmod g+t <file/directory>

COTD: Screen

Overview

Ever wanted to run a long-running command but you can’t seem to get it to complete because you have a shaky connection? Can’t run your command in the background? Want to leave your IRC session open so you can SSH from work and catch the chat room action? Enter screen. Screen allows you to

 

Installation

Well, installation depends on your flavor of Linux. Here are the installation methods for a few common flavors of Linux:

 

up2date

To install screen with up2date issue this command:

# up2date -i screen

 

yum

To install screen with yum issue this command:

# yum -y install screen

 

apt

To install screen with apt issue this command:

# apt-get install screen

 

Usage

Using screen is extremely easy. Here are a few commands to help you understand how it operates.

 

Creating a new screen

Type "screen" to start a new screen. Note that the title of putty (if you’re using putty) tells you which screen you are currently attached to by inserting "[screen 0: bash]  before your normal [email protected]:/path text.
image

You should now run any commands you want to save inside your "screen".

 

Detaching a screen

To detach your current screen simply press "CTRL + AD" (Control plus A then D). You are now presented with a message saying "[detached]". You are now returned to your normal shell outside of your virtual screen.

 

Attaching to an existing screen

Chances are if you are using screen you’ll need to reattach to your detached screen. Type "screen -r" to reattach to your current screen. If multiple screen sessions are active, you’ll see a list of current screens to choose from. Type "screen -r PID" to reattach to that screen.
image
Multiple screens to choose from

Video Overview

Here’s a quick demonstration of screen.

10 Substantial Events in Linux History

Here are a few substantial events in Linux history:

1991 – Linus Torvalds posts his first message about his free operating system resembling MINIX. He mentions that the operating system will probably never support anything other than AT-hard drives.

1992 – Andrew Tanenbaum, a computer scientist and author of the MINIX kernel, wrote a post in response to Linus’ post in 1991. He said "Linux is Obselete" which sparked the debate about the structure of Linux. Hundreds of people were on-board with Linux at Linus’ campus; then thousands were developing and perfecting the code … soon to become hundreds of thousands.

image 1993 – The compressed kernel source was no around 800K and Linux had passed multiple revisions (15+)

1994 – Linus opened Linux version 1.1; beginning the stable development of the Linux we know today.

1994 – Caldera was formed in October 1994. I won’t go much into detail about Caldera – about all they contributed was a dual-processor Pentium machine for the development of the SMP-based kernel.

1996 – The 2.0 kernel was released and included many enhancements. The list looks like this: Multi-Architecture support (x86 and Alpha). This kernel also had support for SMP. The kernel is now around 5MByte compressed.

1996 – KDE was founded. The choice to go with the Qt toolkit was a sketchy one. Qt, at the time, did not use a free software license. The GNU team was concerned about this.

1997 – In August 1997, two projects were started to help KDE. The Harmony Toolkit was the replacement for the previously pay-only Qt libraries. GNOME was also founded but build without Qt and only built upon free software)

1999 – The kernel has doubled in size. The new kernel 2.2 release included a number of features. Finer-grained locking for improved multi-processor support. IBM announced an large project in support of the Linux operating system.

2004 – Microsoft published documents evaluating the use of Windows Vs. Linux with the name "Get the Facts" on their website. RedHat, Novell and IBM published articles in response to Microsoft’s "bent" version of the truth.

Managing Your EXIM Queue

Have you ever had a hard time with spammers or recently logged in to one of your Linux servers only to find 10-20+ exim processes hogging all of your resources? Check out the commands below to help you manage your queue.

 
Count The Messages

Count the number of messages in your queue. If this number is abnormally large; it’s usually a good indication of some malicious activity on your server.
# exim -bpc

 

View The Messages

After finding out you have 1000’s of messages in your exim queue it might be a good idea to stop the exim service.
# service exim stop

 

After stopping the service run this command to see what’s in the queue:
# exim -bp | less

 

This will pipe the results to the less command to enable you to page through and search the results. Find one of the message ID’s that seem to be duplicates (spam) and copy it to your clipboard.
A message ID looks like this: 1KBB28-0006UC-Fl

 

Run this command to view the header of the suspect message:
# find /var/spool/exim/input -name "1KBB28-0006UC-Fl-H" -exec cat {} \;

or this method with exim
# exim -Mvh 1KBB28-0006UC-Fl

 

The header can show you exactly where the message is originating as well as the recipient. If the spammer is connecting directly to your SMTP server you can also gather the IP address here.

 

Removing The Messages

Let’s get the messages out of the queue. For our example, we’ll use the address [email protected] as the sending address. To remove all emails in the queue for [email protected] run this command:

# exim -bp | awk ‘$4[email protected] { print $3 }’ | xargs exim -Mrm

e2label: Label Your Disks

With cheaper storage you might find yourself literally submersed in storage devices. It is possible to have your block devices change (from sdc to sde) when using multiple controllers of the same model (I’ve experienced this with 3Ware controllers). So how do you know where to mount your devices after a reboot? Use e2label to label your disks (much like NTFS labels in Windows).

 

Labeling Your Device

To label your device follow these steps:

# e2label /dev/sdb1 awesomedisk

 

Modifying Your fstab

Now you can modify your fstab to reflect your label changes. Instead of using the path to the block device (/dev/sdb1) you can use this syntax: (LABEL=awesomedisk)

Example:

LABEL=awesomedisk                 /mnt/awesomedisk                       ext3    defaults        1 1

 

Check Current Label

To check the label currently set for a block device use this command:
# tune2fs -l /dev/sdb1 | grep "Filesystem volume name:"

Linux SysAdmin Toolbox

imageWorking with Linux over the years has really exposed me to many useful and unique tools. I’ll show you a few of the tools that help me get through my day a bit quicker and more efficiently. 

Moving/Grabbing FIles

Getting files to your Linux system can be easily achieved using the tools below.

wget

The wget command allows you to retrieve files from an FTP/HTTP or HTTPs source. Typically use this command to get new packages or new tarballs to your server.

Usage:
# wget <source>
# wget http://us2.php.net/get/php-5.2.6.tar.gz/from/this/mirror

scp

SCP allows you to transfer files over SSH (TCP Port 22). You can even transfer files between two servers over SSH.

Usage:

# scp <source> [email protected]:/path/on/remote/host

Looking for FIles

Lose a file? Need to list all files modified after a certain date? Want to remove all files with "foobar" in the name? Check out the commands below.

find

The find command allows you to search for a specific file by name, size or modified time. The criteria mentioned before is just part of what you can do with find. To see full functionality type "man find" to view the manual page.

# find / -name foo.atxt (this command finds all files named "foo.txt" starting in the root directory)

Another usage of find which is quite handy is the -exec flag.

# find / -name foo.txt -exec ls -lth {} \;

This command will list all files named foo.txt and run the command "ls -lth" on each file. This can be useful to apply a command to all files that match a certain string.

locate

Use the locate command to find a specific file quickly. This is more primitive than the before mentioned find command. It relies on a database of files which it performs its lookups. To update this database run a command, appropriately named, "updatedb".

Usage:
# locate foo.txt

Sorting Results

Since a Linux Administrator typically needs to sort through massive log files, show certain fields and manipulate output; an admin needs to know about the tools below.

sort

The sort command allows you to sort the results of your output. To sort the results by the first regular character (not a symbol). Here is an example of how to use sort with the locate command:

# locate bin | sort

Piping the output of locate to sort will show all results in order.

uniq

Uniq is short for, you guessed it, unique. Uniq accepts input from STDIN (Standard Input) and shows only unique lines of text. This command basically strips duplicates.

# cat foo.txt | uniq

awk

Awk is a pattern scanning engine. The most common usage is to separate rows of text into chunks. The chunks, by default, are separated by spaces. For instance this sentence: "The quick brown fox" when piped to this command: "awk {‘print $2’}" will print simply "quick".

# echo "The quick brown fox" | awk {‘ print $3 ‘}
brown

This command can be very useful when combining it with long field-driven lists.

wc

wc (Word Count) will count the number of words, newlines and bytes in a file. Typically this command is used with the "-l" flag to count the lines if output. This command accepts input from STDIN (Standard Input).

# cat foo.txt | wc -l
300

This means that the file "foo.txt" contains 300 lines.

Running Commands (that may take a while)

Since the command line interface (CLI) usually offers no progress bar (except for certain apps like wget), you’ll sometimes need to run a program for a long time. Searching through millions of files? Removing a directory containing large quantities of data/files? What if you get disconnected while running one of these commands in the foreground? Use one of the solutions below to run your commands.

Starting Commands in the Background

It is generally useful to start a command, which takes a long time, in the background. Think of these programs as "service" programs which run without a terminal (ssh session, or physical terminal) attached. To start a program in the background simply add a space and the ampersand (&) after the command.

# sleep 1000 &

The above command will be executed in the background. The command waits 1000 seconds to end. Check the progress if your command by issuing "ps aux | grep sleep" to view the PID and status of your command.

To force a command currently running in the foreground to run in the background press "CTRL + Z". You will then be presented the PID. Disregard the "Stopped" message. This indicates the foreground process has stopped and is now running in the background.

screen

Using the program called "screen" allows you to run multiple virtual terminals behind the scenes. This can be useful for programs (and commands) that can’t be ran in the background. Instead of running multiple SSH sessions you can simply type "screen" to initiate a new screen. Press "CTRL + AD" to detach the current screen. Type "screen -r" to reattach to the screen(s) currently active. To install screen on CentOS/RHEL issue the following command:

# yum -y install screen

CentOS 4.x – But I want PHP5 and MySQ5

This sometimes can be an issue. I want PHP5 but I only have CentOS 4.6. What can I do? It seems like the CentOS repository doesn’t have PHP5, only PHP4. Enter “–-enablerepo centosplus”. Just add “—enablerepo centosplus” after yum and you’ll be able to install PHP 5, Apache 2.0 and MySQL 5 in no time.

# yum –enablerepo=centosplus –y install php httpd mysql php-mysql php-gd

The above also works for upgrades by substituting “install” for “upgrade”.

Linux Command: diff – What’s the difference?

Sometimes it’s necessary to quickly determine the difference between two files. The command “diff” can be particularly useful if you find yourself in this situation.

The most simple usage of diff is “diff <oldfile> <newfile>”.

The output looks like this:
image

The output of diff is really how to alter the original file to match the new file. “1a2” means to add a new line to the old file after line 1. This line should be “bar” as signified by the “>” mark.

Often patches are distributed using the diff file. PHPBB usually offers this as a way to patch their open source code. In short, using diff can save you time and offer a way to compare to configuration files to see their differences.

SEO Your Application With mod_rewrite

Generally speaking, many search engines like Google, MSN, and Yahoo like to see your URLs be as pretty as possible. Using applications like PHP will often produce URLs that look like this: http://somesite.com/index.php?section=section1&data=test. These URLs are typically frowned upon by the search engines and can produce mixed results when attempting to achieve the  highest search engine ranking.

There are two methods we can use to make our URL’s a bit more friendly. Many applications today already provide pre-packaged .htaccess files. It’s important for me to explain that there are two methods for setting up mod_rewrite rules. The first (and most common) method is to use a file called “.htaccess” placed in the root directory of your web site. This file is read in to the Apache configuration in real time (The AllowOverride option in your Apache configuration controls this). We’ll cover the .htaccess method here.

The .htaccess Method

This is the most common way to implement mod_rewrite rules. First, open a text file with your favorite editor (if you do not have shell access simply use notepad or vi on your local machine). Remember that a dot (.) before the file name designates this file as hidden in Linux.

For our example we’re going to redirect http://somesite.com/index.php?section=section1&data=test to look like this: http://somesite.com/section1/test.

  1. Edit the .htaccess file you wish to
    # vi /var/www/html/.htaccess
  2. Place the following in the new .htaccess file:
    RewriteEngine On
    RewriteRule ([^/]+)/([^/]+)?$ index\.php?section=$1&data=$2 [L]
  3. Save the file and exit your editor.
    (* Note: If you upload this file with an FTP/SCP program your file will not appear as it is hidden, if you need to delete the file due to a misconfiguration, issue “DELE .htaccess” in the directory you uploaded the file.)

The [L] directive tells mod_rewrite to redirect the request with a 301 redirect.

Troubleshooting

  1. I receive a 500 server error after uploading.
    Check to make sure no typo’s were made. Check the apache error_log for any clues.
  2. The redirect does not take place but the site still shows.
    This is usually because your host does not support mod_rewrite or is not allowing .htaccess files. To check if .htaccess is being parsed open a new .htaccess and put the following inside:
    Redirect /test “http://www.google.com”
    If this works you can safely assume that .htaccess files are being parsed.