Chapter 9. Partitioning and Formating Disks

Chapter 9. Partitioning and Formating Disks

This aspect is critical while installing the operating system, we will see how to split the hard disk and format them in order to meet the requirements.

OBJECTIVES

• Describe most common types of hard disk and data buses
• Explain disk geometry and other partition concepts
• Understand disk devices naming and their associated device nodes
• Understand several partition strategies
• Use blkid and fdisk 
• Backup and restore partitions

COMMON DISK TYPES

• IDE
  • Integrated Drive Electronics, used for consumer mass markets but now a day obsolete
• SATA
  • Serial Advance Technology Advancement, faster and better than IDE and EIDE disks, these drives are the common drive in mass comsumer markets
• SCSI
  • Small Computer Systems Interface, mainly used for servers, they are good working in parallel with the RAID configurations
• SAS
  • Serial Attached SCSI, with data transfers similar to SATA but with overall better performance
• USB
  • Universal Seria Bus, used for peripheral devices
• SSD
  • Solid State Drive, used for storage and a replacement for rotational drives

DISK GEOMETRY

Rotational Disk

• Heads
  • Contains tracks
• Track
  • Contains sectors
• Sector
  • Minimal part of chunked data
• Cinlinder
  • Is a group which consist of the same track on all platters

$ sudo fdisk -l /dev/sda
 
Disk /dev/sda: 1000.2 GB, 1000204886016 bytes 255 heads, 63 sectors/track, 121601 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk identifier: 0x00000001
 
Device Boot      Start       End        Blocks      Id      System
/dev/sda1 *          1      3800      30522368      83       Linux
/dev/sda2         3800      3851        409518      83       Linux
.....
 
Note the use of the -l option which simply lists the partition table without entering interactive mode.

PARTITIONING

Disks are divided into partitions, there can be up to 4 primary partitions, one of them can be a extended partition and therefore can be divided into several logical partitions.

In SATA devices there can be up to 15 partitions, Partitions 1-4 are primary partitions and one of them is extended, within the extended partition we will have to configure the subsequent partitions untill reach the 15 partitions.

WHY PARTITION ?

• Separation
  • To separate operative system primary files from the users files
• Sharing
  • It also might be a good reason to share a users filesystem for example shared across a network NFS (Network File System)
• Security
  • To impose different quotas, permitions and setting in different parts of the system
• Size
  • Some applications have data that can vary in size, thus is important to configure such filesystems in a separate partition to avoid a system crash due to space limitations
• Performance
  • Some files are about to being accessed much more often and thus if critical it is convenient to place such data within a faster disk
• Swap
  • The linux operative system prefer to have a separated swap partition, in order to take advantage of certain features like hibernation

COMMON PARTITIONING

• First partition contains the boot partition /boot
• Second partition contains the root filesystem /
• Third partition contains the swap partition 
• Fourth partition contains the /home directory

PARTITIONING TABLE

The disk partitioning table is containend within the Master Boot Record which is 512 bytes in length.

The partition table is ( 4 maximun primary partitions of 16 bytes ) 64 bytes long and is placed after the 446 byte boot record (There are 2 bytes at the end of the Master Boot Record registry that marks the end of the sector, which always has the value 0x55AA).

The first 446 bytes of the MBR are reserved for the boot loader (such as GRUB). Only one partition on a given disk may be marked active meaning that such partition will contain the data to boot the system.

Each of the 4 partition entries have 16 bytes and describes one of the four possible primary partitions. The information for each is

• Active bit
• Beginning address in cylinder/head/sectors (CHS) format
• Partition type (xfs, ext4, ext3, etc )
• Ending address in CHS
• Start sector
• Number of sectors in partition

NAMING DISK DEVICES AND NODES

The linux kernel interacts with the disks (device nodes) through the virtual file system usually located in /dev directory.

Raw access through the device node is a good way to destroy a filesystem, this example shows how to format a partition on sda9

$ sudo mkfs.ext4 /dev/sda9

​Device nodes for SCSI and SATA disks follow a simple naming convention:

The first hard disk is /dev/sda.
The second hard disk is /dev/sdb.
etc.
Partitions are also easily enumerated as in:

/dev/sdb1 is the first partition on the second disk.
/dev/sdc4 is the fourth partition on the third disk.
In the above sd means SCSI or SATA disk. Back in the days where IDE disks could be found they would have been

/dev/hda3, /dev/hdb etc. 

Doing ls -l /dev will show you the current available disk device nodes

For SCSI devices we need to elaborate a little more on what we mean by first, second hard disk etc. These are determined by the controller number/ID number combination.

The drive designation (a, b, c, etc.) is primarily based on the ID number of the SCSI device rather than its position on the bus itself.

For example, if we had two SCSI controllers with target ID number 1 and 3 on controller 0 and target ID number 2 and 5 on controller 1 (with ID 2 as the last drive):

ID 1 would be /dev/sda
ID 3 would be /dev/sdb
ID 2 (on controller 1) would be /dev/sdc
ID 5 would be /dev/sdd

BLKID AND LSBLK

[abernal@localhost ~]$ sudo blkid /dev/sda*
[sudo] password for abernal: 
/dev/sda: PTUUID="a330d547" PTTYPE="dos"
/dev/sda1: UUID="d7430811-554f-43f7-bc22-18afe9d879c3" TYPE="ext4" PARTUUID="a330d547-01"
/dev/sda2: UUID="TtW6nj-yAGi-uVCv-7xGw-F02s-veeG-7GITqd" TYPE="LVM2_member" PARTUUID="a330d547-02"​

[abernal@localhost ~]$ lsblk

NAME            MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda               8:0    0 10.4G  0 disk 
├─sda1            8:1    0  500M  0 part /boot
└─sda2            8:2    0  9.9G  0 part 
  ├─fedora-root 253:0    0  8.9G  0 lvm  /
  └─fedora-swap 253:1    0    1G  0 lvm  [SWAP]
sr0              11:0    1 56.3M  0 rom  /run/media/abernal/VBOXADDITIONS_5.0.14_105127

​SIZING UP PARTITIONS

Linux systems needs at least two partitions 

/

This partition will contain the filesystem

swap

This partition is an extend of the physical memory

BACKING UP AND RESTORING PARTITION TABLES

Backing up can be easily done with dd, as in:

$ sudo dd if=/dev/sda of=mbrbackup bs=512 count=1

which will back up the MBR on the first disk, including the 64-bit partition table which is part of it.

The MBR can then be restored if necessary by doing:

$ sudo dd if=mbrbackup of=/dev/sda bs=512 count=1

Note the above commands only copy the primary partition table;  they do not deal with any partition tables stored in the other partitions (for extended partitions, etc.)

PARTITION TABLE EDITORS

One of the main partition table editor is

• fdisk

Nevertheless these are other tools

• sfdisk
• parted
• gparted

USING FDISK

This is an interactive tool so it will display the options step by step through the console

[abernal@localhost ~]$ sudo fdisk /dev/sda

In order to check which partitions the system is aware of, type this command

[abernal@localhost ~]$ cat /proc/partitions

LABEL PARTITIONS

Use the command e2label in order to do this

[abernal@localhost ~]$ sudo e2label /dev/sda3 nuevonombre

​LAB 1

1. Create a file full of zeros 1 GB in length:

    $ dd if=/dev/zero of=imagefile bs=1M count=1024

You can make a much smaller file if you like or don’t have that much available space in the partition you are creating the file on.

2. Put a filesystem on it:

     $ mkfs.ext4 imagefile
     mke2fs 1.42.9 (28-Dec-2013)
     imagefile is not a block special device.
     Proceed anyway? (y,n) y
     Discarding device blocks: done
     .....

Of course you can format with a different filesystem, doing mkfs.ext3, mkfs.vfat, mkfs.xfs etc. 

3. Mount it somewhere:

     $ mkdir mntpoint
     $ sudo mount -o loop imagefile mntpoint

You can now use this to your heart’s content, putting files etc. on it. 

4. When you are done unmount it with:

     $ sudo umount mntpoint

​LAB 2

The next level of complication is to divide the container file into multiple partitions, each of which can be used to hold a filesystem, or a swap area.
You can reuse the image file created in the previous exercise or create a new one. 

1. Run fdisk on your imagefile:

    $ sudo fdisk -C 130 imagefile
     Device does not contain a recognized partition table
     Building a new DOS disklabel with disk identifier 0x6280ced3.
     Welcome to fdisk (util-linux 2.23.2).
     Changes will remain in memory only, until you decide to write them.
     Be careful before using the write command.
     Command (m for help):

2. Type m to get a list of commands: Command (m for help): m

     Command action
        a   toggle a bootable flag
        b   edit bsd disklabel
        c   toggle the dos compatibility flag
        d   delete a partition
        g   create a new empty GPT partition table
        G   create an IRIX (SGI) partition table
        l   list known partition types
        m   print this menu
        n   add a new partition
        o   create a new empty DOS partition table
        p   print the partition table
        q   quit without saving changes
        s   create a new empty Sun disklabel
        t   change a partition’s system id
        u   change display/entry units
        v   verify the partition table
        w   write table to disk and exit
        x   extra functionality (experts only)
     Command (m for help):

3. The -C 130 which sets the number of phony cylinders in the drive is only necessary in old versions of fdisk, which unfortunately you will find on RHEL 6. However, it will do no harm on other distributions.
Create a new primary partition and make it 256 MB (or whatever size you would like:

    Command (m for help): n
     Partition type:
        p   primary (0 primary, 0 extended, 4 free)
        e   extended
     Select (default p): p
   Partition number (1-4, default 1): 1
   First sector (2048-2097151, default 2048):
   Using default value 2048
   Last sector, +sectors or +size{K,M,G} (2048-2097151, default 2097151): +256M
   Partition 1 of type Linux and of size 256 MiB is set

4. Add a second primary partition also of 256 MB in size:

   Command (m for help): n
   Partition type:
      p   primary (1 primary, 0 extended, 3 free)
      e   extended
   Select (default p): p
   Partition number (2-4, default 2): 2
   First sector (526336-2097151, default 526336):
   Using default value 526336
   Last sector, +sectors or +size{K,M,G} (526336-2097151, default 2097151): +256M
   Partition 2 of type Linux and of size 256 MiB is set
   Command (m for help): p
   Disk imagefile: 1073 MB, 1073741824 bytes, 2097152 sectors
   Units = sectors of 1 * 512 = 512 bytes
   Sector size (logical/physical): 512 bytes / 512 bytes
   I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x6280ced3
            Device Boot
   imagefile1            2048      526335
   imagefile2          526336     1050623

5. Write the partition table to disk and exit:

   Command (m for help): w
   The partition table has been altered!
   Syncing disks.
   End
     Blocks   Id  System
     262144   83  Linux
     262144   83  Linux

​LAB 3

We are going to experiment more with: 

• Loop devices and losetup
• parted to partition at the command line non-interactively.

We expect that you should read the man pages for losetup and parted before doing the following procedures.
Once again, you can reuse the image file or, better still, zero it out and start freshly or with another file.

1. Associate the image file with a loop device:

     $ sudo losetup -f 
     /dev/loop1
     $ sudo losetup /dev/loop1 imagefile

where the first command finds the first free loop device. The reason to do this is you may already be using one or more loop devices. For example, on the system that this is being written on, before the above command is executed:

$ losetup -a 
     /dev/loop0: []: (/usr/src/KERNELS.sqfs)

a squashfs compressed, read-only filesystem is already mounted using /dev/loop0. (The output of this command will vary with distribution.) If we were to ignore this and use losetup on /dev/loop0 we would almost definitely corrupt the file.

2. Create a disk partition label on the loop device (image file):

     $ sudo parted -s /dev/loop1 mklabel msdos

3. Create three primary partitions on the loop device:

     $ sudo parted -s /dev/loop1 unit MB mkpart primary ext4 0 256
     $ sudo parted -s /dev/loop1 unit MB mkpart primary ext4 256 512
     $ sudo parted -s /dev/loop1 unit MB mkpart primary ext4 512 1024

4. Check the partition table:

$ fdisk -l /dev/loop1

     Disk /dev/loop1: 1073 MB, 1073741824 bytes, 2097152 sectors
     Units = sectors of 1 * 512 = 512 bytes
     Sector size (logical/physical): 512 bytes / 512 bytes
     I/O size (minimum/optimal): 512 bytes / 512 bytes
     Disk label type: dos
     Disk identifier: 0x00050c11
      Device Boot
      /dev/loop1p1
      /dev/loop1p2
      /dev/loop1p3

5.What happens next depends on what distribution you are on. For example, on RHEL 7 and Ubuntu 14.04 you will find new device nodes have been created:

$ ls -l /dev/loop1*

brw-rw---- 1 root disk 7,1Oct 714:54 /dev/loop1 brw-rw---- 1 root disk 259, 0 Oct 7 14:54 /dev/loop1p1 brw-rw---- 1 root disk 259, 3 Oct 7 14:54 /dev/loop1p2 brw-rw---- 1 root disk 259, 4 Oct 7 14:54 /dev/loop1p3
and we will use them in the following. However, on RHEL 6 such nodes do not appear. Instead, you have to do:

$ sudo kpartx -lv /dev/loop1
$ sudo kpartx -av /dev/loop1
$ ls -l /dev/mapper/loop1*
lrwxrwxrwx 1 root root 7 Oct  9 07:12 /dev/mapper/loop1p1 -> ../dm-8
lrwxrwxrwx 1 root root 7 Oct  9 07:12 /dev/mapper/loop1p2 -> ../dm-9
lrwxrwxrwx 1 root root 8 Oct  9 07:12 /dev/mapper/loop1p3 -> ../dm-10

to associate device nodes with the partitions. So in what follows you can replace /dev/loop1p[1-3] with the actual names under /dev/mapper, or even easier you can do:

$ sudo ln -s /dev/mapper/loop1p1 /dev/loop1p1
$ sudo ln -s /dev/mapper/loop1p2 /dev/loop1p2
$ sudo ln -s /dev/mapper/loop1p3 /dev/loop1p3

6. Put filesystems on the partitions:

   $ sudo mkfs.ext3 /dev/loop1p1
   $ sudo mkfs.ext4 /dev/loop1p2
   $ sudo mkfs.vfat /dev/loop1p3

7. Mount all three filesystems and show they are available:

   $ mkdir mnt1 mnt2 mnt3
   $ sudo mount /dev/loop1p1 mnt1
   $ sudo mount /dev/loop1p2 mnt2
   $ sudo mount /dev/loop1p3 mnt3
   $ df -Th
Filesystem
/dev/sda1
....
/dev/loop1p1
/dev/loop1p2
/dev/loop1p3
Type      Size  Used Avail Use% Mounted on
ext4       29G  8.5G   19G  32% /
ext3      233M  2.1M  219M   1% mnt1
ext4      233M  2.1M  215M   1% mnt2
vfat      489M     0  489M   0% mnt3

8. After using the filesystems to your heart’s content you can unwind it all:

   $ sudo umount mnt1 mnt2 mnt3
   $ rmdir mnt1 mnt2 mnt3
   $ sudo losetup -d /dev/loop0