CS 480 M01 Lecture Notes 13 - Oct 17, 2022

Ch 20. Storage

variety of configs: ligthning-fast storage => vast, archival vault
hardware (HDD, SSD, tapes, RAID, ...) and software (device drivers, partitioning, RAID, LVM, virtualizing over network, filesystems implementations)

1. Adding a Disk - (Simple / Traditional) Disk Installation Procedure

   • Write down disk's manufacturer, model ,size, serial #, ...
   • Connect / attach the drive (USB vs SATA or SAS, hot-plugable ....) - ! the new drive may not be on the highest device number!
   • lsblk -o +MODEL,SERIAL
   • (Low Level Formatting - delineating sectors by writing address information and timing marks on platters)
   • Labeling and Partitioning / Installing partition table on the disk
     parted / gparted , fdisk, cfdisk, (or gdisk) - these days it should understand GPT-style partition tables
     fdisk /dev/sdb   =>   g => n => <enter> ... => w
     the device file for the partition is /dev/sdb1
   • (Let the kernel know about the new partition table)
   • Creating Filesystems
     Implementing the file system abstractions and goodies as described later in the chapter
     mke2fs, mkfs, ....
     mkfs -t ext4 -L spare20GB /dev/sdb1 for EXT4 FS with a shorthand label "spare4TB"
   • Creating Mount Points and Mounting Filesystems
     mkdir /extra2
mount /dev/sdb1 /extra2
     before the next step make sure all worked
   • Setting up automatic mounting & swapping
     /etc/fstab


disk partitioning (sda4 is swap ...)

2. Storage Hardware

   essentially the same 50+ years old technology ... ( wikipedia history)
   
   plus finally a new practical alternative few years ago: SSD - Solid State Disks (flash memory based devices)
   and hybrids (HDs with large SSD buffers)....

   both use the same interfaces and protocols

   Characteristics	HDD	SSD (book vs FA17 numbers)
   Size	Terabytes	Gigabytes (4TB)
   Random Access Time	8 ms	0.25 ms
   Sequential Read	100 MB/s	250 MB/s ( up to ~5 GB/s)
   Random Read	2 MB/s	250 MB/s ( up to ~3.5 GB/s)
   Cost	$0.10 / GB	$3 / GB ($0.25 - $2 / GB)
   Reliability	moderate	unknown
   Limited writes	No	Yes ( ~100,000 / page - check DWPD or TBW : 1 DWPD 1 yr warranty 1TB SSD = 365 TBW SSD )
   other		pre-erasing, erased pool / FS usage, alignment (standard 512B disk block too small) ...

   
   HDD reliability : AFR (average annual failure rate) = 1 / MTBF (mean time between failures )
   Drives: Value vs Mass-market performance vs NAS vs Enterprise
   SSD rewritability limits
   SSD flash memory types: SLC / MLC / TLC cells (single / multiple / tripple bits of info stored in each individual flash memory location => price, speed, ...)
   SSD controller & interfaces ...
   SSD reliability ...

3. Interfaces

   • ATA, SATA (Serial ATA)- higher transfer rates, longer cables, hot-swap, command queuing (SATA-III : 6 Gb/s)
   • PCI Express v. 5.0 : 32 GT/s at 3.9 GB/s (×1) 63 GB/s (×16) ; wikipedia page
     GT/s : gigatransfers per second => 64bit wide bus and 1GT/s = 8GB/s)
   • SCSI - multiple disks on a bus (WideUtra3 160MB/s, 12m), daisy chain, target numbers; SAS (Serial Attached SCSI)
   • USB (Universal Serial Bus) 10 GB/s for USB 3.1, 20 GB/s for USB 3.2
   • Fiber Channel - large number of devices, ~100MB/s & up, ...
   • PATA / IDE (Parallel Advanced technology Attachment / Integrated Drive Electronics)
   
   Software does not care anymore ... LBA - Logical Block Addressing

4. Attachement and Low-level Management of Drives

lsblk -o +MODEL,SERIAL
devices /dev/sda, /dev/sdb, ...
Ephemeral device names - Adding a disk or rebooting can change the device name (sdb => sdc - names are assigned in sequence as the kernel enumerates the interfaces and devices)
/dev/disk
by-id
by-label
by-partuuid
by-path
by-uuid
Formatting and bad block management
= low level formatting ( not writing a partition table) - read the book
ATA secure erase since 2000 PATA and SATA disks implemented command to overwrite disk data - NIST-certified
hdparm
interact with the firmware of SATA and SAS hard disks, secure erase, get/set device parameters:
power options
enable or disable noise reduction options
set the read-only flag
print detailed drive information
...
• Hard disk monitoring with SMART
  Hard disks - fault-tolerant systems , use error-correction coding and firmware to hide imperfections from OS
  SATA devices have detailed status reporting sometimes predictive of HW failures - SMART: Self-Monitoring, Analysis, and Reporting Technology
  > 50 operational parameters: Scan error, Reallocation, Off-line reallocation counts; Number of sectors on probation, ...
  smartd daemon - monitors drives continuously plus /etc/smartd.conf and smartctl command

5. Software Side of Storage

   ===> in the picture means "can be built on" ...
   
   
   • storage device - HD, SSD, external RAID (implemented in hardware) ,...
     • allows random access
     • handles block I/O
     • is represented by a device file
   • partition - fixed size subsection of a storage device
     • has its own device file
     • acts much like an independent storage device
     • the same driver for device handles partitioning usually
   • RAID - Redundant Array of Independent (Inexpensive) Disks - combines multiple storage devices and creates one virtual device. Improve reliability, availability, performance, and capacity through combination of various mirroring, striping, and parity schemes (various RAID levels 0-6,10)
   • Volume groups and logical volumes "essentially a supercharged and abstracted version of disk partitioning " Allows to reallocate space dynamically (grow, shrink, move, ...). Aggregate physical devices into a volume group and split the group into logical volumes. Example: use two disks (250GB and 750GB) to create 1TB volume group, split that into two 500GB logical volumes.
     pvcreate, vgcreate, vgdisplay, lvcreate, ...
   • Filesystem - where and how the content of files are stored, namespace representation, corruption resilience, ...
     kernel (swap space) and DB may not need it ... (may have their own structure)
     

6. Disk Partitioning

   less desirable than LVM in most respects - no snapshots, can't / hard to change later, ...
     ... but simple (and Windows and PC BIOS understands it)

   

   Disk device files

   • device: /dev/sda . partitions: /dev/sda1, /dev/sda2, ...
   • ! "dancing names" issue .... - not in our case
   • parted -l
   Ways to divide up disks
   separate partitions for /tmp (memory based perhaps for performance)
   /var/log or /var
   /home
   swap split among several disks for performance, increase with RAM increase
   partitions vs backup - frequently changing data on partitions that are backed up more frequently

   Traditional partitioning

   • Implemented by writing "label" at the beginning of the disk - defines the range of blocks included in each partition
   • Label may have to coexist with other startup info - boot block and other info - disk label
   • Device driver representing the disk reads the label and uses the partition table to calculate the physical location of each partition
   • Typically one device file per partition plus one for the whole disk
   • Hard to change once done and partitions already used.
   • Only two partitioning schemes: MBR and GPT
   • for PC hardware the boot disk must have partition table
   • MBR or GPT partition table makes a disk comprehensible to Windows
   • Partitions have a defined location on the disk => guarantee locality of reference
   • RAID systems use disks or partitions of matched size

   MBR partitioning

   • Old Microsoft standard since 1980s supporting up to 2TB disks only
   • The only option for old PC hardware with Windows
   • MBR label occupies single 512-byte disk block at the very beginning of a partitioned disk:
       Contains boot(strap) code + space to define 4 partitions : primary partitions
   • One partition can be "extended" containing its own subsidiary partition table of "secondary" partitions
   • 32 bits addressable limit for block addresses and sizes => can support up to 2TB large disks (using 512B sectors)
   • Windows partitioning system lets one partition be marked “active.” - boot loaders try to look for it and load OS from it
   • fdisk MS-DOS command also adopted by systems supporting MBR partitioning - many variations ...

   next ... GPT partitioning