Architecture and Design of the Linux Storage Stack: Gain a deep understanding of the Linux storage landscape

Architecture and Design of the Linux Storage Stack
Contributors
About the author
About the reviewer
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Share Your Thoughts
Download a free PDF copy of this book
Part 1: Diving into the Virtual Filesystem
1
Where It All Starts From – The Virtual Filesystem
Technical requirements
Understanding storage in a modern-day data center
Interacting with storage in Linux
Understanding system calls
Explaining the need for a Virtual Filesystem
Describing the VFS
Implementing a common filesystem interface through the VFS
Peeking at the source code
Tracing VFS functions
Explaining the Everything is a file philosophy
Summary
2
Explaining the Data Structures in a VFS
Technical requirements
Data structures in VFS
Inodes – indexing files and directories
Defining an inode in the kernel
Defining inode operations
Tracking file data on disk through inodes
Can a filesystem run out of inodes?
Directory entries – mapping inodes to filenames
Dentry cache
Dentry states
Dentry operations
File objects – representing open files
Defining file operations
Can a process run out of file descriptors?
Superblocks – describing filesystem metadata
Superblock operations in the kernel
Linking
Summarizing the four structures
Page cache
Reading
Summary
3
Exploring the Actual Filesystems Under the VFS
Technical requirements
The Linux filesystem gallery
The diary of a filesystem – the concept of journaling
The burning question – doesn’t journaling adversely affect performance?
The curious case of CoW filesystems
Extended filesystem
Blocks – the lingua franca of filesystems
Is there a more efficient approach to organizing data?
The structural layout of an Ext4 filesystem
Network filesystem
NFS architecture
Comparing NFS with regular block filesystems
FUSE – a unique method for creating filesystems
Summary
Part 2: Navigating Through the Block Layer
4
Understanding the Block Layer, Block Devices, and Data Structures
Technical requirements
Explaining the role of the block layer
Defining block devices
The defining characteristics of a block device
Looking at the representation of block devices
Looking at data structures in the block layer
The register_blkdev function (block device registration)
The block_device structure (representing block devices)
The gendisk structure (representing physical disks)
The buffer_head structure (representing blocks in memory)
The bio structure (representing active block I/Os)
The bio_vec structure (representing vector I/O)
Requests and request queues (representing pending I/O requests)
The journey of an I/O request in the block layer
Summary
5
Understanding the Block Layer, Multi-Queue, and Device Mapper
Technical requirements
Looking at problems with single-request queues
Understanding the multi-queue block I/O framework
Looking at data structures
Looking at the device mapper framework
Looking at multi-tier caching mechanisms in the block layer
Summary
6
Understanding I/O Handling and Scheduling in the Block Layer
Technical requirements
Understanding the I/O handling techniques in block layer
Sorting
Merging
Coalescing
Plugging
Explaining the Linux I/O schedulers
The MQ-deadline scheduler – guaranteeing a start service time
Budget fair queuing – providing proportional disk share
Kyber – prioritizing throughput
None – minimal scheduling overhead
Discussing the scheduling conundrum
Summary
Part 3: Descending into the Physical Layer
7
The SCSI Subsystem
Technical requirements
The device driver model
Explaining the SCSI subsystem
SCSI architecture
The client and server model
Summary
8
Illustrating the Layout of Physical Media
Technical requirements
Understanding mechanical hard drives
Looking at the physical layout and addressing
Looking at bad sectors
Looking at hard drive performance
Understanding where mechanical drives lag
Explaining the architecture of solid-state drives
Looking at the physical layout and addressing
Looking at reads and writes
Erasing, garbage collection, and the illusion of available space
Looking at wear leveling
Looking at bad block management
Looking at SSD performance
Understanding where SSDs lag
Understanding drive endurance
Reinventing SSDs with NVMe
Summary
Part 4: Analyzing and Troubleshooting Storage Performance
9
Analyzing Physical Storage Performance
Technical requirements
How do we gauge performance?
Understanding storage topology
Analyzing physical storage
Understanding disk service time
Disk access patterns
Determining reads/writes ratio and I/O size
Disk cache
IOPS and throughput
Queue depth
Determining disk busyness
I/O wait
Using disk I/O analysis tools
Establish a baseline using top
The iotop utility
The iostat utility
Performance Co-Pilot
The vmstat command
Pressure Stall Index
Summary
10
Analyzing Filesystems and the Block Layer
Technical requirements
Investigating filesystems and the block layer
The different types of filesystem I/O
What causes filesystem latency?
Identifying the target layers
Finding the right tools
Tracing application calls
Tracing VFS calls
Analyzing cache usage
Analyzing filesystems
Analyzing block I/O
Summarizing the tools
Summary
11
Tuning the I/O Stack
Technical requirements
How memory usage affects I/O
Tuning the memory subsystem
Using direct I/O
Controlling the write-back frequency
Tuning the filesystem
Block size
Filesystem I/O alignment
LVM I/O alignment
Journaling
Barriers
Timestamps
Read-ahead
Discarding unused blocks
Choosing the right scheduler
Summary
Index
Why subscribe?
Other Books You May Enjoy
Packt is searching for authors like you
Share Your Thoughts
Download a free PDF copy of this book