Windows Internals

This document was uploaded by one of our users. The uploader already confirmed that they had the permission to publish it. If you are author/publisher or own the copyright of this documents, please report to us by using this DMCA report form.

Simply click on the Download Book button.

Yes, Book downloads on Ebookily are 100% Free.

Sometimes the book is free on Amazon As well, so go ahead and hit "Search on Amazon"

The definitive guide to modern Windows internals: new coverage of virtualization, file systems, boot, security, and more. For advanced computing professionals, this is the definitive up-to-date guide to how Windows core components behave “under the hood.” Using it, experienced developers can build more powerful and scalable software, administrators can debug complex system and performance problems, and security researchers can harden their systems. This Seventh Edition is fully updated through the May 2021 (21H1/2104) updates to Windows 10 and Windows Server (2022, 2019, and 2016). It adds extensive content on Hyper-V, plus fully rewritten chapters on the boot process, new storage technologies, and Windows system and management mechanisms. As always, it delivers unparalleled insight based on insider access to Microsoft source code, with hands-on experiments using the latest debugging tools to show you Windows’ internal behaviors firsthand. With Windows 11 introducing new user interface design elements that build upon the same core technologies as Windows 10, readers will be well-prepared for this new chapter of computing. Leading Windows insiders help you: • Discover system mechanisms for serving device drivers and applications, including ALPC, Object Manager, synchronization, WNF, WoW64, and the processor execution model • Explore underlying hardware architecture, including trap processing, segmentation, and side channel vulnerabilities • Understand Windows virtualization, and how virtualization-based security (VBS) protects against OS vulnerabilities • Delve into key management and configuration mechanisms, including the Registry, Windows services, WMI, and Task Scheduling • Explore diagnostic services such as Event Tracing for Windows (ETW) and DTrace • Learn how the cache manager and file system drivers interact to provide reliable support for files, directories, and disks, including on Persistent Memory (NVDIMM) DAX devices. • Understand NTFS, ReFS, and other Windows file systems • Review Windows startup/shutdown operations, and OS components involved in boot flow • Analyze UEFI-based Secure Boot, Measured Boot, and Secure Launch About This Book • For experienced programmers, architects, software quality and performance specialists, administrators, security practitioners, and support professionals • Assumes you are a Windows power user

Author(s): Andrea Allievi, Alex Ionescu, Mark Russinovich, David Solomon
Series: Developer Reference
Edition: 7
Publisher: Microsoft Press
Year: 2021

Language: English
Pages: 912
Tags: MS Windows; Windows Internals; System mechanisms


About the Authors


Foreword


Introduction


Chapter 8 System mechanisms


Processor execution model


Segmentation


Task state segments


Hardware side-channel vulnerabilities


Out-of-order execution


The CPU branch predictor


The CPU cache(s)


Side-channel attacks


Side-channel mitigations in Windows


KVA Shadow


Hardware indirect branch controls (IBRS, IBPB, STIBP, SSBD)


Retpoline and import optimization


STIBP pairing


Trap dispatching


Interrupt dispatching


Line-based versus message signaled–based interrupts


Timer processing


System worker threads


Exception dispatching


System service handling


WoW64 (Windows-on-Windows)


The WoW64 core


File system redirection


Registry redirection


X86 simulation on AMD64 platforms


ARM


Memory models


ARM32 simulation on ARM64 platforms


X86 simulation on ARM64 platforms


Object Manager


Executive objects


Object structure


Synchronization


High-IRQL synchronization


Low-IRQL synchronization


Advanced local procedure call


Connection model


Message model


Asynchronous operation


Views, regions, and sections


Attributes


Blobs, handles, and resources


Handle passing


Security


Performance


Power management


ALPC direct event attribute


Debugging and tracing


Windows Notification Facility


WNF features


WNF users


WNF state names and storage


WNF event aggregation


User-mode debugging


Kernel support


Native support


Windows subsystem support


Packaged applications


UWP applications


Centennial applications


The Host Activity Manager


The State Repository


The Dependency Mini Repository


Background tasks and the Broker Infrastructure


Packaged applications setup and startup


Package activation


Package registration


Conclusion


Chapter 9 Virtualization technologies


The Windows hypervisor


Partitions, processes, and threads


The hypervisor startup


The hypervisor memory manager


Hyper-V schedulers


Hypercalls and the hypervisor TLFS


Intercepts


The synthetic interrupt controller (SynIC)


The Windows hypervisor platform API and EXO partitions


Nested virtualization


The Windows hypervisor on ARM64


The virtualization stack


Virtual machine manager service and worker processes


The VID driver and the virtualization stack memory manager


The birth of a Virtual Machine (VM)


VMBus


Virtual hardware support


VA-backed virtual machines


Virtualization-based security (VBS)


Virtual trust levels (VTLs) and Virtual Secure Mode (VSM)


Services provided by the VSM and requirements


The Secure Kernel


Virtual interrupts


Secure intercepts


VSM system calls


Secure threads and scheduling


The Hypervisor Enforced Code Integrity


UEFI runtime virtualization


VSM startup


The Secure Kernel memory manager


Hot patching


Isolated User Mode


Trustlets creation


Secure devices


VBS-based enclaves


System Guard runtime attestation


Conclusion


Chapter 10 Management, diagnostics, and tracing


The registry


Viewing and changing the registry


Registry usage


Registry data types


Registry logical structure


Application hives


Transactional Registry (TxR)


Monitoring registry activity


Process Monitor internals


Registry internals


Hive reorganization


The registry namespace and operation


Stable storage


Registry filtering


Registry virtualization


Registry optimizations


Windows services


Service applications


Service accounts


The Service Control Manager (SCM)


Service control programs


Autostart services startup


Delayed autostart services


Triggered-start services


Startup errors


Accepting the boot and last known good


Service failures


Service shutdown


Shared service processes


Service tags


User services


Packaged services


Protected services


Task scheduling and UBPM


The Task Scheduler


Unified Background Process Manager (UBPM)


Task Scheduler COM interfaces


Windows Management Instrumentation


WMI architecture


WMI providers


The Common Information Model and the Managed Object Format Language


Class association


WMI implementation


WMI security


Event Tracing for Windows (ETW)


ETW initialization


ETW sessions


ETW providers


Providing events


ETW Logger thread


Consuming events


System loggers


ETW security


Dynamic tracing (DTrace)


Internal architecture


DTrace type library


Windows Error Reporting (WER)


User applications crashes


Kernel-mode (system) crashes


Process hang detection


Global flags


Kernel shims


Shim engine initialization


The shim database


Driver shims


Device shims


Conclusion


Chapter 11 Caching and file systems


Terminology


Key features of the cache manager


Single, centralized system cache


The memory manager


Cache coherency


Virtual block caching


Stream-based caching


Recoverable file system support


NTFS MFT working set enhancements


Memory partitions support


Cache virtual memory management


Cache size


Cache virtual size


Cache working set size


Cache physical size


Cache data structures


Systemwide cache data structures


Per-file cache data structures


File system interfaces


Copying to and from the cache


Caching with the mapping and pinning interfaces


Caching with the direct memory access interfaces


Fast I/O


Read-ahead and write-behind


Intelligent read-ahead


Read-ahead enhancements


Write-back caching and lazy writing


Disabling lazy writing for a file


Forcing the cache to write through to disk


Flushing mapped files


Write throttling


System threads


Aggressive write behind and low-priority lazy writes


Dynamic memory


Cache manager disk I/O accounting


File systems


Windows file system formats


CDFS


UDF


FAT12, FAT16, and FAT32


exFAT


NTFS


ReFS


File system driver architecture


Local FSDs


Remote FSDs


File system operations


Explicit file I/O


Memory manager’s modified and mapped page writer


Cache manager’s lazy writer


Cache manager’s read-ahead thread


Memory manager’s page fault handler


File system filter drivers and minifilters


Filtering named pipes and mailslots


Controlling reparse point behavior


Process Monitor


The NT File System (NTFS)


High-end file system requirements


Recoverability


Security


Data redundancy and fault tolerance


Advanced features of NTFS


Multiple data streams


Unicode-based names


General indexing facility


Dynamic bad-cluster remapping


Hard links


Symbolic (soft) links and junctions


Compression and sparse files


Change logging


Per-user volume quotas


Link tracking


Encryption


POSIX-style delete semantics


Defragmentation


Dynamic partitioning


NTFS support for tiered volumes


NTFS file system driver


NTFS on-disk structure


Volumes


Clusters


Master file table


File record numbers


File records


File names


Tunneling


Resident and nonresident attributes


Data compression and sparse files


Compressing sparse data


Compressing nonsparse data


Sparse files


The change journal file


Indexing


Object IDs


Quota tracking


Consolidated security


Reparse points


Storage reserves and NTFS reservations


Transaction support


Isolation


Transactional APIs


On-disk implementation


Logging implementation


NTFS recovery support


Design


Metadata logging


Log file service


Log record types


Recovery


Analysis pass


Redo pass


Undo pass


NTFS bad-cluster recovery


Self-healing


Online check-disk and fast repair


Encrypted file system


Encrypting a file for the first time


The decryption process


Backing up encrypted files


Copying encrypted files


BitLocker encryption offload


Online encryption support


Direct Access (DAX) disks


DAX driver model


DAX volumes


Cached and noncached I/O in DAX volumes


Mapping of executable images


Block volumes


File system filter drivers and DAX


Flushing DAX mode I/Os


Large and huge pages support


Virtual PM disks and storages spaces support


Resilient File System (ReFS)


Minstore architecture


B+ tree physical layout


Allocators


Page table


Minstore I/O


ReFS architecture


ReFS on-disk structure


Object IDs


Security and change journal


ReFS advanced features


File’s block cloning (snapshot support) and sparse VDL


ReFS write-through


ReFS recovery support


Leak detection


Shingled magnetic recording (SMR) volumes


ReFS support for tiered volumes and SMR


Container compaction


Compression and ghosting


Storage Spaces


Spaces internal architecture


Services provided by Spaces


Conclusion


Chapter 12 Startup and shutdown


Boot process


The UEFI boot


The BIOS boot process


Secure Boot


The Windows Boot Manager


The Boot menu


Launching a boot application


Measured Boot


Trusted execution


The Windows OS Loader


Booting from iSCSI


The hypervisor loader


VSM startup policy


The Secure Launch


Initializing the kernel and executive subsystems


Kernel initialization phase 1


Smss, Csrss, and Wininit


ReadyBoot


Images that start automatically


Shutdown


Hibernation and Fast Startup


Windows Recovery Environment (WinRE)


Safe mode


Driver loading in safe mode


Safe-mode-aware user programs


Boot status file


Conclusion


Contents of Windows Internals, Seventh Edition, Part 1


Index