The AWK Programming Language, 2nd Edition (Early Release)

The goal of this book is to teach you what AWK is and how to use it effectively. AWK was created in 1977 as a simple programming language for writing short programs that manipulate text and numbers with equal ease. It was meant as a scripting language to complement and work well with Unix tools, following the Unix philosophy of having each program do one thing well and be composable with other programs. The computing world today is enormously different from what it was in 1977. Computers are thousands of times faster and have a million times as much memory. Software is different too, with a rich variety of programming languages and computing environments. The Internet has given us more data to process, and it comes from all over the world. We’re no longer limited to the 26 letters of English either; thanks to Unicode, computers process the languages of the world in their native character sets. Even though AWK is nearly 50 years old, and in spite of the great changes in computing, it’s still widely used, a core Unix tool that’s available on any Unix, Linux, or macOS system, and usually on Windows as well. There’s nothing to download, no libraries or packages to import — just use it. It’s an easy language to learn and you can do a lot after only a few minutes of study. AWK is a programming language that makes it possible to handle simple computations with short programs, often only one or two lines long. An AWK program is a sequence of patterns and actions that specify what to look for in the input data and what to do when it’s found. Awk searches a set of files that contain text (but not non-text formats like Word documents, spreadsheets, PDFs and so on) for lines that match any of the patterns; when a matching line is found, the corresponding action is performed. A pattern can select lines by combinations of regular expressions and comparison operations on strings, numbers, fields, variables, and array elements. Actions may perform arbitrary processing on selected lines; the action language looks like C but there are no declarations, and strings and numbers are built-in data types. AWK scans text input files and splits each input line into fields automatically. Because so many things are automatic — input, field splitting, storage management, initialization — Awk programs are usually much shorter than they would be in a more conventional language. Thus one common use of AWK is for the kind of data manipulation suggested above. Programs, a line or two long, are composed at the keyboard, run once, then discarded. In effect, AWK is a general-purpose programmable tool that can replace a host of specialized tools or programs. Perform exploratory data analysis Retrieve, validate, transform, and summarize data-the tasks AWK was originally designed for: Manage small databases, create reports, and write queries Generate text and streamline document preparation Write specialized "little languages" for narrow domains Express, experiment with, and test algorithms For programmers on Unix, Linux, MacOS, and Windows platforms, this updated guide is the most reliable source for learning how to tap into the power of AWK.