Let's Go Further: Advanced patterns for building APIs and web applications in Go

Contents
Introduction
Conventions
About the author
Copyright and disclaimer
Prerequisites
Background knowledge
Go 1.17
Other software
Getting Started
Project Setup and Skeleton Structure
Generating the skeleton directory structure
Hello world!
A Basic HTTP Server
Creating the healthcheck handler
Demonstration
Additional Information
API versioning
API Endpoints and RESTful Routing
Choosing a router
Encapsulating the API routes
Adding the new handler functions
Creating a helper to read ID parameters
Additional Information
Conflicting routes
Customizing httprouter behavior
Sending JSON Responses
Fixed-Format JSON
Additional Information
JSON charset
JSON Encoding
Creating a writeJSON helper method
Additional Information
How different Go types are encoded
Using json.Encoder
Performance of json.Encoder and json.Marshal
Additional JSON encoding nuances
Encoding Structs
Changing keys in the JSON object
Hiding struct fields in the JSON object
Additional Information
The string struct tag directive
Formatting and Enveloping Responses
Relative performance
Enveloping responses
Additional Information
Response structure
Advanced JSON Customization
Customizing the Runtime field
Additional Information
Alternative #1 - Customizing the Movie struct
Alternative #2 - Embedding an alias
Sending Error Messages
Routing errors
Additional Information
System-generated error responses
Parsing JSON Requests
JSON Decoding
Zero values
Additional Information
Supported destination types
Using the json.Unmarshal function
Additional JSON decoding nuances
Managing Bad Requests
Triaging the Decode error
Making a bad request helper
Additional Information
Panicking vs returning errors
Restricting Inputs
Custom JSON Decoding
The json.Unmarshaler interface
Validating JSON Input
Creating a validator package
Performing validation checks
Making validation rules reusable
Database Setup and Configuration
Setting up PostgreSQL
Installing PostgreSQL
Connecting to the PostgreSQL interactive terminal
Creating databases, users, and extensions
Connecting as the new user
Additional Information
Optimizing PostgreSQL settings
Connecting to PostgreSQL
Establishing a connection pool
Decoupling the DSN
Additional Information
Using the DSN with psql
Configuring the Database Connection Pool
Configuring the pool
The SetMaxOpenConns method
The SetMaxIdleConns method
The SetConnMaxLifetime method
The SetConnMaxIdleTime method
Putting it into practice
Configuring the connection pool
SQL Migrations
An Overview of SQL Migrations
Installing the migrate tool
Working with SQL Migrations
Executing the migrations
Additional Information
Migrating to a specific version
Executing down migrations
Fixing errors in SQL migrations
Remote migration files
Running migrations on application startup
CRUD Operations
Setting up the Movie Model
Additional Information
Mocking models
Creating a New Movie
Executing the SQL query
Hooking it up to our API handler
Creating additional records
Additional Information
$N notation
Executing multiple statements
Fetching a Movie
Updating the API handler
Additional Information
Why not use an unsigned integer for the movie ID?
Updating a Movie
Executing the SQL query
Creating the API handler
Using the new endpoint
Deleting a Movie
Adding the new endpoint
Advanced CRUD Operations
Handling Partial Updates
Performing the partial update
Demonstration
Additional Information
Null values in JSON
Optimistic Concurrency Control
Preventing the data race
Implementing optimistic locking
Additional Information
Round-trip locking
Locking on other fields or types
Managing SQL Query Timeouts
Mimicking a long-running query
Adding a query timeout
Timeouts outside of PostgreSQL
Updating our database model
Additional Information
Using the request context
Filtering, Sorting, and Pagination
Parsing Query String Parameters
Creating helper functions
Adding the API handler and route
Creating a Filters struct
Validating Query String Parameters
Listing Data
Updating the application
Filtering Lists
Dynamic filtering in the SQL query
Full-Text Search
Adding indexes
Additional Information
Non-simple configuration and more information
Using STRPOS and ILIKE
Sorting Lists
Implementing sorting
Paginating Lists
The LIMIT and OFFSET clauses
Updating the database model
Returning Pagination Metadata
Calculating the total records
Updating the code
Structured Logging and Error Handling
Structured JSON Log Entries
Creating a custom logger
Additional Information
Integration with the http.Server error log
Third-party logging packages
Panic Recovery
Additional Information
Panic recovery in other goroutines
Rate Limiting
Global Rate Limiting
Enforcing a global rate limit
IP-based Rate Limiting
Deleting old limiters
Additional Information
Distributed applications
Configuring the Rate Limiters
Graceful Shutdown
Sending Shutdown Signals
Intercepting Shutdown Signals
Catching SIGINT and SIGTERM signals
Executing the Shutdown
User Model Setup and Registration
Setting up the Users Database Table
Setting up the Users Model
Adding Validation Checks
Creating the UserModel
Registering a User
Additional Information
Email case-sensitivity
User enumeration
Sending Emails
SMTP Server Setup
Setting up Mailtrap
Creating Email Templates
Sending a Welcome Email
Creating an email helper
Using embedded file systems
Using our mail helper
Checking the email in Mailtrap
Additional Information
Retrying email send attempts
Sending Background Emails
Recovering panics
Using a helper function
Graceful Shutdown of Background Tasks
An introduction to sync.WaitGroup
Fixing our application
User Activation
Setting up the Tokens Database Table
Creating Secure Activation Tokens
Creating the TokenModel and Validation Checks
Additional Information
The math/rand package
Sending Activation Tokens
Additional Information
A standalone endpoint for generating tokens
Activating a User
Creating the activateUserHandler
The UserModel.GetForToken method
Additional Information
Web application workflow
SQL query timing attack
Authentication
Authentication Options
HTTP basic authentication
Token authentication
Stateful token authentication
Stateless token authentication
API-key authentication
OAuth 2.0 / OpenID Connect
What authentication approach should I use?
Generating Authentication Tokens
Building the endpoint
Additional Information
The Authorization header
Authenticating Requests
Creating the anonymous user
Reading and writing to the request context
Creating the authentication middleware
Demonstration
Permission-based Authorization
Requiring User Activation
Demonstration
Splitting up the middleware
Additional Information
In-handler checks
Setting up the Permissions Database Table
Relationship between permissions and users
Creating the SQL migrations
Setting up the Permissions Model
Checking Permissions
Demonstration
Granting Permissions
Updating the permissions model
Updating the registration handler
Cross Origin Requests
An Overview of CORS
Demonstrating the Same-Origin Policy
Demonstration
Simple CORS Requests
Restricting Origins
Supporting multiple dynamic origins
Additional Information
Partial origin matches
The null origin
Authentication and CORS
Preflight CORS Requests
Demonstrating a preflight request
Responding to preflight requests
Updating our middleware
Additional Information
Caching preflight responses
Preflight wildcards
Metrics
Exposing Metrics with Expvar
Creating Custom Metrics
Dynamic metrics
Additional Information
Protecting the metrics endpoint
Removing default metrics
Request-level Metrics
Additional Information
Calculating additional metrics
Recording HTTP Status Codes
Additional Information
Visualizing and analyzing metrics
Building, Versioning and Quality Control
Creating and Using Makefiles
A simple makefile
Environment variables
Passing arguments
Namespacing targets
Prerequisite targets and asking for confirmation
Displaying help information
Phony targets
Managing Environment Variables
Using a .envrc file
Quality Controlling Code
Additional Information
Testing
Module Proxies and Vendoring
Module proxies
Vendoring
Vendoring new dependencies
Additional Information
The ./… pattern
Building Binaries
Reducing binary size
Cross-compilation
Additional Information
Build caching
Managing and Automating Version Numbers
Displaying the version number
Including the build time
Automated version numbering with Git
Using Git tags
Deployment and Hosting
Creating a Digital Ocean Droplet
Creating a SSH key
Adding the SSH key to Digital Ocean
Creating a droplet
Server Configuration and Installing Software
Connecting as the greenlight user
Connecting to the droplet
Additional Information
Future changes to the droplet configuration
Deployment and Executing Migrations
Running the API
Running the API as a Background Service
Restarting after reboot
Disable port 4000
Additional Information
Listening on a restricted port
Viewing logs
Configuring the SMTP provider
Additional unit file options
Using Caddy as a Reverse Proxy
Blocking access to application metrics
Using a domain name
Enabling HTTPS
Additional Information
Scaling the infrastructure
Appendices
Managing Password Resets
Additional Information
Web application workflow
Creating Additional Activation Tokens
Authentication with JSON Web Tokens
JSON Encoding Nuances
Nil and empty slices are encoded differently
Using omitempty on a zero-valued struct doesn’t work
Using omitempty on a zero-value time.Time doesn’t work
Non-ASCII punctuation characters aren’t supported in struct tags
Integer, time.Time and net.IP values can be used as map keys
Angle brackets and ampersands in strings are escaped
Trailing zeroes are removed from floats
Working with pre-computed JSON
The MarshalText fallback
The receiver matters when using MarshalJSON
JSON Decoding Nuances
Decoding into Go arrays
Partial JSON decoding
Decoding into interface{} types
Decoding a JSON number to an interface{}
Struct tag directives
Request Context Timeouts
Feedback