Elasticsearch is a powerful tool for handling and managing large amount of data. It is scalable, reliable, and fast, with various features for data analysis and search. This book is a comprehensive guide to using Elasticsearch to manage data. It starts with an overview of Elasticsearch, detailing its importance in today's world. The book further covers the basics of Elasticsearch, including installation, configuration, and index management. Next, the book covers more advanced topics, such as handling geospatial data and using aggregations to analyze data. It also covers performance optimization and administration.
Author(s): Anurag Srivastava
Edition: 2
Publisher: BPB Online
Year: 2023
Language: English
Pages: 418
Cover
Title Page
Copyright Page
Dedication Page
About the Author
About the Reviewer
Acknowledgement
Preface
Table of Contents
1. Getting Started with Elasticsearch
Introduction
Structure
Objectives
Introduction to data search
What is Elasticsearch, and why is it important for search and analytics
Overview of Elasticsearch architecture and components
Node
Master-eligible node
Dedicated master-eligible node
Voting-only master-eligible node
Data node
Ingest node
Machine learning node
Hot data node
Warm data node
Cold data node
Frozen data node
Cluster
Index
Shards
Documents
Applications and use cases for Elasticsearch
Data search
Data logging and analysis
Application Performance Monitoring
System performance monitoring
Data visualization
Different Elasticsearch clients and their usage scenarios
Java
PHP
Perl
Python
.NET
Ruby
JavaScript
Conclusion
Questions
2. Installing Elasticsearch
Introduction
Structure
Objectives
Introduction to Elasticsearch 8
Improved indexing performance
Search performance enhancements
Cross-cluster search improvements
Security enhancements
Operational enhancements
Installing Elasticsearch
Installing Elasticsearch on Linux or macOS
Installing Elasticsearch on Linux
Installing Elasticsearch on macOS
Installing Elasticsearch using the Debian Package
Installing the Debian package manually
Installing Elasticsearch using the RPM package
Installing the RPM package manually
Installing Elasticsearch on Windows
Starting and verifying the Elasticsearch service
Elasticsearch REST APIs
cat APIs
cat API parameters
Verbose
Help
Headers
Response formats
Sort
cat count API
cat health API
cat indices API
cat master API
cat nodes API
cat shards API
Cluster APIs
Cluster health API
Cluster stats API
Conclusion
Questions
3. Elastic Stack: The Ecosystem of Elasticsearch
Introduction
Structure
Objectives
Overview of Elastic Stack components
Elasticsearch: The search and analytics engine
Logstash: The data processing pipeline
Logstash input plugin
Logstash filter plugin
Logstash output plugin
Kibana: The data visualization tool
Beats: The lightweight data shippers
Filebeat
Configure output
Metricbeat
Packetbeat
Configuring Packetbeat
Winlogbeat
Auditbeat
Heartbeat
Functionbeat
Integration of Elastic Stack components
Fetch Apache logs using Logstash
Conclusion
Questions
4. Preparing Data for Indexing
Introduction
Structure
Objectives
The importance of data preparation before indexing
An introduction to Elasticsearch analyzers
Built-in analyzer
Standard analyzer
Whitespace analyzer
Stop analyzer
Pattern analyzer
Language analyzers
Fingerprint analyzer
Exploring tokenizers in Elasticsearch
Word-oriented tokenizers
Letter tokenizer
Lowercase tokenizer
Classic tokenizer
Partial word tokenizers
Edge n-gram Tokenizer
Structured text tokenizers
Understanding token filters
Exploring character filters in Elasticsearch
HTML strip character filter
Mapping the char filter
Pattern replace character filter
Understanding normalizers
Conclusion
Questions
5. Importing Data into Elasticsearch
Introduction
Structure
Objectives
Why is data important for business
Data shipping
Data ingestion
Data storage
Data visualization
Importing data into Elasticsearch using different Beats
Filebeat
Filebeat modules
Pull Apache logs using Filebeat
Change the index name in Filebeat
Metricbeat
Metricbeat modules
Pull server metrics using Metricbeat
Packetbeat
Pulling network data using Packetbeat
Pulling CSV data using Logstash
Conclusion
Questions
6. Index Management: Creating, Updating, and Deleting Elasticsearch Indices
Introduction
Structure
Objectives
Introduction to Elasticsearch index creation and mapping
Creating an index without any document
Creating index along with the documents
Get mapping of the index
Creating a mapping of the index
Index management in Elasticsearch
Performing operations on Elasticsearch indices
Close index
Delete index
Freeze index
Refresh index
Force merge index
Clear index cache
Flush index
Add lifecycle policy
Elasticsearch index APIs
Index management
Creating an index
Delete index
Get index
Close index
Open index
Index exists API
Shrink index
Freeze index
Unfreeze index
Split index
Clone index
Rollover index
Index settings
Update index settings
Get index settings
Managing Elasticsearch index templates
Creating an index template
Get index template
Delete index template
Index Lifecycle Management in Elasticsearch
Conclusion
Questions
7. Search Capabilities: Mastering Query DSL and Search Techniques
Introduction
Structure
Objectives
URI search
Empty search
Field search
Query DSL
Filters and queries
Query
Query types
Full-text search
match_all
match
match_phrase
multi_match
query_string
Term-level queries
Term query
Terms query
Exists query
Range query
Fuzzy query
Wildcard query
Compound queries
Boolean query
Boosting query
Multi-search
Multi-search API
Search and multi-search templates
Search template
Multi search template
Explain API
Inverse document frequency and term frequency
Inverse document frequency
Term frequency
Profile API
Conclusion
Questions
8. Handling Geo with Elasticsearch
Introduction
Structure
Objectives
Introduction to Geospatial search
Geo data types in Elasticsearch
Geo point data
Creating mapping
Saving geo point data
Geo shape data
Creating mapping
Saving geo point data
Point
LineString
Polygon
MultiPoint
MultiLineString
MultiPolygon
GeometryCollection
Envelope
Circle
Geo query and filter DSL
Geo-distance queries
Geo-polygon queries
Geo-bounding box queries
Geo-shape queries
Use case
Restaurant search
Geo aggregation
Conclusion
Questions
9. Analyzing Data with Elasticsearch Aggregations
Introduction
Structure
Objectives
Introduction to Elasticsearch aggregations
Bucket aggregation
Range aggregation
Composite aggregation
Terms
Histogram
Date histogram
Terms aggregation
Filter aggregation
Filters aggregation
Geo distance aggregation
Metrics aggregation
Min aggregation
Max aggregation
Avg aggregation
Sum aggregation
Value count aggregation
Stats aggregation
Extended stats aggregation
Percentiles aggregation
Matrix aggregation
Matrix stats aggregation
Pipeline aggregation
Avg bucket aggregation
Max bucket aggregation
Sum bucket aggregation
Conclusion
Questions
10. Performance Tuning
Introduction
Structure
Objectives
Elasticsearch performance optimization strategies
Optimizing Elasticsearch for largescale data
Tuning Elasticsearch indexing speed
Bulk requests instead of a single request
Smart use of Elasticsearch cluster
Increasing the refresh interval
Disabling replicas
Using auto-generated IDs
Tweaking the indexing buffer size
Utilizing faster hardware
Allocating memory to the filesystem cache
Tuning Elasticsearch search speed
Document modeling
Searching fewer fields if possible
Pre-index data
Mapping of identifiers as keywords
Forcing merge on read-only indices
Using filter instead of query
Increasing the replica count
Fetching only the required fields
Using faster hardware
Allocating memory to the filesystem cache
Avoiding stop words in the search
Avoiding script query
Tuning Elasticsearch for disk usage
Shrink index
Force merge
Disabling unrequired features
Disabling indexing for fields
Disabling norms for text fields
Disabling positions for text fields
Avoiding dynamic string mappings
Disabling _source
Optimizing numeric field types
Elasticsearch best practices
Explicitly defining Elasticsearch index mapping
Optimizing Elasticsearch cluster capacity
Avoiding split-brain problem
Enabling slow query log
Conclusion
Questions
11. Administration: Managing Elasticsearch Clusters
Introduction
Structure
Objectives
Elasticsearch security
Configuring TLS
Elasticsearch cluster passwords
Configuring role-based access using Kibana
Creating users
Creating roles
Index aliases
Creating a repository and snapshot
Creating the repository
Taking the snapshot
Restoring a snapshot
Elastic Common Schema
Why do we need a common schema?
Introduction to elastic common schema
ECS general guidelines
ECS field name guidelines
Getting started with ECS
Scaling Elasticsearch cluster
Vertical scaling
Horizontal scaling
Monitoring Elasticsearch
Conclusion
Questions
Index
