Understand modern data fabric and data mesh concepts using AI-based self-service data discovery and delivery capabilities, a range of intelligent data integration styles, and automated unified data governance—all designed to deliver "data as a product" within hybrid cloud landscapes.
This book teaches you how to successfully deploy state-of-the-art data mesh solutions and gain a comprehensive overview on how a data fabric architecture uses artificial intelligence (AI) and machine learning (ML) for automated metadata management and self-service data discovery and consumption. You will learn how data fabric and data mesh relate to other concepts such as data DataOps, MLOps, AIDevOps, and more. Many examples are included to demonstrate how to modernize the consumption of data to enable a shopping-for-data (data as a product) experience.
By the end of this book, you will understand the data fabric concept and architecture as it relates to themes such as automated unified data governance and compliance, enterprise information architecture, AI and hybrid cloud landscapes, and intelligent cataloging and metadata management.
What You Will Learn
Discover best practices and methods to successfully implement a data fabric architecture and data mesh solution
Understand key data fabric capabilities, e.g., self-service data discovery, intelligent data integration techniques, intelligent cataloging and metadata management, and trustworthy AI
Recognize the importance of data fabric to accelerate digital transformation and democratize data access
Dive into important data fabric topics, addressing current data fabric challenges
Conceive data fabric and data mesh concepts holistically within an enterprise context
Become acquainted with the business benefits of data fabric and data mesh
Who This Book Is For
Anyone who is interested in deploying modern data fabric architectures and data mesh solutions within an enterprise, including IT and business leaders, data governance and data office professionals, data stewards and engineers, data scientists, and information and data architects. Readers should have a basic understanding of enterprise information architecture.
Author(s): Eberhard Hechler, Maryela Weihrauch , Yan (Catherine) Wu
Publisher: Apress
Year: 2023
Language: English
Pages: 413
Table of Contents
About the Authors
About the Technical Reviewer
Acknowledgments
Introduction
Foreword
Chapter 1: Evolution of Data Architecture
Introduction
Data Architectures: Values and Challenges
Enterprise Data Warehouse (EDW)
Big Data, Data Lake, and Data Lakehouse
Key Takeaways
References
Chapter 2: Terminology: Data Fabric and Data Mesh
Introduction
Data Fabric Concept
Data Fabric Framework
AI-Infused Data Fabric
Data Mesh Concept
Relationship: Data Fabric and Data Mesh
Data Product
Key Takeaways
References
Chapter 3: Data Fabric and Data Mesh Use Case Scenarios
Introduction
Automated and Consistent Governance
Include IBM zSystems Data in AI Governance
Unified View of Data Across a Hybrid Cloud
Provide a Comprehensive View of Customers, Vendors, and Other Parties
Unlock the Trustworthy AI Concept
Key Takeaways
References
Chapter 4: Data Fabric and Data Mesh Business Benefits
Introduction
Business Requirements and Pain Points for Data Management and Consumption
Benefits of a Data Fabric and Data Mesh for Technical Teams Managing Data
Benefits of a Data Fabric and Data Mesh for Business Teams Consuming Data
Key Takeaways
References
Chapter 5: Key Data Fabric and Data Mesh Capabilities
Introduction
Knowledge Catalog
Active Metadata
Data Curation
Semantic Knowledge Graphs
Self-Service Capabilities
Trustworthy AI
Introduction
Model Fairness
Drift Detection
Model Explainability
Model Quality Metrics
Intelligent Information Integration
Key Takeaways
References
Chapter 6: Relevant ML and DL Concepts
Introduction to AI, ML, and DL
ML and DL Industry Use Cases
Data Exploration and Preparation
Model Selection, Training, and Evaluation
Model Deployment
Natural Language Processing (NLP)
Key Takeaways
References
Chapter 7: AI and ML for a Data Fabric and Data Mesh
Introduction
General Overview
Cataloging
AI-Infused Understanding of Assets
Asset Discovery
Asset Profiling
Automatic Asset Quality Assessment
Asset Access
AI/ML for Entity Matching
AI/ML to Activate the Digital Exhaust
AI/ML for Semantic Enrichment
Key Takeaways
References
Chapter 8: AI for Entity Resolution
Introduction
Introducing Entity Matching
Traditional Entity Resolution Approaches
Use of AI to Resolve Entity Challenges
The Benefits and Cost of an AI-Based Solution
Considerations for MDM Solutions
Key Takeaways
References
Chapter 9: Data Fabric and Data Mesh for the AI Lifecycle
Introduction to the AI Lifecycle
Key Aspects: DataOps, ModelOps, MLOps
Case Study 1: Consolidating Fragmented Data in a Hybrid Cloud Environment
Case Study 2: Operationalizing AI
Accelerate MLOps with AutoAI
Deployment Patterns for AI Engineering
Key Takeaways
References
Chapter 10: Data Fabric Architecture Patterns
Introduction
Data Fabric and Data Mesh Evolution
Data Consumption Patterns
Data Fabric for a Data Mesh Solution
Data Mesh Self-Service Capabilities
Data Mesh Architecture Overview Diagram
Intelligent Information Integration Styles
Key Takeaways
References
Chapter 11: Data Fabric Within an Enterprise Architecture
Introduction
What Is Enterprise Architecture?
What Is Application Architecture?
Data Fabric as a Data Architecture
Sample of a Data Fabric Within an Enterprise Architecture
Key Takeaways
References
Chapter 12: Data Fabric and Data Mesh in a Hybrid Cloud Landscape
Introduction
What Is Hybrid Cloud?
Key Challenges for Data Architecture
Data Fabric and Data Mesh in Hybrid Cloud
Data Fabric Architecture in Hybrid Cloud
Data Mesh Solution in Hybrid Cloud
Benefits of Data Fabric and Data Mesh for Hybrid Cloud
Key Takeaways
References
Chapter 13: Intelligent Cataloging and Metadata Management
Introduction to Metadata Management
Key Aspects of Intelligent Cataloging
Build an Intelligent Catalog by Automating Data Discovery and Enrichment
Find Data Assets with Semantic Search and Recommendation
Provide Data Insight and Provenance as Data Flows Across the Enterprise
Key Takeaways
References
Chapter 14: Automated Data Fabric and Data Mesh Aspects
Introduction
Intelligent Automation of Metadata
Automated Analysis and Profiling of Data
Automated Tagging, Annotation, and Labeling
Automated Data Quality Assessment
Key Takeaways
References
Chapter 15: Data Governance in the Context of Data Fabric and Data Mesh
Introduction
Importance of Data and AI Governance
Key Aspects of Data and AI Governance
Establishing a Data Governance Foundation with a Data Fabric Architecture
Establishing Automated Regulation with a Data Fabric Architecture
Automatic Enforcement of Data Regulations in Data Fabric
Automate Quality Analysis with Data Fabric
Key Takeaways
References
Chapter 16: Sample Vendor Offerings
Introduction
IBM Cloud Pak for Data
Amazon Web Services
Microsoft Azure
Denodo
Informatica
Key Takeaways
References
Chapter 17: Data Fabric and Data Mesh Research Areas
Introduction
AI-Based Augmented Insight
AI-Infused Automated AI Governance
Hyper-automated Data and AI Fabric
Key Takeaways
References
Chapter 18: In Summary and Onward
Data Fabric and Data Mesh Summarized
Where to Go from Here
Key Takeaways
Part IV: Current Offerings and Future Aspects
Capture.PNG
