فهرست مطالب

• Foreword
• Preface
• Acknowledgments
• About this book
• About the author
• About the cover illustration
• Part 1: Introducing data engineering with Snowflake
• 1. Data engineering with Snowflake
• 1.1 Snowflake for data engineering
• 1.2 Responsibilities of a Snowflake data engineer
• 1.3 Building data pipelines
• 1.4 Data engineering with Snowflake applications
• 1.5 Summary
• 2. Creating your first data pipeline
• 2.1 Setting up your Snowflake account
• 2.2 Staging a CSV file
• 2.3 Loading data from a staged file into a target table
• 2.4 Transforming data with SQL commands
• 2.5 Automating the process with tasks
• 2.6 Summary
• Part 2: Ingesting, transforming, and storing data
• 3. Best practices for data staging
• 3.1 Creating external stages
• 3.2 Viewing stage metadata with directory tables
• 3.3 Preparing data files for efficient ingestion
• 3.4 Building pipelines with external tables
• 3.5 Summary
• 4. Transforming data
• 4.1 Ingesting semistructured data from cloud storage
• 4.2 Flattening semistructured data into relational tables
• 4.3 Encapsulating transformations with stored procedures
• 4.4 Adding logging to stored procedures
• 4.5 Building robust data pipelines
• 4.6 Summary
• 5. Continuous data ingestion
• 5.1 Comparing bulk and continuous data ingestion
• 5.2 Preparing files in cloud storage
• 5.3 Configuring Snowpipe with cloud messaging
• 5.4 Transforming data with dynamic tables
• 5.5 Summary
• 6. Executing code natively with Snowpark
• 6.1 Introducing Snowpark
• 6.2 Creating a Snowpark procedure in a worksheet
• 6.3 Using the SQL API from a local development environment
• 6.4 Generating a date dimension in Snowpark Python
• 6.5 Working with data frames
• 6.6 Ingesting data from a CSV file into a Snowflake table
• 6.7 Transforming data with data frames
• 6.8 Summary
• 7. Augmenting data with outputs from large language models
• 7.1 Configuring external network access
• 7.2 Calling an API endpoint from a Snowpark function
• 7.3 Deriving customer review sentiments
• 7.4 Interpreting order emails using LLMs to save time
• 7.5 Summary
• 8. Optimizing query performance
• 8.1 Getting data from the Snowflake Marketplace
• 8.2 Performing analysis of geographical data
• 8.3 Understanding Snowflake micro-partitions
• 8.4 Optimizing storage with clustering
• 8.5 Improving query performance with search optimization
• 8.6 General tips for improving query performance
• 8.7 Summary
• 9. Controlling costs
• 9.1 Understanding Snowflake costs
• 9.2 Sizing virtual warehouses
• 9.3 Optimizing performance with data caching
• 9.4 Reducing query queuing
• 9.5 Monitoring compute consumption
• 9.6 Summary
• 10. Data governance and access control
• 10.1 Role-based access control
• 10.2 Securing data with row access policies
• 10.3 Protecting sensitive data with masking policies
• 10.4 Summary
• Part 3: Building data pipelines
• 11. Designing data pipelines
• 11.1 Designing data pipelines
• 11.2 Building a sample data pipeline
• 11.3 Summary
• 12. Ingesting data incrementally
• 12.1 Comparing data ingestion approaches
• 12.2 Preserving history with slowly changing dimensions
• 12.3 Detecting changes with Snowflake streams
• 12.4 Maintaining data with dynamic tables
• 12.5 Summary
• 13. Orchestrating data pipelines
• 13.1 Orchestrating with Snowflake tasks
• 13.2 Sending email notifications
• 13.3 Orchestrating with task graphs
• 13.4 Monitoring data pipeline execution
• 13.5 Troubleshooting data pipeline failures
• 13.6 Summary
• 14. Testing for data integrity and completeness
• 14.1 Data testing methods
• 14.2 Incorporating data testing steps in the pipeline
• 14.3 Applying the Snowflake data metric functions
• 14.4 Alerting users when data metrics exceed thresholds
• 14.5 Detecting data volume anomalies
• 14.6 Summary
• 15. Data pipeline continuous integration
• 15.1 Separating the data engineering environments
• 15.2 Database change management
• 15.3 Configuring Snowflake to use Git
• 15.4 Using the Snowflake CLI command line interface
• 15.5 Connecting to Snowflake securely
• 15.6 Applying what we learned in real-world scenarios
• 15.7 Summary
• Appendix A—Configuring your Snowflake environment
• Appendix B—Snowflake objects used in the examples
• Index