Outline detalhado do curso
Module 1 - Dataform Core Components
Topics:
- SQL workflow
- Repositories and workspaces
- Default files and folders
- Compiled graphs
Objectives:
- Understand the components of Dataflow core.
Module 2 - Table Definitions and Dependencies
Topics:
- Declare a data source.
- Create a table.
- Create an incremental table.
- Set partitioning and clustering options.
- Create an empty table.
- Create an external BigLake table.
- Create views and materialized views.
- Define dependencies.
Objectives:
- Create tables and views in BigQuery using Dataform
Module 3 - Document BigQuery Tables and Views
Topics:
- Use column descriptions.
- Use globally defined JavaScript constants.
- Add labels.
Objectives:
- Document BigQuery tables and views.
Activities:
- Lab: Build SQL Workflows with Dependencies in Dataform
Module 4 - BigQuery Security Settings
Topics:
- IAM dataset and table/view access
- Column-level security
- Row-level security
Objectives:
- Understand BigQuery security settings using Dataform
Module 5 - Assertions
Topics:
- Use built-in assertions.
- Create manual assertions.
Objectives:
- Use assertions to validate data in Dataform workflows.
Activities:
- Lab: Work with Assertions and BigQuery Security Settings in Dataform.
Module 6 - SQL Workflow Executions
Topics:
- Dataform code lifecycle.
- What happens during compilation.
- Customize and schedule compilation results.
- Execute workflows (UI, Cloud Scheduler, Cloud Composer).
- Logging and monitoring.
Objectives:
- Execute Dataform SQL workflows in an automated fashion.
Activities:
- Lab: Automate and Monitor SQL Workflow Executions in Dataform
Module 7 - Advanced Use Cases
Topics:
- Create a BigLake table after file upload using Cloud Run functions.
- Build a Machine Learning pipeline with BigQuery ML.
- Work with Slowly Changing Dimensions Type 2.
Objectives:
- Explore additional use cases for Dataform.
Activities:
- Lab: Create a BigLake Table with Dataform Using Cloud Run Functions.