week_3_data_warehouse

Data Warehouse and BigQuery

Slides
Big Query basic SQL

Data Warehouse

Data Warehouse and BigQuery

Partitoning and clustering

Partioning and Clustering
Partioning vs Clustering

Best practices

BigQuery Best Practices

Internals of BigQuery

Internals of Big Query

Advance

ML

BigQuery Machine Learning
SQL for ML in BigQuery

Important links

• BigQuery ML Tutorials
• BigQuery ML Reference Parameter
• Hyper Parameter tuning
• Feature preprocessing

Deploying ML model

BigQuery Machine Learning Deployment
Steps to extract and deploy model with docker

Workshop

Note that this is required to complete prior to the homework as the external tables in BigQuery were not set up after running the Airflow DAGs in Week 2.

Alternatively, you can copy the gcs_to_bq.py script to the Airflow folder in Week 2 and run it to create the tables in BigQuery.

• Integrating Bigquery with Airflow (+ Week 2 Review) - Video

• Setup: Copy over the airflow directory (i.e. the Dockerized setup) from week_2_data_ingestion:

  cp ../week_2_data_ingestion/airflow airflow
  

  Also, empty the logs directory, if you find it necessary.

• DAG: gcs_to_bq_dag.py

Homework

Community notes

Did you take notes? You can share them here.

• Notes by Alvaro Navas
• Isaac Kargar's blog post
• Add your notes here (above this line)