you cant mention data warehouse on GCP without placing big query in that sentence. big query by google cloud is a fully managed, enterprise-grade data warehouse that analyzes data through built-in features like ML & BI.
in this exercise, I deployed an example data warehouse with looker dashboards making virtualization easier.
deployment options
this exercise can either be deployed on the console or through Terraform. For deployment using the console cloud build clones the GitHub repository and everything is abstracted to you all you do I just wait and boom deployed.
with terraform on the other case where you will get your hands dirty and also understand what actually is happening in the background while the build is going on. if you understand terraform you can change a few variables in the terraform.tfvars file to our liking.
with terraform I also customized the table and used my own data instead of the data given to me and it worked like a charm. I just used google provided table schema so as to avoid building what is already built from Scratch.
changes however should be made with cost implications in mind because the default costs only $0.60 a month.
Required IAM roles and permissions need to be enabled for this to work.
services used in this build
- cloud function-this build uses a 2nd gen Python function that is triggered by a change in a storage bucket.
- big query-data warehouse
- cloud storage data is sent to cloud storage after a cloud function trigger.
- looker studio-prepares beautiful dashboards and visualizations and is compatible with most Google products like Sheets, big query etc…
after finishing the project make sure you delete the deployment either from the console or through Terraform to avoid future costs.
also, check out my previous building of a dynamic web application on google cloud
