Welcome to Fast.BI! This guide will walk you through deploying a complete data development platform on Google Cloud Platform using our interactive CLI tool. The entire process takes about 30-45 minutes and creates a production-ready data platform with modern tools like Apache Airflow, dbt, Superset, and more.
Fast.BI creates a complete data development platform with:
- π§ Infrastructure Services: Kubernetes cluster, load balancer, DNS, certificates, monitoring
- π Data Services: Apache Airflow, dbt, Superset, DataHub, MinIO, PostgreSQL
- π Security: Keycloak SSO, encrypted secrets, secure networking
- π Monitoring: Prometheus, Grafana, centralized logging
- π CI/CD: GitLab/GitHub runners, automated deployments
- Python β₯3.9 - For running the Fast.BI CLI
- kubectl - Kubernetes command-line tool
- gcloud CLI - Google Cloud command-line tool
- Terraform - Infrastructure as Code tool
- Terragrunt - Terraform wrapper
- Helm - Kubernetes package manager
- GCP Account with billing enabled
- Custom Domain (e.g., yourcompany.com)
- GitHub/GitLab Account for repositories
- Keycloak Instance (or use our hosted version)
1
Create GCP Account
Visit cloud.google.com and sign up for a new account.
Free Credits Available! New GCP accounts receive $300 in free credits, which is enough to run Fast.BI for approximately 14 days for testing and evaluation.
2
Enable Billing
You must enable billing to use the Fast.BI CLI, even with free credits.
- Go to the GCP Billing Console
- Click "Link a billing account" or create a new one
- Add a payment method (required even for free tier)
- Note your Billing Account ID (format: XXXXXX-XXXXXX-XXXXXX)
3
Install gcloud CLI
# macOS (using Homebrew)
brew install google-cloud-sdk
# Linux
curl https://sdk.cloud.google.com | bash
exec -l $SHELL
# Windows
# Download from: https://cloud.google.com/sdk/docs/install
4
Authenticate with GCP
gcloud auth login
gcloud auth application-default login
Important: You need to create two Git repositories before starting the deployment. These are mandatory for the platform to function.
1
Create Data Models Repository
This repository will store your dbt models and data transformations.
- Create a new repository in GitHub/GitLab (e.g.,
dbt-data-models)
- Initialize with a basic dbt project structure
- Note the repository URL (e.g.,
https://github.com/mycompany/dbt-data-models.git)
2
Create Data Orchestration Repository
This repository will store your Airflow DAGs and workflow definitions.
- Create a new repository in GitHub/GitLab (e.g.,
data-orchestration-dags)
- Initialize with a basic Airflow project structure
- Note the repository URL (e.g.,
https://github.com/mycompany/data-orchestration-dags.git)
3
Configure Deploy Keys (Recommended)
Deploy keys provide secure, read-only access to your repositories.
- Go to your repository settings
- Navigate to "Deploy keys" or "SSH keys"
- Click "Add deploy key"
- Give it a title (e.g., "Fast.BI Platform Access")
- Copy the SSH public key that the CLI will generate
- Paste it into the deploy key field
- Enable "Write access" if needed for CI/CD
1
Access Keycloak Admin Console
After Phase 4 completes, you'll access Keycloak at: https://login.{customer}.{domain}
2
Import Realm Configuration
- Log in to Keycloak admin console
- Click "Import" in the realm dropdown
- Upload the generated realm file from the CLI
- Configure your organization's administrator user
1
Clone the Repository
git clone https://github.com/fast-bi/data-development-platform.git
cd data-development-platform
2
Start the CLI
python cli.py
π Fast.BI Platform Deployment CLI
π No existing deployment found
What would you like to do?
1. Start new deployment
2. Continue existing deployment
3. Exit
Enter choice (1-3): 1
π Basic Configuration (needed for all phases)
? Enter customer tenant name: mycompany
? Enter admin email: admin@mycompany.com
? Select cloud provider: gcp
? Enter project region: us-central1
? Enter domain name (e.g., fast.bi): mycompany.com
π GCP Infrastructure Configuration
Select deployment type:
1. basic
2. advanced
Enter choice (1-2): 1
Enter GCP billing account ID: 01AXXX-XXXX-XXXX
Enter GCP parent folder ID: [Leave empty if not using folders]
Enter whitelisted IP addresses: [Leave empty for 0.0.0.0/0]
ποΈ State Management Configuration
Choose the state backend type:
β’ LOCAL: Infrastructure files saved locally (no Fast.BI dependencies)
β’ REMOTE: Infrastructure files saved in GCS bucket (team collaboration)
Select Terraform state backend type:
1. local
2. remote
Enter choice (1-2): 1
π GKE Deployment Type Configuration
Choose the GKE deployment type:
β’ ZONAL: Single zone (cheaper, faster, good for demos/dev)
β’ MULTIZONE: Multi-zone (production-ready, better with spot instances)
Select GKE deployment type:
1. zonal
2. multizone
Enter choice (1-2): 1
What Gets Created:
- GCP Project:
fast-bi-mycompany
- VPC Network with subnets
- GKE Cluster with node pools
- Firewall rules and load balancer
- Service accounts for platform services
π PHASE 2: Generate Platform Secrets
π Secrets Generation Configuration
Select vault method:
1. local_vault
2. external_infisical
Enter choice (1-2): 1
Select data analysis platform:
1. superset
2. lightdash
3. metabase
4. looker
Enter choice (1-4): 1
Select data warehouse platform:
1. bigquery
2. snowflake
3. redshift
Enter choice (1-3): 1
Select orchestrator platform:
1. Airflow
2. Composer
Enter choice (1-2): 1
Select git provider:
1. github
2. gitlab
3. bitbucket
4. fastbi
Enter choice (1-4): 2
Secrets Generated:
- SSH keys for repository access
- Database passwords and credentials
- API keys and service account tokens
- Platform configuration secrets
π PHASE 3: Configure Data Platform Repositories
β ATTENTION: Deploy keys are mandatory for the Data Orchestrator repository, even if 'access_token' was selected. Please ensure they are added.
Enter data models repository URL: https://github.com/mycompany/dbt-data-models.git
Enter data orchestration repository URL: https://github.com/mycompany/data-orchestration-dags.git
π SSH Public Key for Deploy Keys:
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC... [Generated SSH Public Key]
β οΈ IMPORTANT: Add this SSH public key as a deploy key to BOTH repositories!
Repository Setup Required:
- Copy the SSH public key shown above
- Go to your GitHub/GitLab repository settings
- Add the key as a deploy key with read/write access
- Repeat for both repositories
π§ PHASE 4: Deploy Infrastructure Services
β
1.0_secret_operator - Deployed successfully
β
2.0_cert_manager - Deployed successfully
β
3.0_external_dns - Deployed successfully
β
4.0_traefik_lb - Deployed successfully
β
5.0_stackgres_postgresql - Deployed successfully
β
6.0_log_collector - Deployed successfully
β
7.0_services_monitoring - Deployed successfully
β
8.0_cluster_cleaner - Deployed successfully
β
9.0_idp_sso_manager - Deployed successfully
β
10.0_cluster_pvc_autoscaller - Deployed successfully
Keycloak Setup Required: After this phase completes, you'll need to import the realm configuration into Keycloak. The CLI will provide the realm file and access URL.
π PHASE 5: Deploy Data Services
β
1.0_cicd_workload_runner - Deployed successfully
β
2.0_object_storage_operator - Deployed successfully
β
3.0_data-cicd-workflows - Deployed successfully
β
4.0_data_replication - Deployed successfully
β
5.0_data_orchestration - Deployed successfully
β
6.0_data_modeling - Deployed successfully
β
7.0_data_dcdq_meta_collect - Deployed successfully
β
8.0_data_analysis - Deployed successfully
β
9.0_data_governance - Deployed successfully
β
10.0_user_console - Deployed successfully
Data Platform Components:
- Apache Airflow: Workflow orchestration and scheduling
- dbt: Data transformation and modeling
- Superset: Business intelligence and visualization
- DataHub: Data catalog and governance
- MinIO: S3-compatible object storage
- PostgreSQL: Metadata and configuration database
β¨ PHASE 6: Finalize Deployment
π Deployment Finalization Configuration
Current Git provider: github
Enter Git repository URL for deployment files: https://github.com/mycompany/fast-bi-deployment.git
Enter Git access token: [Your GitHub Personal Access Token]
Cleanup local temporary files after saving to repository?
1. Yes
2. No
Enter choice (1-2): 2
π DEPLOYMENT COMPLETED SUCCESSFULLY!
============================================================
Customer: mycompany
Domain: mycompany.com
Cloud Provider: gcp
Terraform State: local
Repository: https://github.com/mycompany/fast-bi-deployment.git
π Deployed Services:
π§ Infrastructure Services:
β
1.0_secret_operator
β
2.0_cert_manager
β
3.0_external_dns
β
4.0_traefik_lb
β
5.0_stackgres_postgresql
β
6.0_log_collector
β
7.0_services_monitoring
β
8.0_cluster_cleaner
β
9.0_idp_sso_manager
β
10.0_cluster_pvc_autoscaller
π Data Services:
β
1.0_cicd_workload_runner
β
2.0_object_storage_operator
β
3.0_data-cicd-workflows
β
4.0_data_replication
β
5.0_data_orchestration
β
6.0_data_modeling
β
7.0_data_dcdq_meta_collect
β
8.0_data_analysis
β
9.0_data_governance
β
10.0_user_console
π Access Information:
- IDP Console: https://login.mycompany.mycompany.com
- Fast.BI Platform: https://mycompany.mycompany.com
π ENCRYPTION KEY
========================================
β οΈ IMPORTANT: Save this encryption key securely!
Deployment files in the repository are encrypted for security.
You will need this key to decrypt the files later.
Generated encryption key: Ycs_XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX=
π‘ Tip: Store this key in a secure password manager or vault.
- Keycloak Admin: Generated during deployment (check CLI output)
- Database: Auto-configured with secure passwords
- Service Accounts: Automatically managed
Free Tier Cost Management:
- $300 Free Credits: Typically lasts 14 days for testing
- Monitor Usage: Check GCP Console billing dashboard
- Destroy Promptly: Remove environment when done testing to avoid charges
- Set Budget Alerts: Configure billing alerts in GCP Console
1
Destroy Environment
python cli.py --destroy --destroy-confirm
This will completely remove all resources and stop billing.
- Verify billing account is enabled
- Check GCP quotas and limits
- Ensure proper IAM permissions
- Check kubeconfig is valid
- Verify cluster has sufficient resources
- Check service account permissions
- Verify domain nameserver configuration
- Check External DNS service status
- Ensure domain is properly configured
1
Check Deployment Status
python cli.py --show-config
2
View Service Status
kubectl get pods --all-namespaces
3
Check Logs
kubectl logs -n {namespace} {pod-name}
4
Clean Up Local Files
If you need to clean up local deployment files (Terraform state, temporary files, etc.):
python cli.py --cleanup
This will remove local deployment files but won't affect your deployed infrastructure. Use this to free up disk space or clean up after a failed deployment.
After successful deployment:
- Configure Users: Set up SSO authentication in Keycloak
- Connect Data Sources: Configure data replication in Airbyte
- Create Data Models: Build dbt models in your data repository
- Set Up Dashboards: Create visualizations in your BI platform
- Monitor Performance: Use built-in monitoring and alerting
Need Help? Check the troubleshooting section above or open an issue on GitHub.