Deploy OpenShift 3.11 Container Platform on Google Cloud Platform using Terraform

Over the past few days I have converted the OpenShift 3.11 infrastructure on Amazon AWS to run on Google Cloud Platform. I have kept the similar VPC network layout and instances to run OpenShift.

Before you start you need to create a project on Google Cloud Platform, then continue to create the service account and generate the private key and download the credential as JSON file.

Create the new project:

Create the service account:

Give the service account compute admin and storage object creator permissions:

Then create a storage bucket for the Terraform backend state and assign the correct bucket permission to the terraform service account:

Bucket permissions:

To start, clone my openshift-terraform github repository and checkout the google-dev branch:

git clone https://github.com/berndonline/openshift-terraform.git
cd ./openshift-terraform/ && git checkout google-dev

Add your previously downloaded credentials json file:

cat << EOF > ./credentials.json
{
  "type": "service_account",
  "project_id": "<--your-project-->",
  "private_key_id": "<--your-key-id-->",
  "private_key": "-----BEGIN PRIVATE KEY-----

...

}
EOF

There are a few things you need to modify in the main.tf and variables.tf before you can start:

...
terraform {
  backend "gcs" {
    bucket    = "<--your-bucket-name-->"
    prefix    = "openshift-311"
    credentials = "credentials.json"
  }
}
...
...
variable "gcp_region" {
  description = "Google Compute Platform region to launch servers."
  default     = "europe-west3"
}
variable "gcp_project" {
  description = "Google Compute Platform project name."
  default     = "<--your-project-name-->"
}
variable "gcp_zone" {
  type = "string"
  default = "europe-west3-a"
  description = "The zone to provision into"
}
...

Add the needed environment variables to apply changes to CloudFlare DNS:

export TF_VAR_email='<-YOUR-CLOUDFLARE-EMAIL-ADDRESS->'
export TF_VAR_token='<-YOUR-CLOUDFLARE-TOKEN->'
export TF_VAR_domain='<-YOUR-CLOUDFLARE-DOMAIN->'
export TF_VAR_htpasswd='<-YOUR-OPENSHIFT-DEMO-USER-HTPASSWD->'

Let’s start creating the infrastructure and verify afterwards the created resources on GCP.

terraform init && terraform apply -auto-approve

VPC and public and private subnets in region europe-west3:

Created instances:

Created load balancers for master and infra nodes:

Copy the ssh key and ansible-hosts file to the bastion host from where you need to run the Ansible OpenShift playbooks.

scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ./helper_scripts/id_rsa -r ./helper_scripts/id_rsa [email protected]$(terraform output bastion):/home/centos/.ssh/
scp -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ./helper_scripts/id_rsa -r ./inventory/ansible-hosts  [email protected]$(terraform output bastion):/home/centos/ansible-hosts

I recommend waiting a few minutes as the cloud-init script prepares the bastion host. Afterwards continue with the pre and install playbooks. You can connect to the bastion host and run the playbooks directly.

ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ./helper_scripts/id_rsa -l centos $(terraform output bastion) -A "cd /openshift-ansible/ && ansible-playbook ./playbooks/openshift-pre.yml -i ~/ansible-hosts"
ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -i ./helper_scripts/id_rsa -l centos $(terraform output bastion) -A "cd /openshift-ansible/ && ansible-playbook ./playbooks/openshift-install.yml -i ~/ansible-hosts"

After the installation is completed, continue to create your project and applications:

When you are finished with the testing, run terraform destroy.

terraform destroy -force 

Please share your feedback and leave a comment.

Please share!


Also published on Medium.

3 Replies to “Deploy OpenShift 3.11 Container Platform on Google Cloud Platform using Terraform”

  1. Getting errors related to GCP permission when run “terraform init && terraform apply -auto-approve”:

    Error: Error applying plan:

    5 errors occurred:
    * google_compute_address.master: 1 error occurred:
    * google_compute_address.master: Error creating Address: googleapi: Error 403: Required ‘compute.addresses.create’ permission for ‘projects/opneshift-paas/regions/us-east1/addresses/master-address’, forbidden

    * google_compute_http_health_check.infra: 1 error occurred:
    * google_compute_http_health_check.infra: Error creating HttpHealthCheck: googleapi: Error 403: Required ‘compute.httpHealthChecks.create’ permission for ‘projects/opneshift-paas/global/httpHealthChecks/infra-basic-check’, forbidden

    * google_compute_http_health_check.master: 1 error occurred:
    * google_compute_http_health_check.master: Error creating HttpHealthCheck: googleapi: Error 403: Required ‘compute.httpHealthChecks.create’ permission for ‘projects/opneshift-paas/global/httpHealthChecks/master-basic-check’, forbidden

    * google_compute_address.infra: 1 error occurred:
    * google_compute_address.infra: Error creating Address: googleapi: Error 403: Required ‘compute.addresses.create’ permission for ‘projects/opneshift-paas/regions/us-east1/addresses/infra-address’, forbidden

    * google_compute_network.vpc: 1 error occurred:
    * google_compute_network.vpc: Error creating Network: googleapi: Error 403: Required ‘compute.networks.create’ permission for ‘projects/opneshift-paas/global/networks/vpc’, forbidden

    1. Seems like your google account has not enough permission to create the infrastructure. Check if you have Compute Admin and Storage Object creator roles assigned.

  2. It appears you need to add the following to the ‘helper_scripts/ansible-hosts.template.txt’ for GCE:

    [OSEv3:vars]

    openshift_cloudprovider_kind=gce

    This repairs the bastion terraform output being empty and enables the storage provisioners as well as a number of other GCE specific components.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.