Hi all, I would like to start by saying this article is as much for me as it is for you. It contains notes on how to do some things.
Our ForgeOps team has been worked quite a lot on providing different deployment strategies for ForgeOps. I would like to particularly like to thank Paul Schroeder and Guillaume Andru for their work on the subject.
Please note the forgeops repository is in continuous development and might have changed since this post was written.
So there are a few things that we need before you can use the cluster-up.sh scripts to create a cluster that can host ForgeOps.
- We need a GCP account
- We need the repository itself
- We need the prerequisites installed
Setup
Prerequisite Software
- Python | 3.9.16
- Kubernetes Client | 1.26
- Kubernetes Context Switcher | 0.9.4
- Kustomize | 4.5.5
- Skaffold | 1.39.1
- Helm | 3.10.2
- Google Cloud SDK | 392.0.0
- Docker Desktop | 4.9.1
You won’t need all of this just for the creation of a cluster, but it will be useful for the rest of your work with ForgeOps. Please bear in mind these are MacOS requirements. On Linux you need to find your equivelent package names. These are all available via homebrew for Linux as well, and are popular enough to be available via apt,pacman and dnf. In addition to multi-distro package managers such as snap and flatpack.
Clone the Repository
$ git clone https://github.com/ForgeRock/forgeops.git
And we are going to fetch the 7.2.0 release using:
$ cd /path/to/forgeops
$ git checkout release/7.2.0
Set up Account
Start by logging in to GCP.
Agree to the terms and conditions
Once that is done we need to get registered in our gcloud tool.
$ gcloud init
Welcome! This command will take you through the configuration of gcloud.
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).
You must log in to continue. Would you like to log in (Y/n)? Y
Your browser has been opened to visit:
https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8085%f2F&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+htwefdstpds%3A%2F%2Fwwerww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2aasFwww.googleapis.com%2Fauth%2Fargfappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcomherpute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=n3ucW1LXKF1tglhCnWMKBptFfilgfdsI58&access_type=offline&code_challenge=eGlSdVvaYZiKUJYYKvUT2Q156r_bOxJAPa-FVkO-0a4&code_challenge_method=S256
Visit the browser page and authenticate.
To take a quick anonymous survey, run:
$ gcloud survey
You are logged in as: [us-xlv22lusy5ssa@cloudsharelabs.com].
Pick cloud project to use:
[1] pr-xlv22lusy5ssa
[2] Enter a project ID
[3] Create a new project
Please enter numeric choice or text value (must exactly match list item): 1
Your current project has been set to: [pr-xlv22lusy5ssa].
Do you want to configure a default Compute Region and Zone? (Y/n)? Y
Which Google Compute Engine zone would you like to use as project default?
If you do not specify a zone via a command line flag while working with Compute
Engine resources, the default is assumed.
[1] us-east1-b
[2] us-east1-c
[3] us-east1-d
[4] us-east4-c
[5] us-east4-b
[6] us-east4-a
[7] us-central1-c
[8] us-central1-a
[9] us-central1-f
[10] us-central1-b
[11] us-west1-b
[12] us-west1-c
[13] us-west1-a
[14] europe-west4-a
[15] europe-west4-b
[16] europe-west4-c
[17] europe-west1-b
[18] europe-west1-d
[19] europe-west1-c
[20] europe-west3-c
[21] europe-west3-a
[22] europe-west3-b
[23] europe-west2-c
[24] europe-west2-b
[25] europe-west2-a
[26] asia-east1-b
[27] asia-east1-a
[28] asia-east1-c
[29] asia-southeast1-b
[30] asia-southeast1-a
[31] asia-southeast1-c
[32] asia-northeast1-b
[33] asia-northeast1-c
[34] asia-northeast1-a
[35] asia-south1-c
[36] asia-south1-b
[37] asia-south1-a
[38] australia-southeast1-b
[39] australia-southeast1-c
[40] australia-southeast1-a
[41] southamerica-east1-b
[42] southamerica-east1-c
[43] southamerica-east1-a
[44] asia-east2-a
[45] asia-east2-b
[46] asia-east2-c
[47] asia-northeast2-a
[48] asia-northeast2-b
[49] asia-northeast2-c
[50] asia-northeast3-a
Did not print [57] options.
Too many options [107]. Enter "list" at prompt to print choices fully.
Please enter numeric choice or text value (must exactly match list item): 1
Your project default Compute Engine zone has been set to [us-east1-b].
You can change it by running [gcloud config set compute/zone NAME].
Your project default Compute Engine region has been set to [us-east1].
You can change it by running [gcloud config set compute/region NAME].
Created a default .boto configuration file at [/home/forgerock/.boto]. See this file and
[https://cloud.google.com/storage/docs/gsutil/commands/config] for more
information about configuring Google Cloud Storage.
Your Google Cloud SDK is configured and ready to use!
* Commands that require authentication will use us-xlv22lusy5ssa@cloudsharelabs.com by default
* Commands will reference project `pr-xlv22lusy5ssa` by default
* Compute Engine commands will use region `us-east1` by default
* Compute Engine commands will use zone `us-east1-b` by default
Run `gcloud help config` to learn how to change individual settings
This gcloud configuration is called [default]. You can create additional configurations if you work with multiple accounts and/or projects.
Run `gcloud topic configurations` to learn more.
Some things to try next:
* Run `gcloud --help` to see the Cloud Platform services you can interact with. And run `gcloud help COMMAND` to get help on any gcloud command.
* Run `gcloud topic --help` to learn about advanced features of the SDK like arg files and output formatting
* Run `gcloud cheat-sheet` to see a roster of go-to `gcloud` commands.
Now we should be able to use the gcloud, however we also need to register our applications to allow them access.
We do this by running
$ gcloud auth application-default login
Configuration
We will start by exploring the .../forgeops/cluster/gke directory.
auto-up.sh config-connector.sh medium.sh scale-gke.sh
cluster-down.sh gcp-bucket.yaml mini.sh small.sh
cluster-up.sh large.sh multi-cluster.sh
Inspect ./small.sh. It contains a set of environmental variables:
# Source these values for a small cluster
# Change cluster name to a unique name that can include alphanumeric characters and hyphens only.
export NAME="small"
# cluster-up.sh retrieves the region from the user's gcloud config.
# NODE_LOCATIONS refers to the zones to be used by CDM in the region. If your region doesn't include zones a,b or c then uncomment and set the REGION, ZONE and NODE_LOCATIONS appropriately to override:
# export REGION=us-east1
# export NODE_LOCATIONS="$REGION-b,$REGION-c,$REGION-d"
# export ZONE="$REGION-b" # required for cluster master
# PRIMARY NODE POOL VALUES
export MACHINE=e2-standard-8
export PREEMPTIBLE_NODE=false
# DS NODE POOL VALUES
export CREATE_DS_POOL=false
export DS_MACHINE=n2-standard-8
# Values for creating a static IP
export CREATE_STATIC_IP=false # set to true to create a static IP.
# export STATIC_IP_NAME="" # uncomment to provide a unique name(defaults to cluster name). Lowercase letters, numbers, hyphens allowed.
export DELETE_STATIC_IP=false # set to true to delete static IP, named above, when running cluster-down.sh
Please note this is where the node and region for deployment are selected. We can make changes, such as setting the cluster name to community-cluster and enabling the zone we deploy this to, ideally the same one as the one we logged in to.
The updated cluster-up.sh script should look like this:
# Source these values for a small cluster
# Change cluster name to a unique name that can include alphanumeric characters and hyphens only.
export NAME="community-cluster"
# cluster-up.sh retrieves the region from the user's gcloud config.
# NODE_LOCATIONS refers to the zones to be used by CDM in the region. If your region doesn't include zones a,b or c then uncomment and set the REGION, ZONE and NODE_LOCATIONS appropriately to override:
export REGION=us-east1
export NODE_LOCATIONS="$REGION-b,$REGION-c,$REGION-d"
export ZONE="$REGION-b" # required for cluster master
# PRIMARY NODE POOL VALUES
export MACHINE=e2-standard-8
export PREEMPTIBLE_NODE=false
# DS NODE POOL VALUES
export CREATE_DS_POOL=false
export DS_MACHINE=n2-standard-8
# Values for creating a static IP
export CREATE_STATIC_IP=false # set to true to create a static IP.
# export STATIC_IP_NAME="" # uncomment to provide a unique name(defaults to cluster name). Lowercase letters, numbers, hyphens allowed.
export DELETE_STATIC_IP=false # set to true to delete static IP, named above, when running cluster-down.sh
Let’s then set these variables to be active using:
$ source ./small.sh
Execution
To bring up the cluster we can run:
$ ./cluster-up.sh
ForgeRock staff are required to provide metadata.
Are you a ForgeRock employee?[y/n]
This prompt exists for people using the devlopment cluster. We will simply select no.
n
The cluster should then be deployed.
Deploying to region: us-east1
WARNING: The `--enable-stackdriver-kubernetes` flag is deprecated and will be removed in an upcoming release. Please use `--logging` and `--monitoring` instead. For more information, please read: https://cloud.google.com/stackdriver/docs/solutions/gke/installing.
Default change: During creation of nodepools or autoscaling configuration changes for cluster versions greater than 1.24.1-gke.800 a default location policy is applied. For Spot and PVM it defaults to ANY, and for all other VM kinds a BALANCED policy is used. To change the default values use the `--location-policy` flag.
Note: The Pod address range limits the maximum size of the cluster. Please refer to https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr to learn how to optimize IP address allocation.
Creating cluster community-cluster in us-east1... Cluster is being configured..
.â Ź
Creating cluster community-cluster in us-east1... Cluster is being health-check
ed (master is healthy)...done.
Created [https://container.googleapis.com/v1beta1/projects/pr-qtr40355wroqe/zones/us-east1/clusters/community-cluster].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-east1/community-cluster?project=pr-qtr40355wroqe
CRITICAL: ACTION REQUIRED: gke-gcloud-auth-plugin, which is needed for continued use of kubectl, was not found or is not executable. Install gke-gcloud-auth-plugin for use with kubectl by following https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
kubeconfig entry generated for community-cluster.
NAME LOCATION MASTER_VERSION MASTER_IP MACHINE_TYPE NODE_VERSION NUM_NODES STATUS
community-cluster us-east1 1.24.9-gke.3200 35.185.55.102 e2-standard-8 1.24.9-gke.3200 3 RUNNING
W0308 07:44:08.219872 58670 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
storageclass.storage.k8s.io/fast created
W0308 07:44:11.769650 58697 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
Warning: snapshot.storage.k8s.io/v1beta1 VolumeSnapshotClass is deprecated; use snapshot.storage.k8s.io/v1 VolumeSnapshotClass
volumesnapshotclass.snapshot.storage.k8s.io/ds-snapshot-class created
W0308 07:44:12.261865 58703 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
namespace/prod created
W0308 07:44:13.216670 58723 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
clusterrolebinding.rbac.authorization.k8s.io/cluster-admin-binding created
We can log into the cluster using the gcloud commands:
$ gcloud container clusters get-credentials community-cluster --region us-east1 --project pr-qtr40355wroqe
$ kubectl get nodes
NAME STATUS ROLES AGE VERSION
gke-community-cluster-default-pool-7121bd5f-vgqg Ready <none> 4m3s v1.24.9-gke.3200
gke-community-cluster-default-pool-91354983-6pk0 Ready <none> 4m2s v1.24.9-gke.3200
gke-community-cluster-default-pool-d8826c8f-2znj Ready <none> 4m26s v1.24.9-gke.3200
Our ForgeOps compatible cluster is ready to use!
Next Steps:
- Installing a CDK environment in order to create suitable config for your staging environments
- Deploying a CDM environment for testing and validation