How To Set up a 7.2 ForgeOps Cluster using cluster-up.sh on GCP

Hi all, I would like to start by saying this article is as much for me as it is for you. It contains notes on how to do some things.

Our ForgeOps team has been worked quite a lot on providing different deployment strategies for ForgeOps. I would like to particularly like to thank Paul Schroeder and Guillaume Andru for their work on the subject.

Please note the forgeops repository is in continuous development and might have changed since this post was written.

So there are a few things that we need before you can use the cluster-up.sh scripts to create a cluster that can host ForgeOps.

  1. We need a GCP account
  2. We need the repository itself
  3. We need the prerequisites installed

Setup

Prerequisite Software

  • Python | 3.9.16
  • Kubernetes Client | 1.26
  • Kubernetes Context Switcher | 0.9.4
  • Kustomize | 4.5.5
  • Skaffold | 1.39.1
  • Helm | 3.10.2
  • Google Cloud SDK | 392.0.0
  • Docker Desktop | 4.9.1

You won’t need all of this just for the creation of a cluster, but it will be useful for the rest of your work with ForgeOps. Please bear in mind these are MacOS requirements. On Linux you need to find your equivelent package names. These are all available via homebrew for Linux as well, and are popular enough to be available via apt,pacman and dnf. In addition to multi-distro package managers such as snap and flatpack.

Clone the Repository

$ git clone https://github.com/ForgeRock/forgeops.git

And we are going to fetch the 7.2.0 release using:

$ cd /path/to/forgeops

$ git checkout release/7.2.0

Set up Account

Start by logging in to GCP.

Agree to the terms and conditions

Once that is done we need to get registered in our gcloud tool.

$ gcloud init


Welcome! This command will take you through the configuration of gcloud.

Your current configuration has been set to: [default]

You can skip diagnostics next time by using the following flag:
  gcloud init --skip-diagnostics

Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.                                            
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).

You must log in to continue. Would you like to log in (Y/n)? Y


Your browser has been opened to visit:

    https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=32555940559.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8085%f2F&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+htwefdstpds%3A%2F%2Fwwerww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2aasFwww.googleapis.com%2Fauth%2Fargfappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcomherpute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=n3ucW1LXKF1tglhCnWMKBptFfilgfdsI58&access_type=offline&code_challenge=eGlSdVvaYZiKUJYYKvUT2Q156r_bOxJAPa-FVkO-0a4&code_challenge_method=S256

Visit the browser page and authenticate.

To take a quick anonymous survey, run:
  $ gcloud survey

You are logged in as: [us-xlv22lusy5ssa@cloudsharelabs.com].

Pick cloud project to use: 
 [1] pr-xlv22lusy5ssa
 [2] Enter a project ID
 [3] Create a new project

Please enter numeric choice or text value (must exactly match list item): 1

Your current project has been set to: [pr-xlv22lusy5ssa].

Do you want to configure a default Compute Region and Zone? (Y/n)? Y

Which Google Compute Engine zone would you like to use as project default?
If you do not specify a zone via a command line flag while working with Compute 
Engine resources, the default is assumed.
 [1] us-east1-b
 [2] us-east1-c
 [3] us-east1-d
 [4] us-east4-c
 [5] us-east4-b
 [6] us-east4-a
 [7] us-central1-c
 [8] us-central1-a
 [9] us-central1-f
 [10] us-central1-b
 [11] us-west1-b
 [12] us-west1-c
 [13] us-west1-a
 [14] europe-west4-a
 [15] europe-west4-b
 [16] europe-west4-c
 [17] europe-west1-b
 [18] europe-west1-d
 [19] europe-west1-c
 [20] europe-west3-c
 [21] europe-west3-a
 [22] europe-west3-b
 [23] europe-west2-c
 [24] europe-west2-b
 [25] europe-west2-a
 [26] asia-east1-b
 [27] asia-east1-a
 [28] asia-east1-c
 [29] asia-southeast1-b
 [30] asia-southeast1-a
 [31] asia-southeast1-c
 [32] asia-northeast1-b
 [33] asia-northeast1-c
 [34] asia-northeast1-a
 [35] asia-south1-c
 [36] asia-south1-b
 [37] asia-south1-a
 [38] australia-southeast1-b
 [39] australia-southeast1-c
 [40] australia-southeast1-a
 [41] southamerica-east1-b
 [42] southamerica-east1-c
 [43] southamerica-east1-a
 [44] asia-east2-a
 [45] asia-east2-b
 [46] asia-east2-c
 [47] asia-northeast2-a
 [48] asia-northeast2-b
 [49] asia-northeast2-c
 [50] asia-northeast3-a
Did not print [57] options.
Too many options [107]. Enter "list" at prompt to print choices fully.

Please enter numeric choice or text value (must exactly match list item): 1



Your project default Compute Engine zone has been set to [us-east1-b].
You can change it by running [gcloud config set compute/zone NAME].

Your project default Compute Engine region has been set to [us-east1].
You can change it by running [gcloud config set compute/region NAME].

Created a default .boto configuration file at [/home/forgerock/.boto]. See this file and
[https://cloud.google.com/storage/docs/gsutil/commands/config] for more
information about configuring Google Cloud Storage.
Your Google Cloud SDK is configured and ready to use!

* Commands that require authentication will use us-xlv22lusy5ssa@cloudsharelabs.com by default
* Commands will reference project `pr-xlv22lusy5ssa` by default
* Compute Engine commands will use region `us-east1` by default
* Compute Engine commands will use zone `us-east1-b` by default

Run `gcloud help config` to learn how to change individual settings

This gcloud configuration is called [default]. You can create additional configurations if you work with multiple accounts and/or projects.
Run `gcloud topic configurations` to learn more.

Some things to try next:

* Run `gcloud --help` to see the Cloud Platform services you can interact with. And run `gcloud help COMMAND` to get help on any gcloud command.
* Run `gcloud topic --help` to learn about advanced features of the SDK like arg files and output formatting
* Run `gcloud cheat-sheet` to see a roster of go-to `gcloud` commands.

Now we should be able to use the gcloud, however we also need to register our applications to allow them access.

We do this by running

$ gcloud auth application-default login

Configuration

We will start by exploring the .../forgeops/cluster/gke directory.


auto-up.sh       config-connector.sh  medium.sh         scale-gke.sh
cluster-down.sh  gcp-bucket.yaml      mini.sh           small.sh
cluster-up.sh    large.sh             multi-cluster.sh

Inspect ./small.sh. It contains a set of environmental variables:

# Source these values for a small cluster

# Change cluster name to a unique name that can include alphanumeric characters and hyphens only.
export NAME="small"

# cluster-up.sh retrieves the region from the user's gcloud config.
# NODE_LOCATIONS refers to the zones to be used by CDM in the region. If your region doesn't include zones a,b or c then uncomment and set the REGION, ZONE and NODE_LOCATIONS appropriately to override:
# export REGION=us-east1
# export NODE_LOCATIONS="$REGION-b,$REGION-c,$REGION-d"
# export ZONE="$REGION-b" # required for cluster master

# PRIMARY NODE POOL VALUES
export MACHINE=e2-standard-8
export PREEMPTIBLE_NODE=false

# DS NODE POOL VALUES
export CREATE_DS_POOL=false
export DS_MACHINE=n2-standard-8

# Values for creating a static IP
export CREATE_STATIC_IP=false # set to true to create a static IP.
# export STATIC_IP_NAME="" # uncomment to provide a unique name(defaults to cluster name).  Lowercase letters, numbers, hyphens allowed.
export DELETE_STATIC_IP=false # set to true to delete static IP, named above, when running cluster-down.sh

Please note this is where the node and region for deployment are selected. We can make changes, such as setting the cluster name to community-cluster and enabling the zone we deploy this to, ideally the same one as the one we logged in to.

The updated cluster-up.sh script should look like this:

# Source these values for a small cluster

# Change cluster name to a unique name that can include alphanumeric characters and hyphens only.
export NAME="community-cluster"

# cluster-up.sh retrieves the region from the user's gcloud config.
# NODE_LOCATIONS refers to the zones to be used by CDM in the region. If your region doesn't include zones a,b or c then uncomment and set the REGION, ZONE and NODE_LOCATIONS appropriately to override:
export REGION=us-east1
export NODE_LOCATIONS="$REGION-b,$REGION-c,$REGION-d"
export ZONE="$REGION-b" # required for cluster master

# PRIMARY NODE POOL VALUES
export MACHINE=e2-standard-8
export PREEMPTIBLE_NODE=false

# DS NODE POOL VALUES
export CREATE_DS_POOL=false
export DS_MACHINE=n2-standard-8

# Values for creating a static IP
export CREATE_STATIC_IP=false # set to true to create a static IP.
# export STATIC_IP_NAME="" # uncomment to provide a unique name(defaults to cluster name).  Lowercase letters, numbers, hyphens allowed.
export DELETE_STATIC_IP=false # set to true to delete static IP, named above, when running cluster-down.sh

Let’s then set these variables to be active using:

$ source ./small.sh

Execution

To bring up the cluster we can run:

$ ./cluster-up.sh

ForgeRock staff are required to provide metadata.
Are you a ForgeRock employee?[y/n]

This prompt exists for people using the devlopment cluster. We will simply select no.

n

The cluster should then be deployed.

Deploying to region: us-east1
WARNING: The `--enable-stackdriver-kubernetes` flag is deprecated and will be removed in an upcoming release. Please use `--logging` and `--monitoring` instead. For more information, please read: https://cloud.google.com/stackdriver/docs/solutions/gke/installing.
Default change: During creation of nodepools or autoscaling configuration changes for cluster versions greater than 1.24.1-gke.800 a default location policy is applied. For Spot and PVM it defaults to ANY, and for all other VM kinds a BALANCED policy is used. To change the default values use the `--location-policy` flag.
Note: The Pod address range limits the maximum size of the cluster. Please refer to https://cloud.google.com/kubernetes-engine/docs/how-to/flexible-pod-cidr to learn how to optimize IP address allocation.
Creating cluster community-cluster in us-east1... Cluster is being configured..
.⠏                                                                             
Creating cluster community-cluster in us-east1... Cluster is being health-check
ed (master is healthy)...done.                                                 
Created [https://container.googleapis.com/v1beta1/projects/pr-qtr40355wroqe/zones/us-east1/clusters/community-cluster].
To inspect the contents of your cluster, go to: https://console.cloud.google.com/kubernetes/workload_/gcloud/us-east1/community-cluster?project=pr-qtr40355wroqe
CRITICAL: ACTION REQUIRED: gke-gcloud-auth-plugin, which is needed for continued use of kubectl, was not found or is not executable. Install gke-gcloud-auth-plugin for use with kubectl by following https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
kubeconfig entry generated for community-cluster.
NAME               LOCATION  MASTER_VERSION   MASTER_IP      MACHINE_TYPE   NODE_VERSION     NUM_NODES  STATUS
community-cluster  us-east1  1.24.9-gke.3200  35.185.55.102  e2-standard-8  1.24.9-gke.3200  3          RUNNING
W0308 07:44:08.219872   58670 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
storageclass.storage.k8s.io/fast created
W0308 07:44:11.769650   58697 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
Warning: snapshot.storage.k8s.io/v1beta1 VolumeSnapshotClass is deprecated; use snapshot.storage.k8s.io/v1 VolumeSnapshotClass
volumesnapshotclass.snapshot.storage.k8s.io/ds-snapshot-class created
W0308 07:44:12.261865   58703 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
namespace/prod created
W0308 07:44:13.216670   58723 gcp.go:119] WARNING: the gcp auth plugin is deprecated in v1.22+, unavailable in v1.26+; use gcloud instead.
To learn more, consult https://cloud.google.com/blog/products/containers-kubernetes/kubectl-auth-changes-in-gke
clusterrolebinding.rbac.authorization.k8s.io/cluster-admin-binding created

We can log into the cluster using the gcloud commands:

$ gcloud container clusters get-credentials community-cluster --region us-east1 --project pr-qtr40355wroqe

$ kubectl get nodes

NAME                                               STATUS   ROLES    AGE     VERSION
gke-community-cluster-default-pool-7121bd5f-vgqg   Ready    <none>   4m3s    v1.24.9-gke.3200
gke-community-cluster-default-pool-91354983-6pk0   Ready    <none>   4m2s    v1.24.9-gke.3200
gke-community-cluster-default-pool-d8826c8f-2znj   Ready    <none>   4m26s   v1.24.9-gke.3200

Our ForgeOps compatible cluster is ready to use!

Next Steps:

  • Installing a CDK environment in order to create suitable config for your staging environments
  • Deploying a CDM environment for testing and validation
1 Like