Managing Artifacts For Multiple ForgeOps Deployment Environments
You may be familiar with the ForgeOps repository which provides artifacts for quickly deploying a single evaluation environment of the entire ForgeRock Identity Platform. However, a single environment is almost never enough. Most organizations need to deploy three or more environments.
To support a DevOps deployment of the ForgeRock Platform, three kinds of artifacts must be organized separately in version control for each environment: product configuration, docker images, and deployment manifests. ForgeOps release/7.2 and earlier support config-profile directories by product to let you keep exported configuration separate. After building custom docker images, the forgeops tool adds the image tags to an image-defaulter component making it easier for the deployment scripts to identify which image tag to deploy. However, there is only one image-defaulter component; because of this, it is not clear how to build separate docker images for multiple deployment environments. Deployment manifests are generated by combining base manifests and overlays which enables different configuration per environment, but they only generate into into a single kustomize/deploy directory.
ForgeOps --deploy-env parameter
To make it easier to keep these artifacts organized separately by deployment environment, a new forgeops tool parameter, --deploy-env, is being introduced. This parameter enables the forgeops tool to organize artifacts in separate deploy-[environment] directories under kustomize to make it easier to understand which manifests are used to deploy multiple environments.
Suppose you wanted to set up multiple environments with names like dev, test, staging, and production. The following directories would contain the source and generated deployment artifacts.
-
Base manifests: kustomize/base/[product]
-
Overlays: kustomize/overlay/[environment]
-
Deployment Manifests: kustomize/deploy-[environment]
The following examples illustrate some common use cases for the --deploy-env parameter with an example production environment.
Initial Setup
Before using the --deploy-env parameter, we need to copy the kustomize/deploy directory as kustomize/deploy-production. By default, the image-defaulter and generated deployment manifests are located in the kustomize/deploy directory. When multiple deployment environments are defined, each environment will have its own image-defaulter located in kustomize/deploy-[environment]/image-defaulter. The kustomize manifests generated by forgeops install or forgeops generate with an environment specified are also generated in the kustomize/deploy-[environment] directory.
Build Custom Docker Images
Execute the following to build docker images using configuration from docker/[product]/config-profiles/production directories and write the image tags to the kustomize/deploy-production image-defaulter:
forgeops build --config-profile production --deploy-env production --push-to none
Specify --push-to none with the forgeops build command to push the Docker image to the Docker registry embedded in the Minikube cluster.
Before image build
The newName and newTag entries refer to public docker images.
kind: Component
apiVersion: kustomize.config.k8s.io/v1alpha1
commonLabels:
app.kubernetes.io/part-of: forgerock
images:
- name: am
newName: us-docker.pkg.dev/forgeops-public/images/am
newTag: 7.2.0
- name: amster
newName: us-docker.pkg.dev/forgeops-public/images/amster
newTag: 7.2.0
- name: ds
newName: us-docker.pkg.dev/forgeops-public/images/ds
newTag: 7.2.0
- name: ds-cts
newName: us-docker.pkg.dev/forgeops-public/images/ds-cts
newTag: 7.2.0
- name: ds-idrepo
newName: us-docker.pkg.dev/forgeops-public/images/ds-idrepo
newTag: 7.2.0
- name: git-server
newName: us-docker.pkg.dev/forgeops-public/images/git-server
- name: idm
newName: us-docker.pkg.dev/forgeops-public/images/idm
newTag: 7.2.0
- name: ig
newName: us-docker.pkg.dev/forgeops-public/images/ig
newTag: 7.2.0
After image build
The newName entries now refer to images in the local docker registry
kind: Component
apiVersion: kustomize.config.k8s.io/v1alpha1
commonLabels:
app.kubernetes.io/part-of: forgerock
images:
- name: am
newName: am
- name: amster
newName: us-docker.pkg.dev/forgeops-public/images/amster
newTag: 7.2.0
- name: ds
newName: ds
- name: ds-cts
newName: ds-cts
- name: ds-idrepo
newName: ds-idrepo
- name: git-server
newName: us-docker.pkg.dev/forgeops-public/images/git-server
- name: idm
newName: idm
- name: ig
newName: ig
Generate Deployment Manifests
Generated kustomize manifests are a combination of kustomize/base directories and kustomize/overlay directories. Custom deployment overlays can be created per environment in kustomize/overlay/[environment] to store patches to the base overlays. These overlays can be specified when running forgeops with install and generate parameters using the --custom [path to overlay directory] parameter.
Execute the following to generate deployment manifests for the production environment using the production config-profiles and overlays:
forgeops generate --deploy-env production --config-profile production --custom /path/to/forgeops/kustomize/overlay/production
Before generate
kustomize/deploy-production
├── .gitignore
└── image-defaulter
└── kustomization.yaml
After generate
kustomize/deploy-production
├── .gitignore
├── apps
│ ├── amster-files.yaml
│ └── kustomization.yaml
├── base
│ └── kustomization.yaml
├── ds
│ └── kustomization.yaml
├── image-defaulter
│ └── kustomization.yaml
├── secrets
│ └── kustomization.yaml
└── ui
└── kustomization.yaml
Install An Environment
Deploying multiple environments requires additional Kubernetes considerations. Multiple CDM deployments typically require multiple Kubernetes clusters each with their own namespaces. You must remember to set the Kubernetes context and namespace before executing forgeops install to deploy. The namespace can also be set using the forgeops install parameter -n or --namespace, but the context must always be set separately.
The forgeops install option first generates deployment manifests by combining base and overlay manifests then the generated manifests are applied to the current Kubernetes context and namespace.
Execute the following to install the production using production config-profiles and overlays:
forgeops install --deploy-env production --config-profile production --custom /path/to/forgeops/kustomize/overlay/production --namespace production
The resulting manifest directory structure will look the same as after the generate option.
Clean Manifests
When you have made overlay changes and wish to delete old manifests before generating new ones, you can use the forgeops clean parameter. By default, this parameter deletes manifests in kustomize/deploy while leaving the image-defaulter directory and .gitignore file.
Execute the following to delete old manifests for a deployment environment:
forgeops clean --deploy-env production
This will delete old manifests in the kustomize/deploy-production directory while leaving the image-defaulter sub-directory and .gitignore file.
Before clean
kustomize/deploy-production
├── .gitignore
├── apps
│ ├── amster-files.yaml
│ └── kustomization.yaml
├── base
│ └── kustomization.yaml
├── ds
│ └── kustomization.yaml
├── image-defaulter
│ └── kustomization.yaml
├── secrets
│ └── kustomization.yaml
└── ui
└── kustomization.yaml
After clean
kustomize/deploy-production
├── .gitignore
└── image-defaulter
└── kustomization.yaml