Introduction
This blog post is the continuation of the previous article on bulk updates.
Batch operations are similar to bulk operations, the goal being to effect multiple object operations through a single request. The difference is that, while bulk operations apply the same single operation to multiple objects, batch operations are a collection of multiple different operations on a single object each.
As such, the batch operation feature discussed here corresponds to the concept of Bulk Operations as defined in the SCIM 2.0 standard. A potential future evolution of the bulk/batch custom endpoints could be to combine them into one and add support for a SCIM-compatible schema and syntax. Feel free to reply to or comment on this post to let us know if that would be a useful extension.
Overview
This endpoint accepts a list of objects describing resource requests, each specifying the CPUD operation to be performed, and the resource to be applied to. The request parameters correspond to those documented in IDM’s Scripting Function Reference:
| Operation | Required | Optional |
|---|---|---|
| CREATE |
"resourcePath" , {object data}
|
"resourceName" , {params} , "rev"
|
| PATCH |
"resourcePath" , "resourceName" , [{operations }]
|
{params} , "rev"
|
| UPDATE |
"resourcePath" , "resourceName" , {object data} |
{params} , "rev"
|
| DELETE |
"resourcePath" , "resourceName"
|
"rev" |
The list of requests is POSTed to the batchOperation endpoint in the requests fields of the JSON body:
POST https://<tenant fqdn>/openidm/batchOperation?_action=batch
{
"requests": [
{
operation: < PATCH | CREATE | DELETE | UPDATE >,
resourcePath: <resourcePath>,
params: { params },
resourceName: <resource name>,
rev: <revision>,
data: <request data> // object data or array of patch operations
},
{ ... }
]
}
The inner workings are similar to asynchronous bulk operations: The batch of requests is stored in a managed/bulkOperation object, then picked up by the taskscanner for background processing.
Note: The bulkOperation object is being used to persist both deferred bulk as well as batch operations. Eventually, both features might get combined into one endpoint in a future version of this example.
Installation
To install the custom endpoint, you can either copy the script code in the following section into the custom endpoint script editor in the ID Cloud Platform Admin UI (Option 1), or you prepare a configuration file containing the script source, or a file reference, and place it in IDM’s conf directory, or submit it via a REST PUT (Option 2). If you need more details on either option, please take a look at the article on bulk operations.
Option 1: Platform Admin UI
Paste this code into the editor for custom endpoint scripts:
/*
* Copyright 2012-2023 ForgeRock AS. All Rights Reserved
*
* DISCLAIMER: This code is provided to you expressly as an example (“Sample Code”).
* It is the responsibility of the individual recipient user, in his/her sole discretion,
* to diligence such Sample Code for accuracy, completeness, security, and final determination for appropriateness of use.
*
* ANY SAMPLE CODE IS PROVIDED ON AN “AS IS” IS BASIS, WITHOUT WARRANTY OF ANY KIND.
* FORGEROCK AND ITS LICENSORS EXPRESSLY DISCLAIM ALL WARRANTIES, WHETHER EXPRESS, IMPLIED, OR STATUTORY,
* INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
*
* FORGEROCK SHALL NOT HAVE ANY LIABILITY ARISING OUT OF OR RELATING TO ANY USE, IMPLEMENTATION, INTEGRATION,
* OR CONFIGURATION OF ANY SAMPLE CODE IN ANY PRODUCTION ENVIRONMENT OR FOR ANY COMMERCIAL DEPLOYMENT(S).
*
*/
/*
* This endpoint schedules a list of object operations,
* specified as a list of requests.
*
* Dependencies:
* Requires managed/bulkOperation object to be defined in the IDM schema.
* If not defined, this routine defines the object and exits without scheduling any operations.
*
*
* Syntax:
* POST <path to endpoint>
*
* Parameters:
* _action = batch // (*) required
* name = <custom display name for the operation> //
*
* Body:
* {
* requests : [
* {
* operation: < PATCH | CREATE | DELETE | UPDATE >, // (*) required
* resourcePath: <resourcePath>, // (*) required
* params: { params },
* resourceName: <resource name>,
* data: { <request content> },
rev: <resource version>
* },
* ...
* ]
* }
*
* Returns: bulkOperation object
*/
( function(){
const _dbg = "FRDEBUG: deferredBatchOperation ENDPOINT: ";
var info = function( text, args ) { logger.info( "{}{}: {}", _dbg, text, args ? args : "" ) }, isNonEmptyArray = function (field) { return Array.isArray(field) && field.length};
logger.info("{}Received request:{} for resource path {}.",_dbg,String(request.method).toUpperCase(),request.resourcePath);
logger.info("{}Parameters: {}.",_dbg,request.additionalParameters);
const scheduledTaskName = "deferredBatchOperationScannerTask";
const schedulerEndpoint = "scheduler/job";
const scheduledTask = [ schedulerEndpoint, scheduledTaskName].join('/');
const taskscannerEndpoint = "taskscanner";
const objectPath = "managed/bulkOperation";
const bulkOperationType = "BATCH";
const createBulkOperationObject = function( request ) {
// Validate parameters
if ( !isNonEmptyArray(request.content.requests) ) {
info("ERROR: Bad request, request list not specified.");
return { code : 400, message : "Bad request - request list not specified." };
}
if ( !request.action || !(["batch"].includes(request.action.toLowerCase() ) )) {
info("ERROR: Unsupported action",request.action || "<none>" );
return { code : 400, message : "Bad request - unsupported action: " + request.action };
}
logger.info("{}ACTION:{} on resource path:{} name:{}.",_dbg,request.action,request.resourcePath,request.resourceName);
// Prepare bulkOperation object to be picked up by the task scanner
const bulkOperationObject = {
name: request.additionalParameters.name,
request: request,
timeSubmitted: new Date().toISOString(),
isExpired: false,
type: bulkOperationType
};
return openidm.create(
objectPath,
null,
bulkOperationObject
);
}
const createOrTriggerTask = function( taskName ) {
const qf = "_id eq \"" + scheduledTaskName + "\"";
const existingTask = openidm.query( schedulerEndpoint, { _queryFilter: qf } , ["_id"] );
info("Existing task:", JSON.stringify(existingTask));
// Create task scanner job unless already defined (e.g. via persisted schedule)
if ( !existingTask.resultCount) {
// this is the actual worker script performing the bulk operation
const execScript = "\/*\r\n * Copyright 2012-2023 ForgeRock AS. All Rights Reserved\r\n *\r\n * DISCLAIMER: This code is provided to you expressly as an example (\u201CSample Code\u201D).\r\n * It is the responsibility of the individual recipient user, in his\/her sole discretion,\r\n * to diligence such Sample Code for accuracy, completeness, security, and final determination for appropriateness of use.\r\n *\r\n * ANY SAMPLE CODE IS PROVIDED ON AN \u201CAS IS\u201D IS BASIS, WITHOUT WARRANTY OF ANY KIND.\r\n * FORGEROCK AND ITS LICENSORS EXPRESSLY DISCLAIM ALL WARRANTIES, WHETHER EXPRESS, IMPLIED, OR STATUTORY,\r\n * INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.\r\n *\r\n * FORGEROCK SHALL NOT HAVE ANY LIABILITY ARISING OUT OF OR RELATING TO ANY USE, IMPLEMENTATION, INTEGRATION,\r\n * OR CONFIGURATION OF ANY SAMPLE CODE IN ANY PRODUCTION ENVIRONMENT OR FOR ANY COMMERCIAL DEPLOYMENT(S).\r\n *\r\n *\/\r\n\r\n \/*\r\n * This script is executed by the task scanner on each qualifying managed\/bulkOperation object, which contains\r\n * the original request.\r\n *\r\n * Note that the task scanner populates the objectID and input variables in the invocation context.\r\n *\r\n *\/\r\n\r\n( function(){\r\n\r\n const _dbg = \"FRDEBUG: deferredBatchOperation EXEC: \";\r\n const info = function( text, args ) { logger.info( \"{}{}: {}\", _dbg, text, args ? args : \"\" ) }, isNonEmptyArray = function (field) { return Array.isArray(field) && field.length};\r\n\r\n info(\"Executing task on deferred bulk operation\", objectID);\r\n\r\n const executeAction = function( request ) {\r\n \/\/ the original request.content ends up in request.content.object when saving it in the managed object !\r\n if ( !( isNonEmptyArray(request.content.object.requests) ) ) {\r\n info(\"ERROR: Bad request, requests not specified.\");\r\n return { code : 400, message : \"Bad request - no valid request content.\" };\r\n }\r\n if ( !request.action || !([\"batch\"].includes(request.action.toLowerCase() ) )) {\r\n info(\"ERROR: Unsupported action\",request.action || \"<none>\" );\r\n return { code : 400, message : \"Bad request - unsupported action: \" + request.action };\r\n }\r\n\r\n\r\n logger.info(\"{}ACTION:{} on resource path:{} name:{}.\",_dbg,request.action,request.resourcePath,request.resourceName);\r\n\r\n info(\"Processing requestList with numEntries\", request.content.object.requests.length);\r\n\r\n let itemsProcessed = 0;\r\n let itemsFailed = 0;\r\n\r\n request.content.object.requests.forEach( function(item) {\r\n\r\n let objectId = item.resourcePath + ( (!!item.resourceName) ? (\"\/\"+item.resourceName) : \"\");\r\n info(\"objectId is\", objectId);\r\n\r\n switch ( item.operation.toUpperCase() )\r\n {\r\n case \"PATCH\":\r\n try {\r\n openidm.patch(\r\n objectId,\r\n item.rev || null,\r\n item.data,\r\n item.params || null,\r\n [\"_id\"]\r\n )._id ? itemsProcessed++ : itemsFailed++;\r\n } catch (e) {\r\n info(\"PATCH operation failed with error\", e);\r\n itemsFailed++;\r\n }\r\n break;\r\n\r\n case \"DELETE\":\r\n try {\r\n openidm.delete( objectId, item.rev || null, item.params || null, [\"_id\"] )._id ? itemsProcessed++ : itemsFailed++ ;\r\n } catch (e) {\r\n info(\"DELETE operation failed with error\", e);\r\n itemsFailed++;\r\n }\r\n break;\r\n\r\n case \"CREATE\":\r\n try {\r\n openidm.create(\r\n item.resourcePath,\r\n item.resourceName || null,\r\n item.data,\r\n item.params || null,\r\n [\"_id\"]\r\n )._id ? itemsProcessed++ : itemsFailed++;\r\n\r\n } catch (e) {\r\n info(\"CREATE operation failed with error\", e);\r\n itemsFailed++;\r\n }\r\n break;\r\n\r\n case \"UPDATE\":\r\n try {\r\n openidm.update(\r\n objectId,\r\n item.rev || null,\r\n item.data,\r\n item.params || null,\r\n [\"_id\"]\r\n )._id ? itemsProcessed++ : itemsFailed++;\r\n\r\n } catch (e) {\r\n info(\"UPDATE operation failed with error\", e);\r\n itemsFailed++;\r\n }\r\n break;\r\n\r\n default:\r\n info(\"ERROR: Unknown operation\", item.operation);\r\n break;\r\n }\r\n } );\r\n \/\/ Update bulkOperation object with completion details\r\n input.timeCompleted = new Date().toISOString();\r\n input.status = \"done\";\r\n input.isExpired = true;\r\n input.itemsProcessed = itemsProcessed;\r\n input.itemsFailed = itemsFailed;\r\n\r\n openidm.update( objectID, null, input);\r\n\r\n return {\r\n method: request.method,\r\n action: request.action,\r\n itemsProcessed: itemsProcessed,\r\n itemsFailed: itemsFailed\r\n };\r\n }\r\n\r\nif (input.type === \"BATCH\") {\r\n return executeAction( input.request );\r\n} else {\r\n throw { code : 400, message : \"BATCH script launched with wrong operation type: \" + input.type };\r\n}\r\n\r\n\r\n} )();\r\n"
const task = {
"enabled" : true,
"type" : "simple",
"persisted": false,
"concurrentExecution" : true,
"invokeService" : "taskscanner",
"invokeContext" : {
"waitForCompletion" : false,
"numberOfThreads" : 1,
"scan" : {
"_queryFilter" : "/type eq \"BATCH\" AND !(timeStarted pr) AND !(timeCompleted pr)",
"object" : "managed/bulkOperation",
"taskState" : {
"started" : "/timeStarted",
"completed" : "/timeCompleted"
},
"recovery" : {
"timeout" : "10m"
}
},
"task" : {
"script" : {
"type" : "text/javascript",
"source" : execScript
}
}
}
}
info("Creating new task...");
return openidm.create( schedulerEndpoint, scheduledTaskName, task, null, ["_id"] )._id;
} else {
info("Triggering existing task", scheduledTaskName);
return openidm.action("taskscanner", "execute", null, { "name" : scheduledTaskName } );
}
}
const updateSchema = function() {
info("Schema for bulkOperation not found. Updating...");
const boSchema = {
name : "bulkOperation",
type: "object",
schema : {
"$schema": "http://forgerock.org/json-schema#",
"description": "Holds bulkOperations for background processing",
"icon": "fa-stack-overflow",
"order": [
"name",
"timeSubmitted",
"timeStarted",
"timeCompleted",
"status",
"itemsProcessed",
"itemsFailed",
"isExpired",
"request",
"type"
],
"properties": {
"isExpired": {
"searchable": false,
"title": "Expired",
"type": "boolean",
"userEditable": true,
"viewable": true
},
"name": {
"searchable": true,
"title": "Name",
"type": "string",
"userEditable": true,
"viewable": true
},
"request": {
"order": [],
"properties": {},
"searchable": false,
"title": "Request",
"type": "object",
"userEditable": true,
"viewable": true
},
"status": {
"searchable": true,
"title": "Status",
"type": "string",
"userEditable": true,
"viewable": true
},
"itemsProcessed": {
"searchable": true,
"title": "Items successfully processed",
"type": "number",
"userEditable": true,
"viewable": true
},
"itemsFailed": {
"searchable": true,
"title": "Items failed",
"type": "number",
"userEditable": true,
"viewable": true
},
"timeCompleted": {
"searchable": true,
"title": "Completion time",
"type": "string",
"userEditable": true,
"viewable": true
},
"timeStarted": {
"searchable": true,
"title": "Start Time",
"type": "string",
"userEditable": true,
"viewable": true
},
"timeSubmitted": {
"description": "Time when this bulk request was submitted",
"format": null,
"isVirtual": false,
"searchable": true,
"title": "Submission time",
"type": "string",
"userEditable": true,
"viewable": true
},
"type": {
"searchable": true,
"title": "Operation type",
"description": "Type of bulk request (BULK or BATCH)",
"type": "string",
"userEditable": true,
"viewable": true
},
},
"required": [],
"title": "Bulk Operations",
"type": "object"
}
};
var m = openidm.read("config/managed");
m.objects.push(boSchema);
try {
openidm.update("config/managed", null, m);
if (openidm.read("schema/managed/bulkOperation") ) {
info("Schema updated & verified successfully.");
} else {
info("Schema NOT yet verified");
}
}
catch (e) {
info("Failed to update schema");
throw { code : 500, message : "Updating schema with bulkOperation object failed: " + e }
}
return true;
}
// Main function
if (request.method === "action") {
if (!openidm.read("schema/managed/bulkOperation")) {
updateSchema();
return {
status: "Schema updated"
}
} else {
var bo = createBulkOperationObject( request );
info("Created BO object", JSON.stringify(bo));
info("Create Trigger returned", createOrTriggerTask(scheduledTaskName));
return bo;
}
} else if (request.action === "create") {
if (!openidm.read("schema/managed/bulkOperation")) {
updateSchema();
return {
status: "Schema updated"
}
} else {
return {
status: "Schema already updated"
}
}
} else {
throw { code : 400, message : "Unsupported request type: " + request.method };
}
}
)();
Option 2: Config file or REST
Create this endpoint config file, or send it via REST:
{
"context":"deferredBatchOperation/*",
"type" : "text/javascript",
"source" : "\/*\r\n * Copyright 2012-2023 ForgeRock AS. All Rights Reserved\r\n *\r\n * DISCLAIMER: This code is provided to you expressly as an example (\u201CSample Code\u201D).\r\n * It is the responsibility of the individual recipient user, in his\/her sole discretion,\r\n * to diligence such Sample Code for accuracy, completeness, security, and final determination for appropriateness of use.\r\n *\r\n * ANY SAMPLE CODE IS PROVIDED ON AN \u201CAS IS\u201D IS BASIS, WITHOUT WARRANTY OF ANY KIND.\r\n * FORGEROCK AND ITS LICENSORS EXPRESSLY DISCLAIM ALL WARRANTIES, WHETHER EXPRESS, IMPLIED, OR STATUTORY,\r\n * INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.\r\n *\r\n * FORGEROCK SHALL NOT HAVE ANY LIABILITY ARISING OUT OF OR RELATING TO ANY USE, IMPLEMENTATION, INTEGRATION,\r\n * OR CONFIGURATION OF ANY SAMPLE CODE IN ANY PRODUCTION ENVIRONMENT OR FOR ANY COMMERCIAL DEPLOYMENT(S).\r\n *\r\n *\/\r\n\r\n \/*\r\n * This endpoint schedules a list of object operations,\r\n * specified as a list of requests.\r\n *\r\n * Dependencies:\r\n * Requires managed\/bulkOperation object to be defined in the IDM schema.\r\n * If not defined, this routine defines the object and exits without scheduling any operations.\r\n *\r\n *\r\n * Syntax:\r\n * POST <path to endpoint>\r\n *\r\n * Parameters:\r\n * _action = batch \/\/ (*) required\r\n * name = <custom display name for the operation> \/\/\r\n *\r\n * Body:\r\n * {\r\n * requests : [\r\n * {\r\n * operation: < PATCH | CREATE | DELETE | UPDATE >, \/\/ (*) required\r\n * resourcePath: <resourcePath>, \/\/ (*) required\r\n * params: { params },\r\n * resourceName: <resource name>,\r\n * data: { <request content> },\r\n rev: <resource version>\r\n * },\r\n * ...\r\n * ]\r\n * }\r\n *\r\n * Returns: bulkOperation object\r\n *\/\r\n\r\n( function(){\r\n\r\n const _dbg = \"FRDEBUG: deferredBatchOperation ENDPOINT: \";\r\n var info = function( text, args ) { logger.info( \"{}{}: {}\", _dbg, text, args ? args : \"\" ) }, isNonEmptyArray = function (field) { return Array.isArray(field) && field.length};\r\n\r\n logger.info(\"{}Received request:{} for resource path {}.\",_dbg,String(request.method).toUpperCase(),request.resourcePath);\r\n logger.info(\"{}Parameters: {}.\",_dbg,request.additionalParameters);\r\n\r\n const scheduledTaskName = \"deferredBatchOperationScannerTask\";\r\n const schedulerEndpoint = \"scheduler\/job\";\r\n const scheduledTask = [ schedulerEndpoint, scheduledTaskName].join('\/');\r\n const taskscannerEndpoint = \"taskscanner\";\r\n const objectPath = \"managed\/bulkOperation\";\r\n const bulkOperationType = \"BATCH\";\r\n\r\n const createBulkOperationObject = function( request ) {\r\n\r\n \/\/ Validate parameters\r\n if ( !isNonEmptyArray(request.content.requests) ) {\r\n info(\"ERROR: Bad request, request list not specified.\");\r\n return { code : 400, message : \"Bad request - request list not specified.\" };\r\n }\r\n if ( !request.action || !([\"batch\"].includes(request.action.toLowerCase() ) )) {\r\n info(\"ERROR: Unsupported action\",request.action || \"<none>\" );\r\n return { code : 400, message : \"Bad request - unsupported action: \" + request.action };\r\n }\r\n\r\n logger.info(\"{}ACTION:{} on resource path:{} name:{}.\",_dbg,request.action,request.resourcePath,request.resourceName);\r\n\r\n \/\/ Prepare bulkOperation object to be picked up by the task scanner\r\n const bulkOperationObject = {\r\n name: request.additionalParameters.name,\r\n request: request,\r\n timeSubmitted: new Date().toISOString(),\r\n isExpired: false,\r\n type: bulkOperationType\r\n };\r\n\r\n return openidm.create(\r\n objectPath,\r\n null,\r\n bulkOperationObject\r\n );\r\n }\r\n\r\n const createOrTriggerTask = function( taskName ) {\r\n\r\n const qf = \"_id eq \\\"\" + scheduledTaskName + \"\\\"\";\r\n const existingTask = openidm.query( schedulerEndpoint, { _queryFilter: qf } , [\"_id\"] );\r\n info(\"Existing task:\", JSON.stringify(existingTask));\r\n\r\n \/\/ Create task scanner job unless already defined (e.g. via persisted schedule)\r\n if ( !existingTask.resultCount) {\r\n\r\n \/\/ this is the actual worker script performing the bulk operation\r\n const execScript = \"\\\/*\\r\\n * Copyright 2012-2023 ForgeRock AS. All Rights Reserved\\r\\n *\\r\\n * DISCLAIMER: This code is provided to you expressly as an example (\\u201CSample Code\\u201D).\\r\\n * It is the responsibility of the individual recipient user, in his\\\/her sole discretion,\\r\\n * to diligence such Sample Code for accuracy, completeness, security, and final determination for appropriateness of use.\\r\\n *\\r\\n * ANY SAMPLE CODE IS PROVIDED ON AN \\u201CAS IS\\u201D IS BASIS, WITHOUT WARRANTY OF ANY KIND.\\r\\n * FORGEROCK AND ITS LICENSORS EXPRESSLY DISCLAIM ALL WARRANTIES, WHETHER EXPRESS, IMPLIED, OR STATUTORY,\\r\\n * INCLUDING WITHOUT LIMITATION, THE IMPLIED WARRANTIES OF MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.\\r\\n *\\r\\n * FORGEROCK SHALL NOT HAVE ANY LIABILITY ARISING OUT OF OR RELATING TO ANY USE, IMPLEMENTATION, INTEGRATION,\\r\\n * OR CONFIGURATION OF ANY SAMPLE CODE IN ANY PRODUCTION ENVIRONMENT OR FOR ANY COMMERCIAL DEPLOYMENT(S).\\r\\n *\\r\\n *\\\/\\r\\n\\r\\n \\\/*\\r\\n * This script is executed by the task scanner on each qualifying managed\\\/bulkOperation object, which contains\\r\\n * the original request.\\r\\n *\\r\\n * Note that the task scanner populates the objectID and input variables in the invocation context.\\r\\n *\\r\\n *\\\/\\r\\n\\r\\n( function(){\\r\\n\\r\\n const _dbg = \\\"FRDEBUG: deferredBatchOperation EXEC: \\\";\\r\\n const info = function( text, args ) { logger.info( \\\"{}{}: {}\\\", _dbg, text, args ? args : \\\"\\\" ) }, isNonEmptyArray = function (field) { return Array.isArray(field) && field.length};\\r\\n\\r\\n info(\\\"Executing task on deferred bulk operation\\\", objectID);\\r\\n\\r\\n const executeAction = function( request ) {\\r\\n \\\/\\\/ the original request.content ends up in request.content.object when saving it in the managed object !\\r\\n if ( !( isNonEmptyArray(request.content.object.requests) ) ) {\\r\\n info(\\\"ERROR: Bad request, requests not specified.\\\");\\r\\n return { code : 400, message : \\\"Bad request - no valid request content.\\\" };\\r\\n }\\r\\n if ( !request.action || !([\\\"batch\\\"].includes(request.action.toLowerCase() ) )) {\\r\\n info(\\\"ERROR: Unsupported action\\\",request.action || \\\"<none>\\\" );\\r\\n return { code : 400, message : \\\"Bad request - unsupported action: \\\" + request.action };\\r\\n }\\r\\n\\r\\n\\r\\n logger.info(\\\"{}ACTION:{} on resource path:{} name:{}.\\\",_dbg,request.action,request.resourcePath,request.resourceName);\\r\\n\\r\\n info(\\\"Processing requestList with numEntries\\\", request.content.object.requests.length);\\r\\n\\r\\n let itemsProcessed = 0;\\r\\n let itemsFailed = 0;\\r\\n\\r\\n request.content.object.requests.forEach( function(item) {\\r\\n\\r\\n let objectId = item.resourcePath + ( (!!item.resourceName) ? (\\\"\\\/\\\"+item.resourceName) : \\\"\\\");\\r\\n info(\\\"objectId is\\\", objectId);\\r\\n\\r\\n switch ( item.operation.toUpperCase() )\\r\\n {\\r\\n case \\\"PATCH\\\":\\r\\n try {\\r\\n openidm.patch(\\r\\n objectId,\\r\\n item.rev || null,\\r\\n item.data,\\r\\n item.params || null,\\r\\n [\\\"_id\\\"]\\r\\n )._id ? itemsProcessed++ : itemsFailed++;\\r\\n } catch (e) {\\r\\n info(\\\"PATCH operation failed with error\\\", e);\\r\\n itemsFailed++;\\r\\n }\\r\\n break;\\r\\n\\r\\n case \\\"DELETE\\\":\\r\\n try {\\r\\n openidm.delete( objectId, item.rev || null, item.params || null, [\\\"_id\\\"] )._id ? itemsProcessed++ : itemsFailed++ ;\\r\\n } catch (e) {\\r\\n info(\\\"DELETE operation failed with error\\\", e);\\r\\n itemsFailed++;\\r\\n }\\r\\n break;\\r\\n\\r\\n case \\\"CREATE\\\":\\r\\n try {\\r\\n openidm.create(\\r\\n item.resourcePath,\\r\\n item.resourceName || null,\\r\\n item.data,\\r\\n item.params || null,\\r\\n [\\\"_id\\\"]\\r\\n )._id ? itemsProcessed++ : itemsFailed++;\\r\\n\\r\\n } catch (e) {\\r\\n info(\\\"CREATE operation failed with error\\\", e);\\r\\n itemsFailed++;\\r\\n }\\r\\n break;\\r\\n\\r\\n case \\\"UPDATE\\\":\\r\\n try {\\r\\n openidm.update(\\r\\n objectId,\\r\\n item.rev || null,\\r\\n item.data,\\r\\n item.params || null,\\r\\n [\\\"_id\\\"]\\r\\n )._id ? itemsProcessed++ : itemsFailed++;\\r\\n\\r\\n } catch (e) {\\r\\n info(\\\"UPDATE operation failed with error\\\", e);\\r\\n itemsFailed++;\\r\\n }\\r\\n break;\\r\\n\\r\\n default:\\r\\n info(\\\"ERROR: Unknown operation\\\", item.operation);\\r\\n break;\\r\\n }\\r\\n } );\\r\\n \\\/\\\/ Update bulkOperation object with completion details\\r\\n input.timeCompleted = new Date().toISOString();\\r\\n input.status = \\\"done\\\";\\r\\n input.isExpired = true;\\r\\n input.itemsProcessed = itemsProcessed;\\r\\n input.itemsFailed = itemsFailed;\\r\\n\\r\\n openidm.update( objectID, null, input);\\r\\n\\r\\n return {\\r\\n method: request.method,\\r\\n action: request.action,\\r\\n itemsProcessed: itemsProcessed,\\r\\n itemsFailed: itemsFailed\\r\\n };\\r\\n }\\r\\n\\r\\nif (input.type === \\\"BATCH\\\") {\\r\\n return executeAction( input.request );\\r\\n} else {\\r\\n throw { code : 400, message : \\\"BATCH script launched with wrong operation type: \\\" + input.type };\\r\\n}\\r\\n\\r\\n\\r\\n} )();\\r\\n\"\r\n\r\n const task = {\r\n \"enabled\" : true,\r\n \"type\" : \"simple\",\r\n \"persisted\": false,\r\n \"concurrentExecution\" : true,\r\n \"invokeService\" : \"taskscanner\",\r\n \"invokeContext\" : {\r\n \"waitForCompletion\" : false,\r\n \"numberOfThreads\" : 1,\r\n \"scan\" : {\r\n \"_queryFilter\" : \"\/type eq \\\"BATCH\\\" AND !(timeStarted pr) AND !(timeCompleted pr)\",\r\n \"object\" : \"managed\/bulkOperation\",\r\n \"taskState\" : {\r\n \"started\" : \"\/timeStarted\",\r\n \"completed\" : \"\/timeCompleted\"\r\n },\r\n \"recovery\" : {\r\n \"timeout\" : \"10m\"\r\n }\r\n },\r\n \"task\" : {\r\n \"script\" : {\r\n \"type\" : \"text\/javascript\",\r\n \"source\" : execScript\r\n }\r\n }\r\n }\r\n }\r\n info(\"Creating new task...\");\r\n return openidm.create( schedulerEndpoint, scheduledTaskName, task, null, [\"_id\"] )._id;\r\n\r\n } else {\r\n\r\n info(\"Triggering existing task\", scheduledTaskName);\r\n return openidm.action(\"taskscanner\", \"execute\", null, { \"name\" : scheduledTaskName } );\r\n\r\n }\r\n }\r\n\r\n const updateSchema = function() {\r\n\r\n info(\"Schema for bulkOperation not found. Updating...\");\r\n const boSchema = {\r\n name : \"bulkOperation\",\r\n type: \"object\",\r\n schema : {\r\n \"$schema\": \"http:\/\/forgerock.org\/json-schema#\",\r\n \"description\": \"Holds bulkOperations for background processing\",\r\n \"icon\": \"fa-stack-overflow\",\r\n \"order\": [\r\n \"name\",\r\n \"timeSubmitted\",\r\n \"timeStarted\",\r\n \"timeCompleted\",\r\n \"status\",\r\n \"itemsProcessed\",\r\n \"itemsFailed\",\r\n \"isExpired\",\r\n \"request\",\r\n \"type\"\r\n ],\r\n \"properties\": {\r\n \"isExpired\": {\r\n \"searchable\": false,\r\n \"title\": \"Expired\",\r\n \"type\": \"boolean\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"name\": {\r\n \"searchable\": true,\r\n \"title\": \"Name\",\r\n \"type\": \"string\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"request\": {\r\n \"order\": [],\r\n \"properties\": {},\r\n \"searchable\": false,\r\n \"title\": \"Request\",\r\n \"type\": \"object\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"status\": {\r\n \"searchable\": true,\r\n \"title\": \"Status\",\r\n \"type\": \"string\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"itemsProcessed\": {\r\n \"searchable\": true,\r\n \"title\": \"Items successfully processed\",\r\n \"type\": \"number\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"itemsFailed\": {\r\n \"searchable\": true,\r\n \"title\": \"Items failed\",\r\n \"type\": \"number\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"timeCompleted\": {\r\n \"searchable\": true,\r\n \"title\": \"Completion time\",\r\n \"type\": \"string\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"timeStarted\": {\r\n \"searchable\": true,\r\n \"title\": \"Start Time\",\r\n \"type\": \"string\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"timeSubmitted\": {\r\n \"description\": \"Time when this bulk request was submitted\",\r\n \"format\": null,\r\n \"isVirtual\": false,\r\n \"searchable\": true,\r\n \"title\": \"Submission time\",\r\n \"type\": \"string\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n \"type\": {\r\n \"searchable\": true,\r\n \"title\": \"Operation type\",\r\n \"description\": \"Type of bulk request (BULK or BATCH)\",\r\n \"type\": \"string\",\r\n \"userEditable\": true,\r\n \"viewable\": true\r\n },\r\n },\r\n \"required\": [],\r\n \"title\": \"Bulk Operations\",\r\n \"type\": \"object\"\r\n }\r\n };\r\n var m = openidm.read(\"config\/managed\");\r\n m.objects.push(boSchema);\r\n\r\n try {\r\n openidm.update(\"config\/managed\", null, m);\r\n \/\/ wait a bit\r\n \/\/await new Promise( (r1) => { setTimeout( () => { r1(\"DING!!\"); }, 3000); } );\r\n if (openidm.read(\"schema\/managed\/bulkOperation\") ) {\r\n info(\"Schema updated & verified successfully.\");\r\n } else {\r\n info(\"Schema NOT yet verified\");\r\n }\r\n }\r\n catch (e) {\r\n info(\"Failed to update schema\");\r\n throw { code : 500, message : \"Updating schema with bulkOperation object failed: \" + e }\r\n }\r\n\r\n return true;\r\n }\r\n\r\n\/\/ Main function\r\n\r\nif (request.method === \"action\") {\r\n\r\n if (!openidm.read(\"schema\/managed\/bulkOperation\")) {\r\n updateSchema();\r\n return {\r\n status: \"Schema updated\"\r\n }\r\n } else {\r\n var bo = createBulkOperationObject( request );\r\n info(\"Created BO object\", JSON.stringify(bo));\r\n info(\"Create Trigger returned\", createOrTriggerTask(scheduledTaskName));\r\n return bo;\r\n }\r\n } else if (request.action === \"create\") {\r\n if (!openidm.read(\"schema\/managed\/bulkOperation\")) {\r\n updateSchema();\r\n return {\r\n status: \"Schema updated\"\r\n }\r\n } else {\r\n return {\r\n status: \"Schema already updated\"\r\n }\r\n }\r\n } else {\r\n throw { code : 400, message : \"Unsupported request type: \" + request.method };\r\n}\r\n\r\n}\r\n)();\r\n"
}
Examples
Request
POST /openidm/deferredBatchOperation?_action=batch&name=Batchy%20Operation HTTP/1.1
Content-Type: application/json
Authorization: Bearer <admin access token>
Host: <tenant fqdn>
{
"requests":[
{
"operation":"patch",
"resourcePath":"managed/alpha_role",
"resourceName":"batchRole1",
"data":[{
"operation":"replace",
"field":"description",
"value":"PATCHED!!"
}]
},
{
"operation":"delete",
"resourcePath":"managed/alpha_role",
"resourceName":"batchRole2"
},
{
"operation":"update",
"resourcePath":"managed/alpha_role",
"resourceName":"batchRole3",
"rev":"",
"data":{
"name":"batchRole3",
"description":"UPDATED !! - Batch Role 3"
}
},
{
"operation":"create",
"resourcePath":"managed/alpha_role",
"resourceName":"batchRole4",
"data":{
"name":"batchRole4",
"description":"Batch Role 4"
}
}
]}
Response
{
"request": {
"fields": [],
"resourcePath": "",
"resourceVersion": null,
"preferredLocales": {
"locales": [
""
],
"preferredLocale": ""
},
"content": {
"object": {
"requests": [
{
"operation": "patch",
...
},
...
]
},
"pointer": {
"empty": true,
"value": "/"
},
"map": true,
"boolean": false,
"collection": false,
"list": false,
"notNull": true,
"number": false,
"string": false,
"null": false
},
"action": "batch",
"requestType": "ACTION",
"resourcePathObject": {
"empty": true
},
"additionalParameters": {
"name": "Batchy Operation"
}
},
"name": "Batchy Operation",
"isExpired": false,
"type": "BATCH",
"timeSubmitted": "2023-10-10T13:32:43.352Z",
"_rev": "8da074b0-45c0-46a0-adc3-551ca72bbf25-12958",
"_id": "26d283bf-30f3-4c51-9943-57fa554d4527"
}