What Triggers the getLastToken Method in a Connector During Sync

thank you for this, it was very helpful. I see better now how they work. One other question, I notice when I trigger a sync the getLastToken isn’t called. What triggers that?

1 Like

How are you triggering the sync? Executing the livesync via the REST API directly or via the admin UI as shown in the link @patrick_diligent shared should both show you the latest token which getLastToken returned, either in the response from the call or via the UI.

@mwtech In fact getLatestSyncToken is called at the first ever livesync or after the sync token is reset. This call should return a token for the latest change in the resource, and not the last token returned by the previous livesync run.

So @rob.kimball to see getLatestSyncToken in action, reset the sync token this way (change according to connector name/object type):

DELETE /repo/synchronisation/pooledSyncStage/SYSTEMMSSQLACCOUNT

where the connector name is “mssql” and object type is “account”

Then launch a livesync: POST /system/?_action=liveSync&source=system/mssql/account

and you should see a trace for the getLatestSyncToken - but no sync occurring.

So if no update occurred in the resource since the sync token has been reset (e.g deleted), the livesync will not sync anything. That’s why it is important to have an initial reconciliation while livesync is already enabled.

So in this particular example above, that’s what shows in the log file after I reset the sync token:

Aug. 16, 2023 11:55:06 am DEBUG o.i.f.api.operations.SyncApiOp: Enter: getLatestSyncToken(ObjectClass: __ACCOUNT__)	Method: getLatestSyncToken
Aug. 16, 2023 11:55:06 am DEBUG o.i.f.api.operations.SyncApiOp: Return: SyncToken: 44002	Method: getLatestSyncToken

Get Latest Sync Token is invoked first (no sync token yet in the DB), then the Sync operation shows up, but did not find anything to sync (normal, no changes occurred since resetting the sync token), and return the sync token passed into the operation, untouched.



Thanks for the correction there, and sorry for the misinformation. Guess that shows it’s been a couple of years since I last wrote a connector!

thank you both of you. it is very appreciated. One thing I’m trying to wrap my head around is though, if getlatestsynctoken is getting the current latest token, it should always result in a no change sync, what is the use case for that? I can see a use case to clear the token and get EVERYTHING(like a full recon situation)…I guess livesync isn’t’ meant to be treated that way?

Based on what @patrick_diligent said, getLatestSyncToken is only called if IDM does not have a sync token. IDM needs to know where to start querying from on the target system, and that is what the token is for. Once IDM has acquired a sync token, it persists that in the repo and uses that for subsequent liveSync operations.

For example, let’s say objects in your target system have an attribute named updateDate which tracks when the object was last updated and you are using this value for the sync token. When IDM doesn’t have a sync token for the system and getLatestSyncToken is called, your connector would find the object with the most recent updateDate value and return that value as the sync token. On subsequent liveSync calls to the connector IDM will pass in the sync token and the connector will query the target system for records with an updateDate greater than the sync token value and return those objects along with an updated syncToken reflecting what the most recent updateDate value now is. IDM then persists that new syncToken value, and that is used for the next liveSync call.

liveSync is usually used once an initial reconciliation has taken place, as a means of keeping IDM up-to-date with the latest data from the target system.


To be clear in the order in which to apply reconciliation and livesync in the initial provisioning:

  1. Configure a livesync schedule
  2. Once the first livesync run has occurred (sync token established), launch a reconciliation

This to avoid missing updates happening after a reconciliation task complete (e.g sync of a single object in the reconciliation run) and before the first livesync run (as the sync token will be initialised with the latest change - which can be potentially later than the latest change captured by reconciliation if the source is subject to update traffic during the process). This is particularly important when there is no plan to establish a reconciliation schedule in production (which is a valid proposition for very large datasets e.g several 10M identities) but just rely on livesync for keeping the identities in sync.


If you can’t tell, I come from an Oracle IDM background so some of the concepts are a bit new to me. So it sounds like… if bringing a new app on. once the connector is there. you’d do an initial recon(executequery) to bring in the records. Then configure livesync schedule to keep things up to date(sync) If the volume is excessively large you’d NOT do regular full recons and the ONLY incremental is done via livesync, otherwise livesync is for incremental and reconciliation is for full. Do I understand that all ok?

Hi Rob,

Always perform a full reconciliation after the first livesync run (the one that initialises the sync token) so that to capture all changes, then livesync will keep identities in sync as long as there is no network interruptions (so be sure to configure retries on error). If not, when the system is under update traffic, some changes will be forever missed (until the same resources are updated of course).

Whether or not to schedule reconciliations for a large dataset depend on many factors - the most important one is how long it takes (which depends on dataset size and system performances) - and secondly business requirement - what it the acceptable sync time? When livesync is in place, these matters are of a less concern, what matters then is recovering from livesync errors, where different strategies could be adopted. Could be one of the following, to name the least:

  • Schedule a full reconciliation to recover missed changes (can take a long time)
  • Capture missed changes from the log, and perform individual reconById actions. Means, have a robust monitoring solution in place.


1 Like