[OPENIDM-14314] Performance degradation when using query _fields param and returnByDefault is enabled Created: 28/Jan/20 Updated: 23/Jul/20 Resolved: 23/Jul/20
|Component/s:||Module - Relationships, Module - Roles|
|Reporter:||Anastasios Kampas||Assignee:||Dirk Hogan|
|Remaining Estimate:||Not Specified|
|Time Spent:||Not Specified|
|Original Estimate:||Not Specified|
|Support Ticket IDs:|
It's been reported that when effectiveRoles and effectiveAssignments returnbyDefault is enabled and when roles are not queried by the client, IDM will still try to get the data from the DB even though it's not needed.
This can have a serious effect on Database performance whe multiple relationships are linked to the user. It's been observed that when the flag is disabled, the response times are 85% faster.
This is linked to
|Comment by Dirk Hogan [ 28/Jan/20 ]|
Anastasios Kampas from my reading of the code, setting returnByDefault to false on effectiveRoles and effectiveAssignments should prevent the corresponding scripts from being run. Are you sure the database activity is due to the invocation of these scripts? You could tell for sure by putting console log statements in these scripts, and see if these logs appear when returnByDefault is set to false on effectiveRoles and effectiveAssignments.
FWIW, I am working on https://bugster.forgerock.org/jira/browse/OPENIDM-11601, which will replace the effectiveRoles and effectiveAssignment script invocations on every managed user GET invocation with config-driven logic which will calculate and persist these (or any other configured) virtual properties only when the connected graph topology changes, as dictated by the relationship signaling configuration. Thus you will get up-to-date state in the virtual properties which will only be calculated when necessary, and not on every GET as is currently the case. So you will get your performance improvement without the peril of stale data.
|Comment by Anastasios Kampas [ 28/Jan/20 ]|
The customer has indeed disabled the flag as a workaround however the expectation is that since _fields=_id is requested, the relationships do not need to be calculated.
This could end up being a duplicate of
|Comment by Dirk Hogan [ 22/May/20 ]|
This issue is fixed by https://bugster.forgerock.org/jira/browse/OPENIDM-11601
The bottom line: effectiveRoles and effectiveAssignments will only be re-calculated when any part of the graph topology user->roles->assignments is changed, not on every GET of the user. So when you add a role to a user, or an assignment to a role which is assigned to a user, the user will be notified of this change, and virtual properties re-calculated, and persisted. When the user is next read, the virtual properties will not be re-calculated. So you can leave returnByDefault as true, and these virtual properties will be re-calculated and persisted only when the user's role and assignment graph topology changes - not on every read.
|Comment by Chris Drake [ 22/Jul/20 ]|
|Comment by Lana Frost [ 23/Jul/20 ]|
Reopening to add to release notes