[OPENIDM-5914] Role is still showing as assigned in effectiveRoles attribute on query-all output if role is unassigned via the admin UI Created: 01/Jun/16  Updated: 12/Feb/18  Resolved: 07/Feb/18

Status: Closed
Project: OpenIDM
Component/s: Module - Roles
Affects Version/s: OpenIDM 4.0.0, OpenIDM 4.5.0, OpenIDM 5.0.0
Fix Version/s: OpenIDM 6.0.0

Type: Bug Priority: Major
Reporter: Andy Itter Assignee: Alin Brici
Resolution: Fixed Votes: 0
Labels: LEWIS, regroom, release-notes
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified

Issue Links:
Relates
is related to OPENIDM-9253 Improve relationship abstraction behi... Closed
Target Version/s:
Support Ticket IDs:
Verified Version/s:

 Description   

If a role is assigned to a user using the openidm/managed/user endpoint and then subsequently deleted via the OpenIDM Admin UI then the role is present in the effectiveRoles attribute when using managed/user?_queryId=query-all but not when getting the user individually using managed/user/a8...5c

To reproduce:

0). Add a role to a user:

curl --header "X-OpenIDM-Username: openidm-admin" --header "X-OpenIDM-Password: openidm-admin" --header "Content-Type: application/json" --header "If-Match: *" --request PATCH --data '[{"operation": "replace", "field": "/roles/-", "value": {"_ref" : "managed/role/bc...2b"}}]' "http://localhost:8081/openidm/managed/user/a8...5c"

1). Use the OpenIDM 4 Admin UI and remove the role assignment from the user.

2). Get the individual user:

curl --header "X-OpenIDM-Username: openidm-admin" --header "X-OpenIDM-Password: openidm-admin" --request GET "http://localhost:8081/openidm/managed/user/a8...5c"

{"_id":"a8...c","_rev":"7","mail":"test@example.com","sn":"user","givenName":"test","userName":"testuser","accountStatus":"active","effectiveRoles":[],"effectiveAssignments":[]}

Note the role is not present as expected.

3). Get all user objects:

curl --header "X-OpenIDM-Username: openidm-admin" --header "X-OpenIDM-Password: openidm-admin" --request GET "http://localhost:8081/openidm/managed/user?_queryId=query-all"

{"result":[{"_id":"a8...5c","_rev":"7","mail":"test@example.com","sn":"user","givenName":"test","userName":"testuser","accountStatus":"active","effectiveRoles":[{"_ref":"managed/role/bc...2b"}],"effectiveAssignments":[]}],"resultCount":1,"pagedResultsCookie":null,"totalPagedResultsPolicy":"NONE","totalPagedResults":-1,"remainingPagedResults":-1}

Note the role is still present in effectiveRoles.

Expected behaviour:

The content of effectiveRoles when getting the user individually or when using query-all should be consistent.



 Comments   
Comment by Andy Itter [ 01/Jun/16 ]

Additionally, note that if the role is unassigned from the user using the following rather than the Admin UI:

curl --header "X-OpenIDM-Username: openidm-admin" --header "X-OpenIDM-Password: openidm-admin" --header "Content-Type: application/json" --request PATCH --data '[{"operation": "remove", "field": "/roles/0"}]' "http://localhost:8081/openidm/managed/user/a8...5c"

...then the effectiveRoles output is as expected for the two scenarios.

Note that the observed behaviour mentioned in the initial description is the same on the snapshot build.

Comment by Laurent Bristiel [ 08/Jun/16 ]

I could reproduce this bug in 4.0.0 (strange bug!) but not in 4.5 and 5.0, so it was fixed somehow in the meantime.

Comment by Andy Itter [ 28/Jun/16 ]

Tested this again myself just now. This issue is still present in the current 4.5 RC2 build:

 OpenIDM version "4.5.0-RC2" (revision: c1e824c) jenkins-OpenIDM - Sustaining - Release-20 origin/sustaining/4.5.x
  • Create user in the UI
  • Create a role in the UI
  • Add the user to the role in the UI (go to the role to do this)
  • Get the individual user with via REST - the role is present (use /managed/user/_id) as expected
  • Remove role from the user via the UI (by accessing the role - accessing the user will not work)
  • Check again via REST (/managed/user/_id) the role is no longer visible - as expected
  • Check via REST (use /managed/user?_queryId=query-all) the role is still visible in effectiveRoles.
Comment by Dirk Hogan [ 28/Jun/16 ]

When a provisioning role is removed from a user, a DELETE is performed upon either:
1. managed/user/user-in-question/roles/grant-id – this occurs when the role is deleted by navigating to the user in the Admin UI
2. managed/role/role-in-question/members/grant-id – this occurs when the role is deleted by navigating to the role in the Admin UI.

Keep in mind that #1 does not cause the behavior documented in this defect, and #2 does.

Both of these requests are fielded by RelationshipProvider#deleteInstance. In both cases, RelationshipProvider#deleteInstance will:
1. delete the relationship grant in the relationship table
2. persist the managed object 'owning' the relationship (either the user or role, corresponding to the DELETE invocation above, respectively)
3. trigger a sync check on the 'opposite' end of the relationship, so that assignments are updated on target systems corresponding to this now-removed role grant. In other words, when a grant is deleted on managed/role/members the referenced managed user will be synced; when a grant is deleted on managed/user/roles, the referenced user will by synced.

Note that effectiveAssignments are persisted in managed user so that sync diff logic can handle various delete scenarios, especially when target systems are offline, or implicit sync disabled, at the time where the delete takes place. This is crucial, and explains why this defect only occurs when the role grant is deleted from roles in the Admin UI: in this case, only the role is persisted, and the managed user is synced. This means that the effectiveRoles array in the managed user is never updated to reflect the role grant deletion. In this case, the managed user's effectiveRoles array will contain the role reference (reflecting the previous grant), but the relationship table will have no record of this role grant. A GET on managed/user with _queryId=query-all simply returns the state in the managed user table, which returns the outdated effectiveRoles state. However, when the role grant is updated via the user in the Admin UI, the user is persisted in the repo, and the effectiveRoles array updated/persisted accordingly, which is reflected in the correct query response.

I don't think that this defect can be addressed without a significant relationship refactor, but there is a work-around. If executeOnRetrieve=true is appended to the query, the correct state is returned:
http://localhost:8080/openidm/managed/user?_queryId=query-all&executeOnRetrieve=true
Note also that the effectiveRoles which are truly 'in effect' are always calculated, and only the calculated roles used against any target systems. As far as I can tell, the outdated effectiveRoles state in the managed user table does not have any significant harmful impact. So this defect, at least in the 4.5 time-frame, might best be handled with release-notes indicating that:
1. roles are best deleted from the managed user if possible
2. the executeOnRetrieve=true parameter should be added when all users are queried if #1 cannot be adhered-to.
Lana Frost, the above, release-note-centric approach, will be adopted for 4.5. The larger issue will be be handled in the 5.0 release time-frame.

Comment by Dirk Hogan [ 09/Sep/16 ]

Re-evaluating this issue in the 5.0.0 time-frame, I can corroborate my analysis above. The only correction I would make is I'm not sure we persist effectiveRoles in order to help with the sync diff logic - I'm not sure why this state is persisted. Also any GET automatically executes the onRetrieve script, whereas a query will only execute the onRetrieve script if executeOnRetrieve is set to true. See https://bugster.forgerock.org/jira/browse/OPENIDM-1732 for more context.

I had hoped that relationships would be refactored in the 5.0.0 time-frame, but we will have to wait until 5.5 or 6.0. The fundamental issue is that in persisting effectiveRoles, we essentially persist some of the relationship graph in a graph vertex, state which is not updated when the relationship graph is mutated. That being said, it appears that the effectiveRoles array was intended to be a calculated field, and it is an established idiom to pass the executeOnRetrieve to query requests to obtain the updated state of calculated fields. Which begs the question why effectiveRoles are ever persisted in the first place - a question I cannot answer.

Comment by andi [ 09/Sep/16 ]

Virtual fields by default are persistet to
a) allow detection of things getting removed with an incoming update, to trigger delete events
and potentially
b) To allow searches on currently effective (calculated) values

I'm not sure if b) is currently used, and a) may only currently be in use for assignment detection, it's very well possible that our default scripts don't need to do role assignment delete detection.

Comment by Dirk Hogan [ 12/Sep/16 ]

Thanks for the info andi.

For a bit more context regarding the storage of effectiveAssignments, here's an excerpt from an email from Jake Feasel, from a while back:
/excerpt-start
Chad and I started discussing the rationale for why virtual properties are persisted at all. The reason seems to be so that, during a change to a role (or now, an assignment) the previous state could be compared with the current state and the difference used to calculate any action that may be necessary.

For example, imagine an assignment was configured to provide two entries within ldapGroups using the "merge with target" assignment operation: "cn=Group1" and "cn=Group2". All users with that assignment will have those two groups (among others, possibly). Also, all users with that assignment will have a copy of the assignment persisted in their record, as a result of the virtual property storage.

Then, if the assignment definition is changed so that it only provides "cn=Group1" (omitting "cn=Group2"), the desired behavior is that all users which had been granted the assignment in its earlier state would be have "cn=Group2" removed from their ldapGroups property. This gets us to the sole reason (as far as I know) that we persist the effectiveAssignments value in the managed/user record - so that we can compare the last version that was saved for the user with the version that now exists, and apply the difference.
/excerpt-end

Because the relationship refactor is not making it into the 5.0 release, I will change the target version to 5.5.

Comment by Alin Brici [ 07/Feb/18 ]

Has been fixed during our recent refactor of relationships. 

Comment by Laurent Bristiel [ 12/Feb/18 ]

checked OK in OpenIDM version "6.0.0-SNAPSHOT" (revision: d5c6fa2)

Generated at Fri May 25 09:37:02 BST 2018 using JIRA 7.3.6#73017-sha1:51437cf70ba5689aadb808c1cc05a46d676f5739.