[OPENIDM-14776] Very low throughput from QueryUserFilterNoLinkExpansion test in GKE cloud Created: 20/May/20  Updated: 23/Jun/20  Resolved: 23/Jun/20

Status: Closed
Project: OpenIDM
Component/s: Devops, Module - Relationships, Performance
Affects Version/s: 7.0.0
Fix Version/s: None

Type: Bug Priority: Critical
Reporter: Tinghua Xu Assignee: Dirk Hogan
Resolution: Fixed Votes: 0
Labels: LEWIS, Performance
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

IDM master runs in a GKE cloud


Target Version/s:

 Description   

We noticed that the throughput for QueryUserFilterNoLinkExpansion test is only 2 users/second in GKE cloud while on bare metal machine its about 1400 users/second.

Gary figured out the cause should be from search operations from IDM using fr-idm-uuid in the filter, fr-idm-uuid is not indexed and so response times are high.

Example, in this case over 3s.

ds-idrepo-0 ds {"eventName":"DJ-LDAP","client":{"ip":"10.44.32.96","port":45864},"server":{"ip":"10.44.32.97","port":1389},"request":{"protocol":"LDAP","operation":"SEARCH","connId":136,"msgId":180304,"dn":"ou=people,ou=identities","scope":"one","filter":"(&(fr-idm-uuid=03c16303-839a-4b26-98ea-2712aa0d9278)(objectClass=person)(objectClass=organizationalPerson)(objectClass=inetOrgPerson)(objectClass=iplanet-am-user-service)(objectClass=devicePrintProfilesContainer)(objectClass=kbaInfoContainer)(objectClass=fr-idm-managed-user-explicit)(objectClass=forgerock-am-dashboard-service)(objectClass=inetuser)(objectClass=iplanet-am-auth-configuration-service)(objectClass=iplanet-am-managed-person)(objectClass=iPlanetPreferences)(objectClass=oathDeviceProfilesContainer)(objectClass=pushDeviceProfilesContainer)(objectClass=sunAMAuthAccountLockout)(objectClass=sunFMSAML2NameIdentifier)(objectClass=webauthnDeviceProfilesContainer))","attrs":["objectClass","fr-idm-uuid","etag","uid","cn","userPassword","inetUserStatus","fr-idm-effectiveRole","fr-idm-effectiveAssignment","postalCode","st","street","displayName","description","co","l","givenName","sn","telephoneNumber","mail","organizationName","fr-idm-lastSync","fr-idm-consentedMapping","fr-idm-kbaInfo","fr-idm-preferences","iplanet-am-user-alias-list"]},"transactionId":"5903b008-407d-4576-87f8-732278230421-1249908/4","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":3226,"elapsedTimeUnits":"MILLISECONDS","additionalItems":{"unindexed":null},"nentries":1},"timestamp":"2020-05-20T16:53:10.787Z","_id":"ca7f2207-0511-45c0-baf8-522e5e2a85a1-26222","source":"audit","topic":"ldap-access","level":"INFO"}

He did a test using his own docker image for forgeops/docker/7.0/ds/idrepo and the throughput get back to 900 users/second. here is the docker image diff:

git diff docker/7.0/ds/idrepo/bin/setup.sh
diff --git a/docker/7.0/ds/idrepo/bin/setup.sh b/docker/7.0/ds/idrepo/bin/setup.sh
index 83033de5..d837b880 100755
--- a/docker/7.0/ds/idrepo/bin/setup.sh
+++ b/docker/7.0/ds/idrepo/bin/setup.sh
@@ -34,4 +34,11 @@ setup-profile --profile ${CONFIG} \
 ##    set-password-storage-scheme-prop --scheme-name "Salted SHA-512" --set enabled:true
 ##    set-password-policy-prop --policy-name "Default Password Policy" --set default-password-storage-scheme:"Salted SHA-512"
 #EOF
+dsconfig --offline --no-prompt --batch <<EOF
+create-backend-index \
+          --backend-name amIdentityStore \
+          --set index-type:equality \
+          --type generic \
+          --index-name fr-idm-uuid
+EOF
 

He suspects that "it could be misconfig in rest2ldap, maybe rest2ldap should not make such search if fr-idm-uuid is defined as a naming attribute"



 Comments   
Comment by Tinghua Xu [ 22/May/20 ]

Gary Williams, Matt has this PR https://stash.forgerock.org/projects/OPENDJ/repos/opendj/pull-requests/5496/ for OPENDJ-7081,  they look as the same issue, right?

Comment by Tinghua Xu [ 23/Jun/20 ]

Resolving https://bugster.forgerock.org/jira/browse/OPENDJ-7081 fixed the issue.

Generated at Mon Mar 01 09:28:24 UTC 2021 using Jira 7.13.12#713012-sha1:6e07c38070d5191bbf7353952ed38f111754533a.