Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-14776

Very low throughput from QueryUserFilterNoLinkExpansion test in GKE cloud

    Details

    • Target Version/s:

      Description

      We noticed that the throughput for QueryUserFilterNoLinkExpansion test is only 2 users/second in GKE cloud while on bare metal machine its about 1400 users/second.

      Gary figured out the cause should be from search operations from IDM using fr-idm-uuid in the filter, fr-idm-uuid is not indexed and so response times are high.

      Example, in this case over 3s.

      ds-idrepo-0 ds {"eventName":"DJ-LDAP","client":{"ip":"10.44.32.96","port":45864},"server":{"ip":"10.44.32.97","port":1389},"request":{"protocol":"LDAP","operation":"SEARCH","connId":136,"msgId":180304,"dn":"ou=people,ou=identities","scope":"one","filter":"(&(fr-idm-uuid=03c16303-839a-4b26-98ea-2712aa0d9278)(objectClass=person)(objectClass=organizationalPerson)(objectClass=inetOrgPerson)(objectClass=iplanet-am-user-service)(objectClass=devicePrintProfilesContainer)(objectClass=kbaInfoContainer)(objectClass=fr-idm-managed-user-explicit)(objectClass=forgerock-am-dashboard-service)(objectClass=inetuser)(objectClass=iplanet-am-auth-configuration-service)(objectClass=iplanet-am-managed-person)(objectClass=iPlanetPreferences)(objectClass=oathDeviceProfilesContainer)(objectClass=pushDeviceProfilesContainer)(objectClass=sunAMAuthAccountLockout)(objectClass=sunFMSAML2NameIdentifier)(objectClass=webauthnDeviceProfilesContainer))","attrs":["objectClass","fr-idm-uuid","etag","uid","cn","userPassword","inetUserStatus","fr-idm-effectiveRole","fr-idm-effectiveAssignment","postalCode","st","street","displayName","description","co","l","givenName","sn","telephoneNumber","mail","organizationName","fr-idm-lastSync","fr-idm-consentedMapping","fr-idm-kbaInfo","fr-idm-preferences","iplanet-am-user-alias-list"]},"transactionId":"5903b008-407d-4576-87f8-732278230421-1249908/4","response":{"status":"SUCCESSFUL","statusCode":"0","elapsedTime":3226,"elapsedTimeUnits":"MILLISECONDS","additionalItems":{"unindexed":null},"nentries":1},"timestamp":"2020-05-20T16:53:10.787Z","_id":"ca7f2207-0511-45c0-baf8-522e5e2a85a1-26222","source":"audit","topic":"ldap-access","level":"INFO"}
      

      He did a test using his own docker image for forgeops/docker/7.0/ds/idrepo and the throughput get back to 900 users/second. here is the docker image diff:

      git diff docker/7.0/ds/idrepo/bin/setup.sh
      diff --git a/docker/7.0/ds/idrepo/bin/setup.sh b/docker/7.0/ds/idrepo/bin/setup.sh
      index 83033de5..d837b880 100755
      --- a/docker/7.0/ds/idrepo/bin/setup.sh
      +++ b/docker/7.0/ds/idrepo/bin/setup.sh
      @@ -34,4 +34,11 @@ setup-profile --profile ${CONFIG} \
       ##    set-password-storage-scheme-prop --scheme-name "Salted SHA-512" --set enabled:true
       ##    set-password-policy-prop --policy-name "Default Password Policy" --set default-password-storage-scheme:"Salted SHA-512"
       #EOF
      +dsconfig --offline --no-prompt --batch <<EOF
      +create-backend-index \
      +          --backend-name amIdentityStore \
      +          --set index-type:equality \
      +          --type generic \
      +          --index-name fr-idm-uuid
      +EOF
       

      He suspects that "it could be misconfig in rest2ldap, maybe rest2ldap should not make such search if fr-idm-uuid is defined as a naming attribute"

        Attachments

          Activity

            People

            • Assignee:
              dhogan Dirk Hogan
              Reporter:
              Tinghua.Xu Tinghua.Xu [X] (Inactive)
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: