Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-8087

Rest2Ldap optimization: simplify filters where possible

    XMLWordPrintable

    Details

      Description

      The following search request has been observed being sent from IDM to DS via Rest2Ldap:

      response: {
         elapsedTimeUnits: "MILLISECONDS"
         status: "SUCCESSFUL"
         elapsedTime: 12608937  <----- 3.5h
         statusCode: "0"
         nentries: 652763       <----- 650k users
         additionalItems: {
         unindexed: null
         }
      }
      operation: "SEARCH"
      dn: "ou=user,o=alpha,o=root,ou=identities"
      scope: "one"
      connId: 453
      msgId: 2571
      protocol: "LDAP"
      filter: "(&(objectClass=person)(objectClass=organizationalPerson)(objectClass=inetOrgPerson)(objectClass=iplanet-am-user-service)(objectClass=devicePrintProfilesContainer)(objectClass=deviceProfilesContainer)(objectClass=kbaInfoContainer)(objectClass=fr-idm-managed-user-explicit)(objectClass=forgerock-am-dashboard-service)(objectClass=inetuser)(objectClass=iplanet-am-auth-configuration-service)(objectClass=iplanet-am-managed-person)(objectClass=iPlanetPreferences)(objectClass=oathDeviceProfilesContainer)(objectClass=pushDeviceProfilesContainer)(objectClass=sunAMAuthAccountLockout)(objectClass=sunFMSAML2NameIdentifier)(objectClass=webauthnDeviceProfilesContainer)(objectClass=fr-idm-hybrid-obj)(objectClass=fr-ext-attrs)(objectClass=top))"
      attrs: [
         0: "objectClass"
         1: "fr-idm-uuid"
         2: "etag"
         3: "co"
         4: "mail"
         5: "fr-attr-str1"
         6: "fr-idm-managed-user-memberoforgid"
         7: "fr-attr-str2"
         8: "fr-attr-idate5"
         9: "fr-attr-str3"
         10: "fr-attr-idate4"
         11: "postalCode"
         12: "fr-attr-str4"
         13: "fr-attr-idate3"
         14: "fr-attr-str5"
         15: "fr-attr-istr5"
         16: "fr-attr-istr4"
         17: "userPassword"
         18: "fr-attr-istr3"
         19: "fr-attr-istr2"
         20: "fr-attr-istr1"
         21: "fr-attr-imulti3"
         22: "fr-attr-int5"
         23: "fr-attr-imulti4"
         24: "fr-attr-int4"
         25: "fr-idm-consentedMapping"
         26: "fr-attr-imulti5"
         27: "fr-attr-int3"
         28: "fr-attr-int2"
         29: "fr-attr-imulti1"
         30: "fr-attr-imulti2"
         31: "fr-attr-int1"
         32: "givenName"
         33: "st"
         34: "street"
         35: "telephoneNumber"
         36: "l"
         37: "displayName"
         38: "fr-idm-effectiveAssignment"
         39: "description"
         40: "inetUserStatus"
         41: "fr-attr-multi1"
         42: "fr-attr-date3"
         43: "fr-attr-date2"
         44: "fr-attr-multi3"
         45: "fr-attr-date5"
         46: "fr-attr-multi2"
         47: "fr-attr-date4"
         48: "fr-idm-lastSync"
         49: "fr-attr-multi5"
         50: "iplanet-am-user-alias-list"
         51: "fr-attr-multi4"
         52: "fr-idm-kbaInfo"
         53: "fr-attr-iint4"
         54: "fr-attr-iint3"
         55: "isMemberOf"
         56: "fr-attr-iint2"
         57: "sn"
         58: "fr-attr-iint1"
         59: "fr-attr-date1"
         60: "fr-attr-iint5"
         61: "fr-idm-preferences"
         62: "cn"
         63: "uid"
         64: "fr-attr-idate2"
         65: "fr-attr-idate1"
         66: "fr-idm-effectiveRole"
      ]
      

      The filter contains an AND filter containing many (21) objectClass equality filters where the objectClass is a super type of other objectClasses. Object classes are quite expensive to normalize and large filters like these are expensive to evaluate for positive cases because each filter component must be evaluated. There's no need to specify super-classes in filters. We may even be able to eliminate them when compiling the filter, both in the client and server.

      Note that, in this particular example, I expect that all sub-filters were evaluated against the objectClass index and all of them were all IDs. That's 21 index queries for each search, which may or may not be a problem depending on how frequently the searches are performed.

        Attachments

          Issue Links

            Activity

              People

              ondrej.fuchsik Ondrej Fuchsik
              matthew Matthew Swift
              Yannick Lecaillez Yannick Lecaillez
              Ondrej Fuchsik Ondrej Fuchsik
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: