Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-2731

Middle and final substring indexes fail to return candidates, resulting in an unindexed search.

    Details

    • Support Ticket IDs:
    • Sprint:
      OpenDJ Bugfix Sprint 77

      Description

      A simple substring search that worked with 2.6.x now fails with result=50 and an unindexed search with OpenDJ 3.0.0.

      [29/Feb/2016:16:51:33 -0700] SEARCH REQ conn=0 op=1 msgID=2 base="dc=example,dc=com" scope=sub filter="(&(givenname=*jon*)(sn=*farrell*))" attrs="dn"
      [29/Feb/2016:16:51:33 -0700] SEARCH RES conn=0 op=1 msgID=2 result=50 message="You do not have sufficient privileges to perform an unindexed search" nentries=0 unindexed etime=91
      

      Working Test: 2.6.3

      ———————————————————————————————
      
      opendj; bin/$ status
      Mon Feb 29 16:49:03 MST 2016
      
                --- Server Status ---
      Server Run Status:        Started
      Open Connections:         1
      
                --- Server Details ---
      Host Name:                opendj.forgerock.com
      Administrative Users:     cn=Directory Manager
      Installation Path:        /opt/instances/opendj
      Version:                  OpenDJ 2.6.3
      Java Version:             1.8.0_45
      Administration Connector: Port 4444 (LDAPS)
      
                --- Connection Handlers ---
      Address:Port : Protocol : State
      -------------:----------:---------
      --           : LDIF     : Disabled
      0.0.0.0:636  : LDAPS    : Disabled
      0.0.0.0:1389 : LDAP     : Enabled
      0.0.0.0:1689 : JMX      : Disabled
      0.0.0.0:8080 : HTTP     : Disabled
      
                --- Data Sources ---
      Base DN:     dc=example,dc=com
      Backend ID:  userRoot
      Entries:     8795
      Replication: 
      
      Server PID:  27766
      
      
      
      opendj; bin/$ date; ./ldapsearch --port 1389 -D "uid=openam,ou=admins,dc=example,dc=com" -w password --baseDN dc=example,dc=com "(&(givenname=*jon*)(sn=*farrell*))" dn
      Mon Feb 29 16:49:08 MST 2016
      dn: uid=jonfarrell,ou=People,dc=example,dc=com
      
      [29/Feb/2016:16:49:08 -0700] CONNECT conn=8 from=127.0.0.1:57325 to=127.0.0.1:1389 protocol=LDAP
      [29/Feb/2016:16:49:08 -0700] BIND REQ conn=8 op=0 msgID=1 version=3 type=SIMPLE dn="uid=openam,ou=admins,dc=example,dc=com"
      [29/Feb/2016:16:49:08 -0700] BIND RES conn=8 op=0 msgID=1 result=0 authDN="uid=openam,ou=admins,dc=example,dc=com" etime=1
      [29/Feb/2016:16:49:08 -0700] SEARCH REQ conn=8 op=1 msgID=2 base="dc=example,dc=com" scope=wholeSubtree filter="(&(givenname=*jon*)(sn=*farrell*))" attrs="dn"
      [29/Feb/2016:16:49:08 -0700] SEARCH RES conn=8 op=1 msgID=2 result=0 nentries=1 etime=2
      [29/Feb/2016:16:49:08 -0700] UNBIND REQ conn=8 op=2 msgID=3
      [29/Feb/2016:16:49:08 -0700] DISCONNECT conn=8 reason="Client Unbind"
      

      Upgrade the 2.6.3 instance to 3.0.0

      ———————————————————————————————
      
      >>>> Performing upgrade
      
        Changing matching rule for 'userCertificate' and 'caCertificate' to
      ...
      ...
      ...
        Archiving concatenated schema.......................................   100%     
      
      >>>> OpenDJ was successfully upgraded from version 2.6.3.12667 to
      3.0.0.185acee3ba68d8da1782007eebacb3701dc996d6
      
      
      >>>> Performing post upgrade tasks
      
      Feb 29, 2016 4:50:55 PM org.forgerock.i18n.slf4j.LocalizedLogger info
      INFO: Loaded extension from file '/opt/instances/opendj/lib/extensions/snmp-mib2605.jar' (build 3.0.0, revision 185acee3ba68d8da1782007eebacb3701dc996d6)
      ...
      ...
      ...
      INFO: Rebuild complete. Processed 8795 entries in 1 seconds (average rate 6171.9/sec)
        Rebuilding all indexes..............................................   100%     
      
      >>>> Post upgrade tasks complete
      

      Failed Test: 3.0.0

      ———————————————————————————————
      
      opendj; bin/$ ./ldapsearch --port 1389 -D "uid=openam,ou=admins,dc=example,dc=com" -w password --baseDN dc=example,dc=com "(&(givenname=*jon*)(sn=*farrell*))" dn
      
      SEARCH operation failed
      Result Code:  50 (Insufficient Access Rights)
      Additional Information:  You do not have sufficient privileges to perform an unindexed search
      
      [29/Feb/2016:16:51:33 -0700] CONNECT conn=0 from=127.0.0.1:57345 to=127.0.0.1:1389 protocol=LDAP
      [29/Feb/2016:16:51:33 -0700] BIND REQ conn=0 op=0 msgID=1 version=3 type=SIMPLE dn="uid=openam,ou=admins,dc=example,dc=com"
      [29/Feb/2016:16:51:33 -0700] BIND RES conn=0 op=0 msgID=1 result=0 authDN="uid=openam,ou=admins,dc=example,dc=com" etime=6
      [29/Feb/2016:16:51:33 -0700] SEARCH REQ conn=0 op=1 msgID=2 base="dc=example,dc=com" scope=sub filter="(&(givenname=*jon*)(sn=*farrell*))" attrs="dn"
      [29/Feb/2016:16:51:33 -0700] SEARCH RES conn=0 op=1 msgID=2 result=50 message="You do not have sufficient privileges to perform an unindexed search" nentries=0 unindexed etime=91
      [29/Feb/2016:16:51:33 -0700] UNBIND REQ conn=0 op=2 msgID=3
      [29/Feb/2016:16:51:33 -0700] DISCONNECT conn=0 reason="Client Unbind"
      

      For reference, openam did not have the unindexed search privilege.

      dn: uid=openam,ou=admins,dc=example,dc=com
      objectClass: top
      objectClass: inetOrgPerson
      objectClass: organizationalPerson
      objectClass: person
      sn: openam
      cn: openam
      userPassword: password
      uid: openam
      ds-privilege-name: subentry-write
      ds-privilege-name: update-schema
      ds-privilege-name: password-reset
      

      2.6.3 Index status

      opendj; bin/$ ./dbtest list-index-status  --backendID userRoot  --baseDN dc=example,dc=com
      Index Name                 Index Type  JE Database Name                             Index Valid  Record Count  Undefined  95%  90%  85%
      ---------------------------------------------------------------------------------------------------------------------------------------
      id2children                Index       dc_example_dc_com_id2children                true         11            1          0    0    0
      id2subtree                 Index       dc_example_dc_com_id2subtree                 true         11            2          0    0    0
      uniqueMember.equality      Index       dc_example_dc_com_uniqueMember.equality      true         0             0          0    0    0
      mail.equality              Index       dc_example_dc_com_mail.equality              true         2402          0          0    0    0
      mail.substring             Index       dc_example_dc_com_mail.substring             true         19476         12         0    0    0
      aci.presence               Index       dc_example_dc_com_aci.presence               true         1             0          0    0    0
      telephoneNumber.equality   Index       dc_example_dc_com_telephoneNumber.equality   true         0             0          0    0    0
      telephoneNumber.substring  Index       dc_example_dc_com_telephoneNumber.substring  true         0             0          0    0    0
      givenName.equality         Index       dc_example_dc_com_givenName.equality         true         2389          0          0    0    0
      givenName.substring        Index       dc_example_dc_com_givenName.substring        true         8270          0          0    0    0
      member.equality            Index       dc_example_dc_com_member.equality            true         0             0          0    0    0
      ds-sync-hist.ordering      Index       dc_example_dc_com_ds-sync-hist.ordering      true         332           0          0    0    0
      ds-sync-conflict.equality  Index       dc_example_dc_com_ds-sync-conflict.equality  true         0             0          0    0    0
      entryUUID.equality         Index       dc_example_dc_com_entryUUID.equality         true         8795          0          0    0    0
      sn.equality                Index       dc_example_dc_com_sn.equality                true         3228          0          0    0    0
      sn.substring               Index       dc_example_dc_com_sn.substring               true         12197         0          0    0    0
      cn.equality                Index       dc_example_dc_com_cn.equality                true         6             1          0    0    0
      cn.substring               Index       dc_example_dc_com_cn.substring               true         59            1          0    0    0
      objectClass.equality       Index       dc_example_dc_com_objectClass.equality       true         28            16         0    0    0
      uid.equality               Index       dc_example_dc_com_uid.equality               true         7808          0          0    0    0
      
      Total: 20
      
      Index: id2children
      Undefined keys: [4]
      
      Index: id2subtree
      Undefined keys: [1] [4]
      
      Index: mail.substring
      Undefined keys: [.co.us] [.state] [.us] [ate.co] [e.co.u] [j.us] [co.us] [s] [state.] [tate.n] [te.co.] [us]
      
      Index: objectClass.equality
      Undefined keys: [deviceprintprofilescontainer] [forgerock-am-dashboard-service] [inetorgperson] [inetuser] [iplanet-am-auth-configuration-service] [iplanet-am-managed-person] [iplanet-am-user-service] [iplanetpreferences] [mycoperson] [organizationalperson] [person] [sunamauthaccountlockout] [sunfederationmanagerdatastore] [sunfmsaml2nameidentifier] [sunidentityserverlibertyppservice] [top]
      
      Index: cn.substring
      Undefined keys: [_]
      
      Index: cn.equality
      Undefined keys: [_]
      

      3.0.0 Index status

      opendj; bin/$ ./backendstat show-index-status --backendID userRoot --baseDN dc=example,dc=com
      Index Name                                        Raw DB Name                                                          Index Valid  Record Count  Over Entry Limit  95%  90%  85%
      ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      uniqueMember.uniqueMemberMatch                    /dc=example,dc=com/uniqueMember.uniqueMemberMatch                    true         0             0                 0    0    0
      mail.caseIgnoreIA5Match                           /dc=example,dc=com/mail.caseIgnoreIA5Match                           true         2402          0                 0    0    0
      mail.caseIgnoreIA5SubstringsMatch:6               /dc=example,dc=com/mail.caseIgnoreIA5SubstringsMatch:6               true         19476         12                0    0    0
      telephoneNumber.telephoneNumberSubstringsMatch:6  /dc=example,dc=com/telephoneNumber.telephoneNumberSubstringsMatch:6  true         0             0                 0    0    0
      telephoneNumber.telephoneNumberMatch              /dc=example,dc=com/telephoneNumber.telephoneNumberMatch              true         0             0                 0    0    0
      aci.presence                                      /dc=example,dc=com/aci.presence                                      true         1             0                 0    0    0
      ds-sync-hist.changeSequenceNumberOrderingMatch    /dc=example,dc=com/ds-sync-hist.changeSequenceNumberOrderingMatch    true         332           0                 0    0    0
      cn.caseIgnoreMatch                                /dc=example,dc=com/cn.caseIgnoreMatch                                true         6             1                 0    0    0
      cn.caseIgnoreSubstringsMatch:6                    /dc=example,dc=com/cn.caseIgnoreSubstringsMatch:6                    true         59            1                 0    0    0
      objectClass.objectIdentifierMatch                 /dc=example,dc=com/objectClass.objectIdentifierMatch                 true         28            16                0    0    0
      entryUUID.uuidMatch                               /dc=example,dc=com/entryUUID.uuidMatch                               true         8795          0                 0    0    0
      uid.caseIgnoreMatch                               /dc=example,dc=com/uid.caseIgnoreMatch                               true         7808          0                 0    0    0
      givenName.caseIgnoreMatch                         /dc=example,dc=com/givenName.caseIgnoreMatch                         true         2389          0                 0    0    0
      givenName.caseIgnoreSubstringsMatch:6             /dc=example,dc=com/givenName.caseIgnoreSubstringsMatch:6             true         8270          0                 0    0    0
      member.distinguishedNameMatch                     /dc=example,dc=com/member.distinguishedNameMatch                     true         0             0                 0    0    0
      sn.caseIgnoreMatch                                /dc=example,dc=com/sn.caseIgnoreMatch                                true         3228          0                 0    0    0
      sn.caseIgnoreSubstringsMatch:6                    /dc=example,dc=com/sn.caseIgnoreSubstringsMatch:6                    true         12197         0                 0    0    0
      ds-sync-conflict.distinguishedNameMatch           /dc=example,dc=com/ds-sync-conflict.distinguishedNameMatch           true         0             0                 0    0    0
      
      Total: 18
      
      Index: /dc=example,dc=com/cn.caseIgnoreMatch
      Over index-entry-limit keys: [_]
      
      Index: /dc=example,dc=com/objectClass.objectIdentifierMatch
      Over index-entry-limit keys: [2.5.6.0] [2.5.6.6] [2.5.6.7] [deviceprintprofilescontainer] [forgerock-am-dashboard-service] [inetorgperson] [inetuser] [iplanet-am-auth-configuration-service] [iplanet-am-managed-person] [iplanet-am-user-service] [iplanetpreferences] [mycoperson] [sunamauthaccountlockout] [sunfederationmanagerdatastore] [sunfmsaml2nameidentifier] [sunidentityserverlibertyppservice]
      
      Index: /dc=example,dc=com/mail.caseIgnoreIA5SubstringsMatch:6
      Over index-entry-limit keys: [.co.us] [.state] [.us] [ate.co] [e.co.u] [j.us] [co.us] [s] [state.] [tate.n] [te.co.] [us]
      
      Index: /dc=example,dc=com/cn.caseIgnoreSubstringsMatch:6
      Over index-entry-limit keys: [_]
      

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                fabiop Fabio Pistolesi
                Reporter:
                lee.trujillo Lee Trujillo
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: