Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-2422

Allow the maximum 'idSetLimit' for getIDSetFromScope based 'indexes' to be configured

    Details

    • Type: Improvement
    • Status: Done
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 3.0.0
    • Fix Version/s: 3.0.0
    • Component/s: core server
    • Labels:
      None
    • Support Ticket IDs:

      Description

      Assume the following scenario:

      dc=example,dc=com
       -> ou=BigTree (5 million entries)
       -> ou=MediumTree (5000 entries)
       -> ou=SmallTree (50 entries)
      

      In this simplified case, also assume that all objects only consist of the following 3 possibilities.

      objectclass=typeA

      objectclass=typeB

      objectclass=typeC

      ou=BigTree contains lots of entries, at least 100k of each of these 3 types.

      Now a user wants to search with base ou=MediumTree for objectclass=typeB. There may only be a handful of these types in this subtree.

      Unfortunately, since there are many objectclass=typeB in BigTree, the objectclass.equality index is of no use here as it has gone completely all-ids on this backend.

      The only remaining way of reducing the search candidates is to use the scope, which in theory could give back a set of 5000 ids.

      However, there is currently (DJ3.0.0) a hard coded entry-limit on this mechanism of SCOPE_IDSET_LIMIT (4096) entries. This search is therefore unindexed and there is nothing the administrator can do about it. Despite 5000 only being a little greater than 4096, the search would need to go through all 5+million entries

      In 2.6.x this limit was governed by the id2subtree/id2children indexes and these were influenced by the default index-entry-limit. While not ideal, it was therefore possible to manipulate it.

      Although this problem can (and probably should) be fixed by better database design (split into multiple backends, ensure more uniqueness in the entries themselves so more specific searches can be used) - it would be useful if, in rare cases where this situation has occurred, the limit could simply be increased slightly instead.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ludo Ludovic Poitou
                Reporter:
                ian.packer Ian Packer [X] (Inactive)
                Dev Assignee:
                Ludovic Poitou
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: