Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-8085

Rest2Ldap optimization: avoid reading referenced resources when resolving references



    • Improvement
    • Status: Dev backlog
    • Blocker
    • Resolution: Unresolved
    • 7.1.0
    • None
    • common-repo, performance, rest
    • None


      Here is example output when reading the virtual resources representing the role references in a parent resource:

      $ http -p b GET 'http://admin:password@localhost:8080/api/managed/user/bob/roles?_queryFilter=true'
          "pagedResultsCookie": null,
          "remainingPagedResults": -1,
          "result": [
                  "_id": "bob-1-role-id",
                  "_refObject": {
                      "_id": "bob-1-role-id",
                      "field": "value role-1 of bob"
                  "_refResourceId": "role-1",
                  "_refResourceRev": "660fd181-9593-3cc4-a38f-9a9e1eed746f",
                  "_rev": "023a8930-9ffa-40ad-afd7-4fd55cef605b-1"
                  "_id": "bob-2-role-id",
                  "_refObject": {
                      "_id": "bob-2-role-id",
                      "field": "value role-3 of bob"
                  "_refResourceId": "role-3",
                  "_refResourceRev": "5d0479cf-7fc4-3a1c-8699-ddeaa0184b70",
                  "_rev": "023a8930-9ffa-40ad-afd7-4fd55cef605b-1"
          "resultCount": 2,
          "totalPagedResults": -1,
          "totalPagedResultsPolicy": "NONE"

      Note that reading the main resource /api/managed/user/bob is expected to generate similar output. It doesn't at the moment, but we can assume for the sake of this issue that it does.

      Each reference includes the _refResourceId and the _refResourceRev of the referenced resource. Usually, where DN based naming is used, the _refResourceId can be inferred from the DN contained in the associated LDAP attribute. There's no need to read it from the referenced entry. The requirement to return the _refResourceRev forces a secondary read, "JOIN", of the referenced entry. Reading a resource containing 100 references triggers 100 additional reads. Furthermore, the _refResourceRev value is useless unless additional fields of the referenced resource are returned because it represents an unknown state. Even if the client application is maintaining a cache of local resources, then the _refResourceRev is redundant since a change in value indicates that the cached resource needs to be refreshed, but in that case just proactively read its content once rather than twice - once to read the revision and a second time to read the remaining content.

      Suggested fix:

      This issue can be closed once Rest2Ldap avoids redundant reads, by not returning the _refResourceRev unless requested.

      The location of the potential optimization is in AbstractReferencePropertyMapper#filteredRead and org.forgerock.opendj.rest2ldap.AbstractReferencePropertyMapper#readEntry. In particular, in readEntry Rest2Ldap performs a base object search with a known DN (where the ID is usually specified in the DN) and 3 requested attributes - objectClass, uid and etag. Once the _refResourceId field is not requested by default, the readEntry method could be optimized for the following condition:

      • if the referenced resource using DN based naming
      • if the requested LDAP attributes only contains objectClass and the naming attribute (i.e. the RDN attribute type).

      We could optimize by faking up a successful search response. We could even detect this use case in the underlying LDAP client socket implementation which already does some caching IIRC.


      The existing contract for IDM expects the _refResourceId to be returned by default, so this optimization would result in a change in behavior, which may require a resource API version change in IDM. CC/ Brendan Miller Andi Egloff Jason Lemay




            Unassigned Unassigned
            matthew Matthew Swift
            0 Vote for this issue
            0 Start watching this issue