Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-15322

Query on relationship endpoint takes much longer time to return with external DS as repo


    • Target Version/s:


      The test preloads 30K managed users first, with each user has 10 roles. Using external DS as repo, the following query would take about 28 seconds to return when external DS is used as repo. 

      curl  --header "X-OpenIDM-Username: openidm-admin"  --header "X-OpenIDM-Password: openidm-admin" --request GET "http://localhost:8080/openidm/managed/user/A-A-10518/roles?_queryFilter=true&_fields=*_ref" 


      Checked on DS log, there are some unindexed searches taking about 3 seconds, one is as below:

        "eventName": "DJ-LDAP",
        "client": {
          "ip": "",
          "port": 60014
        "server": {
          "ip": "",
          "port": 31389
        "request": {
          "protocol": "LDAP",
          "operation": "SEARCH",
          "connId": 347,
          "msgId": 8910,
          "dn": "ou=user,ou=managed,dc=openidm,dc=forgerock,dc=com",
          "scope": "one",
          "filter": "(fr-idm-managed-user-roles:nameAndOptionalJsonEqualityMatchingRule:=uid=role27,ou=role,ou=managed,dc=openidm,dc=forgerock,dc=com)",
          "attrs": [
        "transactionId": "ff07a83a-b8d6-49fb-857f-a79948eda2ff-14602554",
        "response": {
          "status": "SUCCESSFUL",
          "statusCode": "0",
          "elapsedTime": 3218,
          "elapsedTimeUnits": "MILLISECONDS",
          "additionalItems": {
            "unindexed": null
          "nentries": 4290
        "timestamp": "2020-08-25T00:16:20.129Z",
        "_id": "ff07a83a-b8d6-49fb-857f-a79948eda2ff-14602581"

      Attached are the ds access log with the query run and the output with command on the log:

      jq 'select(.response.elapsedTime>=1000)'  ldap-access.audit.json


      To reproduce the symptom using Pyforge, run the following command:

      ./run-pybot.py -n -v -c perf -s relationship.QueryRelationShipEndpoint -t IDM_Query_Relationship_Endpoint_Fields_Ref OpenIDM 

      then the query above.



      The same query returned in less or about one second using MySQL as repo.

      Check the query output between MySQL and DS(not the same but not significantly different):

      -rw-rw-r-- 1 testuser testuser  8253228 Aug 25 02:16 ds.out
      -rw-rw-r-- 1 testuser testuser  7591678 Aug 25 01:52 mysql.out 


        1. ds-log.zip
          1.79 MB
        2. jq.out
          10 kB

          Issue Links



              • Assignee:
                jason Jason Lemay
                Tinghua.Xu Tinghua Xu
              • Votes:
                0 Vote for this issue
                4 Start watching this issue


                • Created: