Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-13258

Significant recon throughput degradation introduced from the changes from OPENIDM-13101

    XMLWordPrintable

    Details

    • Target Version/s:
    • Verified Version/s:
    • Story Points:
      2
    • Sprint:
      2019.7 - IDM

      Description

      We see significant recon throughput degradation in some of our MySQL based recon tests, one test is recon from Managed to DJ:
      Before the changes from OPENIDM-13101, in commit 69fdc1d1a, we have: (Duration is in second and tps is throughput)

      20190518 01:49:32.243 - INFO - # success         : 100000
      20190518 01:49:32.243 - INFO - # current duration: 198.55
      20190518 01:49:32.244 - INFO - # current tps     : 503.651473
      

      Since OPENIDM-13101(commit 5d0e3fe), we have:

      20190518 01:40:15.663 - INFO - # failures        : 0
      20190518 01:40:15.663 - INFO - # success         : 5121
      20190518 01:40:15.663 - INFO - # current duration: 2048.73
      20190518 01:40:15.663 - INFO - # current tps     : 2.499597
      

      It's a significant drop.
      The symptom is seen in recon from IDM to DJ tests, and from DJ to IDM recon update tests(didn't see it in recon from DJ to IDM create tests though) and didn't see it in DJ based repo tests.

      The symptom can be seen from the very beginning of the test, for a long time after recon starts, little/no progress were made:

      20190518 01:13:16.614 - INFO - # failures        : 0
      20190518 01:13:16.614 - INFO - # success         : 1
      02520190518 01:13:16.614 - INFO - # current duration: 429.68
      20190518 01:13:16.614 - INFO - # current tps     : 2.385496
      

      To reproduce using PyForge:
      Change stress parameters as follow:

      NUM_USERS                  : "50",
      DURATION                   : "2000",
      

      and OpenIDM parameters:

            
      VERSION                   : "7.0.0-SNAPSHOT",
      JAVA_HOME                 : "/usr/java/jdk11",
      REPO_TYPE                 : "mysql",
      

      And OpenDJ

      VERSION                    : "6.5.0"
      

      and run the test as:

      python3 run-pybot.py -c stress -s idm.regular_recon.ReconManUserToLDAP -t IDM_Recon_Create_Managed_User_To_LDAP OpenIDM
      

        Attachments

          Activity

            People

            Assignee:
            jason.vincent jason vincent
            Reporter:
            Tinghua.Xu Tinghua Xu
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Dates

              Created:
              Updated:
              Resolved: