Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-13985

Performance degradation in creating managed users introduce by commit 015ba931

    Details

    • Target Version/s:
    • Verified Version/s:
    • Story Points:
      3
    • Sprint:
      2020.05 - IDM, 2020.06 - IDM

      Description

      Noticed about 6% throughput drop and 7% mean response times drop in creating simple managed users using PUT test introduced by the commit 015ba931:
      Before the commit 015ba931, Gatling tool has the following report for the test:

      ---- Global Information --------------------------------------------------------
      > request count                                     163199 (OK=163199 KO=0     )
      > min response time                                     27 (OK=27     KO=-     )
      > max response time                                    555 (OK=555    KO=-     )
      > mean response time                                    55 (OK=55     KO=-     )
      > std deviation                                         13 (OK=13     KO=-     )
      > response time 50th percentile                         53 (OK=53     KO=-     )
      > response time 95th percentile                         74 (OK=74     KO=-     )
      > response time 99th percentile                         90 (OK=90     KO=-     )
      > response time 99.9th percentile                      138 (OK=138    KO=-     )
      > mean requests/sec                                542.189 (OK=542.189 KO=-     )
      ---- Response Time Distribution ------------------------------------------------
      > t < 800 ms                                        163199 (100%)
      > 800 ms < t < 1200 ms                                   0 (  0%)
      > t > 1200 ms                                            0 (  0%)
      > failed                                                 0 (  0%)
      

      With commit 015ba931: We have

      ---- Global Information --------------------------------------------------------
      > request count                                     152746 (OK=152746 KO=0     )
      > min response time                                     30 (OK=30     KO=-     )
      > max response time                                    559 (OK=559    KO=-     )
      > mean response time                                    59 (OK=59     KO=-     )
      > std deviation                                         13 (OK=13     KO=-     )
      > response time 50th percentile                         57 (OK=57     KO=-     )
      > response time 95th percentile                         78 (OK=78     KO=-     )
      > response time 99th percentile                         97 (OK=97     KO=-     )
      > response time 99.9th percentile                      157 (OK=157    KO=-     )
      > mean requests/sec                                507.462 (OK=507.462 KO=-     )
      ---- Response Time Distribution ------------------------------------------------
      > t < 800 ms                                        152746 (100%)
      > 800 ms < t < 1200 ms                                   0 (  0%)
      > t > 1200 ms                                            0 (  0%)
      > failed                                                 0 (  0%)
      

      Note: The test used the external DJ(latest DJ master build) as repo, if the changes are not repo dependent, the issue should be applicable to JDBC repo as well(see degradation for JDBC repo as well in Jenkins test job runs)

      To reproduce the symptom using Pyforge:
      1. Set Pyforge configurations as below:

      [Stress]
      num_users = 10
      duration = 60
      concurrency = 10
      max_throughput = -1
      test_duration = 300
      preload_users = 20
      preload_duration = 300
      test_concurrency = 30
      
      [OpenIDM]
      host_name = ${Default:host_name}
      java_home = /usr/java/jdk11
      java_args = -Xms6g -Xmx6g
      version = 7.0.0-SNAPSHOT
      previous_version = 6.5.0
      protocol = http
      repo_type = external_dj
      use_docker_repo = False
      jvm_debug_options = ${Default:jvm_debug_options}
      cloud = ${Default:cloud}
      enable_metrics = True
      enable_policy_enforcement = False
      enable_jetty_config = False
      
      [IDMExternalDJ]
      host_name = ${Default:host_name}
      java_home = ${Default:java_home}
      java_args = ${Default:java_args}
      version = 7.0.0-SNAPSHOT
      ldap_port = 31389
      base_dn = "dc=openidm,dc=forgerock,dc=com"
      admin_port = 34444
      

      2. run the test as:

      run-pybot.py -v -c perf -s simple_managed_user.CreateManUsersPut OpenIDM
      

      3. Check results in pyforge/latest/debug.txt

      Attached are the metrics for the two("good"/"bad") commits.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cgdrake Chris Drake
                Reporter:
                Tinghua.Xu Tinghua.Xu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: