Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-10079

Slower sync operation when external DJ is used as repo

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: OpenIDM 6.0.0
    • Fix Version/s: OpenIDM 6.0.0
    • Labels:
    • Environment:
      IDM 6.0.0 master runs on a CentOS7 with DJ 6.0.0 M1 as repo and external resource, Java 8.

      Description

      With MySQL as IDM repo, we can perform 100 users/second over 300 seconds period in the sync create from managed to DJ and got good response times. With external DJ as repo, we not do 50 users/second(either requests timeout or high response times in case requests don't timeout). Attached are data from a case requests didn't timeout but response times are high

      ---- Global Information --------------------------------------------------------
      > request count                                      15000 (OK=15000  KO=0     )
      > min response time                                     36 (OK=36     KO=-     )
      > max response time                                  50028 (OK=50028  KO=-     )
      > mean response time                                  8537 (OK=8537   KO=-     )
      > std deviation                                      13658 (OK=13658  KO=-     )
      > response time 50th percentile                        211 (OK=211    KO=-     )
      > response time 95th percentile                      41175 (OK=41174  KO=-     )
      > response time 99th percentile                      48875 (OK=48875  KO=-     )
      > response time 99.9th percentile                    49832 (OK=49832  KO=-     )
      > mean requests/sec                                  42.98 (OK=42.98  KO=-     )
      ---- Response Time Distribution ------------------------------------------------
      > t < 800 ms                                          8695 ( 58%)
      > 800 ms < t < 1200 ms                                 135 (  1%)
      > t > 1200 ms                                         6170 ( 41%)
      > failed                                                 0 (  0%)
      

      Noticed that when requests timeout, access(e.g. query to get all managed users) would hang.

      To reproduce using Pyforge:
      Do necessary setup(num_users, duration, IDM repo) and run test
      python -u run-pybot.py -c stress -i gatling -s *SyncManUsersToLdap -t sync_create OpenIDM

        Attachments

        1. dj_logs.zip
          4.76 MB
        2. openidm.restapi.CreatePut_20180129-182544.out
          50 kB
        3. openidm0.log.0
          55 kB
        4. sync.jstack
          3.76 MB
        5. sync.metrics
          10 kB

          Issue Links

            Activity

              People

              Assignee:
              jim.mitchener Jim Mitchener
              Reporter:
              Tinghua.Xu Tinghua Xu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: