Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-10048

Low throughput in recon create when external DJ is used as IDM repo

    XMLWordPrintable

    Details

      Description

      When external DJ is used as repo, recon create from DJ to managed(external DJ as repo) has low throughput, it's about 20 users/second. It's very low if compared with MySQL as repo and in the same test env, where the throughput is about 360 users/second.
      The test reconciles 200K users from DJ to IDM managed, attached are IDM log from the test and the metrics collected halfway during the recon(around 50K users reconciled), the debug.txt has the recon progress if needed.

      To reproduce using Pyforge:
      1. config config.cfg with appropriate parameters(test 50K users is good enough) and IDM repo type.
      2. run test:
      python -u run-pybot.py -c stress -s *ReconLDAPToManUser -t check_idm_time_and_throughput_in_recon_create OpenIDM
      Notes:
      1. Create with PUT and create with POST on managed users with external DJ as rep can have dispatch rate/throughput 300 users/second in the same test env.
      2. Test shows the recon didn't make progress during some 3*33 seconds period.
      3. recon from managed(external DJ as repo) to DJ showed similar slowness(throughput around 20 users/second).
      4. recon update looks fine when used external DJ as repo, from DJ to managed has 430 users/second throughput, from managed to DJ has throughput 630 users/second
      5. The external DJ repo process has a high CPU usage all the time(1266%), see attached screenshot.

        Attachments

        1. debug.txt
          294 kB
        2. openidm0.log.0
          88 kB
        3. recon-at-45-minutes.metrics
          13 kB
        4. Screen Shot 2018-01-22 at 1.55.56 PM.png
          Screen Shot 2018-01-22 at 1.55.56 PM.png
          501 kB

          Issue Links

            Activity

              People

              Assignee:
              jason Jason Lemay
              Reporter:
              Tinghua.Xu Tinghua Xu
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: