Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-11601

OpenIDM Needs to Rework how it Queries the repo for Assignments and Roles

    XMLWordPrintable

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: OpenIDM 5.0.0, OpenIDM 6.0.0
    • Fix Version/s: 7.0.0
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Verified Version/s:
    • Story Points:
      5
    • Sprint:
      2019.16 - IDM, 2019.17 - IDM, 2020.01 - IDM, 2020.02 - IDM, 2020.03 - IDM, 2020.04 - IDM, 2020.05 - IDM, 2020.06 - IDM, 2020.07 - IDM
    • Support Ticket IDs:

      Description

      Description:
      We are seeing poor performance in our load testing of IDM 6.0 with Postgresql as the repo. We are testing with about 70k users.

      Troubleshooting:
      In looking at the performance tuning information in Postgresql and using p6spy (https://github.com/p6spy/p6spy) we noticed a couple of things that seem to be contributing to the poor performance.

      1. IDM is querying for conditional roles, individual role, and individual assignments hundreds of thousands of times each during the course of a recon (on average 8-10 queries per user). We reviewed the queries with our DBA but he wasn't able to offer any help in speeding this up.

      2. When a user is being recon'd from Banner to managed/user, the onRecon.groovy script does not query for all assignments so the implicit syncs that are triggered to the downstream systems like AD have to make an additional query for their assignments.

      Request:
      Reduce the number of unnecessary queries IDM makes to the repo through caching or another mechanism so IDM can scale better.

      Blockage:
      Can't proceed in our efforts to upgrade to OpenIDM 6.0 with the current performance levels. Our workaround looks like it may work.

      Workaround:
      Role and Assignment data is static for us and changes maybe once every six months or so.
      1. We have a script now that will cache this static data into the properties section of script.json.
      2. onRecon.groovy doesn't do anything any longer.
      3. conditionalRoles.js, effectiveRoles.js, and effectiveAssignments.js now use the global variables from script.json instead of querying the repo.

        Attachments

          Issue Links

            Activity

              People

              Assignee:
              dhogan Dirk Hogan
              Reporter:
              jeremy.barras Jeremy Barras [X] (Inactive)
              Votes:
              0 Vote for this issue
              Watchers:
              11 Start watching this issue

                Dates

                Created:
                Updated:
                Resolved: