Uploaded image for project: 'OpenAM'
  1. OpenAM
  2. OPENAM-8023

OAuth2 load: contention at com.sun.identity.sm.ServiceConfigImpl.getInstance level (perf regression compared to AM12.0.2)

    Details

    • Support Ticket IDs:

      Description

      Following OPENAM-8004, we run again the same Oauth2 load with latest build including the fix for that bug.

      Performance looks a bit better now but throughput is going down pretty quickly (for instance CRUD AM read/sec goes down from 2.5k/sec 1k/sec in about 20 minutes, while we can easily sustain 2.8k/sec with AM12.0.2 during 10h)

      Doing jstack we see many threads stuck at

      "http-bio-8081-exec-998" #1518 daemon prio=5 os_prio=0 tid=0x00007f7a44836000 nid=0x20e2f waiting on condition [0x00007f792e99e000]
         java.lang.Thread.State: WAITING (parking)
      	at sun.misc.Unsafe.park(Native Method)
      	- parking to wait for  <0x0000000605bdfa28> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
      	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireQueued(AbstractQueuedSynchronizer.java:870)
      	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquire(AbstractQueuedSynchronizer.java:1199)
      	at java.util.concurrent.locks.ReentrantLock$NonfairSync.lock(ReentrantLock.java:209)
      	at java.util.concurrent.locks.ReentrantLock.lock(ReentrantLock.java:285)
      	at com.sun.identity.sm.ServiceConfigImpl.getInstance(ServiceConfigImpl.java:570)
      	at com.sun.identity.sm.ServiceConfigImpl.getInstance(ServiceConfigImpl.java:484)
      	at com.sun.identity.sm.ServiceConfigManagerImpl.getOrganizationConfig(ServiceConfigManagerImpl.java:257)
      	at com.sun.identity.sm.ServiceConfigManager.getOrganizationConfig(ServiceConfigManager.java:277)
      	at org.forgerock.openam.scripting.datastore.ScriptConfigurationDataStore.getOrgConfig(ScriptConfigurationDataStore.java:318)
      	at org.forgerock.openam.scripting.datastore.ScriptConfigurationDataStore.getSubOrgConfig(ScriptConfigurationDataStore.java:347)
      	at org.forgerock.openam.scripting.datastore.ScriptConfigurationDataStore.get(ScriptConfigurationDataStore.java:156)
      	at org.forgerock.openam.scripting.service.ScriptConfigurationService.get(ScriptConfigurationService.java:96)
      	at org.forgerock.openam.oauth2.OpenAMScopeValidator.getScriptConfiguration(OpenAMScopeValidator.java:388)
      	at org.forgerock.openam.oauth2.OpenAMScopeValidator.getOIDCClaimsExtensionScript(OpenAMScopeValidator.java:374)
      	at org.forgerock.openam.oauth2.OpenAMScopeValidator.getUserInfo(OpenAMScopeValidator.java:243)
      	at org.forgerock.openam.oauth2.OpenAMOAuth2ProviderSettings.getUserInfo(OpenAMOAuth2ProviderSettings.java:446)
      	at org.forgerock.openidconnect.UserInfoServiceImpl.getUserInfo(UserInfoServiceImpl.java:91)
      	at org.forgerock.openidconnect.restlet.UserInfo.getUserInfo(UserInfo.java:76)
      	at sun.reflect.GeneratedMethodAccessor102.invoke(Unknown Source)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:483)
      	at org.restlet.resource.ServerResource.doHandle(ServerResource.java:520)
      	at org.restlet.resource.ServerResource.get(ServerResource.java:742)
      	at org.restlet.resource.ServerResource.doHandle(ServerResource.java:617)
      	at org.restlet.resource.ServerResource.doNegotiatedHandle(ServerResource.java:678)
      	at org.restlet.resource.ServerResource.doConditionalHandle(ServerResource.java:356)
      	at org.restlet.resource.ServerResource.handle(ServerResource.java:1043)
      	at org.restlet.resource.Finder.handle(Finder.java:236)
      	at org.restlet.routing.Filter.doHandle(Filter.java:150)
      	at org.restlet.routing.Filter.handle(Filter.java:197)
      	at org.restlet.routing.Filter.doHandle(Filter.java:150)
      	at org.restlet.routing.Filter.handle(Filter.java:197)
      	at org.restlet.routing.Router.doHandle(Router.java:422)
      	at org.forgerock.openam.rest.service.RestletRealmRouter.doHandle(RestletRealmRouter.java:121)
      	at org.restlet.routing.Router.handle(Router.java:639)
      	at org.restlet.routing.Filter.doHandle(Filter.java:150)
      	at org.restlet.routing.Filter.handle(Filter.java:197)
      	at org.restlet.routing.Filter.doHandle(Filter.java:150)
      	at org.restlet.routing.Filter.handle(Filter.java:197)
      	at org.restlet.routing.Filter.doHandle(Filter.java:150)
      	at org.restlet.engine.application.StatusFilter.doHandle(StatusFilter.java:140)
      	at org.restlet.routing.Filter.handle(Filter.java:197)
      	at org.restlet.routing.Filter.doHandle(Filter.java:150)
      	at org.restlet.routing.Filter.handle(Filter.java:197)
      	at org.restlet.engine.CompositeHelper.handle(CompositeHelper.java:202)
      

      See full jstack in attachment

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jamesphillpotts James Phillpotts
                Reporter:
                sberthol Sebastien Bertholet [X] (Inactive)
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: