Uploaded image for project: 'OpenDJ'
  1. OpenDJ
  2. OPENDJ-1678

authrate: Contention at AuthenticatedUsers level

    Details

    • Type: Bug
    • Status: Done
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 2.6.2
    • Fix Version/s: Not applicable
    • Component/s: None
    • Labels:
      None

      Description

      Doing authrate load with the following command on a 8 cores box:
      sberthol@lab01-fr:bin$ ./authrate -p 3389 -D '%2$s' -w password -f -c 100 -b "ou=people,dc=openam,dc=forgerock,dc=com" -s one -g "rand(0,999)" "(uid=user.%d)"
      ------------------------------------------------------------------------------
      Throughput Response Time
      (ops/second) (milliseconds)
      recent average recent average 99.9% 99.99% 99.999% err/sec bind time %
      ------------------------------------------------------------------------------
      5731.4 5753.4 17.341 17.341 390.733 449.863 454.444 0.0 40.4
      10827.1 8532.1 9.037 11.570 304.966 413.519 454.444 0.0 67.2
      9646.0 8882.4 10.165 11.090 81.890 408.668 450.277 0.0 60.5
      9702.4 9078.7 10.133 10.845 70.554 400.285 450.277 0.0 60.5
      9622.5 9184.7 10.178 10.709 65.356 394.911 449.863 0.0 60.8
      9507.6 9237.4 10.303 10.641 62.975 390.733 449.863 0.0 60.8
      9623.5 9291.5 10.181 10.574 59.746 381.731 437.095 0.0 60.7
      9654.6 9336.1 10.139 10.519 56.674 374.471 437.095 0.0 60.6
      9538.9 9358.4 10.275 10.491 54.772 350.892 429.157 0.0 60.9
      9564.1 9378.7 10.251 10.467 53.650 347.238 429.157 0.0 60.8
      9534.7 9392.7 10.272 10.449 53.923 340.185 422.039 0.0 61.0

      => we reach a throughput of about 9.5K auths/sec but the cpu stays 30% idle.

      Looking at a DJ jstack (in attachment), we see that most workers are blocked at AuthenticatedUsers level, trying to put or remove users.

      We might want to try to optimize this part of the code if possible (may have a significant impact on OpenAM auth load for big deployments)

      $ grep AuthenticatedUsers /tmp/jstack.authrate.contention.AuthenticatedUsers
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:121)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.remove(AuthenticatedUsers.java:137)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)
      at org.opends.server.core.AuthenticatedUsers.put(AuthenticatedUsers.java:103)

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                matthew Matthew Swift
                Reporter:
                sberthol Sebastien Bertholet [X] (Inactive)
                Dev Assignee:
                Matthew Swift
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: