Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-13532

Under load IDM container may fail to respond to requests

    Details

    • Target Version/s:
    • Story Points:
      3

      Description

      Using the simple managed users test from Jenny at https://stash.forgerock.org/projects/QA/repos/lodestar/pull-requests/298/overview, we see that IDM container may fail to respond to http requests under certain load.
      One request is to read managed users, the Gatling simulation has a concurrency 40 and test duration 900 seconds, here is one instance of the following failure(from Gatling report):

      19:57:25.505 [WARN ] i.g.h.a.ResponseProcessor - Request 'Managed user Read' failed: status.find.in(200,304,201,202,203,204,205,206,207,208,209), but actually found 404
19:57:25.520 [DEBUG] i.g.h.a.ResponseProcessor -
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
Managed user Read: KO status.find.in(200,304,201,202,203,204,205,206,207,208,209), but actually found 404
=========================
Session:
Session(Managed user Read,14,Map(email -> H-M-11248@example.com, gatling.http.cache.dns -> io.gatling.http.resolver.ShuffleJdkNameResolver@67ebc807, 4db0359d-94a5-49e0-8b43-d92245c61463 -> 2261, username -> H-M-11248, id -> 626e36f2-11d0-4a48-8abf-7c7d56a05cfa, givenname -> Mbwozbno, timestamp.4db0359d-94a5-49e0-8b43-d92245c61463 -> 1562183630532, familyname -> Hrciso, gatling.http.cookies -> CookieJar(Map(CookieKey(route,tinghuaxu.iam.performance-team.forgeops.com,/openidm) -> StoredCookie(route=1562183631.056.10828.74002; path=/openidm; secure; HTTPOnly,true,false,1562183631136), CookieKey(session-jwt,tinghuaxu.iam.performance-team.forgeops.com,/) -> StoredCookie(session-jwt=eyJ0eXAiOiJKV1QiLCJraWQiOiJvcGVuaWRtLWp3dHNlc3Npb25obWFjLWtleSIsImN0eSI6IkpXVCIsImFsZyI6IkhTMjU2In0.ZXlKMGVYQWlPaUpLVjFRaUxDSnJhV1FpT2lKdmNHVnVhV1J0TFd4dlkyRnNhRzl6ZENJc0ltVnVZeUk2SWtFeE1qaERRa010U0ZNeU5UWWlMQ0poYkdjaU9pSlNVMEV4WHpVaWZRLk9mWlUxb2ROVTlqRWJRbkNrX0Uzd3hqcHR5MVAwT0JWcmNEVU5veDRndHNBSkdOYXdRa1F2cVpMM3YyODl3QmVJbXk4UDZFZ0FCc01sdHJ0SS1MUjc2Z2JIc3RCa09YTFd5dTJJRU5sdkNKMDkwY3piMXlCanF6VjZsVVZSckJGdWJtd0U0OHNONWtRdGhTTXI2SE16bG9xendoS1VTbk9rUVRLaTRIQnRnREFnSzQ3UTk1OGVTSkxrUWxjZTM5eTNXUjdZZGUzSzVjeVhVQkZZZDJnZzl6eUpKeFBoRkJPWlFnVGx0RTktaE1waGtFdUUyVE8zcFJ3ek9JUmRic19HZm9VZ2Y3a05yRzlJbDRMSGdPN1NoYXBRV04zUlhYZVZpWHhkU25BUFdfLXJndVVRRE5oaE12YWJfN1lmcDNVUlFHaEptSFN1ZHpSR2lFMW9kNjZBdy5mMGZ0Q2VhWDZ4MGt6ZzVXVzA4Uk5BLjBoek1YS2x3RV9Uem1LbGdnRlFmZkMwLVAtSlF0VWtiTWV6T2Y0Q1pRd05OZFd2VzhyMF9YWEpkQllLUnFlaVo1blhoRWtndHUyOUhVekJLa0tXRVpjc3pmelZabGVkSEdhcGVTeE5FNm94WTF0WnpFMC14bTlNMWE4QWd6QTFfbmdlVFBRb2VnYnhOMGRXVG5NUUxuYlJUTVpseElpcUVKd2VKVXZMQ09sbXgycGs2RUdXTzNtOXQ4UGtiZlBIcmNuR3g5QWdPdUJvSVlhT1NHRzZWa1RyVnF3MEk1RzBwamJFa2RXMzZzX3dLaVBxZDBzc3VYMll5TDJ4d21FczBvVXN3cTVIM2UtbVlVUFNBbFV0SGpkNEFIakNPYkxqSFVCRVJvYzdzYXJLUUVNanR1YWxISzF1ZXdWUkVsVkdhN091Y2stS3laelBxUUhGRmk3Zl8zcGNXeDBrZW05YmxQbkV2UFM0Rl80N0tyRHR5VVZCYUFJN2FnOW5UNl9xNkRYZWNINmE1TUhVOXNVVVJvRENhQ0g4YU9TMTlobS1HbUFkeWhzdTZ6ZlYwMEVIaWg1NzZYYWExWnNPN2s2WGd5ME0xeVRTUVpQZEYwUXAyX0JSV3E0VWhaUTJaaDltVzZSSmZsY1VMNUJMRmxabi1tN1ZJT0IzX1ZxOGI3ejhtMGFpakREMkVnaFBGQ2daRmxwNE84TFNTSlljalg1TlZJUmJoX0oxNG9WMEFpVmVnd2JrRVpUZk9tR1B4WVFnZ3FzZTdGOVhrd1dsNFFYUWt4ZGh2Y216dVY0bmo4V3RRd1BMTktHcXpkenZTQnhDeXotNWE0WVlsLXlfOTRCMThRM3FGNFdGbjNLamZ1eHhkX3lFTlpRMVB5aHhoZHdneHlsYjU3ekgzd3hUeEFQbzVpYmR1Z2lZU1JYX0pKLTJyQUV3X1FYYlNjWVFMSkp4RnFBSTd4cnlZMHdIVUY3TXZSZ09lTDU2Q1lvWXRVLXQ5TVkxT2RaQVZmejBVcVdqX29jd1VxT1ZtQlY4VE5TS3pfU25qbHJNd0Y3djZpbWhUTmdIbzc5THdBMzdWbHllNjExR2VmTGhHRjh3bWs4NzEudDA2ckZwXzlpSUR6b3Q5WmZGQ3BRQQ.9Ka3aB2Yl9n9a3zz_XWecigLtcryPAsUVdyVtT_3IAQ; path=/; HTTPOnly,true,false,1562183813977)))),1562183630532,234,KO,List(ExitAsapLoopBlock(4db0359d-94a5-49e0-8b43-d92245c61463,io.gatling.core.session.package$RichExpression$$$Lambda$379/1268974304@1e7135e7,io.gatling.core.action.Exit@7881702c)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$440/1482429406@1825de23)=========================
HTTP request:
GET https://tinghuaxu.iam.performance-team.forgeops.com:443/openidm/managed/user/626e36f2-11d0-4a48-8abf-7c7d56a05cfa
headers=
Content-Type: application/json
User-Agent: Robot/Gatling
X-OpenIDM-Username: openidm-admin
X-OpenIDM-Password: openidm-admin
Cookie: route=1562183631.056.10828.74002; session-jwt=eyJ0eXAiOiJKV1QiLCJraWQiOiJvcGVuaWRtLWp3dHNlc3Npb25obWFjLWtleSIsImN0eSI6IkpXVCIsImFsZyI6IkhTMjU2In0.ZXlKMGVYQWlPaUpLVjFRaUxDSnJhV1FpT2lKdmNHVnVhV1J0TFd4dlkyRnNhRzl6ZENJc0ltVnVZeUk2SWtFeE1qaERRa010U0ZNeU5UWWlMQ0poYkdjaU9pSlNVMEV4WHpVaWZRLk9mWlUxb2ROVTlqRWJRbkNrX0Uzd3hqcHR5MVAwT0JWcmNEVU5veDRndHNBSkdOYXdRa1F2cVpMM3YyODl3QmVJbXk4UDZFZ0FCc01sdHJ0SS1MUjc2Z2JIc3RCa09YTFd5dTJJRU5sdkNKMDkwY3piMXlCanF6VjZsVVZSckJGdWJtd0U0OHNONWtRdGhTTXI2SE16bG9xendoS1VTbk9rUVRLaTRIQnRnREFnSzQ3UTk1OGVTSkxrUWxjZTM5eTNXUjdZZGUzSzVjeVhVQkZZZDJnZzl6eUpKeFBoRkJPWlFnVGx0RTktaE1waGtFdUUyVE8zcFJ3ek9JUmRic19HZm9VZ2Y3a05yRzlJbDRMSGdPN1NoYXBRV04zUlhYZVZpWHhkU25BUFdfLXJndVVRRE5oaE12YWJfN1lmcDNVUlFHaEptSFN1ZHpSR2lFMW9kNjZBdy5mMGZ0Q2VhWDZ4MGt6ZzVXVzA4Uk5BLjBoek1YS2x3RV9Uem1LbGdnRlFmZkMwLVAtSlF0VWtiTWV6T2Y0Q1pRd05OZFd2VzhyMF9YWEpkQllLUnFlaVo1blhoRWtndHUyOUhVekJLa0tXRVpjc3pmelZabGVkSEdhcGVTeE5FNm94WTF0WnpFMC14bTlNMWE4QWd6QTFfbmdlVFBRb2VnYnhOMGRXVG5NUUxuYlJUTVpseElpcUVKd2VKVXZMQ09sbXgycGs2RUdXTzNtOXQ4UGtiZlBIcmNuR3g5QWdPdUJvSVlhT1NHRzZWa1RyVnF3MEk1RzBwamJFa2RXMzZzX3dLaVBxZDBzc3VYMll5TDJ4d21FczBvVXN3cTVIM2UtbVlVUFNBbFV0SGpkNEFIakNPYkxqSFVCRVJvYzdzYXJLUUVNanR1YWxISzF1ZXdWUkVsVkdhN091Y2stS3laelBxUUhGRmk3Zl8zcGNXeDBrZW05YmxQbkV2UFM0Rl80N0tyRHR5VVZCYUFJN2FnOW5UNl9xNkRYZWNINmE1TUhVOXNVVVJvRENhQ0g4YU9TMTlobS1HbUFkeWhzdTZ6ZlYwMEVIaWg1NzZYYWExWnNPN2s2WGd5ME0xeVRTUVpQZEYwUXAyX0JSV3E0VWhaUTJaaDltVzZSSmZsY1VMNUJMRmxabi1tN1ZJT0IzX1ZxOGI3ejhtMGFpakREMkVnaFBGQ2daRmxwNE84TFNTSlljalg1TlZJUmJoX0oxNG9WMEFpVmVnd2JrRVpUZk9tR1B4WVFnZ3FzZTdGOVhrd1dsNFFYUWt4ZGh2Y216dVY0bmo4V3RRd1BMTktHcXpkenZTQnhDeXotNWE0WVlsLXlfOTRCMThRM3FGNFdGbjNLamZ1eHhkX3lFTlpRMVB5aHhoZHdneHlsYjU3ekgzd3hUeEFQbzVpYmR1Z2lZU1JYX0pKLTJyQUV3X1FYYlNjWVFMSkp4RnFBSTd4cnlZMHdIVUY3TXZSZ09lTDU2Q1lvWXRVLXQ5TVkxT2RaQVZmejBVcVdqX29jd1VxT1ZtQlY4VE5TS3pfU25qbHJNd0Y3djZpbWhUTmdIbzc5THdBMzdWbHllNjExR2VmTGhHRjh3bWs4NzEudDA2ckZwXzlpSUR6b3Q5WmZGQ3BRQQ.9Ka3aB2Yl9n9a3zz_XWecigLtcryPAsUVdyVtT_3IAQ
Host: tinghuaxu.iam.performance-team.forgeops.com
Accept: */*
=========================
HTTP response:
status=
404 Not Found
headers=
Server: nginx/1.15.10
Date: Wed, 03 Jul 2019 19:57:25 GMT
Content-Type: application/json;charset=utf-8
Content-Length: 162
Connection: keep-alive
Cache-Control: no-cache
Content-API-Version: protocol=2.1,resource=1.0
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=15724800; includeSubDomains

body=
{"code":404,"reason":"Not Found","message":"No Results Returned: The search request succeeded but did not return any search result entries when one was expected"}
<<<<<<<<<<<<<<<<<<<<<<<<<

      When manually run curl command on the managed user, the user is actually there:

      curl -k  -H "X-OpenIDM-Username: openidm-admin" -H "X-OpenIDM-Pasiam.performance-team.forgeops.com:443/openidm/managed/user/626e36f2-11d0-4a48-8abf-7c7d56a05cfa |python -mjson.tool
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100   296  100   296    0     0    161      0  0:00:01  0:00:01 --:--:--   161
{
    "_id": "626e36f2-11d0-4a48-8abf-7c7d56a05cfa",
    "_rev": "00000000ad033761",
    "accountStatus": "active",
    "description": "Managed User",
    "effectiveAssignments": [],
    "effectiveRoles": [],
    "givenName": "Mbwozbno",
    "mail": "H-M-11248@example.com",
    "sn": "Hrciso",
    "telephoneNumber": "444-444-4444",
    "userName": "H-M-11248"
}

      Here is an instance with create managed user using POST:

      ================================================================================

20:34:42.873 [WARN ] i.g.h.a.ResponseProcessor - Request 'Managed user create put' failed: status.find.in(200,304,201,202,203,204,205,206,207,208,209), but actually found 400
20:34:42.886 [DEBUG] i.g.h.a.ResponseProcessor - 
>>>>>>>>>>>>>>>>>>>>>>>>>>
Request:
Managed user create put: KO status.find.in(200,304,201,202,203,204,205,206,207,208,209), but actually found 400
=========================
Session:
Session(Managed user create put,4,Map(city -> city1, a36f95dd-3930-4a11-96a3-d6c99bda2d37 -> 2270, email -> H-T-276933@example.com, gatling.http.cache.dns -> io.gatling.http.resolver.ShuffleJdkNameResolver@4f8d42e8, username -> H-T-276933, description -> This is the description for Hpjihu Tbnjazlt, country -> usa, fullname -> Hpjihu Tbnjazlt, roles -> [], givenname -> Tbnjazlt, manager -> [], postalcode -> 27195, timestamp.a36f95dd-3930-4a11-96a3-d6c99bda2d37 -> 1562185700383, familyname -> Hpjihu, gatling.http.cookies -> CookieJar(Map(CookieKey(route,tinghuaxu.iam.performance-team.forgeops.com,/openidm) -> StoredCookie(route=1562185700.668.10826.450737; path=/openidm; secure; HTTPOnly,true,false,1562185700819), CookieKey(session-jwt,tinghuaxu.iam.performance-team.forgeops.com,/) -> StoredCookie(session-jwt=eyJ0eXAiOiJKV1QiLCJraWQiOiJvcGVuaWRtLWp3dHNlc3Npb25obWFjLWtleSIsImN0eSI6IkpXVCIsImFsZyI6IkhTMjU2In0.ZXlKMGVYQWlPaUpLVjFRaUxDSnJhV1FpT2lKdmNHVnVhV1J0TFd4dlkyRnNhRzl6ZENJc0ltVnVZeUk2SWtFeE1qaERRa010U0ZNeU5UWWlMQ0poYkdjaU9pSlNVMEV4WHpVaWZRLllWWWZGTmNNWVR5RlMteXB6Q3ViVjFPa2Y3RjVIZ2RQay0zYWNNazF0TkVGV3RUaVpCUXVNMFVxYU9fc0pTNXhtNHZCZWRYbDcwR3FuS01TV1VlQWFrc0dRQ1Q0LUZtMmdkYUYxX29oQTlsMTF5OVJKYm82RnFRaG44VlpYRXR4YjdNYnJscTdMRTA4LUk2dTBiUDVHbUloMXdycWhTc1hfRTA2VDRCcGtIUENEdUVXUUNtcHE5QUttN2ZDMkk0VVEyUUhWcnNTMUFDeWlqNXg3Tkw1UmhvTEpBQkZrbjhWTWRlZzZqSDQweHM0VkpzSTRVOXV6bUFxVlhuQU9QN2w5X0U1dHlnc3JSNGQyaWxXMXJKVW9jMy1lclBLOVlRS2xqZEY1MWRKQnhUNUdPTVA3Zmk2eGl0Vm9LY3VQazFicUJuNXhQX1A4OFJPMExfN0hTNkJoZy4yeHRjYkdSWlY1elBqOVROTmVtNUNRLmRlVFByNU1MWUNGZTlFTU9fS0NzQkpFMkt2TGptbnRRek9hbldwREtPTkUteEVjR2FFMGdmTXF6Rjd2enFqWGdTNlRyZ2twcHZLVkxpbzQ5Z3p4Yi1YbTdyVGp1S3IzQTI4TWtHV3EzRkVmR2lIb2NRUTNUQVVndWZvbW1jSWFaMkF1Z0ZjR2ZqQnhZeGRGTkp4VHZoMlZFb1Q2cTN5bkc0VFRiNzBzeWMzVzNoRlZ5TlRxaEhrRGdqZnVtMEZ6b2JycklwUmg4UjhSbkRTdHBvaVloT1FYOGVaVHM1c0pqd3lPVHhlSHBMTDVERjhBc3JfQm53eTdKYlE1S1pQN0RZTmZvaVdlQ0p4RkhtdDQ3VXg1VXpCSmlTQy00N3pPRTZtNWQ4NngtblRjTW5NLWwzWmw5N19GVnFodG0zZ3dpZ2tLNXd0VTlNLXE0aFdTang3Z1VIQTg4ZFJ5TGhsSXhrdXBkRmMzdXNrZEFXQ2IwN1JrWXkyLU1HWTlJNzRfSHlJb0h1M2ZLZ0dmT3JHS2VpSjg4RW9oX2Y5QUxZenZsaXlwalhpREQ0TmU4LUJOYlAwV3dGZ0dpZm5SdVd1TUY1Ymt4S21lN1l4VWpxWHhJcTZTbU9aaGcyd1JUVVpXQURvdVV5bzFSbGlsQXgxZWNjbU1jNGFoQndRR2g0UzU5UGZIQVdhQk1ZRkRxd0Y3cDdkaThjX3lQVFBiLVN6bjRCZnE5QnhFQjZEMGpnMEFDcDlBa3pZaGViUFFpUmEySS1sZ2t3WW5kQTZDNXZJclJqYWtqeE5KODFYa1ZsMmRCcWpZVG5PMTR2ZFZmWGVYWGdnX1RRUzBENmw5ZURIVFlJS0FQanlPcTJ5T0VRRDA4eV90RjJBVW1HQVloQ1p2ZHVaVWItNVdBMnZSN1RVM3Y1b3RBODhIYjctT0NEdUE1elZYUmxEaU1DM2FvM3haRXR1RU9CeVRXeVdQYnNsUUdwdUNoWDNEcUxmNTJ5UkZmOTFzX0V1elpENy13a2pnNGRUN19zVVl1OWdaaFRVVVRvTzI2RzFBVkVPTDJQbDIxak1uZnJwV3JGWllPbkxnd2pGbldEOE15U0lKUFd0al8ucC10Y3p0SU1zX0taR0prSzBWV2VsUQ.kOFMcHAihN1TIOscUMvAvn1EI9OdwY4wr_alR4PRbKU; path=/; HTTPOnly,true,false,1562186066026))), password -> Pa_ssw0rd),1562185700383,341,KO,List(ExitAsapLoopBlock(a36f95dd-3930-4a11-96a3-d6c99bda2d37,io.gatling.core.session.package$RichExpression$$$Lambda$387/1108517064@62bf58c4,io.gatling.core.action.Exit@fcbb5e4)),io.gatling.core.protocol.ProtocolComponentsRegistry$$Lambda$444/1622863652@43174d34)
=========================
HTTP request:
POST https://tinghuaxu.iam.performance-team.forgeops.com:443/openidm/managed/user/?_action=create
headers=
X-OpenIDM-Password: openidm-admin
X-OpenIDM-Username: openidm-admin
Accept: application/json
Content-Type: application/json
User-Agent: Robot/Gatling
Cookie: route=1562185700.668.10826.450737; session-jwt=eyJ0eXAiOiJKV1QiLCJraWQiOiJvcGVuaWRtLWp3dHNlc3Npb25obWFjLWtleSIsImN0eSI6IkpXVCIsImFsZyI6IkhTMjU2In0.ZXlKMGVYQWlPaUpLVjFRaUxDSnJhV1FpT2lKdmNHVnVhV1J0TFd4dlkyRnNhRzl6ZENJc0ltVnVZeUk2SWtFeE1qaERRa010U0ZNeU5UWWlMQ0poYkdjaU9pSlNVMEV4WHpVaWZRLllWWWZGTmNNWVR5RlMteXB6Q3ViVjFPa2Y3RjVIZ2RQay0zYWNNazF0TkVGV3RUaVpCUXVNMFVxYU9fc0pTNXhtNHZCZWRYbDcwR3FuS01TV1VlQWFrc0dRQ1Q0LUZtMmdkYUYxX29oQTlsMTF5OVJKYm82RnFRaG44VlpYRXR4YjdNYnJscTdMRTA4LUk2dTBiUDVHbUloMXdycWhTc1hfRTA2VDRCcGtIUENEdUVXUUNtcHE5QUttN2ZDMkk0VVEyUUhWcnNTMUFDeWlqNXg3Tkw1UmhvTEpBQkZrbjhWTWRlZzZqSDQweHM0VkpzSTRVOXV6bUFxVlhuQU9QN2w5X0U1dHlnc3JSNGQyaWxXMXJKVW9jMy1lclBLOVlRS2xqZEY1MWRKQnhUNUdPTVA3Zmk2eGl0Vm9LY3VQazFicUJuNXhQX1A4OFJPMExfN0hTNkJoZy4yeHRjYkdSWlY1elBqOVROTmVtNUNRLmRlVFByNU1MWUNGZTlFTU9fS0NzQkpFMkt2TGptbnRRek9hbldwREtPTkUteEVjR2FFMGdmTXF6Rjd2enFqWGdTNlRyZ2twcHZLVkxpbzQ5Z3p4Yi1YbTdyVGp1S3IzQTI4TWtHV3EzRkVmR2lIb2NRUTNUQVVndWZvbW1jSWFaMkF1Z0ZjR2ZqQnhZeGRGTkp4VHZoMlZFb1Q2cTN5bkc0VFRiNzBzeWMzVzNoRlZ5TlRxaEhrRGdqZnVtMEZ6b2JycklwUmg4UjhSbkRTdHBvaVloT1FYOGVaVHM1c0pqd3lPVHhlSHBMTDVERjhBc3JfQm53eTdKYlE1S1pQN0RZTmZvaVdlQ0p4RkhtdDQ3VXg1VXpCSmlTQy00N3pPRTZtNWQ4NngtblRjTW5NLWwzWmw5N19GVnFodG0zZ3dpZ2tLNXd0VTlNLXE0aFdTang3Z1VIQTg4ZFJ5TGhsSXhrdXBkRmMzdXNrZEFXQ2IwN1JrWXkyLU1HWTlJNzRfSHlJb0h1M2ZLZ0dmT3JHS2VpSjg4RW9oX2Y5QUxZenZsaXlwalhpREQ0TmU4LUJOYlAwV3dGZ0dpZm5SdVd1TUY1Ymt4S21lN1l4VWpxWHhJcTZTbU9aaGcyd1JUVVpXQURvdVV5bzFSbGlsQXgxZWNjbU1jNGFoQndRR2g0UzU5UGZIQVdhQk1ZRkRxd0Y3cDdkaThjX3lQVFBiLVN6bjRCZnE5QnhFQjZEMGpnMEFDcDlBa3pZaGViUFFpUmEySS1sZ2t3WW5kQTZDNXZJclJqYWtqeE5KODFYa1ZsMmRCcWpZVG5PMTR2ZFZmWGVYWGdnX1RRUzBENmw5ZURIVFlJS0FQanlPcTJ5T0VRRDA4eV90RjJBVW1HQVloQ1p2ZHVaVWItNVdBMnZSN1RVM3Y1b3RBODhIYjctT0NEdUE1elZYUmxEaU1DM2FvM3haRXR1RU9CeVRXeVdQYnNsUUdwdUNoWDNEcUxmNTJ5UkZmOTFzX0V1elpENy13a2pnNGRUN19zVVl1OWdaaFRVVVRvTzI2RzFBVkVPTDJQbDIxak1uZnJwV3JGWllPbkxnd2pGbldEOE15U0lKUFd0al8ucC10Y3p0SU1zX0taR0prSzBWV2VsUQ.kOFMcHAihN1TIOscUMvAvn1EI9OdwY4wr_alR4PRbKU
Content-Length: 281
Host: tinghuaxu.iam.performance-team.forgeops.com
compositeByteData={
		    "userName" : "H-T-276933",
		    "givenName" : "Tbnjazlt",
            "sn" : "Hpjihu",
            "mail" : "H-T-276933@example.com",
            "telephoneNumber" : "444-444-4444",
            "password" : "Pa_ssw0rd",
            "description" : "Managed User"
        }
=========================
HTTP response:
status=
400 Bad Request
headers= 
Server: nginx/1.15.10
Date: Wed, 03 Jul 2019 20:34:42 GMT
Content-Type: application/json;charset=utf-8
Content-Length: 120
Connection: keep-alive
Cache-Control: no-cache
Content-API-Version: protocol=2.1,resource=1.0
X-Content-Type-Options: nosniff
Strict-Transport-Security: max-age=15724800; includeSubDomains

body=
{"code":400,"reason":"Bad Request","message":"The referenced object 'internal/role/openidm-authorized', does not exist"}

      I tried to create the managed users manually and it was ok:

      curl -k  -H "Content-Type: application/json" -H "X-OpenIDM-Username: openidm-admin" -H "X-OpenIDM-Password: openidm-admin" --request POST https://tinghuaxu.iam.performance-team.forgeops.com:443/openidm/managed/user/?_action=create --data '{"userName" : "H-T-276933", "givenName" : "Tbnjazlt", "sn" : "Hpjihu", "mail" : "H-T-276933@example.com", "telephoneNumber" : "444-444-4444", "password" : "Pa_ssw0rd", "description" : "Managed User" }'
{"_id":"8cff9653-d9f5-4fcc-85a2-cb75c453013c","_rev":"0000000010dc366c","userName":"H-T-276933","accountStatus":"active","description":"Managed User","givenName":"Tbnjazlt","sn":"Hpjihu","telephoneNumber":"444-444-4444","mail":"H-T-276933@example.com"}

      To reproduce it using pyrock in lodestar:
      1. clone lodestar/pyrock:
      https://stash.forgerock.org/projects/QA/repos/lodestar/browse
      2. cd lodestar/pyrock and run the test as, the test used the default config, two IDM pods, and each of read/update/delete/create POST runs about 15 minutes.
      ./run.py simple_managed_users_ds_shared_repo
      3. Put the URL file:/Users/...../lodestar/results/pyrock/simple_managed_users_ds_shared_repo/global.html on browser
      and check report for read/update/create.

      Note:

      The symptom can be easier to see when I disabled notification service using notificationFactory endpoint, It can be seen with concurrency 35 or 30.
      IDM CPU usage is relative high, attached is one cpu monitoring output during one test, the cpu/request and limit are in the config file attached.

        Attachments

        1. conf.yaml
          4 kB
        2. idm-0.log
          146 kB
        3. idm0-update-error.log
          106 kB
        4. idm-1.log
          145 kB
        5. two-idm-pods-con-40-ds.png
          two-idm-pods-con-40-ds.png
          197 kB

          Issue Links

          relates to

          Bug - A problem which impairs or prevents the functions of the product. OPENDJ-6474 REST: some requests fails when stressing embedded http endpoint with Gatling

          • Critical - Crashes, loss of data, severe memory leak.
          • Done
          mentioned in

          Page Loading...

          Page Loading...

          Page Loading...

          Page Loading...

            Activity

              People

              • Assignee:
                Tinghua.Xu Tinghua Xu
                Reporter:
                Tinghua.Xu Tinghua Xu
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: