Uploaded image for project: 'OpenAM'
  1. OpenAM
  2. OPENAM-17276

AM recorder does not record anymore

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 7.0.0, 7.0.1
    • Fix Version/s: 7.1.0, 7.0.2
    • Component/s: debug logging
    • Labels:
    • Rank:
      1|i03acv:
    • Sprint:
      AM Sustaining Sprint 82, AM Sustaining Sprint 83
    • Story Points:
      1
    • Needs backport:
      No
    • Needs QA verification:
      Yes
    • Functional tests:
      No
    • Are the reproduction steps defined?:
      Yes and I used the same an in the description

      Description

      Bug description

      Doing AM recording sometimes unable to record as the status in recording set to true but no active recording exists

      How to reproduce the issue

      Get a AMADMIN TOKEN and also set the SERVER host.
      Do a recording.

      curl -s -k \
        -X POST \
        -H "iPlanetDirectoryPro: $TOKEN" \
        -H "X-Requested-With: curl" \
        -H "Content-Type: application/json" -d ' {
        "issueID": 99999,
        "referenceID": "configdump",
        "description": "Troubleshooting config",
        "zipEnable": true,
        "configExport": {
         "enable": true,
         "password": "123456789012",
         "sharePassword": false
        },
        "debugLogs": {
         "debugLevel": "MESSAGE",
         "autoStop": {
          "time":  {
           "timeUnit": "MINUTES",
           "value": 5
          },
          "fileSize": {
           "sizeUnit": "GB",
           "value": 1
          }
         }
        },
        "threadDump" : {
         "enable": true,
         "delay" :  {
          "timeUnit": "SECONDS",
          "value": 5
         }
        }
       }' \
       "$SERVER/openam/json/records?_action=start"
      
      ##### Testing
      
      curl 'http://am.lvh.me:8080/openam/json/global-config/realms/Lw' \
        -X 'PUT' \
        -H 'Connection: keep-alive' \
        -H 'Pragma: no-cache' \
        -H 'Cache-Control: no-cache' \
        -H 'accept-api-version: protocol=2.0,resource=1.0' \
        -H 'accept: application/json' \
        -H 'x-requested-with: ForgeRock CREST.js' \
        -H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36' \
        -H 'content-type: application/json' \
        -H 'Origin: http://am.lvh.me:8080' \
        -H 'Referer: http://am.lvh.me:8080/openam/ui-admin/' \
        -H 'Accept-Language: en-GB,en-US;q=0.9,en;q=0.8' \
        -H "iPlanetDirectoryPro: $TOKEN" \
        --data-binary '{"name":"/","active":true,"parentPath":"","aliases":["openam","am.lvh.me","localhost"]}' \
        --compressed \
        --insecure
      
      sleep 5
      
      curl -s -k \
         -X POST \
         -H "iPlanetDirectoryPro: $TOKEN" \
         -H "X-Requested-With: curl" \
         -H "Content-Type: application/json" \
         "$SERVER/openam/json/records?_action=stop"
      

      2. If you check the status

      curl -s -k \
         -X POST \
         -H "iPlanetDirectoryPro: $TOKEN" \
         -H "X-Requested-With: curl" \
         -H "Content-Type: application/json" \
         "$SERVER/openam/json/records?_action=status"
      echo
      

      and if it is like this

      {"recording":true,"record":{"issueID":99999,"referenceID":"configdump","description":"Troubleshooting config","zipEnable":true,"threadDump":{"enable":true,"delay":{"timeUnit":"SECONDS","value":5}},"configExport":{"enable":true,"password":"xxxxxx","sharePassword":false},"debugLogs":{"debugLevel":"message","autoStop":{"time":{"timeUnit":"MILLISECONDS","value":300000},"fileSize":{"sizeUnit":"KB","value":1048576}}},"status":"STOPPED","folder":"/home/am/var/debug/record/99999/configdump/"}}
      

      where the recording state is true but the current record session is stopped.

      3. Running the record session will not change things as this record session is not removed

      4. I know that this may not be consistently reproduced but this had been seen by multiple instance and different tester (environment).

      Expected behaviour
      Future recording can be done
      
      Current behaviour
      Recording not longer works
      

      Detection of the issue

      Checking the recording status will show a stopped but stale AM recording session that is active

      curl -s -k \
         -X POST \
         -H "iPlanetDirectoryPro: $TOKEN" \
         -H "X-Requested-With: curl" \
         -H "Content-Type: application/json" \
         "$SERVER/openam/json/records?_action=status"
      echo
      

      like

      {"recording":true,"record":{"issueID":99999,"referenceID":"configdump","description":"Troubleshooting config","zipEnable":true,"threadDump":{"enable":true,"delay":{"timeUnit":"SECONDS","value":5}},"configExport":{"enable":true,"password":"xxxxxx","sharePassword":false},"debugLogs":{"debugLevel":"message","autoStop":{"time":{"timeUnit":"MILLISECONDS","value":300000},"fileSize":{"sizeUnit":"KB","value":1048576}}},"status":"STOPPED","folder":"/home/am/var/debug/record/99999/configdump/"}}
      

      Work around

      AM Recording will not work until AM is restarted. So that means to get logs one may need to manually enable message debug and all those data collection.

      Code analysis

      When having the ability to consistently replicate this and so it seems that the issues is that when doing a stopRecording the piece of code

              //Set back the debug levels and appenders
              LoggerContext context = (LoggerContext) LoggerFactory.getILoggerFactory();
              for (Logger logger : context.getLoggerList()) {
                  Level previousLevel = previousDebugLevel.get(logger.getName());
                  logger.setLevel(previousLevel);
                  logger.detachAndStopAllAppenders();
                   previousAppenders.get(logger.getName()).forEach(logger::addAppender);
             }
      

      throws a NullPointerExcdeption on the previousAppenders.get(LoggerName)
      return NULL. The problem is that the issue is not detectable w/o debugger

        Attachments

          Activity

            People

            • Assignee:
              chee-weng.chea C-Weng C
              Reporter:
              chee-weng.chea C-Weng C
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: