Uploaded image for project: 'OpenIDM'
  1. OpenIDM
  2. OPENIDM-14432

Restarting IDM cluster generates error message on first node: Scheduled service "scheduler-service-group.liveSync" invocation reported failure:

    Details

    • Target Version/s:
    • Verified Version/s:
    • Story Points:
      2
    • Sprint:
      2020.04 - IDM
    • Support Ticket IDs:
    • Zendesk ID:
      45813

      Description

      A customer is having an issue with the below messages on node1 of their environment when they restart their IDM cluster.

       

      The messages go away after a few minutes and does not occur on the second node.

       

      Executing ./startup.sh...
      Using OPENIDM_HOME:  /app/cams/idm11/openidm
      Using PROJECT_HOME:  /app/cams/idm11/cams
      Using OPENIDM_OPTS:  -server -Xms2g -Xmx2g -d64 -XX:+UseConcMarkSweepGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/app/logs/idm11/idm//gc.log -Dopenidm.scheduler.execute.persistent.schedules=true
      Using LOGGING_CONFIG: -Djava.util.logging.config.file=/app/cams/idm11/cams/conf/logging.properties
      modifying root logger's FileHandler
      modifying root logger's ConsoleHandler
      -> ShellTUI: No standard input...exiting.
      [2020-01-24 03:15:19 6203@ups11app-istp.cams.scotiabank.com DefaultQuartzScheduler_Worker-1 org.forgerock.openidm.quartz.SchedulerServiceJob execute WARNING] Scheduled service "scheduler-service-group.liveSync" invocation reported failure: org.forgerock.json.resource.ServiceUnavailableException: System: system/ldap/account is not available.
      org.forgerock.openidm.quartz.ExecutionException: org.forgerock.json.resource.ServiceUnavailableException: System: system/ldap/account is not available.
          at org.forgerock.openidm.provisioner.api.SystemObjectSetService.execute(SystemObjectSetService.java:494)
          at org.forgerock.openidm.quartz.SchedulerServiceJob.execute(SchedulerServiceJob.java:123)
          at org.quartz.core.JobRunShell.run(JobRunShell.java:223)
          at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
      Caused by: org.forgerock.json.resource.ServiceUnavailableException: System: system/ldap/account is not available.
          at org.forgerock.openidm.provisioner.api.SystemObjectSetService.locateService(SystemObjectSetService.java:562)
          at org.forgerock.openidm.provisioner.api.SystemObjectSetService.liveSync(SystemObjectSetService.java:542)
          at org.forgerock.openidm.provisioner.api.SystemObjectSetService.execute(SystemObjectSetService.java:491)
          ... 3 more[2020-01-24 03:15:19 6203@ups11app-istp.cams.scotiabank.com DefaultQuartzScheduler_Worker-1 org.forgerock.openidm.provisioner.api.SystemObjectSetService auditScheduledService SEVERE] Unable to audit scheduled service AuditEvent [value={ "request": { "protocol": "CREST", "operation": "ScheduledTask", "detail": { "taskName": "scheduler-service-group.liveSync" } }, "transactionId": "d3756505-9d30-479c-b2d8-845b74f8b079-119", "userId": "system", "timestamp": "2020-01-24T20:15:19.695Z", "eventName": "access", "response": { "status": "FAILED", "statusCode": null, "elapsedTime": 75, "elapsedTimeUnits": "MILLISECONDS", "detail": { "code": 500, "reason": "Internal Server Error", "message": "org.forgerock.json.resource.ServiceUnavailableException: System: system/ldap/account is not available." } } }]
      [2020-01-24 03:15:19 6203@ups11app-istp.cams.scotiabank.com DefaultQuartzScheduler_Worker-1 org.forgerock.openidm.quartz.SchedulerServiceJob execute SEVERE] Unable to audit scheduled task scheduler-service-group.liveSync
      org.forgerock.openidm.quartz.ExecutionException: Unable to audit scheduled service
          at org.forgerock.openidm.provisioner.api.SystemObjectSetService.auditScheduledService(SystemObjectSetService.java:510)
          at org.forgerock.openidm.quartz.SchedulerServiceJob.execute(SchedulerServiceJob.java:134)
          at org.quartz.core.JobRunShell.run(JobRunShell.java:223)
          at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:549)
      Caused by: org.forgerock.json.resource.NotFoundException: Resource 'audit/access' not found
          at org.forgerock.json.resource.Router.getBestMatch(Router.java:235)
          at org.forgerock.json.resource.Router.handleCreate(Router.java:260)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:65)
          at org.forgerock.json.resource.Filters$ConditionalFilter.filterCreate(Filters.java:54)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.json.resource.Filters$ConditionalFilter.filterCreate(Filters.java:54)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.json.resource.Filters$ConditionalFilter.filterCreate(Filters.java:54)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.json.resource.Filters$ConditionalFilter.filterCreate(Filters.java:54)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$3.lambda$filterCreate$1(ServletConnectionFactory.java:388)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$3$$Lambda$331/1302578989.get(Unknown Source)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$3.handleRequestWithLogging(ServletConnectionFactory.java:434)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$3.filterCreate(ServletConnectionFactory.java:388)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.openidm.router.filter.PassthroughFilter.filterCreate(PassthroughFilter.java:48)
          at org.forgerock.openidm.router.filter.MutableFilterDecorator.filterCreate(MutableFilterDecorator.java:72)
          at org.forgerock.openidm.router.filter.MutableFilterDecorator.filterCreate(MutableFilterDecorator.java:72)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.json.resource.Filters$ConditionalFilter.filterCreate(Filters.java:54)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.json.resource.Filters$ConditionalFilter.filterCreate(Filters.java:54)
          at org.forgerock.json.resource.FilterChain$Cursor.handleCreate(FilterChain.java:63)
          at org.forgerock.json.resource.FilterChain.handleCreate(FilterChain.java:232)
          at org.forgerock.json.resource.InternalConnection.createAsync(InternalConnection.java:40)
          at org.forgerock.json.resource.AbstractAsynchronousConnection.create(AbstractAsynchronousConnection.java:42)
          at org.forgerock.json.resource.AbstractConnectionWrapper.create(AbstractConnectionWrapper.java:92)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$InternalConnectionWrapper.lambda$create$0(ServletConnectionFactory.java:300)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$InternalConnectionWrapper$$Lambda$330/1049071476.get(Unknown Source)
          at org.forgerock.openidm.metrics.MetricsCollector.time(MetricsCollector.java:51)
          at org.forgerock.openidm.metrics.MetricsCollector.time(MetricsCollector.java:68)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$InternalConnectionWrapper.time(ServletConnectionFactory.java:281)
          at org.forgerock.openidm.servlet.internal.ServletConnectionFactory$InternalConnectionWrapper.create(ServletConnectionFactory.java:300)
          at org.forgerock.json.resource.AbstractConnectionWrapper.create(AbstractConnectionWrapper.java:92)
          at org.forgerock.openidm.provisioner.api.SystemObjectSetService.auditScheduledService(SystemObjectSetService.java:506)
          ... 3 moreOpenIDM version "6.5.0.2" (build: 20191009170628, revision: 657d99b) jenkins-openidm-release-sustaining%2F6.5.x-23
      [2020-01-24 03:15:20 6203@ups11app-istp.cams.scotiabank.com pool-4-thread-1 org.forgerock.openidm.health.HealthService run SEVERE] OpenIDM failure during startup, ACTIVE_NOT_READY: Required services not all started [org.forgerock.openidm.audit, org.forgerock.openidm.authentication, org.forgerock.openidm.config.manage, org.forgerock.openidm.internal.service, org.forgerock.openidm.repo.init]
      OpenIDM ready
      
      
      
      

        Attachments

          Activity

            People

            • Assignee:
              cgdrake Chris Drake
              Reporter:
              marcus.bonin Marcus Bonin
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: