Uploaded image for project: 'OpenAM'
  1. OpenAM
  2. OPENAM-13442

mapPk2Cert.JKSKeyProvider error was encountered during upgrade from 13.5.1 to 5.5.1. Boot.json is not created

    XMLWordPrintable

Details

    • Rank:
      1|hzwibj:

    Description

      Bug description

      Customer encountered the following error exception during the upgrade from OpenAM 13.5.1 to 5.5.1

      Note : OpenAM 13.5.1 is upgraded from the previous version of OpenAM and it is using keystore.jks ( instead of the OOTB keystore.jceks )

      After upgrade, during the first restart, the following error exception was encountered.

       

      amSecurity:08/09/2018 07:12:30:161 AM NZST: Thread[smIdmThreadPool,5,main]: TransactionId[bd09c795-ca71-4b9e-a187-3811a80e2b04-5893]
      ERROR: mapPk2Cert.JKSKeyProvider:
      java.io.IOException: Keystore was tampered with, or password was incorrect
              at com.sun.crypto.provider.JceKeyStore.engineLoad(JceKeyStore.java:879)
              at java.security.KeyStore.load(KeyStore.java:1445)              <========== keystore fails to load 
              at org.forgerock.openam.utils.AMKeyProvider.mapPk2Cert(AMKeyProvider.java:215)
              at org.forgerock.openam.utils.AMKeyProvider.<init>(AMKeyProvider.java:104)
              at com.sun.identity.setup.BootstrapCreator.update(BootstrapCreator.java:125)
              at com.sun.identity.setup.BootstrapCreator.updateBootstrap(BootstrapCreator.java:91)
              at com.sun.identity.common.configuration.ServerConfigXMLObserver.update(ServerConfigXMLObserver.java:108)
              at com.sun.identity.common.configuration.ServerConfigXMLObserver.notifyChanges(ServerConfigXMLObserver.java:80)
              at com.sun.identity.common.configuration.ConfigurationObserver.notifies(ConfigurationObserver.java:185)
              at com.sun.identity.common.configuration.ConfigurationObserver.globalConfigChanged(ConfigurationObserver.java:147)
              at com.sun.identity.sm.ServiceConfigManagerImpl.notifyGlobalConfigChange(ServiceConfigManagerImpl.java:390)
              at com.sun.identity.sm.ServiceConfigManagerImpl.objectChanged(ServiceConfigManagerImpl.java:370)
              at com.sun.identity.sm.SMSNotificationManager.sendNotifications(SMSNotificationManager.java:281)
              at com.sun.identity.sm.SMSNotificationManager$LocalChangeNotifcationTask.run(SMSNotificationManager.java:357)
              at org.forgerock.openam.audit.context.AuditRequestContextPropagatingRunnable.run(AuditRequestContextPropagatingRunnable.java:34)
              at com.iplanet.am.util.ThreadPool$WorkerThread.run(ThreadPool.java:312)
      Caused by: java.security.UnrecoverableKeyException: Password verification failed
      

      Another observation which indicated "Uninitialized keystore" message due to the above error

      ServiceConfigImpl::getInstance: called: ou=server-default,ou=com-sun-identity-servers,ou=default,ou=GlobalConfig,ou=1.0,ou=iPlanetAMPlatformService,ou=services,dc=openam,dc=spark,dc=co,dc=nz
      
      java.security.KeyStoreException: Uninitialized keystore
       at java.security.KeyStore.containsAlias(KeyStore.java:1252)
       at org.forgerock.openam.utils.AMKeyProvider.setSecretKeyEntry(AMKeyProvider.java:556)
       at com.sun.identity.setup.BootstrapCreator.update(BootstrapCreator.java:132)
       at com.sun.identity.setup.BootstrapCreator.updateBootstrap(BootstrapCreator.java:91)
       at com.sun.identity.common.configuration.ServerConfigXMLObserver.update(ServerConfigXMLObserver.java:108)
       at com.sun.identity.setup.AMSetupServlet.init(AMSetupServlet.java:216)
       at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1227)
       at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1140)
       at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1027)
       at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:5038)
       at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5348)
       at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:145)
       at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:753)
       at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:729)
       at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:717)
       at org.apache.catalina.startup.HostConfig.deployWAR(HostConfig.java:940)
       at org.apache.catalina.startup.HostConfig$DeployWar.run(HostConfig.java:1816)
       at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
       at java.util.concurrent.FutureTask.run(FutureTask.java:266)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
       at java.lang.Thread.run(Thread.java:745)
      

       The boot.json was not created but the previous bootstrap file was retained.

      Tracing the code in AMKeyProvider.java reveals that OpenAM tries to write the following entry into two keystore ( jks and jceks ) but fails in jceks keystore.

      Customer has earlier changed the keystore password of keystore.jks ( .storepass and .keypass )

      but did not change the same for keystore.jceks since he is not using it.

      Unfortunately, OpenAM need to write keystore.jceks for the creation of the boot.json.

      The entries are

      dsameuserpwd, Aug 14, 2018, SecretKeyEntry, 
      configstorepwd, Aug 14, 2018, SecretKeyEntry, 

      BootstrapCreator.java

      // write the required boot passwords to the keystore
      amKeyProvider.setSecretKeyEntry(BootstrapData.DSAME_PWD_KEY, dspw); amKeyProvider.setSecretKeyEntry(BootstrapData.CONFIG_PWD_KEY, configStorepw);
      amKeyProvider.store();
      bootConfig.writeConfig(baseDir + "/boot.json");
      
      // We delay deletion of legacy bootstrap until the very end.
      // If there are exceptions, this will leave the bootstrap in place
      // and make the system stil bootable.
      
      if (doMigrate) 
      { 
         bootstrap.delete(); 
      }
      

       

      Workaround

      ==========

      change the keystore.jceks password

      keytool -storepasswd -keystore keystore.jceks -storetype JCEKS 

      Attachments

        Activity

          People

            cristina.herraz Cristina Herraz [X] (Inactive)
            sam.phua Sam Phua
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: