Uploaded image for project: 'OpenAM'
  1. OpenAM
  2. OPENAM-16664

am-base docker-entrypoint.sh script risks am deployment availability

    Details

    • Sprint:
      FRaaS 2020.12 - Muttnik
    • Story Points:
      1
    • Needs QA verification:
      No
    • Functional tests:
      No

      Description

      Bug description

      The am-base Docker image's docker-entrypoint.sh script blocks am start-up until the DS stores which am depends on are available.  Unfortunately, rather than waiting for any DS instance within each store's list to be available, the script requires the first DS instance in each list to be online.

      FRaaS init-containers used to work this way and this led to reduced availability and test instability - see FRAAS-1634.

      Code analysis

      OPENAM source:

      openam-docker/src/main/docker/am-base/docker-entrypoint.sh
      wait_for_datastore() {
        hostname=$(echo $2 | cut -d ":" -f1)
        echo "Waiting for $1 to be available. Trying $hostname:8080/alive endpoint" #What if multiple ds's are set?
        while [[ "$(curl -s -o /dev/null -w ''%{http_code}'' $hostname:8080/alive)" != "200" ]];
        do
                sleep 5;
        done
        echo "$1 is responding"
      }
      
      wait_for_datastore "User Store" "${AM_STORES_USER_SERVERS}"
      wait_for_datastore "CTS" "${AM_STORES_CTS_SERVERS}"
      wait_for_datastore "Application Store" "${AM_STORES_APPLICATION_SERVERS}"
      wait_for_datastore "Policy Store" "${AM_STORES_POLICY_SERVERS}"
      wait_for_datastore "UMA Store" "${AM_STORES_UMA_SERVERS}" 

      Alternative approach implemented in FRaaS:

      deploy/docker/util/waitForPort.sh
      #!/bin/bash
      
      wait_for_port () {
          CHECK_SVCS=$*
          for SVC_GROUP in $CHECK_SVCS; do
              while true; do
                  RESPONDED="false"
                  GROUP=${SVC_GROUP//\// }
                  echo "Waiting for one of $SVC_GROUP"
                  for SVC in ${GROUP}; do
                      HST=$(echo $SVC | cut -d ":" -f 1)
                      PRT=$(echo $SVC | cut -d ":" -f 2)
                      nc -z $HST $PRT
                      if [ $? -ne 0 ]; then
                          sleep 5
                          if (( ${SECONDS} > 600 )); then
                              echo "ABORTING WAITING FOR $SVC, TIMEOUT REACHED."
                              exit 10
                          fi
                      else
                          RESPONDED="true"
                          echo "Service $SVC is up"
                          break
                      fi
                  done 
                  if [ "${RESPONDED}" == "true" ]; then
                      break
                  fi
              done
          done
      }
      
      wait_for_port $*
      
      echo "waitForPort completed in ${SECONDS} seconds."

       

      deploy/docker/util/docker_entrypoint.sh
      #!/bin/bash
      
      case $1 in
          waitForPort)
              shift
              /app/waitForPort.sh $*
          ;;
          *) 
              exec "$@"
          ;;
      esac

        

      deploy/kustomize/base/am/deployment.tmpl.yaml
      ...
            initContainers:
            - args:
              - waitForPort
              - ctsstore-0.ctsstore:1389/ctsstore-1.ctsstore:1389/ctsstore-2.ctsstore:1389
              - userstore-0.userstore:1389/userstore-1.userstore:1389/userstore-2.userstore:1389
              image: gcr.io/fr-saas-registry/util:FRAAS-3451-am-base-8164838bc6ede38e66a8f84e0d8151cd6d2f4f5c
              imagePullPolicy: Always
              name: wait-for-ds
              resources: {}
              terminationMessagePath: /dev/termination-log
              terminationMessagePolicy: File
      ...

       

        Attachments

          Activity

            People

            • Assignee:
              craig.mcdonnell Craig McDonnell
              Reporter:
              craig.mcdonnell Craig McDonnell
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: