[OPENAM-12373] amster transport key makes rest operations too slow for hardcoded timeout limit of 10 seconds Created: 29/Jan/18  Updated: 07/Jul/20  Resolved: 28/Mar/18

Status: Resolved
Project: OpenAM
Component/s: REST SDK
Affects Version/s: 5.5.1
Fix Version/s: 6.0.0, 5.5.2

Type: Bug Priority: Major
Reporter: Simon Harding Assignee: Sean ONeill [X] (Inactive)
Resolution: Fixed Votes: 0
Labels: AME, Must-Fix
Remaining Estimate: Not Specified
Time Spent: Not Specified
Original Estimate: Not Specified
Environment:

Centos 7 Java 8 Tomcat 8


Issue Links:
Duplicate
is duplicated by OPENAM-12371 XUI fails to load Oauth2 Clients >10K Closed
Relates
relates to OPENAM-16456 Amster export-config fails if transpo... Open
relates to OPENAM-16172 Performance issue with Transport key ... Closed
relates to OPENAM-11876 Amster has a timeout limit of 10 seco... Closed
Target Version/s:
Needs backport:
Yes
Needs QA verification:
No
Functional tests:
No
Are the reproduction steps defined?:
No (add reasons in the comment)

 Description   

Bug description

If a transport key is set up to allow the rest interface (and therefore amster) to export passwords, interfaces that list objects with passwords/secrets take too long to respond. This has been observed in amster export-config and simply listing 20 OAuth2 clients in a realm. In the case of amster, export times out and fails. This is made worse by the hard coded timeout in amster.

How to reproduce the issue

  1. Set up AM
  2. Create 20 Oauth2 clients.
  3. Create a transport key as described in the amster installation docs, or using transport-key.sh in the samples folder.
  4. Reboot AM
  5. Visit the OAuth2 clients page in AM.
  6. Wait for about 30 seconds.

Recomend only exporting passwords/secrets on the rest interface if the request is made with a specific parameter. That way having a transport key in place does not slow down the rest of the service, but passwords could still be requester when needed (eg amster export config).

 

 


 Comments   
Comment by Simon Harding [ 05/Feb/18 ]

The am instances wer upgraded to M3s with more cores last week, and now it takes ~40s to list 150 OAuth2 clients. This is still enough to make amster time out.

Comment by Sean ONeill [X] (Inactive) [ 28/Mar/18 ]

Needs backport to 5.5.0 and 5.5.1 amster

I tested it against backstage integration environment as I couldn't reproduce it locally.

Comment by Jonathan Thomas [ 28/Apr/20 ]

This fix makes the amster connection timeout configurable.
Please note that the transport key will still cause a dramatic drop in performance and is only recommended for import-export only.

Comment by Bernhard Thalmayr [ 02/Jun/20 ]

Jonathan Thomas it seems that the option --connection-timeout was only introduced for the connect command ...

am> :help connect
usage: connect [options] <baseurl>
Options:
  -i, --interactive
        If specified you will be prompted for credentials. Defaults to private
        key authentication.
  -k, --private-key
        Path to a private key file or directory containing one of amster_rsa,
        id_rsa or id_ecdsa. Defaults to {USER_HOME}/.ssh.
  -t, --connection-timeout
        The default timeout is 10 seconds. If specified, this parameter sets
        the timeout in seconds.

but not for the export-config command

am> :help export-config
usage: export-config --path <destination directory> [options]
Options:
  --realms <realms>
        List of realms to export, default: all
  --realmEntities <entities>
        List of realm entities to export, default: all
  --globalEntities <entities>
        List of realm entities to export, default: all
  --failOnError
        If specified then export process will halt if an error occurs,
        default: false.
  --listPasswords
        If set, the export process will create a listing of entities with
        password data, in the export directory. default: false.
  --prettyPrint
      returns the output in a easy to read format

How would this help when amster runs into

[main] ERROR org.forgerock.amster.org.forgerock.openam.sdk.http.DefaultErrorHandler - Unhandled server error: [Status: 502 Bad Gateway]#

during export ?!?

Generated at Fri Nov 27 05:23:20 UTC 2020 using Jira 7.13.12#713012-sha1:6e07c38070d5191bbf7353952ed38f111754533a.