The xmlsec library used (in AM6 ----
AME-15681----) which is 2.1.1 causes issues when the ignorelinebreak is enabled. This causes Certificate or Base64 payload to be generated with
Alternative interpretation: With this issue when using -Dorg.apache.xml.security.ignoreLineBreaks=true, the Base64 cert/signature still have line breaks (ie does not seem to work. which may be needed for other SP) (So a library upgrade is needed).
Details steps outlining how to recreate the issue (remove this text)
- Enable -Dorg.apache.xml.security.ignoreLineBreaks=true or -Dcom.sun.org.apache.xml.internal.security.ignoreLineBreaks=true to JVM settings
- Setup a SAML2 federation with Response signing
- Enable Message debug
- Perform a SAML2 SP initiated login
- Observe the IDP Federation logs for the certificate displaying LF as
a) Maybe replace xmlsec-2.1.1.jar with xmlsec-2.1.2.jar (when -Dorg.apache.xml.security.ignoreLineBreaks=true
Tested this this works.
b) Try to see sure none of the settings
Dorg.apache.xml.internal.security.ignoreLineBreaks=false or Dcom.sun.org.apache.xml.internal.security.ignoreLineBreaks=false (but these later xmlsec still sends linebreaks) actually. (This may not work since xmlsec 2.1.1 always add when this is so) this does not seems to not have issues the but it seem for Forgerock AM<->AM (IDP/SP) this is not causing issues as it seems AM can handle this but other vendor SP may not like this. For some SP that is non-permissive, because of the they may not work and maybe need to see if setting the above true changes things - Dcom.sun.org.apache.xml.internal.security.ignoreLineBreaks=true or -Dorg.apache.xml.security.ignoreLineBreaks=true.
xmlsec changes. See https://issues.jboss.org/browse/WFLY-9892 and also xmlsec bug https://issues.apache.org/jira/browse/SANTUARIO-482. But with the fix in 2.1.2 and the switch this should address the issue
May need to have some documentation or decision what to do to ensure this options is more known (DOC bug after this)