[OPENDJ-7176] Filters with malformed attribute descriptions cannot be parsed Created: 06/May/20  Updated: 28/Aug/20  Resolved: 29/May/20

Status: Done
Project: OpenDJ
Component/s: core apis
Affects Version/s: 6.5.2, 6.5.1, 6.5.0, 6.5.3
Fix Version/s: 6.5.4, 7.0.0

Type: Bug Priority: Major
Reporter: Alex Belovski Assignee: Ondrej Fuchsik
Resolution: Fixed Votes: 0
Labels: release-notes

Issue Links:
Backport
is backported by OPENDJ-7238 Backport OPENDJ-7176: Filters with ma... Done
Regression
is caused by OPENDJ-3718 IllegalArgumentException when running... Done
Story Points: 2
Dev Assignee: Chris Ridd
QA Assignee: Ondrej Fuchsik
Support Ticket IDs:
Backports: OPENDJ-7238 (6.5.4)

 Description   

ldapsearches with a search filter that has “_” (underscore) are working in previous versions of DS (3.x), but failing in 6.5.x with the following message:

The attribute description “mobile_login” could not be parsed because it contains an invalid character “” at position 6_ 

If the ldapsearches are performed from AM => DS, the following is observed:   

  1. AM 13.5.1 to DS 3.0 works (when used attribute “mobile_login” as search filter)
  2. AM 13.5.1 to DS 6.5.x fails (when used attribute “mobile_login” as search filter)
  3. AM 6.5.x to DS 6.5.x fails (when used attribute “mobile_login” as search filter)
  4. AM 6.5.x to DS 3.0 fails (when used the attribute “mobile_login” as search filter)

 
A possible solution as per Chris Ridd's notes:
The 6.5 server code only allows well-formed attribute descriptions when parsing a filter. I think we could safely allow malformed attribute descriptions here.

From opendj-core/.../AttributeDescription.java:

static String validateAttributeDescription(final String attributeDescription) {
 if (attributeDescription == null) {
 // Some Filter allows null attribute description and these must perform the null check them-self.
 return null;
 }
 valueOf0(null, attributeDescription, false, NoOpFactory.INSTANCE);
 return attributeDescription;
}

Change the 'false' to a 'true'.

N.B! This has not been tested and setting the flag from false to true will also change the behavior of the server, which may have a detrimental effect. 
 
This affects the client SDK as well, so ldapsearch (and AM) will not even send such a search filter to the server. 

 



 Comments   
Comment by Ludovic Poitou [ 06/May/20 ]

Changing the value from 'false' to 'true' may have side effects on the server side and should not be done blindly.
There are many places where the server parses a string as a filter and must enforce that malformed attributes are not used if the server rejects them.

Comment by Matthew Swift [ 07/May/20 ]

The strict RFC compliance was introduced by the fix for OPENDJ-3718.

Comment by Chris Ridd [ 07/May/20 ]

The LDAP specifications do not permit attribute descriptions to have underscores.

There are two points here. One is that the client-side code is preventing the application from sending a search request with a non-compliant filter. Other SDKs and clients (I tried OpenLDAP /usr/bin/ldapsearch) also refuse to allow non-compliant filters.

Assuming you do manage to send a non-compliant filter to the DS server, the server will correctly reject the search request as malformed. So the second point is this server behaviour.

We would need to change the client behaviour and the server behaviour for this particular customer. There is resistance from engineering to change the server behaviour, so there is consequently very little reason to change the client behaviour.

Thus the issue is being marked as "Won't fix". The customer will need to modify their data to use a valid attribute, and modify their applications to use the valid attribute.

Comment by Ludovic Poitou [ 07/May/20 ]

The server has an option to allow malformed attribute names in data (only for specific characters).
I think that when this option is enabled, the server should allow these attributes to be used in search filters as well.

Comment by Matthew Swift [ 07/May/20 ]

While investigating this issue this morning, Chris and I came to the conclusion that a fix would involve reverting part of the fix for OPENDJ-3718, specifically the calls to org.forgerock.opendj.ldap.AttributeDescription#validateAttributeDescription. Instead, malformed attribute descriptions should be detected at the point where the filter is compiled to a Matcher in the SearchOperation constructor. Compilation takes into account the schema's compatibility options. However, the constructor is invoked quite far down the call-chain, so care would need to be taken to handle the potential LocalizedIllegalArgumentException and bubble it up from there as a protocol error LdapException.

Comment by Chris Ridd [ 19/May/20 ]

I think we've concurred that this does need to be fixed.

SDK change:

  • the Filter class should not check the attribute descriptions (e.g. via schema)
  • clients should be able to send invalid filters without reconfiguration

Server change:

  • attribute description checking must be done as per the global allow-attribute-name-exceptions property
  • LdapReader now cannot "fail fast" if there's a filter with a malformed attribute description
  • we need to defer the filter error checking to the SearchOperation class.

The checking inside the Filter code was done for OPENDJ-3718, so (part of) that change needs reverting.

Comment by Chris Ridd [ 27/May/20 ]

This may impact QA tests that expect certain requests to result in particular errors.

Comment by Ondrej Fuchsik [ 29/May/20 ]

Verified the check is not done now with 7.0.0-SNAPSHOT rev. 851f945337f.

Generated at Sat Nov 28 22:24:58 UTC 2020 using Jira 7.13.12#713012-sha1:6e07c38070d5191bbf7353952ed38f111754533a.