ldap-bind with AD fails when username contains character 'ß' (LATIN SMALL LETTER SHARP S)

[gupta-kanika-dev]

Issue:

ldap-bind with AD fails when username contains character 'ß' (LATIN SMALL LETTER SHARP S)
For e.g.:
username: "Üßàùªñ1"
password: "something"
domain: ABC
ldap binding fails.

LDAP bind result comes like this:
#<OpenStruct extended_response=nil, code=0, error_message="", matched_dn="", message="Success">.
Failed with result code: 49 and error message: 8009030C: LdapErr: DSID-0C090597, comment: AcceptSecurityContext error, data 52e, v4563

Platform and gems:

Rails: 6.0.3.6
Ruby: 2.7.2
rubyntlm: 0.6.3
net-ldap: 0.16.3 (also tested with latest version 0.17.0)

More about scenario:

According to LDAP bind result:
#<OpenStruct extended_response=nil, code=0, error_message="", matched_dn="", message="Success">.
Failed with result code: 49 and error message: 8009030C: LdapErr: DSID-0C090597, comment: AcceptSecurityContext error, data 52e, v4563

• result code: 49 means "Invalid credentials", and data code: 52e means ERROR_LOGON_FAILURE
• While I am sure that username named "Üßàùªñ1" exists in AD. Hence, username and password being passed are correct.
• Via debugging I have confirmed that domain, netbios, username, password inside Net::LDAP is correct but binding with AD fails inside Net::LDAP::AuthAdapter::Sasl#bind .
• The :mechanism => "GSS-SPNEGO" and :method => :sasl is being used for ldap bind here.

Note: It should be noted that binding works well in our setup with normal unicode characters.
However it fails when username contains 'ß' (LATIN SMALL LETTER SHARP S- a unicode character only)

Debugging results:

I could figure out following difference between Rails 5.2 and Rails 6.0(as in Rails 5.2 binding works but not in Rails 6.0):
Till Rails 5.2, ActiveSupport::Multibyte::Chars#upcase was being maintained by Rails community and it provides casemapping for character 'ß' like this:

• "Üßàùªñ1".mb_chars.upcase.to_s => "ÜßÀÙªÑ1"
  notice upper case-mapping for char 'ß' is 'ß' only.

Since Rails 6.0, ActiveSupport::Multibyte::Chars#upcase is deprecated and started calling String#upcase (https://rubydoc.info/stdlib/core/String#upcase-instance_method) internally instead of maintaining its own implementation. It provides case-mapping for character 'ß' like this in Rails 6.0:

• "Üßàùªñ1".mb_chars.upcase.to_s => "ÜSSÀÙªÑ1"
  notice upper case-mapping for char 'ß' is 'SS' only.

Once again we should note here that String#upcase is following case-mapping according to Unicode standards(https://unicode.org/) only. So ideally NTLM authentication should work with it(use of String#upcase) in the authentication process. But its not the case here with ruby net-ldap.

It would be great to have any other thoughts or views about this issue and what can be the ways to fix it.