Connection from client (go-xmpp) leads to Generic server <...> terminating with CRASH REPORT from base64.erl caused by {3,integer,type,bad} in fn format_bs_fail

[mrusme]

Hey there,

I'm using a client based on go-xmpp to connect to my own ejabberd and I'm experiencing a crash within ejabberd's supervisor:

2025-08-01 00:23:49.124787+00:00 [error] <0.781.0> ** Generic server <0.781.0> terminating 
** Last message in was {'$gen_event',
                        {xmlstreamelement,
                         {xmlel,<<"authenticate">>,
                          [{<<"xmlns">>,<<"urn:xmpp:sasl:2">>},
                           {<<"mechanism">>,<<"SCRAM-SHA-512-PLUS">>}],
                          [{xmlel,<<"upgrade">>,
                            [{<<"xmlns">>,<<"urn:xmpp:sasl:upgrade:0">>}],
                            [{xmlcdata,<<"UPGR-SCRAM-SHA-512">>}]},
                           {xmlel,<<"initial-response">>,[],
                            [{xmlcdata,
                              <<"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX">>}]},
                           {xmlel,<<"user-agent">>,[],
                            [{xmlel,<<"software">>,[],
                              [{xmlcdata,<<"go-xmpp">>}]}]},
                           {xmlel,<<"bind">>,
                            [{<<"xmlns">>,<<"urn:xmpp:bind:0">>}],
                            [{xmlel,<<"tag">>,[],
                              [{xmlcdata,<<"go-xmpp">>}]}]}]}}}
** When Server state == #{mgmt_timeout => 300000,mgmt_resend => if_offline,
                          owner => <0.781.0>,resource => <<>>,access => c2s,
                          stream_authenticated => false,
                          mgmt_max_timeout => 300000,mgmt_stanzas_req => 0,
                          tls_enabled => false,
                          stream_timeout => {120000,-576456206539},
                          lang => <<"en">>,
                          tls_options => [compression_none],
                          mgmt_state => inactive,lserver => <<"mydomain">>,
                          mgmt_max_queue => 5000,mgmt_stanzas_in => 0,
                          pres_a => {0,nil},
                          allow_unencrypted_sasl2 => false,
                          stream_version => {1,0},
                          tls_verify => false,tls_required => false,
                          shaper => c2s_shaper,
                          codec_options => [ignore_els],
                          stream_restarted => false,mod => ejabberd_c2s,
                          stream_encrypted => true,
                          ip => {{79,127,217,51},51324},
                          mgmt_ack_timeout => 60000,stream_direction => in,
                          mgmt_stanzas_out => 0,
                          stream_id => <<"15023865986792316637">>,
                          stream_header_sent => true,
                          sasl_channel_bindings =>
                              #{<<"tls-exporter">> =>
                                    <<XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX>>,
                                <<"tls-server-end-point">> =>
                                    <<XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX>>},
                          mgmt_queue_type => ram,
                          csi_queue => {0,#{}},
                          xmlns => <<"jabber:client">>,
                          socket_monitor =>
                              #Ref<0.4002952566.4232577025.253609>,
                          sasl2_stream_from =>
                              {jid,<<"user">>,<<"mydomain">>,<<>>,
                                  <<"user">>,<<"mydomain">>,<<>>},
                          stream_state => wait_for_sasl_request,user => <<>>,
                          socket =>
                              {socket_state,fast_tls,
                                  {tlssock,#Port<0.351>,
                                      #Ref<0.4002952566.4232708097.253617>},
                                  262144,#Ref<0.4002952566.4232708097.253608>,
                                  {state,3000,20000,20000,1754007829123774},
                                  none,
                                  <<"/var/spool/ejabberd/certs/0f0676018e89e3c8d2ba187be4b164349a1ec565">>},
                          zlib => false,csi_state => active,
                          stream_compressed => false,server => <<"mydomain">>}
** Reason for termination == 
** {badarg,[{base64,dec_bin,8,
                    [{file,"base64.erl"},
                     {line,500},
                     {error_info,#{cause => {3,integer,type,bad},
                                   function => format_bs_fail,
                                   module => erl_erts_errors}}]},
            {xmpp_sasl_scram,mech_step,2,
                             [{file,"src/xmpp_sasl_scram.erl"},{line,155}]},
            {xmpp_sasl,server_step,2,[{file,"src/xmpp_sasl.erl"},{line,127}]},
            {xmpp_stream_in,process_sasl2_request,2,
                            [{file,"src/xmpp_stream_in.erl"},{line,1117}]},
            {xmpp_stream_in,handle_info,2,
                            [{file,"src/xmpp_stream_in.erl"},{line,419}]},
            {p1_server,handle_msg,8,[{file,"src/p1_server.erl"},{line,696}]},
            {proc_lib,init_p_do_apply,3,[{file,"proc_lib.erl"},{line,241}]}]}

2025-08-01 00:23:49.126247+00:00 [error] <0.781.0>@proc_lib:crash_report/4:584 CRASH REPORT:
  crasher:
    initial call: xmpp_stream_in:init/1
    pid: <0.781.0>
    registered_name: []
    exception exit: {badarg,
                        [{base64,dec_bin,8,
                             [{file,"base64.erl"},
                              {line,500},
                              {error_info,
                                  #{cause => {3,integer,type,bad},
                                    function => format_bs_fail,
                                    module => erl_erts_errors}}]},
                         {xmpp_sasl_scram,mech_step,2,
                             [{file,"src/xmpp_sasl_scram.erl"},{line,155}]},
                         {xmpp_sasl,server_step,2,
                             [{file,"src/xmpp_sasl.erl"},{line,127}]},
                         {xmpp_stream_in,process_sasl2_request,2,
                             [{file,"src/xmpp_stream_in.erl"},{line,1117}]},
                         {xmpp_stream_in,handle_info,2,
                             [{file,"src/xmpp_stream_in.erl"},{line,419}]},
                         {p1_server,handle_msg,8,
                             [{file,"src/p1_server.erl"},{line,696}]},
                         {proc_lib,init_p_do_apply,3,
                             [{file,"proc_lib.erl"},{line,241}]}]}
      in function  p1_server:terminate/7 (src/p1_server.erl, line 878)
    ancestors: [ejabberd_c2s_sup,ejabberd_sup,<0.126.0>]
    message_queue_len: 1
    messages: [{'$gen_all_state_event',{xmlstreamcdata,<<"\n">>}}]
    links: [<0.568.0>]
    dictionary: [{'$internal_queue_len',1},
                  {rand_seed,{#{type => exsss,next => #Fun<rand.0.65977474>,
                                bits => 58,uniform => #Fun<rand.1.65977474>,
                                uniform_n => #Fun<rand.2.65977474>,
                                jump => #Fun<rand.3.65977474>},
                              [55900905657702313|26400142432100405]}},
                  {already_terminated,true}]
    trap_exit: false
    status: running
    heap_size: 6772
    stack_size: 28
    reductions: 91394
  neighbours:

2025-08-01 00:23:49.131890+00:00 [error] <0.568.0>@supervisor:do_restart/3:759 SUPERVISOR REPORT:
    supervisor: {local,ejabberd_c2s_sup}
    errorContext: child_terminated
    reason: {badarg,
                [{base64,dec_bin,8,
                     [{file,"base64.erl"},
                      {line,500},
                      {error_info,
                          #{cause => {3,integer,type,bad},
                            function => format_bs_fail,
                            module => erl_erts_errors}}]},
                 {xmpp_sasl_scram,mech_step,2,
                     [{file,"src/xmpp_sasl_scram.erl"},{line,155}]},
                 {xmpp_sasl,server_step,2,
                     [{file,"src/xmpp_sasl.erl"},{line,127}]},
                 {xmpp_stream_in,process_sasl2_request,2,
                     [{file,"src/xmpp_stream_in.erl"},{line,1117}]},
                 {xmpp_stream_in,handle_info,2,
                     [{file,"src/xmpp_stream_in.erl"},{line,419}]},
                 {p1_server,handle_msg,8,
                     [{file,"src/p1_server.erl"},{line,696}]},
                 {proc_lib,init_p_do_apply,3,
                     [{file,"proc_lib.erl"},{line,241}]}]}
    offender: [{pid,<0.781.0>},
               {id,undefined},
               {mfargs,{ejabberd_c2s,start_link,undefined}},
               {restart_type,temporary},
               {significant,false},
               {shutdown,5000},
               {child_type,worker}]

I have tested the exact same connection towards a different XMPP server and there it works flawlessly. I'm not sure if this is an actual bug or if I'm simply holding it wrong.

Maybe someone here could point me towards some first steps for debugging this issue?

[badlop]

Update! Problem reported and fixed in #4428

---

This part of the error message is relevant:

** {badarg,[{base64,dec_bin,8,
                    [{file,"base64.erl"},
                     {line,500},
                     {error_info,#{cause => {3,integer,type,bad},
                                   function => format_bs_fail,
                                   module => erl_erts_errors}}]},
            {xmpp_sasl_scram,mech_step,2,
                             [{file,"src/xmpp_sasl_scram.erl"},{line,155}]},

The xmpp erlang library received the scram details from ejabberd, and then in line 155 it calls base64:decode/1 to decode it, and that crashes.

---

I try this myself manually, and works:

Configure ejabberd:

hosts:
  - localhost
  - 日本語

auth_password_format: scram

Then register account, check password:

$ _build/relive/ejabberdctl register user3 日本語 asd
...
$ _build/relive/ejabberdctl check_password user3 日本語 asd
$ echo $?
0

And login with Gajim:

2025-08-01 11:13:57.415956+02:00 [info] (tls|<0.595.0>)
  Accepted c2s SCRAM-SHA-1 authentication for user3@日本語
  by mnesia backend from ::ffff:127.0.0.1
2025-08-01 11:13:57.418155+02:00 [info] (tls|<0.595.0>)
  Opened c2s session for user3@日本語/gajim.SQ3FWUST

Let's check the database content:

ejabberdctl debug
...

ejabberd_auth:get_password(<<"user3">>, unicode:characters_to_binary("日本語")).
[{scram,<<"d7IeZUtUevnhEUBcMT5ZFV/SXo8=">>,
        <<"OqHdJWQWmt2XN1vC47ZjZV1Pdx0=">>,
        <<"WM6jG3rDm3jwacpzyuenWg==">>,sha,4096}]

base64:decode(<<"d7IeZUtUevnhEUBcMT5ZFV/SXo8=">>).
<<119,178,30,101,75,84,122,249,225,17,64,92,49,62,89,21,
  95,210,94,143>>

---

What difference in configuration do you have? Maybe added some options for SASL or SCRAM that I miss to reproduce the problem?

If you try manually to get tthe password bits and decode base64 like I did... does it work or fail to decode?

[mrusme]

@badlop thank you very much for this! It turned out that my configuration didn't have auth_password_format set and, according to the documentation the default is plain. I have changed the setting to scram and am now able to successfully connect.

As a side note, this was on a regular domain. I however appreciate you testing it with a unicode domain to rule things out!

Long story short, it appears that the process crashes when it is configured to store the plain password. While switching to scram solved the issue for me, it might nevertheless be worth looking into why the default will behave that way.

Also, on a second thought, it might make sense to reevalutate the default (plain). This was something I overlooked, but it certainly doesn't help that FreeBSD (and probably other distributions) ships the unmodified example config that doesn't list the setting. I think it would make sense to include the setting with a short comment in that file for people to notice it, even if it is set to plain by default.

What do you think?

[badlop]

it appears that the process crashes when it is configured to store the plain password.

Being more precise: it crashes only in your setup.

I've setup ejabberd from git master, default configuration with no changes at all. Both Tkabber and Gajim can login correctly:

2025-08-01 19:30:15.225991+02:00 [info] The account admin@localhost was registered from IP address ::ffff:127.0.0.1

2025-08-01 19:30:15.246749+02:00 [info] (<0.2201.0>) Accepted connection [::ffff:127.0.0.1]:37227 -> [::ffff:127.0.0.1]:5222
2025-08-01 19:30:15.345236+02:00 [info] (tls|<0.2201.0>) Accepted c2s DIGEST-MD5 authentication for admin@localhost by mnesia backend from ::ffff:127.0.0.1
2025-08-01 19:30:15.410448+02:00 [info] (tls|<0.2201.0>) Opened c2s session for admin@localhost/tka1

2025-08-01 19:30:18.242608+02:00 [info] (<0.2722.0>) Accepted connection [::1]:37692 -> [::1]:5222
2025-08-01 19:30:18.286207+02:00 [info] (tls|<0.2722.0>) Accepted c2s SCRAM-SHA-512 authentication for admin@localhost by mnesia backend from ::1
2025-08-01 19:30:18.288578+02:00 [info] (tls|<0.2722.0>) Opened c2s session for admin@localhost/gajim.4LR553DR

In your crash case there are more factors involved, probably related to your configuration, or your client, that you haven't shared, and consequently the problem cannot be reproduced or investigated.

If you provide details that help to reproduce the problem, then it could be investigated.

[mrusme]

I am happy to provide a full copy of my ejabberd configuration, as well as write a small test client based on go-xmpp that you can go build and run against your test server. However, is there a way I could send you the configuration directly instead of uploading it publicly?

//edit: E.g. to the email address on your GitHub profile?

[badlop]

Yes, to that address or to the one I just sent you an email