[Bug]: OTHERNAME, MD5_CTX symbol collisions with OPENSSL_COEXIST

[vszakats]

Contact Details

No response

Version

ff68099

Description

I did my tests on macOS with Apple clang, but the issue is env-agnostic.

Tested the latest wolfSSL master with -DOPENSSL_COEXIST and
OpenSSL 3.3.2 headers included from the same source. It worked,
except for two symbols, OTHERNAME and MD5_CTX.

Original report: curl/curl#15438 (comment)
PR to bring "coexist" to curl: curl/curl#15596

Reproduction steps

1. build wolfSSL:

# build wolfSSL (based on: https://github.com/microsoft/vcpkg/blob/master/ports/wolfssl/portfile.cmake)
cmake . -DCMAKE_INSTALL_PREFIX=$PWD/_pkg \
  -DWOLFSSL_BUILD_OUT_OF_TREE=ON \
  -DWOLFSSL_EXAMPLES=OFF         \
  -DWOLFSSL_CRYPT_TESTS=OFF      \
  -DWOLFSSL_OPENSSLEXTRA=ON      \
  -DWOLFSSL_TPM=ON               \
  -DWOLFSSL_TLSX=ON              \
  -DWOLFSSL_OCSP=ON              \
  -DWOLFSSL_OCSPSTAPLING=ON      \
  -DWOLFSSL_OCSPSTAPLING_V2=ON   \
  -DWOLFSSL_CRL=ON               \
  -DWOLFSSL_DES3=ON              \
  -DWOLFSSL_ASIO=OFF             \
  -DWOLFSSL_DTLS=OFF             \
  -DWOLFSSL_DTLS13=OFF           \
  -DWOLFSSL_DTLS_CID=OFF         \
  -DWOLFSSL_QUIC=ON              \
  -DWOLFSSL_SESSION_TICKET=ON    \
  '-DCMAKE_C_FLAGS=-DHAVE_EX_DATA -DNO_WOLFSSL_STUB -DWOLFSSL_ALT_CERT_CHAINS -DWOLFSSL_DES_ECB -DWOLFSSL_CUSTOM_OID -DHAVE_OID_ENCODING -DWOLFSSL_CERT_GEN -DWOLFSSL_ASN_TEMPLATE -DWOLFSSL_KEY_GEN -DHAVE_PKCS7 -DHAVE_AES_KEYWRAP -DWOLFSSL_AES_DIRECT -DHAVE_X963_KDF'
cmake --build .
cmake --install .

2. build the minimal test.c:

/* build test:
   clang -isystem /path/to/wolfssl.git/_pkg/include -isystem /usr/local/opt/openssl@3/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -c test.c
 */

#define OPENSSL_COEXIST
#include <openssl/x509v3.h>
#include <openssl/md5.h>
#include <wolfssl/options.h>
#include <wolfssl/ssl.h>

Relevant log output

In file included from test.c:5:
/path/to/wolfssl.git/_pkg/include/wolfssl/ssl.h:224:39: error: typedef redefinition with different types ('struct WOLFSSL_ASN1_OTHERNAME' vs 'struct otherName_st')
typedef struct WOLFSSL_ASN1_OTHERNAME OTHERNAME;
                                      ^
/usr/local/opt/openssl@3/include/openssl/x509v3.h:160:3: note: previous definition is here
} OTHERNAME;
  ^
In file included from test.c:5:
In file included from /path/to/wolfssl.git/_pkg/include/wolfssl/ssl.h:4574:
In file included from /path/to/wolfssl.git/_pkg/include/wolfssl/openssl/asn1.h:27:
In file included from /path/to/wolfssl.git/_pkg/include/wolfssl/openssl/ssl.h:42:
In file included from /path/to/wolfssl.git/_pkg/include/wolfssl/openssl/evp.h:43:
/path/to/wolfssl.git/_pkg/include/wolfssl/openssl/md5.h:61:25: error: typedef redefinition with different types ('WOLFSSL_MD5_CTX' (aka 'struct WOLFSSL_MD5_CTX') vs 'struct MD5state_st')
typedef WOLFSSL_MD5_CTX MD5_CTX;
                        ^
/usr/local/opt/openssl@3/include/openssl/md5.h:46:3: note: previous definition is here
} MD5_CTX;
  ^
2 errors generated.

[anhu]

Hi @vszakats ,

Nice catch!! I will try your reproduction steps and come up with a fix. Please stay tuned.

Warm regards, Anthony

[anhu]

I ran your cmake command that is the first step in your reproduction steps and noticed that the generated options.h did not have OPENSSL_COEXIST so when I double checked your cmake command line, I did not see OPENSSL_COEXIST anywhere there. Forgive me, but I'm a bit confused.

Should it be there?

Warm regards, Anthony

[vszakats]

Thanks for dealing with this Anthony.

I ran your cmake command that is the first step in your reproduction steps and noticed that the generated options.h did not have OPENSSL_COEXIST so when I double checked your cmake command line, I did not see OPENSSL_COEXIST anywhere there. Forgive me, but I'm a bit confused.

Should it be there?

Reading the affected headers, it should not matter, because the
colliding macros are added regardless of settings (build-time or
use-time). Building wolfSSL without MD5 support fixes MD5_CTX,
but often this isn't feasible and it still leaves OTHERNAME.

Also, #define OPENSSL_COEXIST fixed all other collisions, except
these two. So I figured it's a workable way to tell the headers to enable
coexist.

That said I did try making a tailored coexist wolfSSL build. First tried
-DWOLFSSL_OPENSSLCOEXIST=ON but it makes the build fail with:
error: use of undeclared identifier 'WOLFSSL_EVP_CTRL_AEAD_GET_TAG'
With -DWOLFSSL_OPENSSL_COEXIST=ON the build is successful,
but test.c continues to produce the problem. Same with using
CMAKE_C_FLAGS to pass -DOPENSSL_COEXIST (which is never set by
CMakeLists.txt).

In case I missed an option, let me know, and I can re-test.

[anhu]

Ah...I missed that you define it manually in your application. This is likely to break things. Let me see if I can add it to your cmake flags and see what happens for me. Please stay tuned.

Warm regards, Anthony

[anhu]

Hi,

I've spoken with the member of our engineering team that has been working on the openssl coexist feature recently and he will be looking into this. I've assigned it to him. Please expect further information and/or fixes from him by the end of the week.

Warm regards, Anthony

[douzzer]

@vszakats fixed by #8216, currently in flight. Fixes these conflicts, and all other remaining conflicts with OpenSSL low level crypto headers.

[vszakats]

Thank you @douzzer, I can confirm #8216 fixes these issues. Both in this isolated example and when building curl with the vcpkg configuration and wolfSSL + OpenSSL.

$ ./curl -V
curl 8.11.1-DEV (Darwin) libcurl/8.11.1-DEV wolfSSL/5.7.4 (OpenSSL/3.3.2) zlib/1.2.11 libidn2/2.3.7 libpsl/0.21.5 nghttp2/1.63.0
Release-Date: [unreleased]
Protocols: dict file ftp ftps gopher gophers http https imap imaps ipfs ipns mqtt pop3 pop3s rtsp smb smbs smtp smtps telnet tftp ws wss
Features: alt-svc AsynchDNS ECH HSTS HTTP2 HTTPS-proxy IDN IPv6 Largefile libz MultiSSL NTLM PSL SSL threadsafe TLS-SRP UnixSockets