readdir() in PKCS#11 config file read does not guarantee the expected alphabetical order

[andreas-meissner]

When using freeslots to store labelled keys in ATECC608B-TFLXTLS the readout of the PKCS#11 configuration is broken since v3.4 (20221027).
As soon as a key is stored in a freeslot (and the according 0.x.conf is created), the ATECC608B is not recognized anymore by PKCS#11 tools.

To Reproduce
Steps to reproduce the behavior:

1. Set up PKCS#11 environment es described in https://github.com/MicrochipTech/cryptoauthlib/tree/main/app/pkcs11
2. In 0.conf modify the line "freeslots = 2,3,4" to make these key slots available or key storage (note that slot 1 has fixed content on ATECC608B-TFLXTLS)
3. Call p11tool --list-keys -> the fixed keys of ATECC608B-TFLXTLS are shown.
4. Generate a new key with p11tool "pkcs11:token=00ABC" --generate-privkey=ECDSA --bits=288 --curve=secp256r1 --label=testkey -> the config file 0.2.conf is created
5. Again call p11tool --list-keys -> the PKCS#11 token representing the ATECC608B-TFLXTLS is not found anymore.

Expected behavior
The PKCS#11 token representing the ATECC608B-TFLXTLS is still accessible, the fixed keys and the generated key in keyslot 2 are shown.
This worked as intended in v3.3.3

Additional context
Debug output of crytoauthlib shows

p11tool --list-keys

1024:1024:C_Initialize:141:
1024:1024:pkcs11_config_load_objects:892:Opening Configuration: /var/lib/cryptoauthlib/0.2.conf
1024:1024:pkcs11_config_load_objects:956:Trying to load an object configuration without a slot configuration file
1024:1024:C_Initialize:142:CKR_OK(0)
1024:1024:C_GetInfo:159:
1024:1024:C_GetInfo:160:CKR_CRYPTOKI_NOT_INITIALIZED(190)
1024:1024:C_GetSlotList:184:
1024:1024:C_GetSlotList:185:CKR_CRYPTOKI_NOT_INITIALIZED(190)
warning: no token URL was provided for this operation; the available tokens are:

1024:1024:C_GetSlotList:184:
1024:1024:C_GetSlotList:185:CKR_CRYPTOKI_NOT_INITIALIZED(190)

Issue analysis
The issue is raised by a change in pkcs11_config.c -> pkcs11_config_load_objects() in v3.4, where readdir() is used to enumerate over all configuration files in a loop, while within the loop it is expected that the token configuration file (0.conf) is read and parsed before the keyslot configuration files (0.x.conf).

But the behavior of readdir() is implementation specific and there is no guarantee that it returns an alphabetically ordered list of files in a directory (which would be needed to read 0.conf with the slot_id before all other 0.x.conf files that require the slot_id already been set).

Actually, in a lot of implementations readdir() works in traversal order of file generation history, as it is in our case too.

[andreas-meissner]

BTW: code indentation is a mess in pkcs11_config_load_objects() and makes it hard to read.