Ideas for metadata fields for reproducible builds

[RobTillaart]

ideas to extend the yaml file with following fields

- CRC32: 0x12345678                      of the binary build => or even per library / source level
- DATE_CREATED: 2021-01-01
- VERSION: 1.0.3
- AUTHOR: john doe
- EMAIL: john doe @ provider.com
- COPYRIGHT: MIT, GPL, GPS, none
- LOCATION: github.com/johnDoe/Arduino/....

The CRC32 is the 'ultimate' compatibility check. OK other hashes might even be more 'ultimate' but you get the idea.

From the list above the CRC is the only one that can actually be used and affect the build process.
So the other ones may appear as comments?

[Ing-Dom]

a CRC32 on library level can cause problems regarding configuration of libraries in their .h files

[RobTillaart]

@SirSydom
Can you give an example?
It is not clear for me what problem you are pointing at.

[Ing-Dom]

some libraries contain defines in their header files to adapt the library to specific needs. some example I found after quick search: https://github.com/adafruit/Adafruit_NeoPixel/blob/13d4e20f60a7c3a909693759a01a84c781f732b1/Adafruit_NeoPixel.h#L134

This is a general design flaw in the arduino environment - because the library is for ALL you projects (sketches) but the change to its defines may be only valid for one specific sketch (or even on specific version of one specific sketch).

To adress this, one could think about not touching the lib but specifing the defines to the preprocessor directly (which could be done in the propossed idea) - but this will only work if all relevant defines are made to work with that, e.g. with #ifndef FOO #define FOO BAR #endif

So, guarding libraries you use untouched will work with a crc32. But what if the library has to be modified ?

[RobTillaart]

But what if the library has to be modified ?

I see the point made and yes the CRC would produce a mismatch
That in itself should not result in a failing build, however a warning seems to be reasonable / fair.

If the programmer knows it did change the library she/he should expect a CRC warning.

The CRC should not be mandatory, but using it gives that little extra control.

[Ing-Dom]

as I just recognized, there is a proposed solution for the problem:

https://github.com/arduino/tooling-rfcs/blob/profiles/RFCs/0003-build-profiles.md#vendoring-libraries-with-custom-modifications

[RobTillaart]

Sounds like a good generic solution - still, even from such modified library a CRC could be made :)

[cmaglie]

We thought about adding a CRC, but in the end, we opted to not add it in this first iteration because it has some tricky issues to clear out:

• the binary produced by the build may not be always reproducible bit-by-bit because the binary may contain the current date or filesystem paths (in .elf files for example) or debugging info or other artifacts that are system-dependent.
• this means that we should produce source-level CRC, but then we must define an algorithm that scans a directory and produce a single CRC that should be portable across different OS (with different line-endings, filesystem properties...) or, alternatively, store all CRC for all source files (but in this case we will pollute the profiles.yaml with a lot of crcs...)

BTW the idea of adding CRC is still valid IMHO, and we will surely add it in the future.

[RobTillaart]

@cmaglie
Thanks those points make sense, now that you mention it I recall similar problems on MSVC due to build number in dll's and exe's.

Maybe it is possible to get a (weaker) signature from the cyclomatic complexity of the code?
(roughly number of possible paths)

we will surely add it in the future.

Propose to add an appropriate label e.g. FUTURE to the issue.