Convert initializers to NumPower

[SkibidiProduction]

Initializers have been converted to NumPower.

For He, Xavier, LeCun initializers, a division into two types was made. The first type is based on a truncated normal distribution, the second on a unified distribution.

The final distributions of the initializers are made in accordance with their analogs in the keras library.

A fork of the original NumPower is used (https://github.com/RubixML/numpower) due to the fact that there are difficulties with accepting changes and fixing bugs in the original extension.

[andrewdalpino]

Hey @SkibidiProduction overall a pretty good start!

I really like the thoroughness of the unit tests but would encourage you to come up with some more descriptive names for the test methods instead of initializeTest1() initializeTest2() etc. I also like that you broke the initializers into Normal and Uniform versions. Finally, I like that you converted to more modern PHP syntax.

I'm concerned with strict types being unnecessary and slowing down our runtime. I'm also not keen on the idea of introducing additional exception classes without a compelling reason to do so. Also, I think we should stick to calling classes that provide API as "interfaces" and not introduce the "contract" term to mean the same thing as it has the potential to confuse people but provides no additional benefit. Lastly, I think we should avoid creating additional namespace hierarchies unless they are necessary.

[marclaporte]

A fork of the original NumPower is used (https://github.com/RubixML/numpower) due to the fact that there are difficulties with accepting changes and fixing bugs in the original extension.

Once things are tested, would it make sense to propose changes as a PR at https://github.com/NumPower/numpower/pulls ?

So these changes would have more visibility and perhaps lead to more collaboration throughout the ecosystem.

[SkibidiProduction]

Once things are tested, would it make sense to propose changes as a PR at https://github.com/NumPower/numpower/pulls ?

So these changes would have more visibility and perhaps lead to more collaboration throughout the ecosystem.

@marclaporte Yes, but it's better not to change old classes, but to create new ones. Otherwise, tests will fail and this will complicate refactoring.

[andrewdalpino]

Which tests will fail @SkibidiProduction? Are we able to modify the tests along with the changes to the classes so they do not fail?

Once things are tested, would it make sense to propose changes as a PR at https://github.com/NumPower/numpower/pulls ?
So these changes would have more visibility and perhaps lead to more collaboration throughout the ecosystem.

@marclaporte Yes, but it's better not to change old classes, but to create new ones. Otherwise, tests will fail and this will complicate refactoring.

[andrewdalpino]

Looks good, nice work @SkibidiProduction

[SkibidiProduction]

Which tests will fail @SkibidiProduction? Are we able to modify the tests along with the changes to the classes so they do not fail?

@andrewdalpino Initializers now return NDArray instead of Matrix as before. However, other entities, such as layers, optimizers, etc., cannot work with NDArray objects. Moreover, they use Matrix objects as input parameters, and use Matrix objects inside their implementation. Thus, changing the entity of one type used in building a neural network, or even more so changing only one class, will lead to the tests for all related code falling.

[andrewdalpino]

Oh yes @SkibidiProduction I know that, I meant that question in the context of NumPower. @marclaporte suggested we submit our changes to the original NumPower repo when we're done making changes needed for Rubix ML. I think that's a worthwhile thing to do, but it sounded like you suggested there would be issues, specifically with tests.