Change of training parameters.

[beaka]

Hi Jeff, I am testing your example and having to change the size of the array and getting unexpected results.

As I have not coded for 15 years it is confusing me as I have it wrong.

The networks layers:

DECLARE_UNEURAL_LAYER(input_layer, 7);
DECLARE_UNEURAL_LAYER(hidden_layer, 10);
DECLARE_UNEURAL_LAYER(output_layer, 7);

The training data:

fix16_t inputs[7] [5] = {
		{F16(26/40), F16(16/40), F16(10/40), F16(20/40), F16(19/40)},
		{F16(22/40), F16(26/40), F16(28/40), F16(23/40), F16(12/40)},
		{F16(7/40), F16(14/40), F16(34/40), F16(34/40), F16(2/40)},
		{F16(35/40), F16(1/40), F16(23/40), F16(19/40), F16(18/40)},
		{F16(1/40), F16(8/40), F16(26/40), F16(24/40), F16(13/40)},
		{F16(11/40), F16(3/40), F16(1/40), F16(31/40), F16(1/40)},
		{F16(25/40), F16(24/40), F16(31/40), F16(10/40), F16(23/40)},
	};

The expected output:

fix16_t expected_outputs[7][1] = {
		{F16(29/40)},
		{F16(32/40)},
		{F16(17/40)},
		{F16(27/40)},
		{F16(9/40)},
		{F16(11/40)},
		{F16(12/40)},
	};

And the printout:

Allocated 700 bytes of storage for the network
Successfully initialized storage
Successfully attached data
Randomizing Network Weights
Successfully created raw uNeural Network
Allocated 3200 bytes of scratch space
Initial expected/output 0 - 0.986145|0.982849 
Initial expected/output 1 - 0.933624|0.982849 
Initial expected/output 2 - 0.977219|0.982849 
Initial expected/output 3 - 0.951111|0.982849 
Initial expected/output 4 - 0.974335|0.982849 
Initial expected/output 5 - 0.000000|0.982849 
Initial expected/output 6 - 0.000000|0.982849 
Iteration: 0
Global Error: 21.582047
RMSE: 2.322830
Actual output 0 - 0.982681 
Actual output 1 - 0.982681 
Actual output 2 - 0.982681 
Actual output 3 - 0.982681 
Actual output 4 - 0.982681 
Actual output 5 - 0.982681 
Actual output 6 - 0.982681 

Where have I got it wrong?

[Jeff-Ciesielski]

Howdy @beaka, wow, I'm not gonna lie, I forgot this library even existed 😅, it's been nearly a decade since I wrote this! I'm sure there's a bug in my code, I'll see if I can find some time on my lunch breaks this week to have a look

[beaka]

Thanks Jeff, no hurry, just having an investigation into NN's and fixed point for some filtering and the search returned your code.
As I said, have not coded for a long time so it is certain that I have got the bug.

[Jeff-Ciesielski]

Had a quick look, a few things immediately pop out:

1. There's definitely a bug in my code, it breaks down on complex networks (I'm going to fix that to see if I can't get it to learn your function)
2. The weights and expected outputs you're providing are decaying to integers (1 and 0 really) because of the way you're writing them. You need to provide either the numerator or denominator as a float to ensure that F16(29/40) doesn't just turn into 0 because of rounding, it should be F16(29/40.0) instead

[beaka]

Thanks Jeff, the 40.0 fixed it so at least there is some results. Will take this old brain some time to get back into the nuances but yeah I should have remembered that. What would you say is a "complex" network? I really have no Idea about NN's yet. Again no hurry on fixing anything but again thanks for the quick fix.

…

On Thu, 6 Mar 2025 at 02:09, Jeff Ciesielski ***@***.***> wrote: Had a quick look, a few things immediately pop out: 1. There's definitely a bug in my code, it breaks down on complex networks (I'm going to fix that to see if I can't get it to learn your function) 2. The weights and expected outputs you're providing are decaying to integers (1 and 0 really) because of the way you're writing them. You need to provide either the numerator or denominator as a float to ensure that F16(29/40) doesn't just turn into 0 because of rounding, it should be F16(29/40.0) instead — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF56IT6G6OTCUIDKM7ZWGT2S4HUPAVCNFSM6AAAAABYMGYE32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBRGIYTSMZRGI> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: Jeff-Ciesielski]*Jeff-Ciesielski* left a comment (Jeff-Ciesielski/libuneural#2) <#2 (comment)> Had a quick look, a few things immediately pop out: 1. There's definitely a bug in my code, it breaks down on complex networks (I'm going to fix that to see if I can't get it to learn your function) 2. The weights and expected outputs you're providing are decaying to integers (1 and 0 really) because of the way you're writing them. You need to provide either the numerator or denominator as a float to ensure that F16(29/40) doesn't just turn into 0 because of rounding, it should be F16(29/40.0) instead — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF56IT6G6OTCUIDKM7ZWGT2S4HUPAVCNFSM6AAAAABYMGYE32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBRGIYTSMZRGI> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[Jeff-Ciesielski]

I just mean that it learned a simple network (the example), but when provided with a more complex function to learn, it's falling apart and not converging. I'm going to have a look, it's probably a bug in the back propagation code

[beaka]

Oh OK, I am not up on the terms and I thought it was something special I had to look up. Is this the effect of not "converging"? I notice that when I run it there is a lot of zero's appearing. -------------------- 1:0 - d/w/e: 0.017105/-0.678711/-0.011612 Err Sum: -0.011612 1:1 - d/w/e: -0.004547/-0.724960/0.003296 Err Sum: -0.008316 1:2 - d/w/e: -0.039749/0.000000/0.000000 Err Sum: -0.008316 1:3 - d/w/e: -0.014893/0.000000/0.000000 Err Sum: -0.008316 1:4 - d/w/e: -0.023376/0.000000/0.000000 Err Sum: -0.008316 1:5 - d/w/e: -0.012009/0.000000/0.000000 Err Sum: -0.008316 1:6 - d/w/e: -0.059311/0.000000/0.000000 Err Sum: -0.008316 l2 delta-0.008316

…

On Thu, 6 Mar 2025 at 03:01, Jeff Ciesielski ***@***.***> wrote: I just mean that it learned a simple network (the example), but when provided with a more complex function to learn, it's falling apart and not converging. I'm going to have a look, it's probably a bug in the back propagation code — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF56IQQZ65EHHBLOJRAPED2S4NVXAVCNFSM6AAAAABYMGYE32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBRGM4DCMBRG4> . You are receiving this because you were mentioned.Message ID: ***@***.***> [image: Jeff-Ciesielski]*Jeff-Ciesielski* left a comment (Jeff-Ciesielski/libuneural#2) <#2 (comment)> I just mean that it learned a simple network (the example), but when provided with a more complex function to learn, it's falling apart and not converging. I'm going to have a look, it's probably a bug in the back propagation code — Reply to this email directly, view it on GitHub <#2 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAF56IQQZ65EHHBLOJRAPED2S4NVXAVCNFSM6AAAAABYMGYE32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDOMBRGM4DCMBRG4> . You are receiving this because you were mentioned.Message ID: ***@***.***>