GUACAMOLE-1971: Safely multiply values to be used for memory allocation.

[aleitner]

CodeQL has flagged a potential integer overflow so I used guac_mem_ckd_mul_or_die() to safely multiple the values.

[necouchman]

@aleitner and @jmuehlner Should this get fixed in the patch branch, and then applied to the changes in main, as well?

[jmuehlner]

@necouchman this fix is related to the FreeRDP 3 code, which isn't in patch.

[necouchman]

Okay. I asked because the call to _aligned_malloc is in there in the previous version:

guacamole-server/src/protocols/rdp/pointer.c

Lines 42 to 45 in c715f16

	/* Allocate data for image */
	unsigned char* data = _aligned_malloc(pointer->width * pointer->height * 4, 16);

and I think the changes made for FreeRDP3 commits just wrap that call based on whether it should be winpr_aligned_malloc or _aligned_malloc:

guacamole-server/src/protocols/rdp/rdp.h

Lines 55 to 61 in 2aabe28

	#ifdef HAVE_WINPR_ALIGNED
	#define GUAC_ALIGNED_FREE winpr_aligned_free
	#define GUAC_ALIGNED_MALLOC winpr_aligned_malloc
	#else
	#define GUAC_ALIGNED_FREE _aligned_free
	#define GUAC_ALIGNED_MALLOC _aligned_malloc
	#endif

I wasn't sure if it only affects one case and not the previous one...

[jmuehlner]

Ah, good call. Yeah this should be fixed in patch as well, for the old call.

[mike-jumper]

FWIW, I've looked through the FreeRDP code surrounding this, and the values in question are strongly limited to 32x32 or 96x96 (_update_read_pointer_color()) or 384x384 (_update_read_pointer_large()).

It's probably still worth fixing on patch.