MNT: Enable numpy 2+ installs

[greglucas]

Change Summary

Overview

Numpy 2 is more strict about the overflow math calculations, so we needed to update a few cases to handle that as well.

New Dependencies

Bumping numpy to allow 2+

[greglucas]

@maxinelasp, I'm not sure how you want to handle this, so let me know if you want something else. You are calling astype(int32) in the code calling section, but in the tests you pass in a frombuffer() call which is only int8. We could get rid of the type-cast in the calling routine and rely on this, or we could immediately cast the frombuffer in the test too if you'd prefer.

[maxinelasp]

Annoyingly, this is also complicated by the compression code that I'm working on, which also takes in vector data. Is the problem that the 8 bit ints are signed?

[maxinelasp]

If this change passes the tests then I'm ok with it. The bitshifting is unfortunately complicated with different types, but I am also looking to rework the types to be more consistent in my newest change, so that it works with the uncompressed and compressed algorithms.

[greglucas]

Is the problem that the 8 bit ints are signed?

The problem is here:

imap_processing/imap_processing/mag/l1a/mag_l1a_data.py

Lines 349 to 352 in 4a2642b

	x = (
	((vector_data[pos + 0] & 0xFF) << 8)
	| ((vector_data[pos + 1] & 0xFF) << 0)
	) & 0xFFFF

where you are doing math with 0xFFFF which is out of range for those types, so numpy2+ is warning and saying we need to be explicit with what types we want to do the math on.

Within the code you are already casting here:

imap_processing/imap_processing/mag/l1a/mag_l1a.py

Line 185 in 4a2642b

mag_l0.VECTORS.astype(dtype=np.int32), # type: ignore[union-attr]

But in the tests, you aren't:

imap_processing/imap_processing/tests/mag/test_mag_l1a.py

Lines 63 to 65 in 4a2642b

	(primary_vectors, secondary_vectors) = MagL1a.process_vector_data(
	input_data, total_primary_vectors, total_secondary_vectors
	)

So I was saying that we could also update this in those two locations if you wanted as well.

[maxinelasp]

Yeah, ok, in that case either change is fine, and I'll go back once the compression algorithm is done and make sure it's consistent with the types I'm using there.

[subagonsouth]

Looks good to me.

[maxinelasp]

We probably want to have this pinned to a minor version, no? numpy = "^2.1.0 I'm not sure we want random bumps up to, for example, 2.2.0.

[greglucas]

I don't think we should require numpy 2+ because that will likely cause issues with other packages only supporting <=2 because those packages maybe haven't done the fixes themselves yet to be compatible.

I'm personally a fan of more open-ended ranges for packages so that we don't get pinned back and then have to do a massive update later on. When deploying/installing the package as a user, then we can/should use explicit package locks, but in general I have a preference for leaving the testing of the package open so that we don't get stuck way behind the times and miss things.

[maxinelasp]

Ok, since this uses a lockfile I'm alright with that. Did you see if requiring numpy 2+ actually caused issues with packages?

[greglucas]

Yes, it does not cause issues right now, we are able to install numpy 2+. I think this is a difference of whether we are the final "user" (what I think you're getting at with the lockfile for install) or if we expect this package to be used by others (my assumption) and potentially installing other packages besides the ones we require. Pinning to numpy 2+ will affect others and not just ourselves. If we work with numpy 1.26 right now we should let that keep working IMO.

[greglucas]

Merging now, we can follow up on installation pins in subsequent PRs.