fix(build): increase packing performance

[traa]

Implementation is kinda hairy, but overall on large modules like mine (12+k files) it improves packing speed more than 10 times (from 350+ seconds to 20+ seconds on nwn-build pack-only)

[jakkn]

The changes work on my end and I see the performance improvement, but it broke pack for flat folder layout. See comments for explanation and suggested fix. Let me know if you need help with it.

[jakkn]

build.rb supports flat folder layout and the file matching here makes the below code blow up when running

nwn-build --flat extract
nwn-build pack-only

nwn-devbase/build.rb

Line 195 in 649c161

FLAT_LAYOUT = options[:flat] || (SOURCES.size > 0 && Pathname.glob(to_forward_slash SRC_DIR.join("*/")).size == 0) || false # Assume flat layout only on -f or if the source folder contains files but no directories

Array lookups, though slow, were safe in this regard. A quickfix could be an optional join on type, and to pass the value of FLAT_LAYOUT to pack.rake and apply the same logic there. A prettier fix might be to use a hash map, but that would require even more changes and I'm hesitant to rewrite large parts at this point due to the lack of test coverage.

[jakkn]

This reads nicely and is useful. Thumbs up.

[jakkn]

The logic above doesn't look right. This function was a mess already, and I think it needs to be redone in a more sensible way. The two flags make it complex, and there's something not right about conditionals that log an action without actually doing anything.

I would suggest replacing L461-468 with

  nss_changed = update_gffs()
  should_compile = nss_changed && !skip_compile

And this line to

Suggested change

	compile_nss(MODULE_FILE) unless skip_compile
	if should_compile
	compile_nss(MODULE_FILE)
	else
	if skip_compile
	puts "[INFO] Skipping compilation due to configuration"
	else
	puts "[INFO] No change in nss sources detected. Skipping compilation."
	end
	end

Or perhaps you see something more logical and/or compact.