Add pre and post processing scripts for schema generation

[1337joe]

The biggest blocker I'm running into trying to address #9045 is that a number of fields are conditionally removed from the serializers depending on what view is calling it like this:

if issubclass(view.__class__, ListModelMixin):
    self.fields.pop('notes', None)

The schema generator doesn't seem to handle this case at all, so any field that is conditionally removed is excluded from the schema entirely. That leaves me with a schema that doesn't define a notes field so when one is included in a details query the generated code reports it as an error (and certainly offers no way to view the data).

I haven't been able to figure out a programmatic way to detect when schema generation is happening to not remove the fields, nor have a found a configuration that affects this behavior.

The included scripts work, but they aren't integrated in any way. I'm opening this as a draft PR to ask if it's worth finding a way to integrate this into the invoke dev.schema ... command or if there's a better path I should be going down.

The scripts are intended to be used as follows:

./pre_schema.sh
invoke dev.schema --ignore-warnings --filename schema.yml --overwrite
./post_schema.sh schema.yml

[netlify]

✅ Deploy Preview for inventree-web-pui-preview canceled.

Name	Link
🔨 Latest commit	b710934
🔍 Latest deploy log	https://app.netlify.com/sites/inventree-web-pui-preview/deploys/67c615ce85e11900084dab5a

[matmair]

detecting if the script is being run is pretty simple, we do it a bunch:

InvenTree/src/backend/InvenTree/InvenTree/ready.py

Lines 12 to 14 in 8cee2e3

	def isImportingData():
	"""Returns True if the database is currently importing (or exporting) data, e.g. 'loaddata' command is performed."""
	return any(x in sys.argv for x in ['flush', 'loaddata', 'dumpdata'])

[matmair]

I think the main issue here is that the filters are not exposed correctly to the schema, @SchrodingersGat is slowly cleaning that up - that is probably the better way

[1337joe]

There are certainly a bunch of warnings in the schema generation about filters, but I don't see a connection between that and the notes field, which hits a bunch of different objects. If you can point me to a PR where a filter is cleaned up maybe I'll better understand how the filters should be working.

I'll see if adding an isGeneratingSchema ready check gives me a path forward. Thanks for pointing that file out.

[matmair]

not notes specifically but there are dozens of fields part_detail, manufacturer_part_detail, pretty_name

[matmair]

there are also build-in tools to do postprocessing https://drf-spectacular.readthedocs.io/en/latest/customization.html#step-6-postprocessing-hooks

[1337joe]

Yeah, I used a script for post-processing here because I already had to undo the code changes the pre-processing script made. I've already got a working post-processing hook, just trying to test thoroughly as I go before I decide it's ready to merge.