Performance Monitoring

[bckohan]

We need to track performance along several axes to catch performance regressions before they get shipped.

Principally interested in:

1. Number of queries issued
2. Speed of polymorphic model iteration

Would like to be able to point the tests to a version on pypi, a PR or a commit hash.

Related #781

[JohananOppongAmoateng]

Waht benchmarks are you considering? No of queries? Speed? Mermory

[JohananOppongAmoateng]

I dont know if i am the only one who has noticed this but i think there is an obvious increase in the ci time. Havent done any measurements between the versions

[bckohan]

Waht benchmarks are you considering? No of queries? Speed? Mermory

All of the above. I also think comparison to non-polymorphic Django iteration is in order. Users should be able to see what the real penality for polymorphism is when deciding to use this library or not.

[bckohan]

I dont know if i am the only one who has noticed this but i think there is an obvious increase in the ci time. Havent done any measurements between the versions

The number of tests has more than doubled in the past few months. That should account for most of it. #781 probably wouldn't have had a noticeable impact because there are only a few tests that generate more than a handful of rows.

[bckohan]

I added a basic query counting test that should at least catch egregious errors like #781. That test lives in test_performance.py. It'd be nice to add a few more checks in there but I also don't want to bog down CI with too much of this heavy stuff. I think we need a separate framework that runs big data checks and maybe generates comparison plots between versions and vanilla Django.