Improving Version 5 Query Behavior

[bckohan]

I'm starting this discussion as a dumping ground for ideas for how to make polymorphism support first-class in Version 5. What I mean by this is removing the two-phase fetch, check ctype and refetch process with a single fetch process.

My best idea right now:

Django uses INNER JOIN to fetch linked parent tables when fetching multi-table models from a derived model. This also works for polymorphic models when fetching in that direction because the parents must exist. The difference with the polymorphic queries is that you also want to fetch down the hierarchy to find any child model rows. You could do this by adding LEFT OUTER JOIN for every model down the hierarchy as well and basing the python class mapping on one up from the first null table id. A nice side effect of this is that you would also eliminate the need for polymorphic_ctype_id which is a source of a lot of problems - especially in multi-database/multi-tenant setups.

What I do not currently understand is how hard this level of customization would be or if there would be any unanticipated side effects because I'm still getting acquainted with Django's ORM internals. Advice from ORM experts would be greatly appreciated!

[JohananOppongAmoateng]

Eliminating polymorphic_ctype_id would simplify correctness and remove an entire class of bugs, but it likely needs to be an opt-in or long-term goal due to migration and third-party dependency concerns.

[bckohan]

I think the benefits would outweigh the downsides. The complexity of relying on the contenttypes framework is responsible for a plurality of the bugs on the tracker and it forces a solution that breaks the iterator contract of basic Django - requiring many more sql queries than should be necessary. Removing it would mean you couldn't, with an arbitrary row, determine the most derived row without hitting the database again - but if you're asking this question in code you're probably about to hit the database again anyway.

To minimize code disruption we could provide a polymorphic_ctype/_id property that maps back to the correct type and deprecate its access for v6. I think polymorphism should ultimately stand completely apart from the contenttypes framework. They are orthogonal concepts to me. The only thing I can think of that would break in a non-trivial way is the current proxy model behavior, but that behavior is going to be changed by default in version 5 anyway. For users who are relying on the current behavior - which I doubt is that many - they could reimplement (or just switch proxy off for the models in question).

[JohananOppongAmoateng]

That explanation helps a lot, thanks for taking the time to spell it out.

I hadn’t fully appreciated how deeply the contenttypes dependency affects both correctness and query behavior, but framing it in terms of breaking Django’s normal iterator expectations really makes the trade-off clearer. From that angle, the idea that polymorphism should stand apart from contenttypes makes a lot of sense to me. they solve different problems, and coupling them seems to force complexity in places where Django is otherwise quite clean.

The idea of keeping a polymorphic_ctype / _id compatibility layer while deprecating direct access in v6 also sounds like a reasonable way to reduce disruption, especially for third-party code. As a newer contributor, that kind of gradual transition feels much easier to reason about than a hard break.

I don’t have enough context yet to judge the proxy model implications in detail, but if that behavior is already changing in v5, it does seem like a good opportunity to rethink this without adding extra legacy paths. Overall, this makes the long-term direction much clearer to me.

[JohananOppongAmoateng]

Do we have any plans for assisting users of the library we migrating their dbs in the form of a script or management command for automating the migration. I mean considering the breaking changes scheduled in v5 i wiould assume that their models might break because we no longer using the polymorphic_ctype and other breaking changes?

[bckohan]

Do we have any plans for assisting users of the library we migrating their dbs in the form of a script or management command for automating the migration. I mean considering the breaking changes scheduled in v5 i wiould assume that their models might break because we no longer using the polymorphic_ctype and other breaking changes?

If we removed dependency on content types the only real database migration would be the removal of the polymorphic_ctype foreign key and Django should automatically handle that migration for users.