Possible idea for getting rid of the joins

[andybak]

I've used django-polymorphic a couple of times and it's great.

However - I'm trying to optimize the hell out of some code and got to thinking about that pesky INNER JOIN.

I think I've got an idea to do something similar to polymorphism but trading the cost of the join for the cost of some extra storage.

I dropped @chrisglass an email with the following and he suggested I posted it here so @vdboor and @Kronuz could chime in.

Tell me if you think there's anything worthwhile in the following:

1. The superclass should be a concrete model that has the superset of all it's child model's fields.
2. The children just have their own specific fields as normal
3. You add an extra 'type' field to the superclass
4. Each child sets the 'type' field on save to indicate which child that row relates to
5. Each child has a custom manager so that the default queryset filters on the correct 'type'.

Voila. You can query the superclass or the children.

Converting one into the other is fairly easy as you always know the pk will the same.

Caveats

All fields that aren't common to all children must be nullable or have defaults.
I am not sure how reverse relationships to the superclass are going to work.

I can obviously do all this without any special code but some special magic could solve a few things.

1. It could fix reverse relationships
2. If could avoid duplicating the field definitions on the children (although I can already get round this by using abstract mixins)
3. It could automatically handle the type field, setting type on save(), and the manager queryset.
4. It could do some sanity checking.

I'd value your opinion on whether this is worth pursuing. Could it even be a 'mode' within django-polymorphic?

[vdboor]

Hi,

Thanks for jumping in, and great to have a discussion beforehand to see if this is worth pursuing. This is a big idea indeed.

---

There is one thing I don't get about the proposal though:

Tell me if you think there's anything worthwhile in the following:

The superclass should be a concrete model that has the superset of all it's child model's fields.

Currently we have this.

The children just have their own specific fields as normal
You add an extra 'type' field to the superclass

Like the polymorphic_ctype_id field?

Each child sets the 'type' field on save to indicate which child that row relates to

That's what the polymorphic_ctype_id field does.

Each child has a custom manager so that the default queryset filters on the correct 'type'.

That should already happen in the current situation.

---

I think the only difference between our situation and yours, is the fact that you don't inherit the models, but keep them totally separate. That would indeed avoid the INNER JOIN, as that's caused by Django's ORM logic for model inheritance, and not by polymorphic itself. (yes we support only that behavior).

To avoid the JOIN in our current situation, you'd have to optimize the code in the PolymorphicQuerySet._get_real_instances(). For example, by using some model magic/proxy/dynamic model, and merging it together manually. However, that could break custom user code in ChildModel.__init__(), and maybe some other things.

Is that worth pursuing? Maybe, as for as optimization goes. However, you'd also have to take all things into account that can break; and have a solid solution for that. If you can, I think you have a solid case to go for such improvement.