Codejam #6 : Automatic relationship handling

[samuelgarcia]

The actual implementation use list for relation ship.
This is easy to code but not efficient when the number of object is high in models (PyNN, mozaik)

Todd propose NeoContainer that would handle automatic relation ship.

Some are thinking of some kind of indexes but nothing clear at the moment.

here the place to discuss.

[toddrjen]

There are several approaches to handling this:

1. A new class, which is either a subclass of list or uses python's ABCs to create a list-like object. It would override the setitem-related methods to automatically create relationships.
2. Some sort of property that sits between the user and the list of objects and handles the relationship generation. For this approach, I have not been able to find a way to handle things like block.segments[1] = segment, but I am under the impression others in the group have found a way.

The advantage and disadvantage of approach 1 is that, if the list is copied elsewhere, it will still automatically handle relationships. So that means that you can pass around the list in the io class without having to move around the entire neo object. This, however, might lead to unexpected behavior for users who think it is an ordinary list.

[rproepp]

I would prefer refraining from implicit side effects whenever possible. In this case, I don't think that the convenience gain warrants this change. We now have create_many_to_one_relationship on every container.

I see that Samuel lists efficiency as an argument, is that really an issue that would be solved by introducing a proxy or property?

I have code in an internal version of Spyke Viewer that uses the fact that relationships are not enforced. However, it is needed in conjunction with working around performance problems when using Neo with a database, so that use case should not be a strong argument here.

[samuelgarcia]

Do have some draft code for both ?

In any case, a main issue that was discussed in Julich is the performance.
In PyNN and Mozaik they suffer a lot with the list because they handle thousands of SpikeTrain which are link to thousand of Units. We thought of a numpy internal solution to manipulate such big list. I think it compatible with solution 1 and 2.

In the case 2, we would also have some kind of pseudo list proxy, no ? Or the list would be attached as a private attr to neo object ?

[toddrjen]

I have code in an internal version of Spyke Viewer that uses the fact that relationships are not enforced. However, it is needed in conjunction with working around performance problems when using Neo with a database, so that use case should not be a strong argument here.

Even in case 1, there would still be a semi-private internal list that can be access when needed, and I think the pseudo-list should have a method to create all the relationships at once in cases where the internal list is used directly. So if bypassing the automatic relationship generation is needed that can certainly be supported.

That would also simplify the porting process. It would simply require converting existing IO classes to use the internal list initially, and use the manual relationship generation method in place of create_many_to_one_relationship. That wouldn't be much harder than a find-replace. Then the IO classes could be individually ported to use automatic relationship generation in places where it makes sense.

[samuelgarcia]

done