Fixed and improved sorting #3266
Conversation
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
Documentation and Community
|
github-actions
bot
added
internal
| @@ -245,6 +245,8 @@ def apply_sorting( | |||
| UserSchema, getattr(table, "user_id") == UserSchema.id | |||
| ) | |||
|
|
|||
| query = query.add_columns(UserSchema.name) | |||
There was a problem hiding this comment.
Can you explain why this is necessary?
From what I read, this essentially means the query is now a
select(original_query.output, user_name) query? This would be very confusing IMO, that people using the filter model now suddenly get extra items in their query results
There was a problem hiding this comment.
You may have seen this pattern in different places in this PR. This is needed because eventually we need to order by this column and if we don't add this here, the entire thing fails with an SQLOperationalError, simply due to the fact that it can not find the column in the initial selection.
I realize this might be confusing at first glance, but as you can see from the code, I did not have to make any adjustments to the Schema -> Response conversion for the corresponding Schemas. This means that our developers won't have to manage it and also, since this is entirely isolated to our internal code, our users will not be affected by it either.
There was a problem hiding this comment.
Yep but I really don't get why, and I don't think we should rely on "this for some reason currently works so let's hope for the best" in our DB code.
from sqlmodel import SQLModel, Field, create_engine, Session
from sqlalchemy import Column, select, ForeignKey
class TableA(SQLModel, table=True):
id: int = Field(default=None, primary_key=True)
a: str = Field(nullable=False)
__tablename__ = "table_a"
class TableB(SQLModel, table=True):
id: int = Field(default=None, primary_key=True)
fk: int = Field(
sa_column=Column(
ForeignKey(
f"table_a.id",
name="fk",
),
nullable=False,
),
)
b: int = Field(nullable=False)
engine = create_engine("sqlite:///:memory:")
SQLModel.metadata.create_all(engine)
with Session(engine) as session:
a = TableA(a="1")
session.add(a)
session.commit()
b1 = TableB(b=2, fk=a.id)
session.add(b1)
session.commit()
b2 = TableB(b=3, fk=a.id)
session.add(b2)
session.commit()
with Session(engine) as session:
query = select(TableB).distinct()
query = query.join(TableA)
query = query.add_columns(TableA.a).order_by(TableA.a)
query = query.limit(5).offset(0)
print(query)
result = session.exec(query).all()
print(result)This is a simple example to show the behaviour of this add_columns(...) method, and if I run this I get the added columns in the result.
There was a problem hiding this comment.
For future reference, the difference between our sql_zen_store code and the code snippet above was the fact that one used the select function from sqlalchemy while the other used sqlmodel. I have added a comment on the sql_zen_store code regarding this difference, for more info, please check there.
src/zenml/models/v2/base/scoped.py
Outdated
| @@ -524,79 +523,3 @@ def get_custom_filters( | |||
| ) | |||
|
|
|||
| return custom_filters | |||
|
|
|||
| def apply_sorting( | |||
There was a problem hiding this comment.
Why was the tag sorting removed now?
There was a problem hiding this comment.
Unfortunately, the tests showed that the concatenation of the tags alphabetically did not work as intended and tags were sorted randomly. Until we could figure out a solution, I have decided to disable sorting by tags.
There was a problem hiding this comment.
I think the time to figure out a solution is now, and as part of this PR :P
The frontend already has some many version flags to enable/disable features, and the idea behind implementing the sorting was to enable sorting by all columns that are displayed in the frontend tables. If we now leave out one column, this would be even more complexity in the frontend
There was a problem hiding this comment.
I fixed this one. Now, it looks a bit more complicated as I had to sort the tags before, but I am hopeful the new tests with the tag sorting will pass now.
Describe changes
Pre-requisites
Please ensure you have done the following:
developand the open PR is targetingdevelop. If your branch wasn't based on develop read Contribution guide on rebasing branch to develop.Types of changes