Empty Smart Playlists

[elvinos]

Hello,

Thanks so much for this project it is great!
I have been using this to extract playlists from my Rekordbox library automatically and have noticed that smart-playlists are currently all appearing empty, when the database is queried. Are other getting this issue? Any work arounds to see the songs associated to these smart playlists?

Thanks,

Alexander

[dylanljones]

Hi Alexander,

Same for me, The DB does not contain the contents of Smart-Playlists. I am pretty sure these Playlists are filled at runtime, so the contents are by design not stored in the DB. This kind of makes sense, since Rekordbox would need to write to the DB every time tracks in the collection change.
So far pyrekordbox has no parser for the Smart Playlists, but I am working on it and will add it soon:) Until then, you can parse the rules yourself and query the database for the contents according to the rules. Here is some information that I have gathered so far.

The rules are stored as a XML string, e.g.:

<NODE Id="123456789" LogicalOperator="2" AutomaticUpdate="0">
    <CONDITION PropertyName="genre" Operator="1" ValueUnit="" ValueLeft="Genre name" ValueRight=""/>
    <CONDITION PropertyName="artist" Operator="1" ValueUnit="" ValueLeft="Artist name" ValueRight=""/>
</NODE>

The NODE tag contains the ID or the playlist, a logical operator of the (multiple) conditions and AutomaticUpdate, which seems to always be 0. The logical operator is encoded as

• 1: All conditions must be met
• 2: Any condition must be met

Each CONDITION tag contains the property name (artist, album, genre, ...), an operator, a value unit and a left and right value.
The operator is how the value(s) are compared to the property and are encoded as follows:

• 1: equal
• 2: not equal
• 3: greater
• 4: less
• 5: in range
• 6: in last (for example 'date added in last 6 days')
• 7: not in last
• 8: contains
• 9: not contains
• 10: starts with
• 11: ends with

Usually the value is only stored in ValueLeft, ValueRight is only used for the in-range condition (as far as i can tell).

Hope that helps! I will try to implement a handler for the Smart Playlists soon!

[dylanljones]

I added a handler for the smart playlists. You can parse the XML as follows:

from pyrekordbox.db6 import Rekordbox6Database, SmartList

db = Rekordbox6Database()
pl = db.get_playlist(ID=<id of smart playlist>)
smartlist = SmartList()
smartlist.parse(pl.SmartList)
for cond in smartlist.conditions:
    print(cond)

You can create a SQLAlchemy filter clause for queries:

from pyrekordbox.db6 import DjmdContent

filter_clause = smartlist.filter_clause()
for content in db.query(DjmdContent).filter(filter_clause):
    print(content)

You don't have to do the querying manually, I also implemented a new method to return the contents of a playlist directly,
supporting both regular and smart playlists:

from pyrekordbox import Rekordbox6Database

db = Rekordbox6Database()
pl = db.get_playlist(ID=<id of smart playlist>)
for content in db.get_playlist_contents(pl):
    print(content)

[dylanljones]

My bad, there was a stupid bug where only the first column name was used for all conditions...
Also the MyTag condition was a little tricky, but got it to work with sub-queries. However, now the db instance has to be passed to the filter clause method:

Does this fix the issues for you?

[dylanljones]

Never mind, I found a better way without having to do sub-queries:)
Should work as expected now!

[elvinos]

Cheers Dylan!

Have taken another look, definitely moving in the right direction. Still coming across a number of fringe cases where I am getting a blank playlist returned or (now) an error. Expect this is because i use smartlists in my Rekordbox for cross cutting tracks by tags as well us identifying new uncatalogued music in my collection... Outlining the issues I am coming across below, hopefully all easy for you to replicate your end:

1. Error for filter where smart list uses tag in the query type object 'DjmdContent' has no attribute 'myTag'
   • is this due to remapping the myTags relationship to DjmdContent?
2. Smartlists where the query looks for a blank value not returning any values (e.g. Genre == '', Color = No Color)
   • Expect this will require looking for GenreID = 0 (when another table is referenced) in the filter clause logic / adding blank values as a case
3. Multi clause smartlist not always creating a filter query as expected. e.g. duplicates stockDate clause

Let me know if i can be more specific. Will report back if I can find any workarounds in the meantime.

Thanks!

[dylanljones]

Hey Alexander,

glad to hear we are making progress.
The conditions for Mytag did not work, but I can't reproduce the error you are getting. Rekordbox uses the ID of MyTags in the smart list XML, but they left bit shift the ID with a sign change (no idea why lol, typical Pioneer stuff...). The Mytag conditions now work for me.
They do the same for the playlist ID in the XML, fixed that as well.
Also, empty values should work now! Didn't have to move to the ID, simply replacing the empty string with None did the trick.
I also tried to reproduce your third point, but for me multiple conditions on the same property (i used stockDate) works as expected.
Are you using the newest version from GitHub? Can you maybe post the XML string of the playlist that is causing the issues?

pl = db.get_playlist(Name="<playlist name>").one()
print(pl.SmartList)

[elvinos]

Hello again,

Apologies for the delay. Have been doing some thorough testing and seems like the MyTags field and blank values are now both working thanks!

Seem to be a few more fringe cases where things aren't quite working still. Outlining the examples below with accompanying XML hopefully all straight forward tweaks:

1. Multi-clause where a MyTag and Colour = none are the conditions:
   <NODE Id="-1263246057" LogicalOperator="1" AutomaticUpdate="0"><CONDITION PropertyName="myTag" Operator="8" ValueUnit="" ValueLeft="327597401" ValueRight=""/><CONDITION PropertyName="grouping" Operator="2" ValueUnit="" ValueLeft="" ValueRight=""/></NODE>
2. Some MyTag query e.g. if My Tag is = Breaks & Brokendance <- note it seems to sometime work with the & character so not the obvious cause:
   <NODE Id="-1263246057" LogicalOperator="1" AutomaticUpdate="0"><CONDITION PropertyName="myTag" Operator="8" ValueUnit="" ValueLeft="327597401" ValueRight=""/></NODE>
3. Based on DJ play count and Date added (before a certain date):
   <NODE Id="179367549" LogicalOperator="1" AutomaticUpdate="0"><CONDITION PropertyName="counter" Operator="1" ValueUnit="" ValueLeft="" ValueRight=""/><CONDITION PropertyName="stockDate" Operator="4" ValueUnit="" ValueLeft="2021-01-01" ValueRight=""/></NODE>

Hope that is enough to replicated the issues. Issue 2 seems the least intuitive to what the root cause could be...

Also more generally pyrekordbox seems to have a tendency to pull deleted / archived tracks, which in many cases results in larger than expected playlist sizes. Using the following if clause helps to clean this mostly:
if content.rb_data_status == 0 and content.rb_local_data_status == 0 and content.rb_local_deleted == 0: however in some instances there seem to be a few more tracks pulled by pyrekordbox than in my library. Any thoughts on how to further filter to get a 1:1 match?

Really appreciated!