Random games getting dropped though they are NOT removed from the wishlist

[icue]

For example,

• "Cogmind" was dropped in 08cf0d7
• "Hades II" was dropped in 8929cee

They often make it back in the next run, but the drop is not expected in the first place.

[icue]

@Vinfall Curious if you have ever observed this issue. This seems pretty common on my end (almost happens every day).

[Vinfall]

Not exactly as I only have 20-ish games in wishlist (you have 180+). But sometimes the same game would be ordered slightly differently, which is probably related, e.g.

• Super Onion Boy+ in Vinfall/SteamWishlistCalendar@09f0e49#diff-583c804a76e5a5faa1325c7e07c805d1aa92007b612bc099edd577ef6cafed29
• 青天井 & FAIRY TAIL 地下迷城 in Vinfall/SteamWishlistCalendar@05b6ba3#diff-583c804a76e5a5faa1325c7e07c805d1aa92007b612bc099edd577ef6cafed29

I suspect it's caused by one of these but have not tested yet:

• data structure of successful.txt (\t\t?)
• if a page has few items the returned json is ill-formed
• some weird date/timezone issue

[icue]

Thanks for the input!

I've seen order shuffles on my end as well. The game list returned from Steam should already be sorted by the app IDs of the games (at least that's what I observed). That said, I've added a manual sorting logic in 9e87371, which hopefully will prevent the random shuffle.

Maybe it will (magically) solve this issue, too. If not, at least we ruled out a possibility. I'll keep monitoring the changes.

[icue]

Ok, I think I've found the reason.
The URL https://store.steampowered.com/wishlist/profiles/{steamID}/wishlistdata/?p=0 is returning 99 games for me, instead of 100 - one game is missing.
(On a side note, I've found out that using the https://store.steampowered.com/wishlist/id/{steamID}/wishlistdata/ URL for those who have a customized ID, the returned number of games per page would be far from 100 - I've seen numbers like 68, 37,18, etc. It seems so random.)
Also, the returned list is stable in a short duration, but can change over time: Game A might be missing for this request and for an immediate request following it. However, after say 10 minutes, game A might come back and game B might be missing.

I think there's something funky going on with Steam's API. Perhaps the result returned depends on the caller/server's location, etc, but the games missing in my case are not even recently added to my wishlist, so cache is not accountable here.

This is bad, as it means there's no way to tell whether the removal of a game in the result is because the user has actually removed it, or beacuse the returned wishlist is not complete.

I think sending two duplicate requests one after another and combining the game list might at a very low possibility solve this issue, but it certainly is not an elegant solution.

[Vinfall]

It occurs to me that Steam API returns similar results (aka. inconsistent response) for user library regarding id & profiles. Sadly there is no such thing like https://steamcommunity.com/profiles/${STEAM_ID}/games/?tab=all&xml=1 to export everything at once for wishlist...

[icue]

Well, if the web API is just buggy like this, then I guess there's not much we can do.

I bumped into https://github.com/ValvePython/steam today, which might be able to get a user's wishlist, but it requires logging in / api key / stuff like that, which will make things way more complex. So I probably won't migrate to that just for 1 missing game.

[Vinfall]

Yeah, I discovered that a few days ago. Actually it cannot get wishlist data if I understand correctly. Searching wishlist in the docs only returned two functions that are removed.

There is also Steam Web API Documentation and Tester developed by xPaw (SteamDB guy), which requires developer key as well. The thing is, Steam Web API does NOT seem to have the required API at all... It has GetOwnedGames and GetWishlistItemCount (which returns only a number, not a list of wishlisted games) but not GetWishlistedGames or whatever.

[icue]

Ok. So the only way to get wishlist data is through the https://store.steampowered.com/wishlist... URL.

Actually, I found out that even visiting my wishlist from Steam webpage (either through Steam client or on my browser) suffers from the exact same issue - one random game is just missing. And that missing game is the same game missing from the JSON response. And this game changes over time.

I did another round of research and found that this bug has been reported by others as well:

• Reddit
• Steam forum
• Steam guide which covers a lot of wishlist shenanigans, and provides a way to check the missing game from browser - so yes, there is a way to know which app ID is missing.

I think the steps look like this:

1. Scrape the wishlist webpage (https://store.steampowered.com/wishlist/profiles/XXX/), in addition to the JSON, to get g_rgWishlistData and g_Wishlist.rgVisibleApps. Comparing these two, I can get the misisng games.
2. Scrape the store page of the missing games to get their release dates.

Maybe I'll work on it when I have the time, but defnitely not soon.

[Vinfall]

Or buy & play some games to keep wishlist below the 99 threshold XD, that's how I fix it.