Sort order in merged views (folders or "unread") does grouping by feed ressource and does not sort by date only

[rankerson]

IMPORTANT

Read and tick the following checkbox after you have created the issue or place an x inside the brackets ;)

• I have read the CONTRIBUTING.md and followed the provided tips
• I accept that the issue will be closed without comment if I do not check here
• I accept that the issue will be closed without comment if I do not fill out all items in the issue template.

Explain the Problem

I want to view my new items in the newsfeed (either in the overview "unread" or clicking on a folder, which includes several feeds. My expectation would be, that all collected feed items (of all feeds in the respective folder or all feeds, if the view "unread" is choosen) are sorted by date (either ASC or DESC depending on the sort order selected in the settings).

I see now, that the feed items are sorted by date in the corresponding sort order, BUT grouped by feed. So I see first all items of one feed sorted by date and then the items of the next feed sorted by date. My expectation is that all feed items are sorted by date w/o grouping by feed. If my expectation does not fit the idea I can also create a pull request.

Steps to Reproduce

Explain what you did to encounter the issue

1. Create a subfolder with twi different feed sources.
2. Click on the folder and check the sort order of feed items and if they are grouped by feed ressource

System Information

• News app version: 24.0.0
• Nextcloud version: 26.0.7
• Cron type: (system cron/python updater/...) system cron
• PHP version: 8.2.11
• Database and version: mysql, 11.2.2
• Browser and version: Chrome 123.0.6312.86
• OS and version: Linux 6.1.0-17-cloud-amd64 x86_64

I skipped the following log-Output as no error message occured and I guess it is just a wrong resp. unexpected behaviour. If additional logs are needed, do not hesitate to ask for it.

[Grotax]

Hi, I think this grouping is a side effect of the collection time, each feed is fetched one by one and added to the database.

The items are then sorted by the date they were added. And if I remember right this is done because the date that the feeds use is not very reliable.

But I'm not a 100% sure. The logic is in the data mappers or the service for items

[rankerson]

Hi, ah, ok. Understand this, but does it makes sense? I understand, that sometimes the date from the feed itself might not be appropiate, but is this not the exeption?

Cheers
Ranki

[Grotax]

I guess in the end it would be best to leave this up to the user.

Currently the main topic is that the UI is rewritten with vuejs and that the missing features are added and the bugs are fixed after that we could think again about item sorting

[wofferl]

The problem here is that the frontend load 40 items at a time and get it sorted by item id from the backend. In merged views sorting by date would mean that items that are loaded later can be sorted above the actual items viewed.
Sorting by date seems difficult, because you either have to load all items for a view or the backend have to be rewritten to give the items sorted by date, which I think would not that easy.

[SMillerDev]

This bug can be solved, it's just that the people that do the work aren't affected by it. And the people that are affected by it don't do the work to fix it.

[wofferl]

The topic is as old as the news app itself and I wouldn't call the problem a bug either. It is currently implemented this way and it won't be that easy to change it.

#51
#227
#782

You can't compare the app with a normal standalone client that has its own database and synchronizes the articles from the API or fetch it on its own.

The frontend itself only keeps a small relevant part of the articles, so that sorting purely by PubDate is not possible or leads to problems. The frontend also has no influence on when new articles are fetched into the database.

When updating, the backend fetches feed by feed, which means that they are stored in the database in “groups”. The shorter the retrieval interval, the smaller the groups.

As the frontend does not hold its own database, the data is not synchronized, but is requested freshly each time from the API. This happens in groups of 40 articles. The backend delivers the data sorted by ItemID, which corresponds to the date the article was fetched. The frontend only keeps the last ItemID in order to request the next 40 items when scrolling.

If you now simply sort by PubDate in the frontend, this would lead to problems with the infinite scroll, as new items have to be re-sorted when they are loaded. This leads to items being sorted at the top and bottom.

What could be currently done about this in the frontend?

• Ignore it and let the user navigate back when new items appear above in the list?
• Clear the item list after 40 items and display the next 40 items? Which would also only be a partial sort?
• Load always all items, at the cost of performance?

What would be a solution with a change in the backend and the API?

• The backend could sort all items when fetching. But even here there could still be grouping if a feed does not have a proper PubDate.
• The API could deliver the items to the frontend sorted by PubDate. But how would the frontend fetch items in groups than? Asking for 40 items with a newer PubDate of the latest item it got last time won't work here, because new items with an older PubDate could be fetched in between.

The simplest change would be to show the item added date instead of the PubDate in the list or removing it entirely.

If anyone else has a solution, please feel free to share it.

[wofferl]

As a further note, in the beginning the effect of grouping is much bigger because each feed is loaded completely. After the first fetch cycles, however, this is minimized, depending on the feeds.