[META] Features missing from Project Search Pages

[wesruv]

Modules Listing Page:

• RE: #95: Add More Info links on project pages #96 "More Info about Modules" link - link to https://api.backdropcms.org/modules
• Add exposed filters to project listings #169 "Filter" Dropdown, suggested filters are module 'packages' from info file: like Administration, Education, Commerce, etc
• Add exposed sorts to project listings #170 "Sort" Dropdown, some example sorts are: Most Relevant, Most Downloads
• "Downloads" stat
• "Active Installs" stat

Module Page

• Primary Download link - this is kind of there, I'm hacking a release table to make it look like the mock. Think we should have a primary link at the top and a release table with multiple options below the README (for mobile) and the flyout for desktop
• Right sidebar
  1. Version/See all verions bit
  2. Downloads / Active Installs / Total Installs stats
  3. Github links
  4. Clone URL
  5. Categories
• Versions flyout
• Remove tabs from current modules page (View and Releases)
  

Themes

• Need ability to add a thumbnail to themes
• Figure out how to pull contributor(s) in a sane way
• Tabbed page view with tabs that have filters
  1. Ability to filter by theme type (Scaffolding vs Design)

[jenlampton]

"More Info about Modules" link - do we already have or do we need a page to explain modules in detail?

We were going to link this to this page:
https://api.backdropcms.org/modules

[docwilmot]

"Filter" Dropdown, suggested filters are module 'packages' from info file: like Administration, Education, Commerce, etc

Need ability to add a thumbnail to themes

Figure out how to pull contributor(s) in a sane way

Ability to filter by theme type (Scaffolding vs Design)

We'll have issues with all of the above. Discussed over here: #8 (comment)

• "Packages" are useless;
• thumbnails are limited to one tiny thing; it was proposed (can't find the issue) to consider preview[] = path/to/preview.png to enable multiple previews with proper sized images;
• there is no way to get the project author from GitHub since all projects have "backdrop-contrib" as the author;
• similar issue as modules for theme categories.

We'll need info files to start providing a lot more info that they currently do.

I have a PR for adding tags[] = xxx to modules but I think its been relegated to Backdrop 2.0, although I think it wouldnt change anything much since tags = package really.

I know its a lot to ask, but I wonder if we should consider starting this extra .info data now, rather than having to convert again later. I don't see how we can get the functionality @wesruv is suggesting without that being done.

[wesruv]

These are @dariusgarza's wonderful designs, he can speak to intention better than I can :P

It's also likely I'm not remembering everything we talked about in review of these, so don't take my descriptions above as gospel :)

I think it's safe to say we can put off parts of the design or update the design depending on what's plausible. We'll just need to hash out what we've got and how we can make a good user experience out of that.

If we find that the 'project search' is lacking after that process THEN we can work out how we can get better data to make a better UX.

[docwilmot]

I should have said the design ideas are excellent, and I agree the aim of this is to work on the design. Post was just to raise awareness that we don't have access to the data to put into these designs, and probably won't without some other work. Agreed of course that this must go ahead with what we've got.

[klonos]

• "Packages" are useless;

I don't want to get us -yet again- into an endless bike-shading discussion, but since we did implement backdrop/backdrop-issues#494, we might as well provide some similar way of browsing modules by package/category ...helps newcomers a great deal. Especially when we reach a module count of 100s or 1000s.

[klonos]

• thumbnails are limited to one tiny thing; it was proposed (can't find the issue) to consider preview[] = path/to/preview.png to enable multiple previews with proper sized images;

backdrop/backdrop-issues#1127

[klonos]

...and yes! ...excellent designs!

...this must go ahead with what we've got.

I fully agree on this one too.

[klonos]

Filed a separate bug report for the search issue. It needs to be fixed ASAP!

[xmacinfo]

Interesting discussion! It is nice to see the module pages taking form.

[serundeputy]

"Packages" are useless;
thumbnails are limited to one tiny thing; it was proposed (can't find the issue) to consider preview[] = path/to/preview.png to enable multiple previews with proper sized images;
there is no way to get the project author from GitHub since all projects have "backdrop-contrib" as the author;
similar issue as modules for theme categories.

Much of this information can be obtained through the the github API (https://developer.github.com/v3/#current-version) for example:

Project info for webform: https://api.github.com/repos/backdrop-contrib/webform
Release info for webform: https://api.github.com/repos/backdrop-contrib/webform/releases

There is a rate limit on github API requests 5000/hr (https://developer.github.com/v3/#rate-limiting), but we could authorize an a token and write a module that polls the github API say once per day and take the info and update the project nodes.

No need to add all the additional store in *.info files I don't know if that is better or worse, but would allow the .info files to stay as simple as possible.

[jenlampton]

hm. I like this @serundeputy! less work for humans, more work for robots :)

[docwilmot]

Thanks, @serundeputy for that, didnt know you could get all that from the API. Re my statements, I still stand by most of that though.

• "packages are useless" was a bit harsh. That statement was influenced by the suggestion to kill packages and replace them with tags, thus allowing themes and layouts to also have "packages", and allowing multiple categories for modules. There's no simple solution for the former.
• would still be unable to have only one thumbnail with the GitHub API info, can't see any way that helps that
• same API only gives the name of the person who did a release, not the repo owner, but admittedly that for our purposes is good enough indeed.

Finally, why should *.info files stay "simple"? After you write a 20000 line module, a few lines in the *.info file to ensure your project info is accurate shouldnt be a great burden, I'd say. They've always been simple because it wasnt necessary for Drupal, but we're aiming to be better, not same.

[jenlampton]

would still be unable to have only one thumbnail with the GitHub API info, can't see any way that helps that

See backdrop/backdrop-issues#1517

[xmacinfo]

There is a rate limit on github API requests 5000/h

Why not switch to Gitlab?

https://about.gitlab.com/

[jenlampton]

Why not switch to Gitlab?

Because it's not GitHub ;) The fact that GitHub is GitHub is our biggest reason for being on GitHub

[serundeputy]

I did some work on this today. Add a field to the project_module node types called download_count.

I have code to fetch the relevant data from the github API, but I need a github access token created by someone who owns the backdrop-contrib repo so that I can poll the API for the 200+ plus contrib projects nightly and populate the new field.

Once I have that I can put the field into the search view for the /modules page and expose a sort on that column.

Screenie:

[jenlampton]

Can you use the github API module? It already does the access token magic,
I think.

[serundeputy]

@jenlampton

I could/can use the github API module, but the token for that is most likely stored in settings.php which is fine and good. I need two things:

1. The token has to be set up to have read access to the backdrop-contrib repo. The github API module probably has access to /backdrop/backdrop-issues and maybe backdrop/backdrop if it has access to backdrop-contrib then we are good.
2. If the github API token meets those criteria then I just need you to email me the token. If it does not I need someone with admin to backdrop-contrib on github to create a token with read access to that repo and email it to me.

thanks,
~Geoff

[jenlampton]

As a member of the security team, you should have access to backdrop-contrib. Can you find what you need in the UI? I'm a bit lost as to where to set up tokens... I'm not really sure how to get you what you need. Maybe we should do a screenshare if you still don't have access, and you can help me find it? :)

[serundeputy]

I've got the token I needed to authenticate to the github API as backdrop-contrib.

I've written a module to get the download data for each release and aggregate total downloads for modules, themes and layouts. I've added a field_download_count to each of the content types project_module, project_theme and project_layout and the new module borg_project_metrics saves the download count for each project in that field for the node corresponding to the github project.

Layout node page:

Modules search page:

I still need to scope the hook_cron to once daily say 2AM EST?

If this seems like a good course of action I can clean it up and make a PR anyone interested in taking on the front end theming?

thanks,
~Geoff

[jenlampton]

@serundeputy this is amazing!!! I'd be happy to add the front-end theming as soon as you are ready :)

[serundeputy]

@jenlampton I've filed a PR at #284

It adds the field_download_count to the node types project_module, project_theme and project_layout.

It adds a new module borg_project_metrics the module polls github for downloads of each official release of a project and sums up a a total and populates the node fields.

I've updated the /modules, /themes and /layouts views to have the new field.

I think you can take it away for theming; I did not add filters to the views.

~Geoff

[jenlampton]

PR at #284 merged. Module borg_project_metrics enabled. @quicksketch wonders if this shouldn't be something that's merged upstream into project_github https://github.com/backdrop-contrib/project/tree/1.x-1.x/project_github ?

[jenlampton]

hm. Didn't mean to close that!

[jenlampton]

zomg: https://backdropcms.org/project/addanother

[serundeputy]

@jenlampton another PR to support getting downloads for the core node type.
#285
~Geoff

[jenlampton]

Whoops, closing prematurely... again! sorry folks. :)