Problem with hostgroups not properly being deleted and limit of 100 hostgroups

[ShawnMcKee]

We have found that there seems to be a limit of 100 hostgroups in PWA. Once we hit that limit, you can create a new hostgroup but it will not persist. Logging out and logging back into PWA, the hostgroup you created is lost.

Our guess is that the delete process (trashcan icon in the upper right) does not persistently delete hostgroups AND that there is a 100 group limit. We currently see 100 hostgroups on the WLCG instance, even though when we count hostgroups it is around 75. The specific instance is visible at https://psconfig.opensciencegrid.org/#!/hostgroups/58f749d7f5139f0021ac29d0 (assuming you have an account).

We may have to explore manually correcting the data in MongoDB :(

[arlake228]

@ShawnMcKee its possible that this line is the culprit of it not returning at least:

psconfig-web/psconfig-web-admin/perfsonar-psconfig-web-admin/api/admin/controllers/hostgroups.js

Line 52 in 3ce7ccf

.limit(parseInt(req.query.limit) || 100)

Looks like the API only returns 100 hostgroups max. A short-term workaround might be to bump that value.

I see similar limits here and here:

psconfig-web/psconfig-web-admin/perfsonar-psconfig-web-admin/api/admin/controllers/configs.js

Line 69 in 3ce7ccf

.limit(parseInt(req.query.limit) || 100)

psconfig-web/psconfig-web-admin/perfsonar-psconfig-web-admin/api/admin/controllers/hosts.js

Line 52 in 3ce7ccf

.limit(parseInt(req.query.limit) || 100)

As you are aware, I am not an expert on this code. Don't recall if those files live as shownn in the container or are minified. If those indeed are the issue, we can likely spin a new container.

[ShawnMcKee]

Thanks Andy, can we try a new container with a 200 group limit, if it is straightforward to create? I can then restart docker with the new version.

[ShawnMcKee]

Just a follow up that this worked and the "missing" hostgroups now show up. We had one issue because we were originally using the "bleeding-edge" tag which had support to transferring RabbitMQ credentials if the requesting host matched the host config being requested. This configuration was removed and the 5.0.0 tag works for WLCG/OSG.