Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pmseries --load is extremey slow loading into redis. #1869

Open
YFLOPS opened this issue Jan 3, 2024 · 0 comments
Open

pmseries --load is extremey slow loading into redis. #1869

YFLOPS opened this issue Jan 3, 2024 · 0 comments

Comments

@YFLOPS
Copy link

YFLOPS commented Jan 3, 2024

I've noticed any loading of pmseries data from archives is extremely slow. This occurs on three different systems.

Ubuntu 22.04 - PCP 5.3.6 (very slow)
RHEL 8.3 - PCP 5.3.7 (very slow)
Rocky 9.3 - PCP 6.0.5 (slow)

We run jobs though a batch scheduler on clusters as well as in the cloud, and store only the information from PCP per job. Most of them are only minutes or hours long, but sometimes they are a day or two and this can either cause a segfault or take many minutes to import. The 5.x PCP are prone to segfault, but looks to have been addressed in the below bug report that also notices slow Redis load performance. Below are slow imports on PCP 6.0.5, the larger one segfaults on the older versions.

This is a 122MB import for a medium length job.

pmseries: [Info] processed 8714 archive records from node3.cluster

real 1m55.443s
user 1m48.915s
sys 0m7.293s

This is a 1.3GB import for a longer running job.
pmseries: [Info] processed 275313 archive records from node3.cluster

real 43m59.457s
user 41m26.807s
sys 3m12.486s

Red Hat noted this in a bug, but no action was taken and it was closed as errata.

https://bugzilla.redhat.com/show_bug.cgi?id=2027430

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant