RDB testing

[hamadmarri]

This is a testing version for RDB.

https://github.com/hamadmarri/cacule-cpu-scheduler/blob/master/patches/CacULE/RDB/rdb-testing.patch
https://github.com/hamadmarri/linux/tree/cacule-5.12-rdb-testing

Please read the testing guide here: https://github.com/hamadmarri/cacule-cpu-scheduler/tree/master/patches/CacULE/RDB

Thank you

[JohnyPeaN]

Hi,
thanks for the work and detailed explanation. I am testing this on 5.11 becaue 5.12 has some bug with zfs for me. I have some problem with this version. Everything works, but frequency scaling acts weird. It seems that cores get stuck on last highest frequency with ondemand and schedutil governors. It works fine on cacule without RDB, although I have noticed weird acting cores on schedutil now that I'm looking closer.

The top picture is without rdb and bottom with rdb. It happens with and without nohz_full:

https://imgur.com/a/ArM26mx

The frequency graph is on the bottom (only 12 cores from 24 threads 0-11)
governor was changed from ondemand to schedutil whete there is a change.

[hamadmarri]

Hi @JohnyPeaN

I am afraid it's accounting issue. I will double check. What's the monitoring tool are you using here?

Thank you so much

[JohnyPeaN]

The tool is ksysguard in kde plasma. htop and nmon show the same. The simplest output can be achieved with:
watch grep "cpu MHz" /proc/cpuinfo

[hamadmarri]

Hi @JohnyPeaN

How to regenerate this behavior? By manually change governor from ondemand to schedutil?

[JohnyPeaN]

It doesn't work on eighter ondemand, schedutil and conservative. performance can be temporarily used for testing, but it would be better to find cause of the problem. I just switched them halfway to show none of them work (so it isnt just problem with specific governor).

You can switch frequency scaling governors live through /sys

get available governors:
cat /sys/devices/system/cpu/cpu0/cpufreq/scaling_available_governors
conservative ondemand userspace powersave performance schedutil

get active governor for all cores:
cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

set governor for all cores:
echo "ondemand" | sudo tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

If governors are missing from scaling_available_governors, it is possible they are compiled as modules and not loaded.
sudo modprobe cpufreq_ondemand
and ondemand should show up in scaling_available_governors

[dim-geo]

Just some general thoughts (haven't tested this RDB yet) Instead of analysing parents and children of processes for make/gcc to downprioritize them, why don't you suggest to 'nice' them? Noone should compile stuff without nicing the parent process. Nice priorities work ok with cacule/rdb?

[JohnyPeaN]

All of us are probably using nice as you suggest. But it is a good example for testing interactivity detection mechanism, so it works on tasks which behave as background jobs better than cfs even without nicing them. iow. better out-of-the-box experience for desktop.
For me it seems that cacule stays more interactive under heavy load without manually lowering priorities, so the effect of nicing a compile job is less evident, but it still has an effect.

[hamadmarri]

Just some general thoughts (haven't tested this RDB yet) Instead of analysing parents and children of processes for make/gcc to downprioritize them, why don't you suggest to 'nice' them? Noone should compile stuff without nicing the parent process. Nice priorities work ok with cacule/rdb?

Hi @dim-geo

Yes, nice priorities work with cacule/rdb, but you might see some differences on behavior compared with CFS. IS uses vruntime which includes the calculations of priority.

The autogroup is quite interesting, but I think RDB in design won't work well with cfs grouping.
nice 19 to make would help for sure, but I am trying to come up with automatic solution to help interactivity.

Thank you 👍

[hf29h8sh321]

Which kernel version is the patch for?

Edit: I applied regular patch then RDB on 5.12 and there are no patching conflicts. Is this the correct way?

[hamadmarri]

Hi @hf29h8sh321

Rdb patch should be applied on cacule-5.12

Thank you

[hamadmarri]

I tried to make RDB works with AUTOGROUP

https://github.com/hamadmarri/linux/tree/cacule-5.12-rdb-autogroup-testing

https://github.com/hamadmarri/cacule-cpu-scheduler/blob/master/patches/CacULE/RDB/rdb-autogroup-testing.patch

Please test, make sure to disable CONFIG_RDB_TASKS_GROUP

Thank you

[hamadmarri]

The optimal is either without RDB but with autogroup enabled. or with RDB but without autogroup.

If I'm testing RDB,should I use CONFIG_RDB_TASKS_GROUP and not CONFIG_SCHED_AUTOGROUP, or both disabled?

Both disabled.

[JohnyPeaN]

Both disabled.

Thanks, I will try again later and report back.

[JohnyPeaN]

I'm testing on game rdr2 now because there I see difference.

cacule, no background load = flawless

cacule, heavy background load (nice -19) = very choppy gameplay, little longer intervals than cfs, but simmilar behavior (cfs has very frequent drops like 10 per second, cacule lowers the drop frequency to something like 2 per second)

cacule+rdb, no background load = very smooth gameplay, but 1-2s freeze every few minutes.

cacule+rdb, heavy background load (nice -19) = choppy gameplay, but longer intervals smooth and longer freezes between them (1-10s smooth, 1-2s freeze)

I will do some more tests and try to write down more details when I have more time.

[hamadmarri]

I'm testing on game rdr2 now because there I see difference.

cacule, no background load = flawless

cacule, heavy background load (nice -19) = very choppy gameplay, little longer intervals than cfs, but simmilar behavior (cfs has very frequent drops like 10 per second, cacule lowers the drop frequency to something like 2 per second)

cacule+rdb, no background load = very smooth gameplay, but 1-2s freeze every few minutes.

cacule+rdb, heavy background load (nice -19) = choppy gameplay, but longer intervals smooth and longer freezes between them (1-10s smooth, 1-2s freeze)

I will do some more tests and try to write down more details when I have more time.

Hi @JohnyPeaN

Could you please try setting:
sysctl kernel.sched_interactivity_threshold=0

I believe there are some occasional background process that wake up every couple seconds. They would fall within the interactivity threshold and thus the select_task_rq will pick the most non interactive cpu for them which could mess up the tasks balance. By setting sched_interactivity_threshold to 0, the normal CFS select_task_rq will trigger for all wakeup/new tasks, and this could solve the pauses.

Thank you

[JohnyPeaN]

sysctl kernel.sched_interactivity_threshold=0 does solve the occasional freezes on calcule+rdb and no background load.
With background load it does help, freezes still occur but seem to be shorter.