Improve and document test suites#404
Conversation
7bf0a63 to
5e68f83
Compare
This commit improves the 007 prove test: PG oredering is either case sensitive or case insensitive depending on system settings. The testcase uses ordering based on the query string, with some uppercase and some lowercase queries, causing resulting in different outputs on different systems. This change modifies the ordering to use lower(query) to ensure consistent results everywhere. It also adds a TESTING markdown file describing how to run the test suite when building pg ang pgsm from source.
Naeem-Akhter
left a comment
There was a problem hiding this comment.
With these changes said test case is failing on PGDG 12,13,14. It needs to addressed on these PGDG versions.
First for making one expected file following code needs to be commented or removed in pgsm.pm.
if ($PG_MAJOR_VERSION >= 15 and "$filename_without_extension" == "007_settings_pgsm_query_shared_buffer")
{
$expected_filename = "${expected_filename}.${PG_MAJOR_VERSION}";
}
Then keep one expected output file instead of following two:
t/expected/007_settings_pgsm_query_shared_buffer.out
t/expected/007_settings_pgsm_query_shared_buffer.out.15
After doing above changes you will get the following diff on PGDG 14,13,12.
example | copy pgbench_accounts from stdin with (freeze on)
example | copy pgbench_accounts from stdin
Even if you disable utility statements in t/007_settings_pgsm_query_shared_buffer.pl with following addition, you will still some diff on PGDG 12,13.
print $conf "pg_stat_monitor.pgsm_track_utility = 'off'\n";
So, I suspect it will require more investigation and settings changes where test case doesn't fail on any of PPG 11 ~ 15, PGDG 11 ~ 15 and code-coverage workflows.
|
This is highly outdated and we have plans to revamp the test suite in the near future, so I'm closing this. |
This commit improves the 007 prove test: PG oredering is either case sensitive or case insensitive depending on system settings. The testcase uses ordering based on the query string, with some uppercase and some lowercase queries, causing resulting in different outputs on different systems. This change modifies the ordering to use lower(query) to ensure consistent results everywhere.
It also adds a TESTING markdown file describing how to run the test suite when building pg ang pgsm from source.