feat(pgsql): param node_hugepage_count accepts value -1 to set hugepage automatically (only PG 15+)
#541
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
waitingsong
force-pushed
the
pg-hugepage
branch
from
dd562df to
d47c722
Compare
waitingsong
changed the title
node_hugepage_count accepts value -1 to set hugepage automatically (only PG 15+)node_hugepage_count accepts value -1 to set hugepage automatically (only PG 15+)
waitingsong
force-pushed
the
pg-hugepage
branch
from
66093a0 to
ff0de41
Compare
waitingsong
force-pushed
the
pg-hugepage
branch
from
fadefd0 to
7221005
Compare
- check_pg_ready - is_patroni_paused - start postgres - start and stop postgres gracefully - patroni pause|resume gracefully
- patroni_restart - grace_patroni_restart - pg_read_hugepage - pg_write_hugepage
…page automatically (only PG 15+)
nr_hugepages calculated by `shared_memory_size_in_huge_pages`
if `node_hugepage_count` or `node_hugepage_ratio` changed, it will full backup the pg_cls and restart it gracefully
support usage:
- `pig install`
- `bin/pgsql-add <pg_cls>`
- `bin/pgsql-add <pg_cls> <ip>`
only new nodes changed and restarted
- `pgsql.yml -l <pg_cls> -t pg_hugepage`
calculation rules:
1. `shared_memory_size_in_huge_pages` (from PG15+): if `node_hugepage_count` is -1 and huge_pages (from PG) != off
2. if `node_hugepage_count` > 0
- `shared_memory_size_in_huge_pages`: if `shared_memory_size_in_huge_pages` > `node_hugepage_count`
- `node_hugepage_count`
3. disable hugepage if `node_hugepage_count` is 0
4. if `node_hugepage_ratio` > 0
- `shared_memory_size_in_huge_pages`: if `shared_memory_size_in_huge_pages` > pages calculated from `node_hugepage_ratio`
- page value calculated from `node_hugepage_ratio`
All above rules and usages are tested successfully on Rocky Linux 9
move import of pg_hugepage.yml and grace_patroni_restart.yml at bottom
waitingsong
force-pushed
the
pg-hugepage
branch
from
7221005 to
cce0370
Compare
tested scenarios: - `./pgsql.yml -l <pg_cls> -t pt_restart -e "pt_restart=true"` - all nodes have been restarted - `./pgsql.yml -l '<ip>,&<pg_cls>' -t pt_restart -e "pt_restart=true"` - nodes in ansible_play_hosts_all have been restarted, other nodes in the same pg_cls ignored - `./pgsql.yml -l '<replica>,<primary>,&<pg_cls>' -t pt_restart -e "pt_restart=true"` - nodes in ansible_play_hosts_all have been restarted, other nodes in the same pg_cls ignored - switchover triggered when patroni restart primary failed
waitingsong
force-pushed
the
pg-hugepage
branch
from
842058f to
ad05154
Compare
when new_nr_hugepages equals to pg_shared_memory_size_in_huge_pages
- full backup when ansible_play_hosts_all contains primary - full or incremental backup when ansible_play_hosts_all contains no primary tested scenarios: - `./pgsql.yml -l <pg_cls> -t pg_hugepage` - full backup - `./pgsql.yml -l '<replica>,<primary>,&<pg_cls>' -t pg_hugepage` - full backup - `./pgsql.yml -l '<replica>,&<pg_cls>' -t pg_hugepage` - full or incremental backup
waitingsong
force-pushed
the
pg-hugepage
branch
from
ab6d6a2 to
d6205bf
Compare
Vonng
force-pushed
the
main
branch
9 times, most recently
from
d672e7b to
1b4bee5
Compare
Vonng
force-pushed
the
main
branch
8 times, most recently
from
cc69e58 to
fae7528
Compare
Vonng
force-pushed
the
main
branch
2 times, most recently
from
d351d04 to
9690822
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
nr_hugepages calculated by
shared_memory_size_in_huge_pagesif
node_hugepage_countornode_hugepage_ratiochanged, it will full backup the pg_cls and restart it gracefullysupport usage:
pig installbin/pgsql-add <pg_cls>bin/pgsql-add <pg_cls> <ip>service of nodes in ansible_play_hosts_all will be unavailable during the process, other nodes in the same pg_cls ignored
pgsql.yml -t pg_hugepageentire service of all of the postgres clusters will be unavailable during the process
pgsql.yml -l <pg_cls> -t pg_hugepageentire service of this postgres cluster will be unavailable during the process
pgsql.yml -l '<ip>,&<pg_cls>' -t pg_hugepageservice of nodes in ansible_play_hosts_all will be unavailable during the process, other nodes in the same pg_cls ignored
calculation rules:
shared_memory_size_in_huge_pages(from PG15+): ifnode_hugepage_countis -1 and huge_pages (from PG) != offnode_hugepage_count> 0shared_memory_size_in_huge_pages: ifshared_memory_size_in_huge_pages>node_hugepage_countnode_hugepage_countnode_hugepage_countis 0node_hugepage_ratio> 0-
shared_memory_size_in_huge_pages: ifshared_memory_size_in_huge_pages> pages calculated fromnode_hugepage_ratio- page value calculated from
node_hugepage_ratioAll above rules and usages are tested successfully on Rocky Linux 9
patroni restart pg_cls gracefully, without switchover. switchover triggered when patroni restart primary failed.
tested usage:
./pgsql.yml -l <pg_cls> -t pt_restart -e "pt_restart=true"./pgsql.yml -l '<ip>,&<pg_cls>' -t pt_restart -e "pt_restart=true"./pgsql.yml -l '<replica>,<primary>,&<pg_cls>' -t pt_restart -e "pt_restart=true"inner steps:
pg-backup strategy before update hugepage
tested scenarios:
./pgsql.yml -l <pg_cls> -t pg_hugepage./pgsql.yml -l '<replica>,<primary>,&<pg_cls>' -t pg_hugepage./pgsql.yml -l '<replica>,&<pg_cls>' -t pg_hugepage