Pull in latest from Facebook #25
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Summary: Fixes Style/GlobalStdStream in OSS CI Reviewed By: joshuamiller01 Differential Revision: D45399133 fbshipit-source-id: 5f5f489bc38ec894cfc96e10c15f29e3d451f05e
… disruptive chef code Summary: A method to define when a host can run disruptive chef code. By default this should be allowed during provisioning and boot (when the host is presumably drained and unable to be running a workload) Reviewed By: gogsbread Differential Revision: D34600530 fbshipit-source-id: d741d256e47df94bd9360dca7bc628b3a45f6d3b
Summary: Refactor autostart handling for old systemd versions. This makes the code easier to read in differentiating when we're setting up timers the normal way (with modern systemd) versus the old way, and will make future transitioning to a different model (forthcoming) simpler also. This should be a no-op; node['packages'] is ohai-time and reordering the evaluation within a custom resource should be irrelevant. Reviewed By: dafyddcrosby Differential Revision: D45199845 fbshipit-source-id: 12b6ab20c90c59d1c9660f411de9fb29f2b295c4
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff expand actions on CentOS to all other RHEL_Family platforms. This diff needs stamp from [[chefit](https://www.internalfb.com/permission/group/965392640256092), [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697)] Reviewed By: joshuamiller01 Differential Revision: D45261981 fbshipit-source-id: 5eded266a277b639ca0723394d937f222908a983
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff just used latest rhel_family? functions and added more supported platforms. No effects on existing systems. This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45111725 fbshipit-source-id: 6bf7af74b90a741ac71ea19b1bc169f60f4739d0
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff simply add more supported platforms in the metadata This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45262482 fbshipit-source-id: a1e10897b739c29e96123fcdccf8b038111a3bf3
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff expand actions on CentOS to all other RHEL_Family platforms. This diff simply add more supported platforms in the metadate This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45263447 fbshipit-source-id: 200368833ea06464132953e0fb43966a3c0475fe
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff simply add more supported platforms in the metadate This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45263741 fbshipit-source-id: 044c581e251964c725d8fe4741a346ac1d515c3d
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff expand actions on CentOS to all other RHEL_Family platforms. This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45263995 fbshipit-source-id: 2c87394d302f94906232e9e897dd65b8ffca3181
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff simply add more supported platforms in the metadate This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45264038 fbshipit-source-id: 2bd9d1d0cc81ee9cef8a8b0eba9a36c3a04dd8ec
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff simply add more supported platforms in the metadate This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45264076 fbshipit-source-id: 13415cd3211344b0cadc2538478a2457f8c3c38a
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff simply add more supported platforms in the metadate This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45264112 fbshipit-source-id: 112387fbff20bbfedb7a49b926d308a80ef7c4d6
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff simply add more supported platforms in the metadate This diff needs stamp from [[chefit](https://www.internalfb.com/permission/group/965392640256092), [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697)] Differential Revision: D45264277 fbshipit-source-id: 3bfd6e6ec59c03b9c9149e24d4d3c3f4cbaf9e93
Summary: As titled. By checking the [chef infra](https://fburl.com/0rmv1bnc) and [Meta implementation](https://fburl.com/code/tzjadwep), we know: - *platform*, **centos** and **redhat** are two platforms names - *platform_family*, **rhel** is a family name that contains centos, redhat, rocky and more. More context can be found in this [post](https://fburl.com/workplace/1qm17oop). It looks we have Centos only in prod ([1](https://fburl.com/scuba/chef/k4nsvzfe) and [2](https://fburl.com/scuba/chef/t6xy763c)), and CentOS, Redhat and OEL platforms in [IT/Corp](https://fburl.com/scuba/ifs_chef/l0wj716g). And we have no host running with version<7. This diff simply add more supported platforms in the metadate This diff needs stamp from [hm_data_plane](https://www.internalfb.com/permission/group/437546304210697) Differential Revision: D45264375 fbshipit-source-id: 570242a54d1237259ce2babb6940b2b7e1170561
Summary: * Moved existing logic to extract disk model, firmware and disks in mdarray as `FB::Storage` methods * No logic change to hosts with rootfs mounted on single disk. * On hosts where root is mounted on mdarray, we will now encode the sorted models, relative firmwares to aggregation labels. ``` # Separated by __ which is acceptable in ODS <CUSTOM_ROOT_DISK_MODEL>:__RAID__MZ1LB1T9HBLS-000FB__MZ1LB1T9HBLS-000FB__; <CUSTOM_ROOT_DISK_FIRMWARE>:__RAID__EDW76F2Q__EDW76F2Q__; ``` Reviewed By: joshuamiller01 Differential Revision: D45419604 fbshipit-source-id: 25c37ad73ae55c2af609c0bf662400788baf179d
…ive chef resources Summary: Manage timers directly through systemd rather than via native chef resources. This is much more CPU and time performant; managing 50 timer units with the native service chef resource requires 50 expensive shellouts via systemctl (even when no change needs to be made), which we can collapse into a single shellout to gather the state of all 50 units, and then a single call to enable, and start. Reviewed By: dafyddcrosby Differential Revision: D45170124 fbshipit-source-id: 43dc16c23d882dec660816290d12f9b451cbdfbe
…de.rm Summary: If during a loop you modify an element of the node object, and then read from the same ImmutibleHash, it causes some kind of overhead which costs non-trivial amounts of CPU when iterating enough times. In other words: if you modify node.default['something'][...], then read from node['something'], and then loop and do that again and again, you get significant perf overhead. If the read is removed, this overhead is avoided; this diff takes an unnecessary read of `node['fb_timers']['_timer_path']` and captures it in an lvar, which saves ~5ms per iteration in my environment. While we're here, also switch from node.rm to node.rm_default, because that's also much less CPU expensive. Reviewed By: dafyddcrosby Differential Revision: D45459381 fbshipit-source-id: c641c67a3a6868a6bccb63d4b507a1d510295dad
Summary: Currently we do not enforce the fluentbit service is set to run at startup and kept running. One can manually disable/stop the service and it will not self-reenable. Lets add an attribute to control enforcement of the service state. Reviewed By: joshuamiller01 Differential Revision: D44970184 fbshipit-source-id: 482ae4678de51a317db3eec3c6f3e0b835be317b
Summary: Fix require_relative path in libraries_spec to work in GH Differential Revision: D45500711 fbshipit-source-id: 46df08e140fb11ce08e122546df0110f5e9c4146
…ive chef resources 1% -> 2% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501926 fbshipit-source-id: bd522655399aa057425e55824055ebeca0ced08a
…ive chef resources 2% -> 5% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501921 fbshipit-source-id: 981413141baca9bfea9d21731fec5b1da96d2b59
…ive chef resources 5% -> 15% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501924 fbshipit-source-id: 99a79da6c88922783fa855592fc187600fb79dfc
…ive chef resources 15% -> 25% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501922 fbshipit-source-id: 39e06bee2ece139b4bec6ab2e452d5fcae41c44b
…ive chef resources 25% -> 40% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501923 fbshipit-source-id: 51fe2cb4e7a62bc150e39ccdfc4f4e0768148326
…ive chef resources 40% -> 60% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501918 fbshipit-source-id: db700be5de63820f917a589fa53fbd0f85bcdc3d
…ive chef resources 60% -> 80% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501920 fbshipit-source-id: 97895230dffabc6788c657af49d2c0f1ddaf35c0
Summary: Newer rsyslog has a bug in container builds where it exits immediately. While we figure out why (which is more complicated than one would expect), reflect changes in D45715090 into Chef solo runs so that booted container runs (i.e. Chef Solo runs) based on images built with booted container runs (i.e. Tupperware Base Image derivatives, in our case the Sandcastle Worker image) once again work. Reviewed By: wilkotom Differential Revision: D45725790 fbshipit-source-id: a943ef64d8bea056ab2243fa7ac0d9d925450d89
…ive chef resources 80% -> 100% Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Reviewed By: dafyddcrosby Differential Revision: D45501919 fbshipit-source-id: 24f55f4cf5e24faf24107fc1d7289132768dcf69
Summary: Reviewed By: AadityaNair Differential Revision: D45713557 fbshipit-source-id: a4573894ca8b4b6835c39b27f680e9c20ad0bf4b
Summary: Reviewed By: DanielGrzegorzewski Differential Revision: D45709812 fbshipit-source-id: 981a95f5c31130442ce9fda5b66137fac3226f74
Summary: Dracut gets a copy of `/etc/fstab` to know what disks to mount before changeroot, so we need to make sure that any change (like reconfiguring the mount options) is reflected into dracut's copy Differential Revision: D57394916 fbshipit-source-id: 04b745c47bb5098d6856ab0780ec2bc11e9fd7ad
Summary: > Removed over_provisioning_mapping from fb_storage cookbook Differential Revision: D57170934 fbshipit-source-id: 1c91d411dac4c3632065fe390446daea58d6b207
Summary: Just fixing grammar in README Differential Revision: D57492309 fbshipit-source-id: 6b21c4a9218334778b1747b300f552db06af999b
Differential Revision: D48831692 fbshipit-source-id: c6d0508915f7f21d8d4ecaf2a7d5af560bd177bd
Differential Revision: D57674868 fbshipit-source-id: d0c82ffbedb9908684f7a7a26916ef773e833a40
Differential Revision: D57728699 fbshipit-source-id: d340eb5d677cd9342853d32b6ee4895ca24cb326
Summary: This is a very basic PR I know. But it cleans up the deprecation warnings so I thought I'd add it. Pull Request resolved: facebook#239 Test Plan: Imported from GitHub, without a `Test Plan:` line. Checked https://github.com/facebook/chef-cookbooks/pull/239/checks, deprecation notices are gone Differential Revision: D57752486 fbshipit-source-id: aa4accc12e6dcbab47e6a1637502eec5f17f508e
Summary: Just run the previous diff on the rest of the code base to clean everything up. Only github showed something. ``` $ hg show | grep '^[-+] ' | grep -vEe "^- *(owner|group) *'root'\$" -e '^\+ *(owner|group) *node.root_(user|group)$' ``` So it literally just changes what it's supposed to. Differential Revision: D56219068 fbshipit-source-id: 5084f44b47978ad885ff1a98cda9af7efbde3a56
Differential Revision: D49900503 fbshipit-source-id: 6372c56d38d161a8a819fc3d0a0698b8ee423153
Differential Revision: D58093895 fbshipit-source-id: 1079cfbbff3858c39dba2f496c2c721f23a29d81
Differential Revision: D55665988 fbshipit-source-id: 835ced0078917565168f4ce771f8e9d3df894371
Differential Revision: D58749691 fbshipit-source-id: 99a37e1cb13dad77d5cd14e85da4368765f06415
Summary: eag-prod-2 has three different kinds of T16, each with slightly different storage capacities. This diff makes such that each of the smaller capacity machines will format its disks to have partitions passed to the VM that are the same size as the largest machine. Confirmed we currently have only one of these machines for each FBPN. Other anvils should not be impacted. Had to add "s" to the allowed suffixes that can be passed for partition start/end in fb_storage so that we can be precise in partition sizing. Differential Revision: D58845625 fbshipit-source-id: a63b8fb28170053845e66423d27e15d89cd40bec
Differential Revision: D59270448 fbshipit-source-id: 4ae81e326262083fded5f97fa888c50d4e0d597b
Differential Revision: D58844542 fbshipit-source-id: 7fe52e3b5c24b0c3cf79f1b8fa70cad1a36144f5
Differential Revision: D59322935 fbshipit-source-id: 47688f9b05c3b5f2afebf51b13eff4396b10e7c2
Summary: - Let's defer setting up systemd timers till after workload.ready on boxes - These should not be needed for provisioning a new box ready for workload - If there are, we should fix (e.g. explicitly run the timers .service file or command from chef) - Use not_if to shard out and eventually gate all of our infra out of first boot setup of systemd timers - Good for OSS people to do too ... Differential Revision: D59290162 fbshipit-source-id: 52168797949fb4fae5330d301c1b9deb5a0116b4
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290161 fbshipit-source-id: 2c205a1af8baf40085aa20d79b482efeb98b1d04
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290158 fbshipit-source-id: cd718ff4bd76fca1de79cf072477cce66df7237a
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290159 fbshipit-source-id: 7ff92a8ee50230f6f8df22b102a93e96d11685ee
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290160 fbshipit-source-id: 6634fc2395553e8862b708ca57477a54514037cd
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290163 fbshipit-source-id: 14c1aa66d18d99f27a1815732f2f6f8f380ac662
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290157 fbshipit-source-id: 718d22bbaef2d5aa772f06ba8531c22a78090122
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290156 fbshipit-source-id: e00739ab6754daaa0c45cd3a41142935831da447
Summary: bumpsign This diff was created using http://fburl.com/chefshardbump Differential Revision: D59290155 fbshipit-source-id: fbd2e93fae947c7f4999d527c12642251ac3b5f3
PatrickDale
approved these changes
Jul 22, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Pull in latest from Facebook. I'm usually partial to rebase + force push, but we're doing a merge commit here to preserve more history.
I recommend viewing
git show 452a5bc(the merge commit).Context / Why are we making this change?
To not drift too far from upstream.
Testing and QA Plan
We should probably do a code review as a group and see if there are any big changes in recipes we're using. Rollout tbd.
Impact
Keep our fork close to upstream. Get whatever bugfixes and improvements from Facebook.