Add support to enable instance based storage for data nodes #134
Conversation
rishabh6788
requested review from
peterzhuamazon,
gaiksaya,
zelinh,
prudhvigodithi and
Divyaasm
as code owners
| { | ||
| ignoreErrors: false, | ||
| })); | ||
| currentWorkDir = '/mnt/data'; |
There was a problem hiding this comment.
so does this mean that now opensearch will be installed at /mnt/data folder for both instance based storage? and for all others it will be at /home/ec2-user?
There was a problem hiding this comment.
yes, if instance based storage then the nvme1n1 disk will be mounted on /mnt/data path and OS will be started in that path, if not selected then everything will remain as usual.
Mounting existing /home/ec2-user also works but it is not the ideal way.
We should be unmouting it first and then mount it again, and by any chance if that disk is busy it will cause failures and make the process cumbersome.
There was a problem hiding this comment.
Got it make sense. Can we add this information in the readme?
|
@gaiksaya This PR also adds the change to replace ASG Launch configuration with Launch Templates as the former is getting deprecated and doesn't support ec2 instance types released after Dec-31, 2022. See https://aws.amazon.com/blogs/compute/amazon-ec2-auto-scaling-will-no-longer-add-support-for-new-ec2-features-to-launch-configurations/ |
| volume: BlockDeviceVolume.ebs(50, { deleteOnTermination: true, volumeType: props.storageVolumeType }), | ||
| }], | ||
| requireImdsv2: true, | ||
| userData: UserData.forLinux(), |
There was a problem hiding this comment.
confused about forLinux() here. Looks like userdata is optional right? https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_ec2.LaunchTemplate.html#userdata and all installations we are running via cfnInit?
There was a problem hiding this comment.
Not optional when using LaunchTemplate, this needs to be declared. This adds below:
UserData:
Fn::Base64:
Fn::Join:
- ""
- - |-
#!/bin/bash
# fingerprint: 1ef4754d2d6b55e6
(
set +e
/opt/aws/bin/cfn-init -v --region
- Ref: AWS::Region
- " --stack "
- Ref: AWS::StackName
- |-2
--resource dataNodeAsgASGA92884EE -c default
/opt/aws/bin/cfn-signal -e $? --region
- Ref: AWS::Region
- " --stack "
- Ref: AWS::StackName
- |-2
--resource dataNodeAsgASGA92884EE
cat /var/log/cfn-init.log >&2
)
This was getting added previously as part of Launch Configuration.
Thanks! Can you bump the version accordingly (major or minor depending on breaking changes). Would need to consume new version in nightly playgrounds. |
Signed-off-by: Rishabh Singh <[email protected]>
Done! |
Description
This PR adds support to mount instance-based storage disk for data node instance types that support this feature.
These are SSD nvme volumes attached to ec2 instance that provide better iops and throughput compared to ebs volumes.
We have added a new context parameter
useInstanceBasedStorageto allow user to be able to mount these volumes and use them to run OpenSearch service.This is only enabled for data-nodes in both single-node cluster and multi-node cluster mode.
This PR also adds the change to replace ASG Launch configuration with Launch Templates as the former is getting deprecated and doesn't support ec2 instance types released after Dec-31, 2022. See https://aws.amazon.com/blogs/compute/amazon-ec2-auto-scaling-will-no-longer-add-support-for-new-ec2-features-to-launch-configurations/
Issues Resolved
#132
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.