Is it possible to run FMBench on an EC2 instance on endpoints currently not supported on Sagemaker.

[prasad-nair-amd]

For example i want to run an endpoint which is running on intel/amd/nvidia instances which is generally available. ie, instances which are not available on Sagemaker but available on AWS.

[aarora79]

We are working on a new FMBench release which supports hosting both the LLMs and FMBench on the same EC2 instance (so no SageMaker, no S3 dependancy) and this should enable you to test on AMD instances. Note that you can already do this today in a "bring your own endpoint" kind of way, see instructions here and config file examples here. What the next release would enable doing is to deploy the model on EC2 also through FMBench (although we have not yet to tried this on AMD instances so there would be some trial and error involved, in the meantime please take a look at the links above).