[FEATURE] Testing clients compatibility on different platforms

[saratvemulapalli]

Is your feature request related to a problem?
Coming from: opensearch-project/opensearch-ruby#62 (comment)
We test our clients for Linux platform but we do not know if these clients work smoothly on platforms like windows/mac etc.

What solution would you like?
Test all clients with most popular platforms and CPU architectures:

• x64
• ARM
• Linux
• Windows
• MacOS

Tracking for clients:

• opensearch-java - [FEATURE] Testing compatibility on different platforms opensearch-java#207
• opensearch-py - [FEATURE] Testing compatibility on different platforms opensearch-py#186
• opensearch-ruby - [FEATURE] Testing compatibility on different platforms opensearch-ruby#97
• opensearch-dsl-py - [FEATURE] Testing compatibility on different platforms opensearch-dsl-py#74
• opensearch-rs - [FEATURE] Testing compatibility on different platforms opensearch-rs#64
• opensearch-js - [FEATURE] Testing compatibility on different platforms opensearch-js#275
• opensearch-php - [FEATURE] Testing compatibility on different platforms opensearch-php#78
• opensearch-go - [FEATURE] Testing compatibility on different platforms opensearch-go#146
• opensearch-net - [FEATURE] Testing compatibility on different platforms opensearch-net#82

[shyim]

Btw its not possible to test ARM on GitHub. They don't offer hosted runners and needs own runners and QEMU is holy buggy to run it in an CI

[dblock]

@shyim "yet", but it's a good point - we do have our own Jenkins infra, but it's obviously quite a bit of overhead to be running jobs on that for clients (we do for OpenSearch and OpenSearch Dashboards extensively for arm64).

[Xtansia]

I've been doing some investigating into this, and have hit some hurdles due to us needing to spin up a docker container for OpenSearch to run integration tests against:

1. On the GH-hosted Windows runners, Docker is only setup for Windows containers, not Linux containers. Apparently blocked by the need for enabling Hyper-V to be able to run Linux containers: Add Windows Agent with Docker in Linux container mode actions/runner-images#252
2. On the GH-hosted MacOS runners, Docker is not installed, as there's only the "Desktop" variant which has licensing issues: [macOS] Add Docker to macOS actions/runner-images#2150

The MacOS one can be worked around by installing Docker at run-time using something like https://github.com/docker-practice/actions-setup-docker/ to hackily install a specific version of Docker Desktop that has a working unattended install. This is slow and somewhat unreliable, and the resulting docker performance is very poor due to Docker having to run a VM, on a runner that's already a VM. For the Rust client, the MacOS integration test run was approximately 3 times as long as the Ubuntu run (~36min vs ~12mins).