Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dockerize the application #42

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Dockerize the application #42

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions .env.dist
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
BUCKET_NAME_1=test-accessible-bucket
REGION_1=us-west-2
REGION_2=us-east-1
S3COMPAT_ACCESS_KEY=AKIA...
S3COMPAT_SECRET_KEY=SECRET...
END_POINT=my-s3-endpoint.company.com
NOT_ACCESSIBLE_BUCKET=test-not-accessible-bucket
PREFIX_FOR_PAGE_LISTING=test/prefix/
PAGE_LISTING_TOTAL_SIZE=1000
19 changes: 19 additions & 0 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
FROM maven:3.9

ENV BUCKET_NAME_1 test-accessible-bucket
ENV REGION_1 us-west-2
ENV REGION_2 us-east-1
ENV S3COMPAT_ACCESS_KEY AKIA...
ENV S3COMPAT_SECRET_KEY SECRET...
ENV END_POINT my-s3-endpoint.company.com
ENV NOT_ACCESSIBLE_BUCKET test-not-accessible-bucket
ENV PREFIX_FOR_PAGE_LISTING test/prefix/
ENV PAGE_LISTING_TOTAL_SIZE 1000

COPY . /snowflake-s3compat-api-test-suite/
WORKDIR /snowflake-s3compat-api-test-suite/
RUN --mount=type=secret,id=m2settings,dst=/root/.m2/settings.xml \
mvn clean install -DskipTests

ENTRYPOINT ["mvn", "test"]
CMD ["-Dtest=S3CompatApiTest"]
10 changes: 10 additions & 0 deletions Readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,16 @@ If all of your APIs pass the tests in this repo, please refer to our public docu

[Using On-Premises Data in Place with Snowflake](https://www.snowflake.com/blog/external-tables-on-prem/)

## Docker
In situations where no Java development environment is available, the test suite has been dockerized for ease of access. The container image can be built and ran using these commands (be sure to
haev the applicable m2 settings (check dist) and environment variables set (again, check dist):

```
DOCKER_BUILDKIT=1 docker build -t s3compat-api-test-suite --secret id=m2settings,src=/Users/username/.m2/settings.xml .
docker run --rm -it --env-file ./.env s3compat-api-test-suite
```


## Troubleshooting
We expect the storage vendors provide S3-compatible APIs, which should work like S3 APIs. We have observed there are still differences between storage vendors.
Below are some troubleshooting tips. When your tests fail, please refer to the source code to see what are the test cases.
Expand Down
10 changes: 10 additions & 0 deletions m2_settings.xml.dist
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 https://maven.apache.org/xsd/settings-1.0.0.xsd">
<servers>
<server>
<id>github</id>
<username>your github username</username>
<password>your github access token</password>
</server>
</servers>
</settings>