Application for AWS Credits for Projects

[nicole-numfocus]

Project

All Sponsored Projects

Observations

I've been working with the program manager for the AWS Free and Open Source Software Fund and we have been selected to apply for credits on behalf of projects. To apply we must complete this application.

We are not guaranteed to be awarded all credits requested, but there is high likelihood that we will receive a majority of our asks. Additionally, we can ask that our credits be awarded for two years instead of one so there isn't as much burden of reapplying. AWS is also working on codifying the parameters of the program so that we will have a much clearer understanding of the reapplication process going forward.

To complete the application we will need to include specific asks for each project as well as any blanket requests that could benefit multiple projects. Reaching out to each project individually can be a lengthy process and since not all projects need this support, it may not be very efficient. In the past, we have put out a mass call to projects but did not receive many responses.

NumFOCUS staff is happy to coordinate gathering the information for the application but we will need the committee's help to ensure that the technical aspects of the application are completed appropriately. It would also be helpful to us if you all could work with us on gathering the asks. What might be a good approach to getting asks from projects? Are there specific projects that we could focus on or asks that have been made to the committee in the past that could benefit from this support? Are there any blanket needs for multiple projects we could include?

I intended to add this as an item at the next meeting, but since it had to be postponed I thought I would bring it up here so we could get the ball rolling asynchronously.

[aterrel]

Talked at the meeting @arliss-NF and @nicole-numfocus to review old applications to see if anything is relevant and then submit to AWS based on current usage

[InessaPawson]

@nicole-numfocus NumPy could use some AWS credits towards data warehousing. Is it too late to place a request?

[aterrel]

This fell off our task list. Can you give me some parameters around what you are going to be doing, e.g. 100GB on S3 hosting or 1TB of glacier, etc.

[InessaPawson]

@aterrel We need to host datasets for several NumPy tutorials and user survey data. 100GB on S3 hosting should be plenty.

[aterrel]

Hi @InessaPawson , so sorry for the delay!

Okay the real cost is the data transfer. Storing 100 GB is only a few dollars, but transferring is a different story. Data transfer is $.05-.09 a gig, which for 50 transfers of the data would be almost $500. And NumPy is so popular, it will probably have far more transfers a month.

Is there a way to keep the high traffic parts of the tutorial on Github?

I'll go ahead and create a bucket so we can get you started. You will need a username for AWS, do you have a preferred username?

[InessaPawson]

@aterrel Thank you for the update, Andy! Before we set up a bucket with AWS, let me speak to the rest of the NumPy team about all the costs involved in hosting the datasets for the tutorials on AWS.

[aterrel]

@InessaPawson any updates here?

[kp992]

Hello, QuantEcon would like to get some AWS credits for running some of the GPU CI jobs. Our current annual bill is around $7k-8k. Is it still possible to get some credits?

We are also open to other possible options.

[aterrel]

@kp992 please create a new ticket at https://github.com/numfocus/infrastructure/issues

We don't have a template for this at the moment, but just create a blank ticket. I think we are going to just need to put in individual grant applications to AWS for these. We can help with that and make sure that NumFOCUS staff have access if billing goes off plan, but I don't think getting everyone to conform to a single timeline is going to work as all our OSS contrib time is spotty.

[kp992]

Thanks @aterrel

[InessaPawson]

@aterrel Hi, Andy! I apologize for the delayed response. We have given some thought to the proposed option and have an additional question. Has the Committee looked into registering NF projects tutorial data sets with the AWS Open Data Sponsorship Program (https://aws.amazon.com/opendata)? It would be the best place for us to host this data and completely free.

[aterrel]

@InessaPawson we are working with ITK to register for this program and that has gone okay. Let me chat with the AWS employee on that to see if we should have a NumFOCUS version or just individual projects.