Merge branch 'hcg/devops'

datamade · May 6, 2021 · a43b058 · a43b058
2 parents 1d3a300 + 30bac9f
commit a43b058
Show file tree

Hide file tree

Showing 17 changed files with 76 additions and 31 deletions.
diff --git a/README.md b/README.md
@@ -10,9 +10,15 @@ Here at DataMade, we do a lot of computer programming. In the spirit of [better
 
 _In alphabetical order and including links to external repository-based documentation._
 
-- [Amazon Web Services](aws/)
-    - [AWS Relational Database Service (RDS)](aws/rds.md)
 - [Continuous Integration (CI)](ci/)
+- [Deployment](deployment/)
+    - [Heroku](deployment/heroku/)
+        - [Deploy a Django app to Heroku](deployment/heroku/deploy-a-django-app.md)
+        - [Scheduling tasks](deployment/heroku/scheduling-tasks.md)
+    - [Netlify](deployment/netlify/)
+        - [Deploy a static site to Netlify](deployment/netlify/README.md#deploy-a-static-site-to-netlify)
+    - [Domain names](deployment/domain-names.md)
+    - [Amazon Web Services](deployment/aws/)
 - [Django](django/)
     - [Translation](django/translation.md)
     - [Wagtail](django/wagtail/)
@@ -25,16 +31,11 @@ _In alphabetical order and including links to external repository-based document
     - [GNU Make and preferred Unix utilities](https://github.com/datamade/data-making-guidelines)
     - [Schema validation](etl/schema-validation.md)
 - [GatsbyJS](gatsby/)
-- [Heroku](/heroku/)
-    - [Deploy a Django app to Heroku](/heroku/deploy-a-django-app.md)
-    - [Scheduling tasks](/heroku/scheduling-tasks.md)
 - [Logging](logging/)
     - [Sentry](logging/sentry.md)
     - [Slack](logging/slack.md)
 - [Mapping](mapping/)
     - [Google APIs (Maps, Geocoding)](mapping/google-apis.md)
-- [Netlify](netlify/)
-    - [Deploy a static site to Netlify](netlify/README.md#deploy-a-static-site-to-netlify)
 - [PostgreSQL](postgres/)
     - [A quick and dirty introduction to `sqlalchemy`](postgres/quick-n-dirty-sqlalchemy.md)
     - [Interacting with a remote database](postgres/Interacting-with-a-remote-database.md)

diff --git a/deployment/README.md b/deployment/README.md
@@ -0,0 +1,15 @@
+# Deployment
+
+DataMade generally uses Heroku for dynamic deployments and Netlify for static
+deployments. However, we maintain a number of legacy applications deployed on
+Amazon Web Services (AWS) infrastructure.
+
+This directory contains documents relevant to all three platforms, as well as
+miscellaneous deployment-related activities.
+
+## Contents
+
+- [AWS](./aws)
+- [Heroku](./heroku)
+- [Netlify](./netlify)
+- [Domain names](domain-names.md)
diff --git a/deployment/aws/README.md b/deployment/aws/README.md
@@ -0,0 +1,11 @@
+# Amazon Web Services
+
+[Until 2020](https://datamade.us/blog/why-were-switching-to-heroku/), AWS was
+DataMade's deployment platform of choice. We've sunset documentation of [our
+custom deployment framework and practices](https://github.com/datamade/deploy-a-site/),
+however we've migrated still-relevant documentation to this directory.
+
+## Contents
+
+- [RDS](rds.md)
+- [Annual server maintenace](annual-server-maintenance.md)
diff --git a/deployment/aws/annual-server-maintenance.md b/deployment/aws/annual-server-maintenance.md
@@ -0,0 +1,15 @@
+# Annual server maintenance
+
+DataMade maintains about two dozen EC2 instances that host a number of legacy
+applications. An inventory of applications deployed on AWS infrastructure can
+be found [here](https://docs.google.com/spreadsheets/d/1_c1_v4IJ5wLpjUt0p0Feq3LXskItu5Ml9J6gZucSqpw/edit?usp=sharing) (internal link).
+
+Our EC2 instances are generally stable and self-sustaining, however [cruft can
+build up over time](https://github.com/datamade/how-to/issues/156#issue-778165823). On an annual basis, perform the following steps to free up
+space so applications can chug on.
+
+1. Truncate system journal files: `sudo journalctl --vacuum-size=100M`
+2. Purge unneeded apt packages: `sudo apt-get autoremove`
+    - You may be prompted to run `apt --fix-broken-install`. Go ahead and do this, selecting "Keep local version" if you see any warnings that a file may have changed. You should then be able to run `autoremove` without issues.
+
+Taken together, these steps should clear up 2-3 GB of space.
diff --git a/aws/rds.md → deployment/aws/rds.md b/aws/rds.md → deployment/aws/rds.md
diff --git a/deployment/domain-names.md b/deployment/domain-names.md
@@ -0,0 +1,26 @@
+# Domain names
+
+DataMade uses [Namecheap](https://www.namecheap.com/) as our domain name
+provider. Currently, domain names we control are under Derek's personal account.
+
+If we are setting up a domain name, Derek should purchase the domain name. After
+purchase, Derek will share administrative privileges for the domain name with
+all partners and lead developers.
+
+## Sharing administrative privileges for a domain
+
+1. In the listing of domain names, click manage for the domain
+name you want to share privileges on.
+
+    ![Manage Domain](../images/account_panel.png)
+
+2. Choose the "Sharing & Transfer" tab.
+
+    ![Sharing and Transfer](../images/sharing_transfer.png)
+
+3. Add the user to the account using their `@datamade.us` email address and
+submit.
+
+4. Give the user full permissions.
+
+    ![Add Manager](../images/add_manager.png)
diff --git a/heroku/README.md → deployment/heroku/README.md b/heroku/README.md → deployment/heroku/README.md
diff --git a/heroku/deploy-a-django-app.md → deployment/heroku/deploy-a-django-app.md b/heroku/deploy-a-django-app.md → deployment/heroku/deploy-a-django-app.md
diff --git a/heroku/research/hec-log.md → deployment/heroku/research/hec-log.md b/heroku/research/hec-log.md → deployment/heroku/research/hec-log.md
diff --git a/heroku/research/jfc-log.md → deployment/heroku/research/jfc-log.md b/heroku/research/jfc-log.md → deployment/heroku/research/jfc-log.md
diff --git a/...ku/research/recommendation-of-adoption.md → ...ku/research/recommendation-of-adoption.md b/...ku/research/recommendation-of-adoption.md → ...ku/research/recommendation-of-adoption.md
diff --git a/heroku/scheduling-tasks.md → deployment/heroku/scheduling-tasks.md b/heroku/scheduling-tasks.md → deployment/heroku/scheduling-tasks.md
diff --git a/netlify/README.md → deployment/netlify/README.md b/netlify/README.md → deployment/netlify/README.md
@@ -1,7 +1,7 @@
 # Netlify
 
 This document records best practices for deploying static sites to
-[Netlify](netlify.com), DataMade's preferred static hosting provider.
+[Netlify](https://netlify.com), DataMade's preferred static hosting provider.
 
 ## Contents
 

diff --git a/domain_names/domain_names.md b/domain_names/domain_names.md
diff --git a/domain_names/account_panel.png → images/account_panel.png b/domain_names/account_panel.png → images/account_panel.png
diff --git a/domain_names/add_manager.png → images/add_manager.png b/domain_names/add_manager.png → images/add_manager.png
diff --git a/domain_names/sharing_transfer.png → images/sharing_transfer.png b/domain_names/sharing_transfer.png → images/sharing_transfer.png