Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add limiter service written in go #9

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -68,3 +68,6 @@ typings/
# IDEs
.vscode
.history

# macOS
.DS_Store
10 changes: 5 additions & 5 deletions api/garden.yml
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
kind: Build
type: container
name: api-build
name: api
description: The backend build for the voting UI

---
kind: Deploy
type: container
name: api
build: api-build
build: api
description: The backend deploy for the voting UI
spec:
args: [python, app.py]
Expand All @@ -25,7 +25,7 @@ spec:
ingresses:
- path: /
port: http
hostname: api.${variables.base-hostname}
hostname: api-${variables.base-hostname}
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've changed the URLs to avoid SSL errors when using this example with the AWS quickstart cluster: https://github.com/garden-io/garden-aws-quickstart

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did we also update DNS?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need to test the local version of this 👍

healthCheck:
httpGet:
path: /health
Expand All @@ -38,6 +38,6 @@ kind: Test
type: container
name: unit
description: Unit test for backend API
build: api-build
build: api
spec:
args: ["echo", "ok"]
args: ["echo", "ok"]
7 changes: 7 additions & 0 deletions limiter/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
FROM golang:alpine as prod
RUN mkdir /app
ADD ./src /app/
WORKDIR /app
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o main .
CMD ["/app/main"]
EXPOSE 8080
8 changes: 8 additions & 0 deletions limiter/Dockerfile.syncmode
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
FROM cosmtrek/air:latest as sync

# We do not copy anything on purpose in sync mode.
# Code will be copied using mutagen as part of Garden's code synchronization feature.
RUN mkdir /app
WORKDIR /app

EXPOSE 8080
54 changes: 54 additions & 0 deletions limiter/garden.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
# Production container to be used without sync mode.
kind: Build
type: container
name: limiter-prod
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before we merge this, we should add "limiter-sync" and "limiter-prod" to the ECR repository default list in https://github.com/garden-io/garden-aws-quickstart

include:
- src
- Dockerfile
exclude:
- src/tmp/*
spec:
dockerfile: "Dockerfile"

---

# The sync container only provides the tools necessary for building the library
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# The sync container only provides the tools necessary for building the library
# The sync container only provides the necessary build tools

# when using it during inner loop development together with Garden's sync mode.
kind: Build
type: container
name: limiter-sync
include:
- Dockerfile.syncmode
spec:
dockerfile: "Dockerfile.syncmode"

---

kind: Deploy
type: container
name: limiter
build:
$if: ${this.mode=='sync'}
$then: limiter-sync
$else: limiter-prod
include: []
stefreak marked this conversation as resolved.
Show resolved Hide resolved
dependencies:
- deploy.api
spec:
sync:
command: [air]
paths:
- target: /app
source: ./src
mode: one-way-replica
exclude: ["tmp/*/**"]
ports:
- name: http
containerPort: 8080
servicePort: 80
ingresses:
- path: /
port: http
hostname: limiter-${var.base-hostname}
env:
PROXY_URL: http://api
44 changes: 44 additions & 0 deletions limiter/src/.air.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
root = "."
testdata_dir = "testdata"
tmp_dir = "tmp"

[build]
args_bin = []
bin = "./tmp/main"
cmd = "go build -o ./tmp/main ."
delay = 0
exclude_dir = ["assets", "tmp", "vendor", "testdata"]
exclude_file = []
exclude_regex = ["_test.go"]
exclude_unchanged = false
follow_symlink = false
full_bin = ""
include_dir = []
include_ext = ["go", "tpl", "tmpl", "html"]
include_file = []
kill_delay = "0s"
log = "build-errors.log"
poll = false
poll_interval = 0
rerun = false
rerun_delay = 500
send_interrupt = false
stop_on_error = false

[color]
app = ""
build = "yellow"
main = "magenta"
runner = "green"
watcher = "cyan"

[log]
main_only = false
time = false

[misc]
clean_on_exit = false

[screen]
clear_on_rebuild = false
keep_scroll = true
1 change: 1 addition & 0 deletions limiter/src/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
tmp
12 changes: 12 additions & 0 deletions limiter/src/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
module example.garden.io/m

go 1.20

require github.com/didip/tollbooth/v7 v7.0.1

require (
github.com/didip/tollbooth v4.0.2+incompatible // indirect
github.com/go-pkgz/expirable-cache v0.1.0 // indirect
github.com/patrickmn/go-cache v2.1.0+incompatible // indirect
golang.org/x/time v0.3.0 // indirect
)
16 changes: 16 additions & 0 deletions limiter/src/go.sum
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/didip/tollbooth v4.0.2+incompatible h1:fVSa33JzSz0hoh2NxpwZtksAzAgd7zjmGO20HCZtF4M=
github.com/didip/tollbooth v4.0.2+incompatible/go.mod h1:A9b0665CE6l1KmzpDws2++elm/CsuWBMa5Jv4WY0PEY=
github.com/didip/tollbooth/v7 v7.0.1 h1:TkT4sBKoQoHQFPf7blQ54iHrZiTDnr8TceU+MulVAog=
github.com/didip/tollbooth/v7 v7.0.1/go.mod h1:VZhDSGl5bDSPj4wPsih3PFa4Uh9Ghv8hgacaTm5PRT4=
github.com/go-pkgz/expirable-cache v0.1.0 h1:3bw0m8vlTK8qlwz5KXuygNBTkiKRTPrAGXU0Ej2AC1g=
github.com/go-pkgz/expirable-cache v0.1.0/go.mod h1:GTrEl0X+q0mPNqN6dtcQXksACnzCBQ5k/k1SwXJsZKs=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4=
golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
46 changes: 46 additions & 0 deletions limiter/src/main.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
package main

import (
"log"
"net/http"
"net/http/httputil"
"net/url"
"os"

"github.com/didip/tollbooth"
)

func main() {
remote, err := url.Parse(os.Getenv("PROXY_URL"))
if err != nil {
panic(err)
}

// Rate limiter: Allow one vote per second.
// Suggestion: Try reducing this to 0.1 to allow one vote every 10 seconds.
// When using garden deploy --sync, the code change will be automatically synchronized
// See also https://docs.garden.io/guides/code-synchronization
limiter := tollbooth.NewLimiter(1, nil)
limiter.SetMessage("Please slow down.")
limiter.SetOnLimitReached(func(w http.ResponseWriter, r *http.Request) {
log.Printf("%s: Limiting: Too many requests", r.URL)
})

// Reverse proxy
handler := func(p *httputil.ReverseProxy) func(http.ResponseWriter, *http.Request) {
log.Println("Hello world!")

return func(w http.ResponseWriter, r *http.Request) {
log.Printf("%s: OK: Forwarding", r.URL)
r.Host = remote.Host
p.ServeHTTP(w, r)
}
}

proxy := httputil.NewSingleHostReverseProxy(remote)
http.Handle("/", tollbooth.LimitFuncHandler(limiter, handler(proxy)))
err = http.ListenAndServe(":8080", nil)
if err != nil {
panic(err)
}
}
2 changes: 1 addition & 1 deletion result/garden.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ spec:
ingresses:
- path: /
port: ui
hostname: result.${var.base-hostname}
hostname: result-${var.base-hostname}
ports:
- name: ui
protocol: TCP
Expand Down
8 changes: 4 additions & 4 deletions vote/garden.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,12 +27,12 @@ spec:
ingresses:
- path: /
port: http
hostname: vote.${var.base-hostname}
hostname: vote-${var.base-hostname}
env:
HOSTNAME: vote.${var.base-hostname}
HOSTNAME: vote-${var.base-hostname}
VUE_APP_USERNAME: ${local.username}
dependencies:
- deploy.api
- deploy.limiter
- deploy.result

---
Expand All @@ -55,7 +55,7 @@ type: container
name: e2e-runner
dependencies: [deploy.vote]
spec:
image: ${actions.deploy.vote.outputs.deployedImageId}
image: ${actions.build.vote.outputs.deploymentImageId}

---
kind: Test
Expand Down
2 changes: 1 addition & 1 deletion vote/vue.config.js
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ module.exports = {
progress: false,
proxy: {
'^/api': {
target: 'http://api',
target: `http://limiter`, // limiter is a rate limiting reverse proxy for the api
changeOrigin: true,
secure: false,
logLevel: 'debug',
Expand Down