-
Notifications
You must be signed in to change notification settings - Fork 182
Create sandbox event endpoint and handlers #784
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 32 commits
Commits
Show all changes
35 commits
Select commit
Hold shift + click to select a range
d8dc71f
WIP sandbox event endpoint boilerplate
0div d7363e9
fix merge conflict with main
0div 587f92e
WIP working sandbox event endpoint boilerplate
0div 61c8fe9
renaming and changes in file structure
0div 67abc7c
fix merge conflict with main
0div b39dd56
WIP abstract event server in its own sandbox submodule; add some head…
0div 4aefc4a
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div 78bfc5a
cosmetic edits
0div 599531e
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div 6f1aa77
address gofumpt errors
0div c47a330
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div 087c421
store events in redis via a catchall endpoint
0div 19a18a7
create event store interface
0div d8ca2e6
fix event store
0div 5aa2b58
go mod tidy & go work sync
0div 59ea4ad
fix merge conflict with main
0div be5c2ad
go mod tidy api
0div eb5ca83
use sorted set for events time-series
0div 3179789
go-lint problematic file
0div 30e2cd8
add redis to start-services action
0div 881fbb0
add redis_url to template-manager iac
0div 89eb6fb
remove proto from redis url in github actions
0div fdc5cdb
update openapi spec for sandbox events endpoint
0div 333bb17
fix merge conflict with main
0div e2e6e2c
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div 6abcc04
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div fd4e9a1
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div b4bb3f6
track Sandbox IPs via store during key Sandbox lifecycle steps relati…
0div 7535b6e
remove sandbox id middleware
0div d25a9f3
use public ip address for sandbox event routing that is not meant to …
0div 60de6a2
fix merge conflict with main
0div 4309142
golint
0div 78ffeba
introduce internal const.go to orchestrator package
0div 0b69432
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div 6d666e6
Merge branch 'main' of https://github.com/e2b-dev/infra into create-i…
0div File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
123 changes: 123 additions & 0 deletions
123
packages/orchestrator/internal/sandbox/event/handlers.go
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,123 @@ | ||
| package event | ||
|
|
||
| import ( | ||
| "context" | ||
| "encoding/json" | ||
| "io" | ||
| "net/http" | ||
| "strings" | ||
|
|
||
| "go.uber.org/zap" | ||
| ) | ||
|
|
||
| type EventHandler interface { | ||
| Path() string | ||
| HandlerFunc(w http.ResponseWriter, r *http.Request) | ||
| } | ||
|
|
||
| type MetricsHandler struct{} | ||
|
|
||
| func (h *MetricsHandler) Path() string { | ||
| return "/metrics" | ||
| } | ||
|
|
||
| func (h *MetricsHandler) HandlerFunc(w http.ResponseWriter, r *http.Request) { | ||
| if r.Method != http.MethodPost { | ||
| http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) | ||
| return | ||
| } | ||
| w.Header().Set("Content-Type", "application/json") | ||
| w.WriteHeader(http.StatusCreated) | ||
| _, err := w.Write([]byte(`{"event_ack":true,"path":"/metrics"}`)) | ||
| if err != nil { | ||
| http.Error(w, "Failed to write response", http.StatusInternalServerError) | ||
| return | ||
| } | ||
| } | ||
|
|
||
| // This is used to track ad-hoc events that are not handled by the event server. | ||
| type DefaultHandler struct { | ||
| store SandboxEventStore | ||
| } | ||
|
|
||
| func (h *DefaultHandler) Path() string { | ||
| return "/" | ||
| } | ||
|
|
||
| func (h *DefaultHandler) HandlerFunc(w http.ResponseWriter, r *http.Request) { | ||
| addr := r.RemoteAddr | ||
| ip := strings.Split(addr, ":")[0] | ||
| sandboxID, err := h.store.GetSandboxIP(ip) | ||
| if err != nil { | ||
| zap.L().Error("Failed to get sandbox ID from IP", zap.Error(err)) | ||
| http.Error(w, "Error handling event", http.StatusInternalServerError) | ||
| return | ||
| } | ||
|
|
||
| zap.L().Debug("Received request from sandbox", zap.String("sandbox_id", sandboxID), zap.String("ip", ip)) | ||
|
|
||
| if r.Method == http.MethodGet { | ||
| events, err := h.store.GetLastNEvents(sandboxID, 10) | ||
| if err != nil { | ||
| zap.L().Error("Failed to get event data for sandbox "+sandboxID, zap.Error(err)) | ||
| http.Error(w, "Failed to get event data for sandbox "+sandboxID, http.StatusInternalServerError) | ||
| return | ||
| } | ||
|
|
||
| eventJSON, err := json.Marshal(events) | ||
| if err != nil { | ||
| zap.L().Error("Failed to marshal event data", zap.Error(err)) | ||
| http.Error(w, "Failed to marshal event data", http.StatusInternalServerError) | ||
| return | ||
| } | ||
|
|
||
| w.Header().Set("Content-Type", "application/json") | ||
| w.WriteHeader(http.StatusOK) | ||
| w.Write(eventJSON) | ||
| return | ||
| } | ||
|
|
||
| if r.Method != http.MethodPost { | ||
| http.Error(w, "Method not allowed", http.StatusMethodNotAllowed) | ||
| return | ||
| } | ||
|
|
||
| // Create event data with path and body | ||
| eventData := SandboxEvent{ | ||
| Path: r.URL.Path, | ||
| } | ||
|
|
||
| body, err := io.ReadAll(r.Body) | ||
| if err != nil { | ||
| http.Error(w, "Failed to read request body", http.StatusInternalServerError) | ||
| return | ||
| } | ||
|
|
||
| zap.L().Info("Received event", zap.String("body", string(body))) | ||
|
|
||
| eventData.Body = make(map[string]any) | ||
| err = json.Unmarshal(body, &eventData.Body) | ||
| if err != nil { | ||
| zap.L().Error("Failed to unmarshal request body", zap.Error(err)) | ||
| http.Error(w, "Failed to unmarshal request body", http.StatusInternalServerError) | ||
| return | ||
| } | ||
|
|
||
| // Store in Redis with sandboxID as key | ||
| err = h.store.AddEvent(sandboxID, &eventData, 0) | ||
| if err != nil { | ||
| zap.L().Error("Failed to store event data", zap.Error(err)) | ||
| http.Error(w, "Failed to store event data", http.StatusInternalServerError) | ||
| return | ||
| } | ||
|
|
||
| w.WriteHeader(http.StatusCreated) | ||
| w.Write([]byte(`{"event_ack":true}`)) | ||
| } | ||
|
|
||
| func NewEventHandlers(ctx context.Context, store SandboxEventStore) []EventHandler { | ||
| return []EventHandler{ | ||
| &MetricsHandler{}, | ||
| &DefaultHandler{store}, | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,48 @@ | ||
| package event | ||
|
|
||
| import ( | ||
| "context" | ||
| "fmt" | ||
| "net/http" | ||
| ) | ||
|
|
||
| // SandboxEventServer handles outbound HTTP requests from sandboxes calling the event.e2b.com endpoint | ||
| type SandboxEventServer struct { | ||
| server *http.Server | ||
| } | ||
|
|
||
| func NewSandboxEventServer(port uint, handlers []EventHandler) *SandboxEventServer { | ||
| mux := http.NewServeMux() | ||
|
|
||
| for _, handler := range handlers { | ||
| mux.HandleFunc(handler.Path(), handler.HandlerFunc) | ||
| } | ||
|
|
||
| server := &http.Server{ | ||
| Addr: fmt.Sprintf(":%d", port), | ||
| Handler: mux, | ||
| } | ||
|
|
||
| return &SandboxEventServer{ | ||
| server: server, | ||
| } | ||
| } | ||
|
|
||
| func (p *SandboxEventServer) Start() error { | ||
| return p.server.ListenAndServe() | ||
| } | ||
|
|
||
| func (p *SandboxEventServer) Close(ctx context.Context) error { | ||
| var err error | ||
| select { | ||
| case <-ctx.Done(): | ||
| err = p.server.Close() | ||
| default: | ||
| err = p.server.Shutdown(ctx) | ||
| } | ||
| if err != nil { | ||
| return fmt.Errorf("failed to shutdown event server: %w", err) | ||
| } | ||
|
|
||
| return nil | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What about running Gin here so we can use a strict Golang client in envd and track all version changes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
sure, we can use gin, but i'm not sure it's more strict than the std lib and also what do u mean by tracking version changes in this case?
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I was thinking that the "metrics server" can be Gin-backed by the OpenAPI scheme, as we are using for the API, so all breaking changes will be clear (because of the re-generated schema). You will also be able to use the generated Go client from EnvD to call metrics service endpoints.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The point was just make it strong types on both caller and receiver
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Agree with @sitole