This repository is an HTTP Reverse Proxy server written in Go. Following is the simple architecture of the design.
Features:
- Logging, Caching, Error Handling(500)
Packages:
- cmd (contains the main file)
- config (contains configuration)
- helpers (contains proxy helpers and utils)
- middlewares (contains logging and caching)
- proxy (contains server, proxy and a handler)
cmd/proxy/main.go
This is the main file or entry file to the server. It is responsible for only 2 simple tasks: loading the environment variables(using config package) and starting the server(utilizes proxy package).
Prerequisite:
You must have Go and git installed on your computer. Visit go.dev and git-scm
Clone this repository:
git clone https://github.com/JaySonani/http-reverse-proxy/
Note: This is a private repository so only users with access will be able to clone this repo.
Navigate to the repo:
cd http-reverse-proxy/
Install dependencies:
go mod tidy
Start the Reverse Proxy Server:
go run cmd/proxy/main.go
Visit the localhost endpoints, for example:
http://localhost:3000/posts
http://localhost:3000/albums
http://localhost:3000/comments
http://localhost:3000/photos
You should see logs in your terminal like this:
Please note that this HTTP Reverse Proxy Server utilizes the existing Backend Server for this technical assessment. The same is:
https://jsonplaceholder.typicode.com/ (Please visit to see the documentation and all available routes)
To build this Reverse Proxy Server, I have used the following resources:
- YouTube: To learn about Proxy, Forward Proxy and Reverse Proxy
- Google image search: To help understand the Reverse Proxy visually, where exactly the Reverse proxy sits
- ChatGPT/Perplexity/Grok:
- To learn more about Reverse Proxy in detail, and various concepts associated with Reverse Proxy.
- To learn about best practices for a Go project and how to implement a Reverse Proxy Server in Go.
- To select a programming language, Python vs Go, for this technical assessment, comparing their pros and cons.
- Step-by-step process using Deep thinking/Reasoning features to plan the development.
- Design decisions:
- Choice of language: Go vs Python, I selected Go as it is faster compared to Python.
- Backend Server: Instead of creating a backend server, I utilized the existing one available freely on the Internet so that I could focus solely on building HTTP Reverse Proxy.
- Middleware: I've included logging and caching under the hood of middleware.
- Logging: I have added logging for every incoming request in the middleware. This makes debugging easier.
- Caching: By utilizing Redis for caching, performance can be improved for frequently accessed data.
- Limitations:
- No security implemented due to time constraints(No authentication).
- Utilized existing Backend Server, could create my own.
- No load balancing
- No Rate limiting or DoS/DDoS protection
- No HTTPS support(missing SSL/TLS)
- Single Point Failure: If the reverse proxy crashes, all traffic to the backend is lost.
- Load Balancing: To scale this, we would need some sort of Load Balancing. Assuming we have multiple instances of our Backend Server, we must determine which server should be presented with any particular incoming request. Rather than only relying on simple logic such as forwarding the request to most idol servers, intelligent and fine-grained algorithms based on headers, cookies or session data could be a better choice.
- Caching: Store frequent responses in Redis or in-memory cache to avoid hitting the backend. Since we already have Redis implemented for caching, it can perform strongly due to its distributed caching and multi-instance deployments.
- Horizontal Scaling: We can spin up multiple instances of the same server. Also, deploy the HTTP Reverse Proxy behind the load balance(or API Gateway) like Nginx
- Vertical Scaling: Increase the compute power of existing instances.
- Containerization & Orchestration: We can make a docker image of the server for easy distribution and deployment. Leveraging the Kubernetes cluster could be a good idea if we have lots of containers.
- Connection Pooling: If we have a database connection, maintaining a pool of connections and reusing them to fulfil incoming requests can reduce TCP handshake overhead.
- Rate Limiting: We can implement Rate limiting that will make sure that only a certain number of requests can be accepted within a certain period of time. This could prevent server overload and prevent DoS/DDoS attacks.
- Whitelisting IP: We can implement a middleware that would check for the Client's IP against a list(ACL-Access Control List) of pre-approved clients. Only if the match is found, the Proxy will go ahead and try to fulfil that request either by looking at the Cache or Backend Server.
- Authentication: We can validate incoming requests using techniques like JWT token or API key.
- Role-based Authentication: There could be a time when only a certain group of people need access to some endpoints. In such cases, implementing role-based authentication is a good idea to prevent exploitation of access.
- Network Security: We can use HTTPs or TLS(Transport Layer Security)/SSL to prevent Man-In-The-Middle attacks.
- Header Injection: We can sanitize/remove suspicious headers from incoming request
- Internet: Google, YouTube, ChatGPT, Perplexity, Grok
- Postman: To test the Reverse Proxy Server using various HTTP requests such as GET or POST
- Cursor: To help in the development process, getting suggestions, fixing errors, speeding up the development, and comment generation.
- Chrome Browser: To test simple GET endpoints
- Excalidraw: To create a system design architecture diagram
- Add security
- Add a load-balancing mechanism
- Response compression for better performance
- Implement REST best practices
- Layered approach for a more efficient and secure system