Instabug-Backend-Challange

This repository contains both the problem statement and my solution to the Instabug Backend Challenge which was sent to me as a part of the process for my application for the Backend Engineer role at Instabug.

Table of Contents

• Problem Statement
• Solution Discussion
• Installation Instructions
• API Endpoints
• TODO

Problem Statement

It’s required to build a chat system. The system should allow creating new applications where each application will have a token(generated by the system) and a name(provided by the client). The token is the identifier that devices use to send chats to that application.

Each application can have many chats. a chat should have a number. Numbering of chats in each application starts from 1 and no 2 chats in the same application may have the same number. The number of the chat should be returned in the chat creation request.

A chat contains messages and messages have numbers that start from 1 for each chat. The number of the message should also be returned in the message creation request. The client should never see the ID of any of the entities. The client identifies the application by its token and the chat by its number along with the application token.

Add an endpoint for searching through messages of a specific chat. It should be able to partially match messages’ bodies. You must use ElasticSearch for this.

The applications table should contain a column called chats_count that contains the number of chats for this application. Similarly, the chats table should contain a column called messages_count that contains the number of messages in this chat. These columns don’t have to be live. However, they shouldn’t be lagging more than 1 hour.

Assume that the system is to receive many requests. It might be running on multiple servers in parallel and thus multiple requests may be processed concurrently. Make sure to handle race conditions. Try to minimize the queries and avoid writing directly to MySQL while serving the requests(especially for the chats and messages creation endpoints). You can use a queuing system to achieve that. It is allowed for chats and messages to take time to be persisted. You should optimize your tables by adding appropriate indices.

Your app should be containerized. We should only write docker-compose up to run the whole stack

Notes

• You’re only asked to build the API. No GUI is required.
• You should use Ruby on Rails(V5) for building the API and the workers. BONUS: You are encouraged to have the endpoints of chats and messages creation as a Golang app.
• The endpoints should be RESTful. You should provide endpoints for creating, updating and reading applications, chats that belong to a specific application(GET /applications/[application_token]/chats) and messages that belong to a specific chat. Make sure to have appropriate indices in place for these endpoints to be optimized.
• Use MySQL as your main datastore. You’re allowed to use any other component you need along with MySQL. You may want to check out REDIS.
• You should add a Readme containing instructions to run your code.
• BONUS: Write specs for your code.

Solution Discussion

I have never worked with ruby nor rails before so probably I have missed a lots of ruby on rails best practices and coding styles.

So as the problem stated it's required to build a chat system, we will be having 3 models within this app which are applications, chats and messages each of them described as following

• Application Model

• 1:m relation with the Chat model

  • token: string (will be generated on a new record creation).
  • name: string (name of the application provided by the user ing the request body).
  • chats_count: integer (Number of chats within this application).

• Chat Model

• 1:m relation with the Message model and belongs to one application

  • application_token: string (Used for faster finding of a specific chat of an application).
  • chat_number: integer (id of the chat for a specific application starts from 1 in each application an counts up).
  • messages_count: integer (number of messages inside this chat).
  • application_id: application (The id of the parent application)

• Message Model

• belongs to one Chat model record.

  • application_token: string
  • chat_number: integer
  • message_number: integer (id of the message for a specific chat starts from 1 for a chat and counts up)
  • message_body: text (The message body provided by the user when creating a message).

a chat should have a number. Numbering of chats in each application starts from 1 and no 2 chats in the same application may have the same number. The number of the chat should be returned in the chat creation request.

Redis was of great help here, given the token of an application we are able to track the current id of the newly created chat, and this was repeated similarly for messages as it was also required to give messages number depending on the parent chat starting from 1 and returning the message number with the response of the message creation request.

The applications table should contain a column called chats_count that contains the number of chats for this application. Similarly, the chats table should contain a column called messages_count that contains the number of messages in this chat. These columns don’t have to be live. However, they shouldn’t be lagging more than 1 hour.

I have also used Redis here for tracking the current number of chats inside each application or messages inside each of the chats, so simply we are using the atomic incr function after a valid chat/message creation request and decr after a valid destroy request, however it's required to update the columns with these values and not to be lagged more than 1 hour, so I have created a cron job that fetch all the current chats/message numbers and updates them accordingly into the db tables.

Assume that the system is to receive many requests. It might be running on multiple servers in parallel and thus multiple requests may be processed concurrently. Make sure to handle race conditions. Try to minimize the queries and avoid writing directly to MySQL while serving the requests(especially for the chats and messages creation endpoints). You can use a queuing system to achieve that. It is allowed for chats and messages to take time to be persisted.

PubSub Scheme inside RabbitMQ was used, so for any create, update or destroy we only validate the request then throw it into a queue to be consumed using background workers (chats worker and message worker) for executing the operations into the MYSQL database.

Add an endpoint for searching through messages of a specific chat. It should be able to partially match messages’ bodies. You must use ElasticSearch for this.

So in the Message model we index the message_body using ES and query_string was used for matching using a simple regex to partially match the message bodies as required. This was a bit simple and straightforward or maybe I am missing something here (Conspiracy theory, it can't be that easy).

Your app should be containerized. We should only write docker-compose up to run the whole stack

So I had to set up all the 5 used services and their configurations inside a docker-compose.yml.

TODO -> Should index token as the id of the application.

Installation Instructions

As required in the task all you need to do to get the API working is by running the following command inside the project directory.

docker-compose up

API Endpoints

The API will be listening on port 4000

Application Endpoints:

• Get all available applications. GET /api/v1/applications

curl -X GET  http://localhost:4000/api/v1/applications

• Get a specific application using the application token. GET /api/v1/applications/:token

 curl -X GET  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9

• Create a new application. POST /api/v1/applications

curl -X POST  http://localhost:4000/api/v1/applications?name=instagram-chat

• Update the name of an existing application. PUT /api/v1/applications/:token

curl -X PUT  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9?name=instabug-chat

• Delete an application. DELETE /api/v1/applications/:token

curl -X DELETE  http://localhost:4000/api/v1/applications/p9VktkiF5m1Fz2TG1bXXdS3D

Chat Endpoints:

• Get all available chats for a specific application. GET /api/v1/applications/:token/chats

curl -X GET  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats

• Get a specific chat. GET /api/v1/applications/:token/chats/:chat_number

curl -X GET  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats/1

• Create a new chat. POST /api/v1/applications/:token/chats

curl -X POST  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/ch
ats

• Delete a chat. DELETE /api/v1/applications/:token/chats/:chat_number

curl -X DELETE  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/
chats/1

Message Endpoints:

• Get all messages of a specific chat. GET /applications/:token/chats/:chat_number/messages

curl -X GET  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats/1/messages

• Get a specific message. GET /applications/:token/chats/:chat_number/messages/:message_number

curl -X GET  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats/1/messages/1

• Create a new message. POST /applications/:token/chats/:chat_number/message

curl -X POST  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats/1/messages?message_body=Hey!

• Update existing message PUT /applications/:token/chats/:chat_number/messages/:message_number

curl -X PUT  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats/1/messages/1?message_body=Hola!

• Delete a message. DELETE /todos/:todo_id/items/:id

curl -X DELETE  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats/1/messages/1

• Search for a message within a certain chat. GET /applications/:token/chats/:chat_number/messages/search

curl -X GET  http://localhost:4000/api/v1/applications/FgxLAPzvCZ9qq7BPPjRL8zd9/chats/1/messages/search?query=search_pattern

TODO

• Fix The error with the workers (only one consumer getting the work now).
• Add code specs (Model specs was added we need to test the API as well).
• Handle more unhappy cases and exceptions for the user requests.