Guru.com-Clone: A Scalable Microservices Application Replicated From Guru

Table of Contents

1. Project Overview
2. Architecture
3. Implementation
   • Databases
     • PostgreSQL
     • Apache Cassandra
     • Firebase
     • Redis
   • Microservices
     • Users Microservice
     • Freelancers Microservice
     • Jobs Microservice
     • Messaging Microservice
     • Notifications Microservice
     • Reports & Feedbacks Microservice
     • Payments Microservice
   • Media Server
   • Message Queues
   • Dockerizing The App
     • Cleaning & Packaging Microservices
     • Building Docker Images
     • Uploading Images on DockerHub
   • Deployment
     • Kubernetes
     • Auto Scalar
     • Service Discovery
     • Load Balancer
     • Web Server
     • CI/CD
   • Logging & Monitoring
     • SLF4J
     • Prometheus
     • Grafana
4. API Documentation
5. Load Testing

Project Overview

Guru.com-Clone is a microservices-driven platform enabling freelancers and clients to engage, collaborate, and accomplish projects, mirroring the functionalities of Guru. Engineered with scalability and resilience in focus, the system leverages a variety of technologies to adeptly manage high volumes of traffic and extensive data.

Architecture

Implementation

Databases

The system utilizes a variety of databases tailored to specific needs:

• PostgreSQL: Employs a robust relational centeralized database for structured data storage and management.
• Apache Cassandra: Utilizes a scalable and distributed NoSQL database for handling large volumes of unstructured data with high availability and fault tolerance.
• Firebase: Integrates a cloud-based database solution.
• Redis: Implements an in-memory data store for caching frequently accessed data, optimizing response times and reducing computational overhead.

Microservices

• All microservices are implemented using Java Spring Boot .
• They consume messages from the RabbitMQ message broker and respond through RabbitMQ as well.
• Requests are cached in Redis, so if the same request is sent more than once, there is no need to recompute the response every time and fetch the data again from the database as it can be retrieved directly from the cache .
• In some cases, a microservice would have to communicate with another one to complete a certain functionality, which is done through RabbitMQ. Every microservice can be scaled up or down independently of other microservices to adapt to the amount of incoming traffic.

Media Server

The Media Server, powered by Firebase, serves as the cornerstone for handling multimedia content within the Guru.com replica platform. Leveraging Firebase's cloud-based storage and real-time database capabilities, this server facilitates seamless uploading, storage, and retrieval of media files, including images, videos, and documents. With Firebase's robust security features and scalability, the Media Server ensures secure and reliable access to multimedia content across devices and platforms.

Message Queues

RabbitMQ serves as the backbone for asynchronous communication between various microservices in this project . The message queues play a pivotal role in facilitating communication, particularly for the Notification Service. This service consumes messages from different sources, including the Freelancer, Job, and Messaging Microservices, each with its dedicated queue. These queues handle a myriad of notifications, such as viewing profiles, job invitations, placing quotes, accepting quotes, and receiving new messages. This structured approach ensures timely and reliable delivery of notifications, enhancing user engagement and platform functionality.

Dockerizing The App

Deployment

In this project, the microservices were deployed locally on a Docker Kubernetes cluster along with Postgres, PgAdmin, RabbitMQ, Redis, Prometheus, and Grafana, enabling efficient management and scalability of the application components. Leveraging Docker containers orchestrated by Kubernetes, each microservice was encapsulated and deployed as a scalable and isolated unit. This deployment approach facilitated seamless integration and testing of the microservices in a controlled environment, allowing for rapid development iterations and ensuring consistency across deployments. Additionally, the use of Kubernetes provided automated deployment, scaling, and management capabilities, empowering the team to efficiently manage and orchestrate the deployment lifecycle of the microservices. Overall, deploying the microservices locally on a Docker Kubernetes cluster enhanced development agility and reliability, laying a solid foundation for future scalability and production deployment.

Logging & Monitoring

API documentation

Authentication

Endpoints require authentication using a bearer token. Include the token in the Authorization header of your requests.

Authorization: Bearer <your_access_token>

• URL: http://scalabol.freelancer/freelancer/{freelancer_id}
• Method: GET

Request Parameters

• freelancer_id (path parameter): The ID of the freelancer to retrieve.
• Example Request

GET http://scalabol.freelancer/freelancer/d5a0c5f5-2911-41bc-be7b-559dd3a064b9

Response

• 200 OK: Returns the details of the requested freelancer.

{
  "id": "d5a0c5f5-2911-41bc-be7b-559dd3a064b9",
  "name": "Freelancer1",
  "profile_views": 78,
  ...
}

• 404 Not Found: If the freelancer with the specified ID does not exist.

• URL: http://scalabol.job/jobs/{job_id}
• Method: GET

Request Parameters

• job_id (path parameter): The ID of the job to retrieve.
• Example Request

GET http://scalabol.job/jobs/64d950c5-cb47-4711-b1bc-cbe88415018f

Response

• 200 OK: Returns the details of the requested job.

{
  "id": "64d950c5-cb47-4711-b1bc-cbe88415018f",
  "title": "E-commerce Mobile App",
  "description": "Sample Description",
  ...
}

• 404 Not Found: If the job with the specified ID does not exist.

Load Testing

We used JMeter to load test our app we configured it to simulate hundred thousands of users' requests. here are some results for different endpoints. Here are a few examples of endpoint performance.