Hunter WebScraping App

This is a personal project that I have made public for interested parties to analyze. This application is a web scraping tool that collects data from the Wvetro website. The data is then stored in a MongoDB database and can be accessed through a RESTful API.

Other applications can consume this API to access the data. The project is still in development, and I plan to add more features in the future.

Stack

The project applies my studies in:

• Node/TypeScript
• Scraping with Puppeteer: Something I was very eager to explore (haha)
• RESTful API
• DTOs
• Interfaces
• Object-Oriented Programming
• Clean Architecture
• Domain-Driven Design
• SOLID Principles
• Service Pattern
• Repository Pattern
• MongoDB (Atlas): Chose MongoDB with Atlas to gain a deeper understanding of the tool.
• Builder Pattern
• Factory Pattern
• Mapper Pattern
• TDD with Vitest: Testing is a new area for me. Previously, I focused more on logic and concepts, but recently I've decided to study and apply it more.

Future plans include:

• Domain Events
• WatchList
• Docker
• User control with:
  • Route authentication
  • Rules

I'm aware that I could use a framework like NestJS, but I chose to implement things more manually for now, though I plan to use it in the future.

Currently, this project also includes WEB and DESKTOP APP projects which are still just ideas. My primary focus is on understanding the backend. However, I plan to create a web app using NextJS and a Windows app using Electron.