TeleSummary
TeleSummary is a telegram bot that summarizes all the messages that are sent by various users in a Telegram group.
Application goals
Especially in the workplace, Telegram has become one of the most used messaging systems among teams, so a system like TeleSummary could help large organizations with distributed work teams monitor internal communications and make decisions about them more quickly.
Technologies
-
Data source: python-telegram-bot.
-
Data ingestion: Fluentd.
-
Data streaming: Kafka.
-
Centralized Service: Zookeeper.
-
Data Processing: Apache Spark.
-
Data Indexing: Elasticsearch.
-
Data Visualization: Kibana.
Project pipeline
Project setup
Requirements
- Docker
- Docker Compose
- Telegram Bot: place the bot token inside a .env file in the root directory:
MY_TOKEN = insertyourtoken
Downloads
- Kafka: download from this link and put it in kafka/setup directory.
- Spark: download from this link and put it in spark/setup directory.
Application launch
- Create container virtual network:
$ docker network create --subnet=10.0.100.0/24 tap
- Launch app:
$ docker compose up
Results
Here is services IP table:
| Container | URL | Description |
|---|---|---|
| Kafka UI | http://localhost:8080 | Open kafka UI to monitor Kafka Topics |
| Elasticsearch | http://localhost:9200 | Open Elasticsearch to manage indexes |
| Kibana | http://localhost:5601 | Open Kibana to view data and create a dashboard |
These are my visualizations:
To import the same:
- Open Kibana.
- Go to "Stack Management" and after go to "Saved Objects".
- Import from file.
- Choose kibana/Dashboard_TAP.ndjson
Author
TeleSummary has been developed by Luca Strano, Computer Science student at Department of Mathematics and Computer Science, University of Catania, for the "Technologies for Advanced Programming" course.
Email: [email protected]