A dockerized application for performing sentiment analysis, tag recognition, and text summarization on YouTube videos.
AudioMood is a dockerized application that offers advanced features for YouTube videos analysis including sentiment analysis (positive, neutral, negative), keyword/tag recognition, and text summarization. Here’s what AudioMood can provide:
.env
file in the main directory and add the line OPENAI_API_KEY=your-api-key
.You first need to clone the repository on your local computer. For the initial run of the application, navigate to its main directory and execute the following command:
docker-compose up --build
Then go to localhost:5000.
To stop the application we recommend using docker-compose down
. To start again your application you won't need the --build
option, so just type docker-compose up
.
The user needs to input a YouTube URL of the video they want to analyze and select the OpenAI Whisper model size for audio-to-text transcription. A producer service in Python will download the video's audio in mp3 format, send it to OpenAI Whisper for full transcription, and forward the entire text to Logstash, which provides data ingestion. The producer will split the text into chunks (about 15 words each) and send each chunk to Logstash for sentiment analysis and keyword recognition. Logstash will distribute each message to a Kafka broker across three different topics: summary, entities, and sentiment. Kafka provides data streaming in KRaft mode.
Three different services in Spark will read data from Kafka topics:
Each Spark service will send data to a different Elasticsearch index for data indexing. In the end, Kibana will then provide data visualization for each feature, and the user will be able to see the embedded Kibana dashboard on their User Interface.
This data engineering project is the final exam for the Technologies for Advanced Programming course, taught by Professor Salvatore Nicotra at the University of Catania, academic year 2023/24.