This repository contains a package with the model built on the well-known "Titanic" dataset.
Many novice data scientists begin their journey in data science by building models on the well-known Titanic dataset. They tend to do that in jupyter notebooks, which is a nice tool for EDA and building simple models. However, when it comes to pushing the built models to production this tool becomes inconvenient.
In fact, there are some steps that should be done in order to prepare the model for production such as organizing the code in modules, writing tests, adding linters and type checks and e.t.c. However, I noticed that the majority of my students are not aware of such steps.
Therefore, I created this repository to teach my students on how to switch from jupyter notebooks to production code and wrap the models into python package, so that it could be used later in different applications such as web application. As an example in this repo the model is built on the Titanic dataset, therefore the built package is called "titanic_model".
This repo is heavily influenced by the excellent course at Udemy "Deployment of Machine Learning Models".
The model parameters are set via configs. The configs are represented by yaml files. The values
for parameters can be set in titanic_model/config.yml
file. The cofigs are parsed and validated
in titanic_model/config/core.py
module using StrictYaml lib for parsing
and Pydantic lib for type checking the values.
The pipeline is set in titanic_model/pipeline.py
file. Training is set in
titanic_model/train_pipeline.py
file. All the data processing steps are made in the same
Scikit-learn style including custom transformations, stored in
titanic_model/processing/features.py
file.
The code for prediction is set in titanic_model/predict.py
file. Before every prediction
the validation of input data is made. The code for validation can be found in
titanic_model/processing/validation.py
file.
The code can be run via the Tox tool. Tox is a
convenient way to set up the environment and python paths automatically and run the
required commands from the command line. The file with description for tox can be found
in tox.ini
file. The following commands can be run from the command line
using tox:
mkdir ./titanic_model/trained_models
and then run tox -e train
tox -e test_package
tox -e typechecks
tox -e stylechecks
In order to install the package run
pip install titanic-model
After that you can make predictions, using the package:
from titanic_model.predict import make_prediction
# Example input
input_dict = {'PassengerId': [0], 'Pclass': [1], 'Name': ['Snyder, Mrs. John Pillsbury (Nelle Stevenson)'],
'Sex': ['female'], 'Age': [23], 'SibSp': [1], 'Parch': [0], 'Ticket': [21228], 'Fare': [82.2667],
'Cabin': ['B45'], 'Embarked': ['S']}
result = make_prediction(input_data=input_dict)
print(result)
Link to the app: https://github.com/Emilien-mipt/titanic-webapp
Link to the corresponding Heroku link: https://titanicwebapp.herokuapp.com/
CI has been added to the project using Github Actions in order to automate package testing step and upload to PyPI step. The files that stand for CI are located in ./github/workflows/
directory. CI.yml
file stands for automatic testing of the package every pull-request and push to the main branch, while PyPI.yml
file is responsible for the automatic upload of the package to the PyPI every time the release is made for the corresponding version of the package.