And end-to-end demo of MLOps in SageMaker
The following prerequisites must be completed to replicate this project.
From the SageMaker Studio UI, create a SageMaker Project using one of the 1st party supported templates. Please use one of the following templates -
*Note - It is recommended to use the first template which leverages SageMaker native services for Source Version Control and CI/CD. This reduces the number of steps needed for an end to end demo. If choosing the other templates, please follow the documentation to complete the template specific prerequisites.
For more information on SageMaker Projects, visit the AWS documentation.
Once the project is created, you will see 2 repositories created - one for the model building code and one for the model deployment code. The model deployment code will remain untouched, the model building code will be changed by replacing pipelines/abalone/
with the code from this repo under sagemaker-pipeline/
datawrangler/abalone-dataset-header.csv
to S3 and note the S3 URIdatawrangler/fs-ingest-wrangler-training-template.flow
with the S3 URI you uploaded the dataset to in the step above.flow
file to SageMaker Studio and open it. This will open up the DataWrangler UI.Export Step
and select Feature Store
. This will generate a notebook.This repository contains 2 folders -
sagemaker-pipeline/
model-monitor
sagemaker-pipeline/pipeline-dw-fs.py
lines 131 to 178 need to be replaced with the code in the notebook created by the DataWrangler Export.step_read_train
sagemaker-pipeline/pipeline.py
with step_read_train
and step_process
from sagemaker-pipeline/pipeline-dw-fs.py
.IF NOT USING FEATURE STORE, IGNORE THE STEPS ABOVE AND FOLLOW THE BELOW STEPS.
pipelines/abalone/
with the code in sagemaker-pipeline/
.-staging
. You can navigate to CodePipeline, and under Pipelines you will see one with your project name and model-deploy
. Click on that Pipeline and you will see a manual approval option. When approved, a new endpoint will be created with the suffix -prod
.To setup Model Monitor
model-monitor/create_endpoint.ipynb
to create an endpoint with DataCapture enabledmodel-monitor/data_quality_monitor.ipynb
to set up a Data Quality Monitoring schedule on the endpoint.Once all the Endpoints have been created, navigate to the Endpoint UI in SageMaker Studio. Click on the endpoint deployed using the notebook in the model monitor folder,
Things to highlight in the demo -