Cerebral – Microsoft Global Hackathon 2024

Introduction

The "Cerebral" Microsoft Global Hackathon 2024 team has laid the foundation for the next generation of the Cerebral solution, an intelligent assistant at the edge that utilizes Generative AI and various Adaptive cloud approach and services. This innovative solution is designed to simplify tasks for different roles across industries such as Retail, Logistics, and Manufacturing.

Demo overview

Cerebral serves as the central intelligence engine that interacts with different personas, showcasing how it assists in various scenarios. By integrating seamlessly across different roles, Cerebral demonstrates its multi-industry, multi-role capabilities, making it a powerful AI assistant for enhancing operational workflows, decision-making processes, and cross-functional collaboration in diverse industries.

• Retail (Lisa): As a store manager, Lisa relies on Cerebral to monitor inventory levels. When Cerebral detects low stock for certain products, it proactively alerts Lisa and helps her pre-fill the order form with the required quantities. This feature saves Lisa time and ensures that the store is always well-stocked, preventing potential sales losses due to out-of-stock items.

• Logistics (Mark): As a warehouse supervisor, Mark uses Cerebral to stay informed about operational issues. In this case, Cerebral detects a problem with a sensor indicating a broken fridge that could lead to a shortage of products (e.g., 10 boxes of cheese). Cerebral notifies Mark about this issue and helps him by pre-filling the service order form. This ensures that Mark can quickly respond to maintenance needs and avoid further product losses, maintaining the warehouse's efficiency.

• Manufacturing (Tom): As an operations technician (OT), Tom benefits from Cerebral's ability to provide step-by-step troubleshooting guidance. Tom can consult Cerebral for documentation and even ask natural language questions to understand how to resolve complex problems, such as fixing an issue with a production machine or addressing food safety concerns. For example, if there's an electricity issue affecting the salmon production line, Cerebral helps guide Tom through the process of identifying and fixing the problem, reducing downtime and ensuring product quality.

Watch our quick 3-minutes video:

Key objectives

The work done during the Hackathon week has focused on the following pillars:

• Expand Cerebral for Multi-Industry and Multi-Role Use: Make Cerebral capable of supporting various industries and roles, adapting to different requirements and workflows.
• Enhance User Experience and Graphical Interface: Improve the user interface and overall user experience to make interactions more intuitive and efficient.
• Enable Proactive Notifications: Implement a proactive notification module based on the most critical components of each business.
• Decouple the Frontend from the Backend: Separate the graphical interface from the backend to enhance flexibility and scalability.
• Expand Real-Time Data Capabilities: Extend the ability to access real-time data not only from time-series databases but also by converting natural language questions into queries for relational databases.
• Maintain a Hybrid Model: Provide the option to choose between using Azure OpenAI or leveraging the PHI-3 model and a vector database at the edge.
• Automate Infrastructure Creation as Code: Implement infrastructure automation using Bicep to streamline the deployment and management process, ensuring that all components are deployed efficiently and consistently across different environments.

Note: Cerebral is part of the Arc Jumpstart program and a key component of future Jumpstart product investments. It's first implementation was showcased at the Microsoft Build 2004 conference as part of "the Adaptive cloud approach: Build and scale apps from cloud to edge". An additional reference implementation can be found in the form of a Jumpstart Drop as team continues to invest in the solution development.

Cerebral architecture overview

The architecture of Cerebral is designed to facilitate seamless integration and real-time interaction across various components, making it a versatile solution for different industrial scenarios.

Key components

1. Cerebral Engine: The Cerebral Engine is composed of several logical layers, each playing a vital role in the overall architecture. It integrates seamlessly with different components to provide a flexible and adaptive AI-driven experience. The web application is developed in ReactJS, allowing it to adapt to multiple scenarios and user requirements across various industries.

   • React JS Web Application: This is the user interface layer that offers an intuitive and user-friendly graphical experience. It enables users to interact with Cerebral's capabilities, presenting data and insights in an accessible format. The use of ReactJS makes the application highly adaptable, capable of supporting various workflows and scenarios.

   • REST API Query Processing Orchestrator: The REST API layer acts as the main communication hub, managing incoming queries and orchestrating data retrieval from multiple sources. It serves as a gateway that allows Cerebral’s functionalities to be accessed through REST APIs, providing the flexibility to embed Cerebral in different processes and making it agnostic to any graphical interface. This approach enables integration with other systems or applications, enhancing Cerebral's interoperability.

   • Proactive Recommendation and Alerts Engine: This component provides proactive alerts and recommendations based on predefined rules or AI insights. It continuously monitors the conditions within databases and generates alerts for critical scenarios. Users can configure natural language questions that the engine converts into queries for time-series or relational databases, enabling analysis of the most critical aspects of their business. This ensures that users are always informed about potential issues or opportunities, helping them make data-driven decisions.

   • RAG (Retrieval Augmented Generation) at the Edge: This module utilizes Chroma vector databases along with SLM/LLM models (such as Phi-3) to deliver intelligent responses. The design pattern of Retrieval Augmented Generation (RAG) allows users to work in environments with limited or no internet connectivity while still benefiting from AI capabilities. This enables the deployment of advanced generative AI models at the edge, ensuring low latency and minimal dependence on cloud connectivity.

   • Database Connectors: The database connectors facilitate seamless integration with various databases, including InfluxDB, MSSQL Lite, and custom solutions. Cerebral's ability to convert natural language questions into database queries means that users can interact with their data in a more intuitive way. This allows non-technical users to extract insights from time-series or relational databases without needing to understand complex query languages.

2. Data Sources: Cerebral’s REST API has the capability to analyze natural language questions posed by users and determine whether they relate to real-time data or document/manual-based information. When the question is identified as related to real-time data, the API consults the appropriate data sources, such as InfluxDB for time-series data or SQLite for relational data.

   • InfluxDB: InfluxDB handles telemetry data from various assets, providing near real-time insights. This data source is ideal for monitoring sensor data, allowing users to keep track of asset conditions and operational metrics.

   • SQLite: SQLite serves as the storage for point-of-sale (POS) and inventory data, enabling relational queries. This data source is critical for accessing structured data related to products, transactions, and inventory levels within a data warehouse.

   • Data Simulator: The data simulator generates both time-series and relational data, providing a testing environment for the Cerebral system. This ensures that users can validate their use cases and workflows in a controlled setting before deploying the solution in a live environment.

3. Azure IoT Operations: Cerebral leverages the capabilities of Azure IoT Operations to facilitate data communication and processing at the edge. It utilizes the MQTT broker as part of the logic for the Retrieval Augmented Generation (RAG) process, ensuring efficient data flow at the edge. Additionally, Azure IoT Operations data flow connectors are employed to transmit data to Azure's data services.

   • MQTT Broker & Data Flow: The MQTT broker collects and manages data from IoT devices, enabling seamless integration with edge-based systems. This enhances Cerebral’s ability to handle data from a wide range of industrial IoT sensors and devices, supporting real-time decision-making processes.

4. Azure Cognitive Services & OpenAI: These services enhance Cerebral’s capabilities by providing advanced AI functionalities.

   • Speech-to-Text & AI-Powered Contextualization: Azure Cognitive Services offer speech-to-text capabilities, enabling users to interact with Cerebral using voice commands. Additionally, Azure OpenAI provides AI-powered data contextualization and query classification, allowing users to ask complex questions in natural language and receive intelligent responses.

5. PowerBI Dashboards & Azure Data Factory: These components provide visualization and data orchestration capabilities that are essential for monitoring and analyzing business operations.

   • Data Visualization & Insights: PowerBI Dashboards allow users to visualize data, making it easier to monitor key performance indicators (KPIs) and gain actionable insights. Azure Data Factory acts as the data orchestration service, enabling the movement and transformation of data from various sources into the dashboard for comprehensive analysis.

Decision tree architecture

The decision tree architecture of Cerebral illustrates how different data types and queries are handled to provide relevant insights and recommendations.

Execution flow

Cerebral uses a bi-directional query workflow to handle various use cases:

• Proactive Alerts: Triggers alerts based on inventory shortages, operational outages, or equipment state.
• Work Item Pre-population: Assists in creating work orders or maintenance requests.
• Error Summarization: Provides summaries of operational errors and suggests troubleshooting steps.

Queries are classified into three main types:

• Near Real-Time Data: InfluxDB queries are executed to retrieve time-series data.
• Relational-Based Data: Queries are converted into SQLite queries to fetch necessary data.
• OpenAI RAG-Based Queries: The query is sent to the backend, where the vector database and LLM are used to generate a comprehensive response.

How to use Cerebral

Cerebral offers a versatile solution designed to support various roles:

• Store Managers: Can use Cerebral to monitor inventory levels and address shortages.
• Logistics Managers: Utilize Cerebral to manage equipment status and streamline work orders.
• Operations Managers: Benefit from error summaries and troubleshooting support to ensure operational efficiency.

Cerebral’s multi-industry, multi-role capabilities make it a powerful assistant that integrates AI into day-to-day operations seamlessly.

Business intelligence data pipeline architecture

The Business Intelligence (BI) data pipeline in Cerebral is designed to provide comprehensive insights into sales, inventory management, and operational data across various retail locations. It integrates edge data processing with cloud-based analytics to deliver both real-time and historical insights for different roles, including local store managers, operations teams, business analysts, and accounting professionals.

Key components and flow

1. Data Ingestion

   • Point-of-Sale (POS) System: The POS system captures sales data, which is then ingested into the Cerebral data pipeline. This data provides valuable insights into sales performance, helping store managers monitor sales trends and inventory turnover.
   • Warehouse Inventory Management: Inventory tracking and management data from the warehouse are ingested into the Cerebral pipeline. This helps maintain accurate inventory levels, ensuring that stock is replenished in a timely manner.

2. Data Storage

   • SQLite Database: The ingested data from both POS and inventory tracking systems is stored in an SQLite database running on a Kubernetes pod. This edge-located database includes:
   • POS Database: Stores detailed sales transactions, allowing for real-time sales analysis.
   • Inventory Database: Maintains data related to inventory levels, stock movements, and warehouse transactions.

3. Cerebral Integration The Cerebral engine processes data stored in the SQLite databases, enabling advanced analytics and generating insights based on sales, inventory, and operational metrics. This allows Cerebral to assist in decision-making and provide real-time recommendations to users.

4. Data Synchronization and Replication

   • Azure Data Factory: There is an asynchronous replication of data from the edge-located SQLite database to Azure Data Factory. This process ensures that data from different locations (such as Toronto and Berlin stores) is consolidated and enriched in the cloud for broader analysis and reporting.
   • The consolidated data is made available for cloud-based analytics, providing a more comprehensive view of operations across multiple retail locations.

5. Data Visualization

   • PowerBI Dashboards: PowerBI is used to create both locally and remotely accessible dashboards:
   • Local Dashboards: Store managers and operational technology (OT) personnel access real-time sales and inventory data through locally accessible dashboards.
   • Remote Dashboards: Business analysts and accounting professionals can access consolidated BI dashboards through Azure Data Factory, providing deeper insights into overall business performance.

Benefits

• Real-Time Inventory and Sales Tracking: The integration of POS and warehouse data allows for real-time tracking of inventory and sales, enabling more effective decision-making and stock management.
• Comprehensive Analytics: By consolidating data from multiple stores and making it available through Azure Data Factory, Cerebral ensures that both local and global business insights are easily accessible.
• Edge-to-Cloud Data Flow: The architecture supports an edge-to-cloud data flow, ensuring that data is processed efficiently at the edge while still enabling advanced analytics and reporting in the cloud.

This architecture ensures that Cerebral provides actionable insights, enabling users across different roles to make informed decisions based on accurate and up-to-date data.

Note: For more information, please refer to the dashboards folder.

Getting started

Follow these steps to set up and use Cerebral in your environment:

1. Deploy the Backend: Set up the Cerebral backend components on an edge-located, Arc-enabled Kubernetes cluster.
2. Connect Data Sources: Integrate InfluxDB, SQLite, and other required data sources.
3. Configure the Frontend: Deploy the React JS web application and connect it to the backend using the provided APIs.
4. Enable Proactive Alerts: Configure the proactive alert engine to monitor critical business components.

Prerequisites

Before deploying Cerebral, several prerequisites must be fulfilled to ensure a successful installation and operation. The system is designed to run on both virtual machines and physical servers that can handle edge computing tasks effectively.

Hardware requirements

1. Linux-Based System: Cerebral requires a Linux-based system, specifically a VM or physical machine running Linux Ubuntu 22.04. This system will perform as an edge server, handling queries directly from the production line and interfacing with other operational systems.

2. Resource Specifications:

   • Minimal Resource Deployment: For deployments using the Language Learning Model (LLM) hosted on Azure, a lighter resource footprint is feasible. A machine with at least 16 GB RAM and 4 CPU cores should suffice.
   • Full Resource Deployment: For on-premises deployments where the System Lifecycle Management (SLM) is also located on-premises, a more robust system is required. It's recommended to use an Azure VM configured to simulate an edge environment with 32 GB RAM and 8 CPU cores.

Software requirements

• Azure CLI: Essential for interacting with Azure services.
• Rancher K3s: Lightweight Kubernetes distribution suitable for edge computing environments.
• Curl: Tool to transfer data from or to a server, used during various installation steps.

Network requirements

To ensure smooth communication and operation of the Cerebral, specific network configurations are necessary. These configurations cater to the infrastructure's hybrid nature, leveraging both Azure services and on-premises components.

Azure Arc-enabled Kubernetes

Cerebral utilizes Azure Arc-enabled Kubernetes to extend Azure management capabilities to Kubernetes clusters anywhere. This integration allows for the management of Kubernetes clusters across on-premises, edge, and multi-cloud environments through Azure's control plane.

Control Plane

The control plane of Cerebral, managed through Azure Arc, requires network configurations that adhere to Azure Arc's networking requirements. It's crucial to ensure that all necessary ports and endpoints are accessible to facilitate command and control operations seamlessly.

Data Plane

For the data plane, which handles the direct processing and movement of operational data:

• Port 443 (HTTPS): This port is used predominantly to secure data transmission across the network, ensuring encrypted communication for all data exchanges between the edge devices and the centralized data services.

Deployment steps

Step 1 - Deploy the example infrastructure

Get the infrastructure as code

• Clone the repo with the Infrastructure as Code

git clone https://github.com/Azure/jumpstart-cerebral

• Navigate to the correct folder.

cd jumpstart-cerebral/infrastructure/bicep

• Edit the sample main.bicepparam file with values for your environment, providing a public SSH key,.name for a log analytics workspace, and option to deploy Azure Bastion.

nano main.bicepparam

Deploy the infrastructure as code

• From the same folder, run the following command to deploy the code.

az group create -n Cerebral -l eastus2
az deployment group create -g Cerebral -f main.bicep -p main.bicepparam

• Once the deployment is complete, SSH into the created virtual machine and continue to the next step.

Step 2 - Install Cerebral

Deploy Namespace, InfluxDB, Simulator, and Redis

• Apply the Cerebral namespace:

kubectl apply -f https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/cerebral-ns.yaml

• Create a directory for persistent InfluxDB data:

sudo mkdir /var/lib/influxdb2
sudo chmod 777 /var/lib/influxdb2

• Deploy InfluxDB:

kubectl apply -f https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/influxdb.yaml

• Configure InfluxDB:

kubectl apply -f https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/influxdb-setup.yaml

• Deploy the data simulator:

kubectl apply -f https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/cerebral-simulator.yaml

• Deploy SQL Server Lite:

kubectl apply -f https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/mssql.yaml

• Configure SQL Server Lite:

kubectl apply -f https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/mssql-setup.yaml

• Validate the implementation:

kubectl get all -n cerebral

Access InfluxDB

Use a web browser to connect to the Server IP of the InfluxDB service to access its interface, for example http://<IP_Server>:8086

Validate that there is a bucket named manufacturing and that it contains a measurement called assemblyline with values.

To access to Grafana use the admin and the password of ArcPassword123!!

Install Redis

• Deploy Redis to store user sessions and conversation history:

kubectl apply -f https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/redis.yaml

Deploy the Cerebral application

• Create an Azure OpenAI service in your subscription and obtain the key and endpoint for use in the application configuration.

• Download the Cerebral application deployment file:

wget https://raw.githubusercontent.com/Azure/jumpstart-cerebral/refs/heads/main/deployment/cerebral-api.yaml

• Edit the file with your Azure OpenAI instance details:

nano cerebral-api.yaml

• Apply the changes and deploy the application:

kubectl apply -f cerebral-api.yaml

Verify all Components

Ensure that all components are functioning correctly by checking the pods and services.

Testing Cerebral

At this point in the setup, "Cerebral" is fully operational using only Azure Open AI for processing queries. Here are the steps to test the functionality of the system:

Accessing the web Interface swagger

Open your web browser and navigate to the web address of your server appended with port 5003. For example, if you are running the server locally, you would go to:

http://localhost:5003

Note on deployment types

• Cloud-Based LLM Deployment: This setup requires minimal resources at the Edge and leverages Azure's robust cloud capabilities for processing and data handling, suitable for scenarios with adequate network connectivity and less stringent data locality requirements.

• On-premises SLM Deployment: This approach is ideal for environments where the integration with on-premises data is required and is requires to have the SLM at the edge. It demands more substantial resources but provides enhanced control over data and processes.

Conclusion

The Cerebral solution is built to be adaptive, efficient, and scalable, making it an indispensable tool across multiple industries. By leveraging the power of Generative AI, real-time data processing, and seamless integration with cloud and edge services, Cerebral offers a robust platform for enhancing operational workflows and decision-making processes.