This repository contains different code samples & other examples related to the Databricks platform & Spark:
-
airflow-dags - Examples of Airflow DAGs for Databricks.
-
database-diagram-builder - tool to generate UML diagram(s) for tables in Databricks/Spark database.
-
dbconnect-maven - skeleton of the Maven project for simple Spark job, and instructions on how to run it via databricks-connect.
-
dbconnect-package-versions-check - tool to checks compatibility of local Databricks connect environment with Databricks cluster.
-
dbconnect-sbt - skeleton of the SBT project for simple Spark job, and instructions on how to run it via databricks-connect.
-
dbsql-with-aad-token - example of querying data on Databricks using python-sql-connector library. Authentication to Databricks is performed using Azure Active Directory tokens issued for Azure Service Principal.
-
dbutils-in-jar - example of using Databricks dbutils in JVM-based code compiled into
.jar
.
-
ip-access-list-analyzer - analyzer/fix tool for Databricks IP Access Lists.
-
kafka-eventhubs-aad-auth - library to support Azure Active Directory authentication for Spark Kafka & EventHubs connectors accessing Event Hubs.
-
pyspark-snippets - functions that simplify development of PySpark code for Databricks
-
simba-jdbc-aad-token - example of querying data on Databricks using JDBC protocol. Authentication to Databricks is performed using Azure Active Directory tokens issued for Azure Service Principal.
-
spring-jdbc-dbsql - example of querying Databricks via JDBC using Spring JDBC.
You can also find more examples of Spark code in the other repositories: