Azure Databricks Learning

Hands-on Manual

1. Table of Contents

Cluster Deployment

• Create Azure Databricks Workspace

• Create First Databricks Cluster

• Get Databricks Personal Access Token

• Get Databricks Instance

• Create Azure Databricks Cluster - Azure Data Lake Storage Credential Passthrough

Environment Configuration

• Development Environment Configuration

• Secret Scope

• Use secrets in Databricks

• Mount ADSL Gen2 to Cluster using service principal and OAuth 2.0

• Provide a static Public IP Address to On-Premises or Other Cloud Infrastructure Firewall

• S2S VPN Between Databricks Cluster and RDS Services

Sample Code

• Use secrets as parameters

• Pass parameters to Notebook

• Read data from IoT Hub

• Read data from SAP HANA

• Read data from JSON

• Read data from Kafka

• Read data from Azure Data Lake Storage Gen2

• Read data from Azure SQL DB

• Read data from Azure Synapse Analyst - SQL Pool

• Write data to Azure Data Lake Storage Gen2

• Write data to Kafka

• Write data to Azure Event Hubs

• Write data to Power BI Streaming Datasets API

Others

• Create Azure Data Lake Storage Gen2 Storage Account
• Set up Azure Private Link with Azure Databricks

2. Appendix