Data Analysis with Spark and Databricks in Azure Synapse
Consultant, Data Scientist
Azure Synapse Workspace provides the ability to use both Apache Spark and Databricks. Which one should you use? The answer of course is “It Depends”. In this session we are going to review what the use cases are which would determine why you would select one tool over another. Here we will examine the costs, the kind of data being analyzed, how much data is processed, variability of data loads, and other variables which determine which solution should be implemented. This session will also review when a Spark based processing tool should be used and when you are better off another tool such as SQL on-demand or an Extract, Load and Transform (ELT) process. The demos will show how to implement each solution in the Azure Synapse Workspace and how each can be used to process data.
Familiarity with Azure