In this video Chris Seferlis gives a quick demonstration of connecting an external data source to my Synapse environment and the differences in performance between connecting as an external table or querying directly off of blob storage.

Then he goes into the difference in external table and how that differs from copying the data right into Synapse or using a CTAS statement.

Coming from a data warehousing and BI background, Franco Patano wanted to have a catalogue of the Lakehouse, including schema and profiling statistics.

He created the Lakehouse Data Profiler notebook using Python and SQL to analyze the data and generate schema and statistics tables. He then uses the new SQL Analytics product from Databricks to dashboard and visualize the data profiling statistics. He discusses how to use these dashboards to optimize JOINs and other operations.

[ Lightning talk from Data + AI Summit 2020]

Microsoft Mechanics shows us a practical use case for Predictive Maintenance, Safety, and Efficiency through Microsoft Azure Synapse.

Find out how Azure Synapse is part of the next-generation data and analytics platform for global aviation tech company, GE Aviation. Jeremy Chapman speaks with Luke Bowman, Senior Product Manager at GE Aviation’s Digital Group, to discuss how they are evaluating Azure Synapse to drive the development of predictive maintenance analytics at scale to help airlines, as well as to get ahead of issues to optimize flight safety and operational efficiency.

If you are new to Azure Synapse, it’s Microsoft’s limitless analytics platform that brings enterprise data warehousing and big data processing together into a single service, removing the traditional constraints for analyzing data of all shapes and sizes.

Databricks recently streamed this tech chat on SCD, or Slowly Changing Dimensions.

We will discuss a popular online analytics processing (OLAP) fundamental – slowly changing dimensions (SCD) – specifically Type-2.

As we have discussed in various other Delta Lake tech talks, the reliability brought to data lakes by Delta Lake has brought a resurgence of many of the data warehousing fundamentals such as Change Data Capture in data lakes.

Type 2 SCD within data warehousing allows you to keep track of both the history and current data over time. We will discuss how to apply these concepts to your data lake within the context of the market segmentation of a climbing eCommerce site.

 

In this video, Chris Seferlis continues discussing the Modern Data Platform in Azure with Part 3: Data Processing.

Tools Discusssed:

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources—at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs. All of this leverages our limitless Azure Data Lake Storage service for any type of data.

Microsoft Mechanics explains.