In this video, Chris Seferlis continues discussing the Modern Data Platform in Azure with Part 3: Data Processing.
Tools Discusssed:
Learn how to use Apache Spark and Delta Lake on Databricks to perform ETL processing, manage late arriving data, and repair corrupted data.
Companies look to support both business analytics and machine learning initiatives within their organization, but often face challenges with complex operations, proprietary technologies, and unreliable data.
Chris Seferlis will be publishing on the modern data warehouse in Azure.
Here he starts with an overview of the stages of a data warehouse and the implications of ELT vs ETL as we move from sources, ingestion, storage, transformation, staging and presentation.
Learn more:
https://azure.microsoft.com/en-in/solutions/architecture/modern-data-warehouse/
Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources—at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs. All of this leverages our limitless Azure Data Lake Storage service for any type of data.
Microsoft Mechanics explains.
Chris Seferlis describes some key differences between the Kappa and Lambda Architectures, advantages and disadvantages of each, and why you might choose one over the other on the Azure platform.
In this episode of Data Driven, Andy and I talk to Trey Johnson about ETL, Data, and fixing up old cars. Also, I apologize for the delay in production for this episode
Press the play button below to listen here or visit the show page at DataDriven.tv
The latest episode of Data Exposed welcomes Alan Yu, a PM in the SQL Server engineering group focusing on SQL Tools. He shows off the latest SQL Server Management Studio 17.3 update.
Alan starts off by highlighting two new features in the latest release: Import Flat File Wizard and XEvent Profiler. Import Flat File Wizard is a new tool that helps streamline the import experience for a user who wants to import .csv or .txt files without specialized domain knowledge. The XEvent profiler is a live viewer window of extended events that can be quickly launched. For the purpose of this demo, Alan chose to focus on showcasing the Import Flat File Wizard.
To learn more about SSMS 17.3, check out this blog post: https://blogs.technet.microsoft.com/dataplatforminsider/2017/10/10/whats-new-in-sql-server-management-studio-17-3/
Download the latest version of SSMS: https://docs.microsoft.com/en-us/sql/ssms/download-sql-server-management-studio-ssms
To learn more about PROSE, check out: https://microsoft.github.io/prose/
