Advancing Analytics highlights the Data and AI updates from the first day of Ignite 2021.
Advancing Analytics answers the question that has come up a few times in my day to day work: it looks great, but how much is this going to cost?
There has been a whole lot of excitement around Azure Purview, the new governance and classification tool in Azure, but also a few worried noises about the cost.
In today’s video, Simon looks through the available cost information for Purview, talks about what that means for the product’s target audience and what we can see about the future state.
The Purview Pricing page can be found here – https://azure.microsoft.com/en-us/pricing/details/azure-purview/
Advancing Analytics takes a closer look at the two new runtimes available for Databricks.
We have not just one but two new Databricks Runtimes currently in preview – 7.6 brings several new features focussing on making Autoloader more flexible, improving performance of Optimize and Structured Streaming.
Runtime 8.0 is a much wider change, seeing the shift to Spark 3.1, introducing new language versions for Python, Scala and R.
This shift brings a large swathe of functionality, performance and feature changes, so take some time to look through the docs.
Simon walks through the high level notes, pulling out some interesting features and improvements.
Simon from Advancing Analytics explores the Atlas API that’s exposed under the covers of the new Azure Purview data governance offering.
There are a couple of different libraries available currently, so don’t be surprised if we see a lot of shifts & changes as the preview matures!
In this video, Simon takes a look at how you can get started with the API in a Databricks Notebook to register a custom lineage between two entities
For more info on the pyapacheatlas library used, see: https://pypi.org/project/pyapacheatlas/
Gain hands-on experience with running interactive queries using Azure Synapse SQL Serverless in this workshop.
Advancing Analytics highlights the newest member of the Azure Synapse family: Azure Purview.
In this video, Simon runs through the basics of getting set up with Purview, how to register data sources and start your data governance journey.
Considering shifting gears into Spark Data Engineering? We have another fun session with Simon Whiteley and Denny Lee as they answer your questions from their last two tech talks.
Considering shifting gears into Spark Data Engineering?
Join this fun session with Simon Whiteley (@mrsiwhiteley) and Denny Lee (@dennylee) as they chat through their meandering journeys from SQL Server & BI to Apache Spark, Delta Lake, and the emerging Data Lakehouse approach.
Be prepared for a geeky, trans-Atlantic event from two data nerds!