Data Lake Storage Gen 2 is the best storage solution for big data analytics in Azure. With its Hadoop compatible access, it is a perfect fit for existing platforms like Databricks, Cloudera, Hortonworks, Hadoop, HDInsight and many more. Take advantage of both blob storage and data lake in one service!
In this video, Azure 4 Everyone introduces to what Azure Data Lake Storage is, how it works and how can you leverage it in your big data workloads. I will also explain the differences between Blob and ADLS.
Sample code from demo: https://pastebin.com/ee7ULpwx
Next steps for you after watching the video
1. Azure Data Lake Storage documentation
2. Transform data using Databricks and ADLS demo tutorial
3. More on multi-protocol access
4. Read more on ACL
In this video, you will see how to use PolyBase in SQL Server 2019 big data cluster to query data from HDFS and join the data with other tables in the database.
Read more about configuring PolyBase to query HDFS.
Get an overview of compute pools in Big Data Clusters.
A career in data is more than data engineering or data science, here’s a great infographic on Twitter about the career options in Big Data.
This video provides an overview of administration experiences for BDC (Big Data Clusters).
In big data clusters, we ensure that management services embedded with the platform provide fast scale and upgrade operations, automatic logs and metrics collection, enterprise grade secure access and high availability.
Gaurav Malhotra joins Scott Hanselman to show how wrangling data flows in Azure Data Factory.
This provides a code-free, serverless environment that simplifies data preparation in the cloud and scales to any data size with no infrastructure management required.
It uses the industry-leading Power Query data preparation technology (also used in Power Platform dataflows, Excel, and Power BI) to prepare and shape the data. Built to handle all the complexities and scale challenges of big data integration, wrangling data flows enable use Apache Spark execution to help you easily prepare data at scale.
Frank is at the Strata NYC conference today working the Microsoft booth. Stop by and say hello!
Press the play button below to listen here or visit the show page at DataDriven.tv
Here’s an interesting look at how Amazon, a web site that originally only sold books, became a member of the trillion dollar valuation club.