Join Ben Weissman and Anna Hoffman on a tour through the possibilities of Big Data Clusters (BDC).

He will give a brief overview about the general architecture and components of a BDC, followed by a demo where he will integrate data from external and internal sources using the same connection for python and T-SQL, and lastly come up with a real-world analytics scenario that will show you how easy it is to use them.

You may not even need petabytes of data to leverage what they have to offer!

Video index:

  • [01:30] It’s not your Grandpa’s SQL Server
  • [02:00] Big Data Clusters Architecture
  • [04:30] Big Data Clusters Real-World Example
  • [06:30] Demo in Azure Data Studio

Ayman El-Ghazali recently presenting this Introduction to Databricks from the perspective of a SQL DBA at the NoVA SQL Users Group.

Code available at:https://github.com/thesqlpro/blogThis is an introduction to Databricks from the perspective of a SQL DBA. Come learn about the following topics:

  • Basics of how Spark works
  • Basics of how Databricks works (cluster setup, basic admin)
  • How to design and code an ETL Pipeline using Databricks
  • How to read/write from Azure Datalake and Database
  • Integration of Databricks into Azure Data Factory pipeline

Code available at:  https://github.com/thesqlpro/blog

In this video, Anna Hoffman and Jeroen ter Heerdt discuss and show PowerShell notebooks and Azure SQL inside Azure Data Studio. Learn how you can leverage PowerShell notebooks and other CLI tools in them to manage your Azure SQL Databases and Managed Instances.

Time index:

  • [00:00] Intro
  • [00:32] Notebooks in Azure Data Studio
  • [01:20] PowerShell kernel for notebooks
  • [02:20] Setting up Azure (Az) module for PowerShell in your notebook
  • [03:17] Running interactive commands in the integrated terminal in Azure Data Studio
  • [04:21] Resources
  • [04:53] Wrap-up

More Data Exposed videos:

https://www.youtube.com/playlist?list=PLlrxD0HtieHieV7Jls72yFPSKyGqycbZR&WT.mc_id=dataexposed-c9-niner

Persistent Log Buffers, sometimes referred to as tail of log caching, uses persistent memory to persist the database log buffer, eliminating bottlenecks that may occur on busy systems waiting for the log buffer to flush to disk.

A process known as log hardening.

Learn more here.

  • [00:00] Intro
  • [00:45] Positioning persistent log buffer
  • [01:13] Persistent memory (PMEM) devices
  • [01:58] Usecase for and benefits of persistent log buffer
  • [02:31] Best practices for SQL Server with PMEM in Windows
  • [03:38] Best practices for SQL Server with PMEM in Linux
  • [04:01] What is persistent log buffer?
  • [04:43] What is forced delayed durability?
  • [05:30] Difference between persistent log buffer and forced delayed durability
  • [06:42] Demo: setting up persistent log buffer
  • [07:54] Wrap-up

Azure Synapse is a limitless analytics service that brings together enterprise data warehousing and Big Data analytics. It gives you the freedom to query data on your terms, using either serverless on-demand or provisioned resources—at scale. Azure Synapse brings these two worlds together with a unified experience to ingest, prepare, manage, and serve data for immediate BI and machine learning needs. All of this leverages our limitless Azure Data Lake Storage service for any type of data.

Microsoft Mechanics explains.

Learn about what is Spark and using it in Big Data Clusters.

Time index

  • [00:00] Introduction
  • [00:30] One-sentence definition of Spark
  • [00:47] Storing Big Data
  • [01:44] What is Spark?
  • [02:35] Language choice
  • [03:27] Unified compute engine
  • [04:57] Spark with SQL Server
  • [05:47] Learning more
  • [06:10] Wrap-up

In this video from Microsoft Developer, learn anout Accelerated Database Recovery, and how it can solve many availability challenges with no operational or application changes.

Find out more about ADR here: https://docs.microsoft.com/en-us/sql/relational-databases/accelerated-database-recovery-concepts?view=sql-server-ver15?WT.mc_id=dataexposed-c9-niner

[00:00] – Introduction
[00:40] – What is Accelerated Database Recovery (ADR)
[01:36] – The current SQL Server Database Recovery process
[03:00] – Database Recovery process with ADR
[04:58] – Benefits of ADR
[06:00] – Enabling ADR
[07:02] – Truncating transaction log strategy with ADR
[07:44] – Summary and wrap-up