Azure Synapse Analytics holds both promise and peril for data professionals of every kind.

Azure Thursday recently posted this great session on Azure Synapse Analytics.

In order to get maximum use of this tool, we need to figure out not only what it can do on its own, but also how it fits together with everything else in Azure. Since Azure Synapse Analytics is a tightly packaged collection of seemingly random pieces, this can seem quite overwhelming.

If you wonder how security works, what piece goes where or how the network plays a part in moving data around, this is the session for you.

In this session we will look into what pieces make up the buffet that is Azure Synapse Analytics, find out how it can work with both structured and unstructured data with ease, and explore some of the architectural patterns that ensure security and performance.

In this session from Ignite 2020, learn how you can build real time BI dashboards with deep granularity using Azure Synapse and Azure Cosmos DB.

Additional resources:

Azure Synapse workspaces can host a Spark cluster.

In addition to providing the execution environment for certain Synapse features such as Notebooks, you can also write custom code that runs as a job inside Synapse hosted Spark cluster.

This video walks through the process of running a C# custom Spark job in Azure Synapse. It shows how to create the Synapse workspace in the Azure portal, how to add a Spark pool, and how to configure a suitable storage account. It also shows how to write the custom job in C#, how to upload the built output to Azure, and then how to configure Azure Synapse to execute the .NET application as a custom job.

Topics/Time index:

  • Create a new Azure Synapse Analytics workspace (0:17)
  • Configuring security on the storage account (1:29)
  • Exploring the workspace (2:42)
  • Creating an Apache Spark pool (3:01)
  • Creating the C# application (4:05)
  • Adding a namespace directive to use Spark (SQL 4:48)
  • Creating the Spark session (5:01)
  • How the job will work (5:22)
  • Defining the work with Spark SQL (6:42)
  • Building the .NET application to upload to Azure Synapse (9:48)
  • Uploading our application to Azyure Synapse (11:45)
  • Using the ZIPed .NET application in a custom Spark job definition (12:39)
  • Testing the custom job (13:36)
  • Monitoring the job (13:56)
  • Inspecting the results (14:25)

If you’re looking to get started with Azure Synapse, Microsoft recently released an Azure Synapse POC template on GitHub that configures a full environment in just a few easy clicks.

Time index:

  • 00:00 – Intro
  • 01:12 – Overview of Azure Synapse
  • 03:13 – Template Overview
  • 06:27 – Deploy the POC Environment
  • 09:45 – Review Deployed Resources
  • 12:03 – Review Logic Apps
  • 15:03 – Synapse Workspace Overview
  • 16:41 – Post-Deployment Steps

Chris Seferlis discusses one of the lesser known and newer Data Services in Azure, Data Explorer.

If you’re looking to run extremely fast queries over large sets of log and IoT data, this may be the right tool for you. I also discuss where it’s not a replacement for Azure Synapse or Azure Databricks, but works nicely alongside them in the overall architecture of the Azure Data Platform.

In this video Chris Seferlis discusses some of the reasons you might want to choose Azure Data Factory over Azure Synapse Workspaces with Synapse Studio.

Even though many of the features overlap, there are still scenarios where I’d use ADF, and pass on the additional features of Synapse. Let me know your thoughts below, please like, comment, share and follow me on Twitter: @bizdataviz

Microsoft Mechanics shows us a practical use case for Predictive Maintenance, Safety, and Efficiency through Microsoft Azure Synapse.

Find out how Azure Synapse is part of the next-generation data and analytics platform for global aviation tech company, GE Aviation. Jeremy Chapman speaks with Luke Bowman, Senior Product Manager at GE Aviation’s Digital Group, to discuss how they are evaluating Azure Synapse to drive the development of predictive maintenance analytics at scale to help airlines, as well as to get ahead of issues to optimize flight safety and operational efficiency.

If you are new to Azure Synapse, it’s Microsoft’s limitless analytics platform that brings enterprise data warehousing and big data processing together into a single service, removing the traditional constraints for analyzing data of all shapes and sizes.

Microsoft Mechanics learns how UK-based data engineering consultant, endjin, is evaluating Azure Synapse for on-demand serverless compute and querying.

Endjin specializes in big data analytics solutions for customers across a range of different industries such as ocean research, financial services, and retail industries.

Host Jeremy Chapman speaks with Jess Panni, Principal and Data Architect at endjin, to discuss how they’re using SQL serverless for on-demand compute as well as visualization capabilities to help customers with big data challenges. If you are new to Azure Synapse, it’s Microsoft’s limitless analytics platform that brings enterprise data warehousing and big data processing together into a single service, removing the traditional constraints for analyzing data of all shapes and sizes.

For more information on endjin and how they help small teams achieve big things, check out their website at https://endjin.com

Watch an introduction to Azure Synapse at https://aka.ms/mechanicssynapse 

Check out other early adopters on our How We Built It series at https://aka.ms/AzureSynapseSeries