One of the new features of Synapse Analytics is Synapse Link – the ability to query a live analytics store within CosmosDB with only tiny amounts of setup. We’ve recently seen it rolled out for the SQL On-Demand endpoint, meaning we can write both Spark and SQL directly over this analytics store!

In today’s video, Simon demonstrates how we can use Synapse Link to build up a Lambda Architecture, which enables near real-time querying with relatively little fuss!

More information on Synapse Link can be found here: https://azure.microsoft.com/en-us/updates/azure-synapse-link-for-azure-cosmos-db-sql-serverless-runtime-support-in-preview/

For the OG Lambda Architecture, check out Nathan Marz’s book “Big Data” here – https://www.manning.com/books/big-data

Learn about seven different database paradigms and what they do best.

Contents:

  • 00:00 Intro
  • 00:45 Key-value
  • 01:48 Wide Column
  • 02:47 Document
  • 04:05 Relational
  • 06:21 Graph
  • 07:22 Search Engine
  • 08:27 Multi-model

The Microsoft Azure channel explains how KPMG Japan uses Azure Arc to build out a seamless data solution.

KPMG Ignition Tokyo, the centerpiece of KPMG Japan’s digital strategy, delivers specialty software solutions to its global clients. With a multi-cloud and hybrid approach, the firm is rolling out its next-generation, AI-based audit software built on Azure, and implementing Azure Arc to deliver seamless solutions for clients across multiple hybrid data estates.

Adam Marczak explains Azure Data Factory Mapping Data Flow in this video.

With Azure Data Factory Mapping Data Flow, you can create fast and scalable on-demand transformations by using visual user interface. In just minutes you can leverage power of Spark with not a single line of code written.

In this episode I give you introduction to what Mapping Data Flow for Data Factory is and how can it solve your day to day ETL challenges. In a short demo I will consume data from blob storage, transform movie data, aggregate it and save multiple outputs back to blob storage.

Sample code and data: https://github.com/MarczakIO/azure4everyone-samples/tree/master/azure-data-factory-mapping-data-flows 

Learn how to extract value from your data to bring the impact of your low-code solutions to a whole new level.

PowerApps already enable creation of useful business applications with minimal effort.

In this session, you will learn about how and why to connect your applications to Azure services responsible for Big Data.

You will see an example of an application that keeps track of NYC taxi logs and provides logistical information for greater business insights. You will leave this session with confident understanding of what Big Data connection options PowerApps provide, how to connect your application to Big Data, as well as how to reference and visualize it.

Additional Resources: Power Apps Devs

Gaurav Sen explains NoSQL databases in this introductory video.

NoSQL is a popular database storage method. It keeps data as key value pairs. The advantages and disadvantages of NoSQL compared with RDBMS (which uses SQL) are discussed here, using the Cassandra architecture as an example.

Video index:

  • 1:08 NoSQL explanation and comparison
  • 10:27 Cassandra Architecture
  • 18:00 Quorum
  • 21:30 Compaction of SST tables

Here’s an interesting idea that combines K8S, AI, Big Data, and HPC.

With the emergence and support of Mobile, IoT and Edge Computing technologies, we are seeing the next wave of workloads running on cloud native platforms — Artificial Intelligence (AI; including Machine Learning and Deep Learning), Big Data, and High-Performance Computing (HPC) — where a large amount of compute resources running “batch jobs” connected to massive data lakes is essential.

Microsoft Mechanics shows us a practical use case for Predictive Maintenance, Safety, and Efficiency through Microsoft Azure Synapse.

Find out how Azure Synapse is part of the next-generation data and analytics platform for global aviation tech company, GE Aviation. Jeremy Chapman speaks with Luke Bowman, Senior Product Manager at GE Aviation’s Digital Group, to discuss how they are evaluating Azure Synapse to drive the development of predictive maintenance analytics at scale to help airlines, as well as to get ahead of issues to optimize flight safety and operational efficiency.

If you are new to Azure Synapse, it’s Microsoft’s limitless analytics platform that brings enterprise data warehousing and big data processing together into a single service, removing the traditional constraints for analyzing data of all shapes and sizes.

The Career Force goes through her top 5 free dataset resources in this video.

  1. Data.gov: https://data.govData.gov is a large dataset aggregator and the home of the US Government’s open data.
  2. FiveThirtyEight: https://data.fivethirtyeight.com/ This is a great resource to not only see datasets, but also see how a well-respected analytics organization provides meaningful insights and commentary on the data.
  3. Kaggle: https://www.kaggle.com/Kaggle  is a great resource not only for free datasets, but for data science topics in general.
  4. Data.World: https://data.world/ There are hundreds of thousands of free datasets for anyone that sets up an account on data.world.
  5. Google Dataset Search: https://datasetsearch.research.google.com/ By accessing thousands of different repositories across the web, Google Dataset Search provides access to almost 25 million different publicly available datasets.