Lex Fridman shared this lecture by Vivienne Sze in January 2020 as part of the MIT Deep Learning Lecture Series.

Website: https://deeplearning.mit.edu
Slides: http://bit.ly/2Rm7Gi1
Playlist: http://bit.ly/deep-learning-playlist

LECTURE LINKS:
Twitter: https://twitter.com/eems_mit
YouTube: https://www.youtube.com/channel/UC8cviSAQrtD8IpzXdE6dyug
MIT professional course: http://bit.ly/36ncGam
NeurIPS 2019 tutorial: http://bit.ly/2RhVleO
Tutorial and survey paper: https://arxiv.org/abs/1703.09039
Book coming out in Spring 2020!

OUTLINE:
0:00 – Introduction
0:43 – Talk overview
1:18 – Compute for deep learning
5:48 – Power consumption for deep learning, robotics, and AI
9:23 – Deep learning in the context of resource use
12:29 – Deep learning basics
20:28 – Hardware acceleration for deep learning
57:54 – Looking beyond the DNN accelerator for acceleration
1:03:45 – Beyond deep neural networks

Here’s my talk from the Azure Data Fest Philly 2020 last week!

Neural networks are an essential element of many advanced artificial intelligence (AI) solutions. However, few people understand the core mathematical or structural underpinnings of this concept. In this session, learn the basic structure of neural networks and how to build out a simple neural network from scratch with Python.Neural networks are an essential element of many advanced artificial intelligence (AI) solutions. However, few people understand the core mathematical or structural underpinnings of this concept. In this session, learn the basic structure of neural networks and how to build out a simple neural network from scratch with Python.

What is the universal inference engine for neural networks?

Microsoft Research just posted this video exploring ONNX.

Tensorflow? PyTorch? Keras? There are many popular frameworks out there for working with Deep Learning and ML models, each with their pros and cons for practical usability for product development and/or research. Once you decide what to use and train a model, now you need to figure out how to deploy it onto your platform and architecture of choice. Cloud? Windows? Linux? IOT? Performance sensitive? How about GPU acceleration? With a landscape of 1,000,001 different combinations for deploying a trained model from some chosen framework into a performant production environment for prediction, we can benefit from some standardization.

While the hype around neural networks may have some ground truth to it, they are not the answer to every single problem.

Here’s a great look at where they may fall short.

Unlike other algorithms, neural networks with their deep learning cannot be programmed directly for the task. Just like a new developing brain, they have the requirement to learn the information. The major advantage of neural networks is its ability to outpace almost every other machine learning algorithm.

Microsoft Research features a talk by Wei Wen on Efficient and Scalable Deep Learning (slides)

In deep learning, researchers keep gaining higher performance by using larger models. However, there are two obstacles blocking the community to build larger models: (1) training larger models is more time-consuming, which slows down model design exploration, and (2) inference of larger models is also slow, which disables their deployment to computation constrained applications. In this talk, I will introduce some of our efforts to remove those obstacles. On the training side, we propose TernGrad to reduce communication bottleneck to scale up distributed deep learning; on the inference side, we propose structurally sparse neural networks to remove redundant neural components for faster inference. At the end, I will very briefly introduce (1) my recent efforts to accelerate AutoML, and (2) future work to utilize my research to overcome scaling issues in Natural Language Processing.

See more on this talk at Microsoft Research:
https://www.microsoft.com/en-us/research/video/efficient-and-scalable-deep-learning/

Samuel Arzt shows off a project where an AI learns to park a car in a parking lot in a 3D physics simulation.

The simulation was implemented using Unity’s ML-Agents framework (https://unity3d.com/machine-learning).

From the video description:

The AI consists of a deep Neural Network with 3 hidden layers of 128 neurons each. It is trained with the Proximal Policy Optimization (PPO) algorithm, which is a Reinforcement Learning approach.

freeCodeCamp.org has a two hour, ad-free tutorial on how to use TensorFlow 2.0 in this full course for beginners.

Course created by Tech with Tim. Check out his YouTube channel: https://www.youtube.com/channel/UC4JX40jDee_tINbkjycV4Sg

Course Contents

  • (0:00:00) What is a Neural Network?
  • (0:26:34) How to load & look at data
  • (0:39:38) How to create a model
  • (0:56:48) How to use the model to make predictions
  • (1:07:11) Text Classification (part 1)
  • (1:28:37) What is an Embedding Layer? Text Classification (part 2)
  • (1:42:30) How to train the model – Text Classification (part 3)
  • (1:52:35) How to saving & loading models – Text Classification (part 4)
  • (2:07:09) How to install TensorFlow GPU on Linux