Here’s a great overview of deep learning, an artificial intelligence function that imitates the working of the human brain in processing data and creating patterns for use in decision making.

Deep learning is a subfield of machine learning concerned with algorithms inspired by the structure and function of the brain called artificial neural network (ANN). It has networks capable of learning unsupervised or unstructured data. Deep learning is often known as deep neural learning or deep neural network.

Yannic Kilcher explains why transformers are ruining convolutions.

This paper, under review at ICLR, shows that given enough data, a standard Transformer can outperform Convolutional Neural Networks in image recognition tasks, which are classically tasks where CNNs excel. In this Video, I explain the architecture of the Vision Transformer (ViT), the reason why it works better and rant about why double-bline peer review is broken.

OUTLINE:

  • 0:00 – Introduction
  • 0:30 – Double-Blind Review is Broken
  • 5:20 – Overview
  • 6:55 – Transformers for Images
  • 10:40 – Vision Transformer Architecture
  • 16:30 – Experimental Results
  • 18:45 – What does the Model Learn?
  • 21:00 – Why Transformers are Ruining Everything
  • 27:45 – Inductive Biases in Transformers
  • 29:05 – Conclusion & Comments

Related resources:

  • Paper (Under Review): https://openreview.net/forum?id=YicbFdNTTy

When you think of “deep learning” you might think of teams of PhDs with petabytes of data and racks of supercomputers.

But it turns out that a year of coding, high school math, a free GPU service, and a few dozen images is enough to create world-class models. fast.ai has made it their mission to make deep learning as accessible as possible.

In this interview fast.ai co-founder Jeremy Howard explains how to use their free software and courses to become an effective deep learning practitioner.

Learn More: