Yannic Kilcher explains why transformers are ruining convolutions.

This paper, under review at ICLR, shows that given enough data, a standard Transformer can outperform Convolutional Neural Networks in image recognition tasks, which are classically tasks where CNNs excel. In this Video, I explain the architecture of the Vision Transformer (ViT), the reason why it works better and rant about why double-bline peer review is broken.

OUTLINE:

  • 0:00 – Introduction
  • 0:30 – Double-Blind Review is Broken
  • 5:20 – Overview
  • 6:55 – Transformers for Images
  • 10:40 – Vision Transformer Architecture
  • 16:30 – Experimental Results
  • 18:45 – What does the Model Learn?
  • 21:00 – Why Transformers are Ruining Everything
  • 27:45 – Inductive Biases in Transformers
  • 29:05 – Conclusion & Comments

Related resources:

  • Paper (Under Review): https://openreview.net/forum?id=YicbFdNTTy

CNBC takes a look at what’s next for the workspace based on what the big tech companies are doing.

Tech offices, from Apple’s 2.8 million square-foot “spaceship” campus, to Facebook’s Menlo Park headquarters complete with a botanical garden, have always pushed the envelope of office space. But coronavirus may make this type of work environment a thing of the past, at least for the near future, as companies try to balance communal work with safety. Here’s a look at how tech companies are changing their offices and work policies as they ease into reopening. 

O’Reilly and TensorFlow teamed up to present the first TensorFlow World last week.

It brought together the growing TensorFlow community to learn from each other and explore new ideas, techniques, and approaches in deep and machine learning.

Presenters in the keynote:

  • Jeff Dean, Google
  • Megan Kacholia, Google
  • Frederick Reiss, IBM
  • Theodore Summe, Twitter
  • Craig Wiley, Google
  • Kemal El Moujahid, Google