Yannic Kilcher explains the paper “Hopfield Networks is All You Need.”

Hopfield Networks are one of the classic models of biological memory networks. This paper generalizes modern Hopfield Networks to continuous states and shows that the corresponding update rule is equal to the attention mechanism used in modern Transformers. It further analyzes a pre-trained BERT model through the lens of Hopfield Networks and uses a Hopfield Attention Layer to perform Immune Repertoire Classification.

Content outline:

  • 0:00 – Intro & Overview
  • 1:35 – Binary Hopfield Networks
  • 5:55 – Continuous Hopfield Networks
  • 8:15 – Update Rules & Energy Functions
  • 13:30 – Connection to Transformers
  • 14:35 – Hopfield Attention Layers
  • 26:45 – Theoretical Analysis
  • 48:10 – Investigating BERT
  • 1:02:30 – Immune Repertoire Classification

The Verge explores Ikea’s push into smart home technology and why that’s a big deal.

For seven years, Ikea has treated the smart home as a hobby. That’s changing now that Björn Block’s Home Smart division has been promoted to the same importance as Living Room, Bedroom, and all the other Ikea businesses that have come to define the company. Ikea faces the challenge of teaming up with Google, Amazon, Apple, and other tech giants while also battling them for primacy in the home.

Read the full feature here: http://bit.ly/38VyVH9