Learning Transformer

퍼블리셔 Sungjun Kim AppTrack Ninja.

Attention, one step at a time

A Transformer, built by hand and shown one step at a time. Twenty-three screens follow "Attention Is All You Need" (Vaswani et al., 2017) in the paper's own order: a word becoming numbers, the three vectors, the scores, the softmax, four heads merged into one, the mask, the second attention, and the loop that writes one word at a time. What this is not: a trained model. Every weight here was set by hand, so that each one can be read in a single line. Nine numbers stand for a word instead of 512. One encoder layer and one decoder layer instead of six of each. Four heads, four words of vocabula…

카테고리: Education, Reference

의 App Store 순위, 스크린샷, 추정, 유사 앱. Learning Transformer.