A Hub for Transformer Blogs and Papers
A curated collection of resources about transformer models, including illustrated guides, GNN connections, and architectural improvements.
A curated collection of resources about transformer models and related topics.
Transformers Explained
“The Illustrated Transformer” contains intuitive animations of how transformer models work. Additional resources cover Universal Transformers, which introduces recurrence concepts, and comparative analyses between transformers and RNNs/CNNs for translation tasks.
GNNs: Similarities and Differences
“Transformers are Graph Neural Networks” bridges transformer architectures with graph neural network concepts, highlighting structural similarities and key differences between the two paradigms.
Transformer Improvements
DeepMind’s work on combining Neural Turing Machines with transformer architectures aims to enhance long-term memory capabilities in deep learning systems, addressing one of the key limitations of standard transformers.