You could have invented transformers from memory
A Hacker News discussion suggests that the concept of transformer models could have been conceived from basic principles, highlighting the foundational nature of the architecture. The post examines how the transformer’s attention mechanism and self‑attention layers can be derived from first‑principles, implying that the breakthrough might have been anticipated by anyone familiar with sequence modeling.