Hypernetworks, also known as dynamic networks, are neural networks in which the weights of at least some of the layers vary dynamically based on the input. Such networks have composite architectures in which one network predicts the weights of another network. I will briefly describe the early days of dynamic layers and present recent results from diverse domains: 3D reconstruction from a single image, image retouching, electrical circuit design, decoding block codes, graph hypernetworks for bioinformatics, and action recognition in video. Finally, I will present a new hypernetwork-based model for the role of feedback in neural computations.