The best bridge is one that just stands there, whatever the weather.” “Bridges and buildings are all designed to be indifferent to their environment, to withstand fluctuations, not to adapt to them. Natural systems adapt, and become a part of their environment. But in natural systems, where the concept of emergence plays a big role, we see complex designs that emerge due to self-organization, and such designs are usually sensitive and responsive to changes in the world around them. I think we’re building neural network systems the same way we are building bridges and buildings. I believe that the way we are currently doing deep learning is like engineering. Left: The Confederation Bridge in Canada.
Lots of sweat and labor had to go into producing these amazing results.Įngineered vs Emerged Bridges. Modern networks are often even more sophisticated, and require a pipeline that spans network architecture and careful training schemes. 2012), the winner of the ImageNet competition in 2012.Įven if we look at the early AlexNet from 2012, which made deep learning famous when it won the ImageNet competition back then, we can see the careful engineering decisions that were involved in its design. Neural network architecture of AlexNet (Krizhevsky et al. But with all of these advances, the impressive feats in deep learning required a substantial amount of sophisticated engineering effort.ĪlexNet. They’re impacting our everyday lives, from performing predictive tasks such as recommendations, facial recognition and object classification, to generative tasks such as machine translation and image, sound, video generation. Unless you’ve been living under a rock, you would’ve noticed that artificial neural networks are now used everywhere. (Figure: Emergence of encirclement tactics in MAgent.) We survey ideas from complex systems such as swarm intelligence, self-organization, and emergent behavior that are gaining traction in ML.
0 kommentar(er)
