Despite the outstanding success of deep neural networks in real-world
applications, most of the related research is
empirically driven and a mathematical foundation is almost completely
missing. One central task of a neural network
is to approximate a function, which for instance encodes a
classification task. In this talk, we will be concerned
with the question, how well a function can be approximated by a neural
network with sparse connectivity. Using methods
from approximation theory and applied harmonic analysis, we will derive
a fundamental lower bound on the sparsity of
a neural network. By explicitly constructing neural networks based on
certain representation systems, so-called
$\alpha$-shearlets, we will then demonstrate that this lower bound can
in fact be attained. Finally, we present
numerical experiments, which surprisingly show that already the standard
backpropagation algorithm generates deep
neural networks obeying those optimal approximation rates. This is joint
work with H. Bolcskei (ETH Zurich), P. Grohs
(Uni Vienna), and P. Petersen (TU Berlin).