Modular/Multiple Neural networks (MNNs) revolve around training smaller, independent networks that can feed into each other or another higher network https://ift.tt/2O42qv7 In principle, the hierarchical organization could allow us to make sense of more complex problem spaces and reach a higher functionality, but it seems difficult to find examples of concrete research done in the past regarding this. I've found a few sources: https://ift.tt/2FStPAD https://ift.tt/2P4f6SM A few concrete questions I have: Are there any tasks where MNNs have shown better performance than large single nets? Could MNNs be used for multimodal classification, i.e. train each net on a fundamentally different type of data, (text vs image) and feed forward to a higher level intermediary that operates on all the outputs? From a software engineering perspective, aren't these more fault tolerant and easily isolatable on a distributed system? Has there been any work into dynamically adapting the top
