Multidimensional Neural Network

In Fully Connected Backpropagation Neural Networks, with many layers and many neurons in layers there is problem known as Gradient Vanishing Problem. Solution to lower its magnitude is to use Not Fully Connected Neural Network, when that is the case than with which neurons from previous layer neuron is connected has to be considered. The simplest solution would be to use Cartesian Coordinate System, and treat layers as one dimensional lines or two dimensional rectangles or three, four, five ... dimensional cuboids. In that model each neuron in layer is connected to neurons in its surrounding in previous layer.