Optimal Neural Decision Boundaries for Maximal Information Transmission

Tatyana Sharpee(1) and William Bialek(2)

(1) Sloan--Swartz Center for Theoretical Neurobiology and Department of Physiology,
University of California at San Francisco, San Francisco, CA 9414.

(2) Department of Physics and the Lewis--Sigler Institute for Integrative
Genomics, Princeton University, Princeton, New Jersey 08544

We consider here how to optimally encode multidimensional signals
using a spike or its absence as two possible outcomes in order to
maximize the mutual information transmitted about those signals. Our goal
is to understand the optimal shape of contours in the input space that
would separate stimuli leading to a spike from stimuli that do not elicit
spikes, and how this optimal shape changes with the average rate of
firing a spike. Our assumption is that most of the variability in neural
response will be caused by stimuli near the spiking decision boundary,
and that the width of the uncertainty region is much smaller than the
characteristic length scales over which the probability distribution of
inputs changes. Stimuli that are far away from the decision boundary will
elicit an almost certain response.

      For a single neuron, the problem of maximizing information
transmission given an average spike probability is equivalent to
minimizing the noise entropy. The noise entropy is proportional to an
integral over the probability distribution of inputs along the decision
contour. Solving the variation problem with respect to the contour's
shape, we derive a general equation for the decision boundary valid for
arbitrary probability distributions that relates its curvature, scaled by
the noise level, to the probability distribution of inputs. Solving this
equation for Gaussian inputs, we show that the optimal contours in this
case are straight lines. In other words, neurons that are optimally
designed to process Gaussian inputs should be sensitive to only one
stimulus dimension, which is in the direction perpendicular to the
decision boundary.

       However, neurons in several sensory modalities, as well as
Hodgkin-Huxley model neurons, have been shown to be sensitive to several
stimulus dimensions. Is this a sign of neural non-optimally? Signals
derived from natural environment can sometimes be more closely
approximated by an exponential, as opposed to a Gaussian, distribution.
As an example, we considered two-dimensional exponentially distributed
inputs ~exp(-|x|-|y|). The optimal decision contours in this case can be
solved for exactly. They have different shapes depending on the average
spike probability, but are almost always curved. At extreme spike
probabilities (either close to 0 or to 1), the optimal contours are
similar to squares with rounded corners. For spike probabilities near ½,
the optimal contours extend to infinity, resembling a wedge with a
rounded corner.  The ubiquity of non-Gaussian signals in nature,
particularly of the exponential distributions considered here, makes
these results relevant for neurons across different sensory modalities.

      This work was supported by the Swartz Foundation and NIMH.