Neuron2vec

Embedding is the key.

What is neuron2vec?

The story starts from word2vec. In language processing, they developed a method to represent words in vectors so that we can actually calculate how similar two words are or even predict the next word using a given context.

Suppose we have found the vector representations of two words dog and cat, which are $\mathbf v_1$ and $\mathbf v_2$, the similarity between these two words are calculated as

$$ \cos(\mathbf v_1,\mathbf v_2) = \frac{\mathbf v_1\cdot \mathbf v_2}{\left|\mathbf v_1\right| \left| \mathbf v_2\right|}. $$

For $\cos(\mathbf v_1,\mathbf v_2)\sim 1$ the two words dog and cat are very similar to each other. Otherwise they are very different.

Using this method people can also predict which words are usually used together to phrase something. For example, we can predict which word is the most probable word following I like.

This use case is what we would like to work out a copycat in neuroscience. More explicitly, what we would like to predict is that given a neuron $n$ with parameters ${\alpha,\beta,\cdots}$ $n(\alpha,\beta,\cdots)$, which neuron with other parameters $n(\alpha’,\beta’,\cdots)$ would it connect to.

The mysterious vectorization

How exactly we map a word into a vector is quite complicated. However, for the case of neurons, the procedure is quite easy (after several days of thinking).

To be simple and clear, we assume that only one kind of neurons is used in a network, which can be described using a fucntion $n(\alpha,\beta,\cdots)$. All we need to do is to map the function onto a vector space. Artificial neural network can finish this task without any trouble. Or we could just develop a function that does the task with a huge amount of parameters, which is not so convinient.

How to train the ANN?

We need data from computational neuroscience.

1. All the synapse data from a neural network.
2. We train the ANN according to the cosine distance we defined.
3. Use the ANN to predict what would a given neuron connect to and with what synapse weight.