4
$\begingroup$

I'd like to implement the following function $f$ in a neural net:

$$ f(i,j)=s_i-s_j $$

where $1\leq i,j\leq N$ are integers and the $s_i\in\mathbb{R}$ are learnable parameters.

What's the best way to do this?

EmbeddingLayer[1, n, "Input" -> 2] appears promising; it maps integers pair of integers $(i,j)$ to learnable real numbers $(s_i,s_j)$. However, I haven't figured out how to subtract them afterwards.

Attempt 1

I tried this:

NetChain[{ EmbeddingLayer[1, n, "Input" -> 2], ThreadingLayer[Subtract] }] 

which fails with

NetChain::ninctyp2: Incompatible types for output of layer 1, EmbeddingLayer[1,\[Ellipsis]], and input to layer 2, ThreadingLayer[Subtract,\[Ellipsis]]; a 2*1 matrix is not compatible with a pair of tensors, respectively. 

It fails because the ThreadingLayer expects two scalar inputs, not one input that's a 2-vector, but am not sure how to massage the data into the right form.

Attempt 2

This appears to work, but seems hokey:

NetGraph[{ EmbeddingLayer[1, n, "Input" -> 2], PartLayer[1], PartLayer[2], ThreadingLayer[Subtract] }, {1 -> 2, 1 -> 3, {2, 3} -> 4}] 

enter image description here

It sends the first layer output in two directions, pulls out the first and second elements, and recombines them later. I feel like there's got to be a simpler way than this.

$\endgroup$

1 Answer 1

2
$\begingroup$

Since $s_i-s_j$ is essentially a dot product with the vector {1,-1}, you could use:

net = NetGraph[ { ConstantArrayLayer["Array" -> {{1, -1}}], EmbeddingLayer[1, n, "Input" -> 2], DotLayer[] }, {1 -> 3, 2 -> 3}] 

enter image description here

$\endgroup$

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.