clab / clab/dynet

Variable sized inputs

Open
#1,647 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

Hi everyone 👍

I'm quite new to DyNet and I'm having a bit of a problem implementing a neural network with **variable sized inputs**. I started with the **XOR** example provided below ([Link from DyNet examples ](https://dynet.readthedocs.io/en/latest/tutorials_notebooks/tutorial-1-xor.html#Dynamic-Networks)) The problem is that although in the code, the size of input is dynamically assigned **"len(inputs)"**, it does not work if we change the input size of the learning set because the weight matrix W is already fixed which causes errors while calculating the output . Any suggestions please? I would be very grateful. Thank you
```

import dynet as dy
# create training instances, as before
def create_xor_instances(num_rounds=2000):
questions = []
answers = []
for round in range(num_rounds):
for x1 in 0,1:
for x2 in 0,1:
answer = 0 if x1==x2 else 1
questions.append((x1,x2))
answers.append(answer)
return questions, answers

questions, answers = create_xor_instances()

# create a network for the xor problem given input and output
def create_xor_network(W, V, b, inputs, expected_answer):
dy.renew_cg() # new computation graph
x = dy.vecInput(len(inputs))
x.set(inputs)
y = dy.scalarInput(expected_answer)
output = dy.logistic(V*(dy.tanh((W*x)+b)))
loss = dy.binary_log_loss(output, y)
return loss

m2 = dy.ParameterCollection()
W = m2.add_parameters((8,2))
V = m2.add_parameters((1,8))
b = m2.add_parameters((8))
trainer = dy.SimpleSGDTrainer(m2)

seen_instances = 0
total_loss = 0
for question, answer in zip(questions, answers):
loss = create_xor_network(W, V, b, question, answer)
seen_instances += 1
total_loss += loss.value()
loss.backward()
trainer.update()
if (seen_instances > 1 and seen_instances % 100 == 0):
print("average loss is:",total_loss / seen_instances)
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.