clab / clab/dynet

Initializing a parameter with a pre-trained one

Open
#1,163 5 comments 0 reactions 0 assignees View on GitHub
fix needs confirmation
Dominant language
C++
Stars
3.4k
Forks
701
PR merge metrics
No merged PRs in 30d

Description

Hi,
So I have two stages in which I train my model. Say I have encoder-decoder architectures in both stages with parameters A and B respectively (some shared, some separate). I train parameters A in the first stage. In the second stage, I want to train B but I want the parameters in B that are separate to be initialized to the values I got for those parameters belonging to A. Is there any way that can be done in C++ Dynet i.e. initialize a parameter/builder with values obtained for another parameter/builder?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading DyNet's C++ parameter and builder APIs, focusing on how parameters are created and initialized. Trace whether values from a trained parameter can be used when creating a separate parameter in the second encoder-decoder stage. Done means documenting or defining a supported way to initialize B's separate parameters from A's trained values.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.