linkedin / linkedin/dagli

How to: Feed Output of final dense layer to LSTM using the examples=placeholders as time series data?

Open
#9 18 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
352
Forks
38
PR merge metrics
No merged PRs in 30d

Description

I try to do feature matching on html-document-nodes, and use features of these document nodes (tagname, text, class, length,..) as placeholder (struct). Currently, the network consists of some fasttext and lstm layers, and the final layer is a dense layer, and all nodes are processed independently of each other. The network lacks a time series like connection of the nodes in a document (the classification of a node may depend on the classification of the previous node).

Do I understand right, that something like that would be possible in dl4j?
https://deeplearning4j.konduit.ai/deeplearning4j/reference/recurrent-layers#inference-predictions-one-step-at-a-time
So, before each document the recurrrent state could be cleared?

How would I do that with dagli?

Btw, I recognized a strong performance (accuracy) drop, when using fasttext with multi-label classification, instead of multiple fasttext instances with single-boolean classification.

Contributor guide

Open the contributing guide

Research direction

Start with the linked Deeplearning4j recurrent-layer inference reference and compare its one-step-at-a-time state handling with DAGLI's described dense/LSTM setup. Done would be an agreed, documented way to feed node outputs as a sequence and reset recurrent state per document, or a clear statement that DAGLI does not support it.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.