huggingface / huggingface/knockknock

Telegram crashes tensorflow model

Open
#60 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.8k
Forks
223
PR merge metrics
No merged PRs in 30d

Description

Hello,
I recently installed knockknock and I am using it with the telegram functionality. It works and sends me messages, however when my tensorflow model is done training, knockknock crashes the entire process with the below error (taken from the telegram message received):

```
Your training has crashed ☠️
Machine name: TomasPC
Main call: auto_train_ML_model
Starting date: 2020-07-10 23:32:02
Crash date: 2020-07-10 23:51:43
Crashed training duration: 0:19:40.329927

Here's the error:
nan

Traceback:
Traceback (most recent call last):
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\knockknock\telegram_sender.py", line 55, in wrapper_sender
value = func(*args, **kwargs)
File "d:\AutoKerasProject\AutoKeras. AutoModel. Time Series. Timeseriesgenerator.py", line 125, in auto_train_ML_model
print(y_train)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\autokeras\auto_model.py", line 444, in evaluate
dataset = self._process_xy(x, y, False)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\autokeras\auto_model.py", line 307, in _process_xy
y = self._adapt(y, fit, self._heads, self._output_adapters)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\autokeras\auto_model.py", line 275, in _adapt
source = adapter.transform(source)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\autokeras\engine\adapter.py", line 88, in transform
return self.convert_to_dataset(dataset)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\autokeras\adapters\output_adapter.py", line 101, in convert_to_dataset
dataset = self.label_encoder.encode(dataset)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\autokeras\encoders.py", line 128, in encode
data))).reshape(-1, 1)
File "D:\Anaconda\envs\tf-gpu\lib\site-packages\autokeras\encoders.py", line 127, in
return np.array(list(map(lambda x: self._label_to_int[x],
KeyError: nan
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with knockknock/telegram_sender.py at wrapper_sender, line 55, then trace the reported call into the AutoKeras paths shown in the traceback. Reproduce the Telegram notification with a TensorFlow training run that raises KeyError: nan, and determine whether knockknock or the underlying model code is responsible. Done means the ownership and expected process behavior are documented and verified.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.