tensorflow / tensorflow/privacy
InvalidArgumentError with Keras ImagedataGenerator
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2k
- Forks
- 477
- Avg merge
- 22h 12m
- Merged PRs (30d)
- 1
Description
Is there any guidance how to choose parameters in conjunction with the Keras ImagedataGenerator?
I experience InvalidArgumentError during the last quarter of the 1st epoch.
I guess the issue is related to the combination of the following parameters:
- training samples: 10057
- batch size of train_generator / valididation_generator: 32 / 32
- step_size_train: 314
- step_size_valid: 126
- num_microbatches: 16
InvalidArgumentError Traceback (most recent call last)
in
5 callbacks=[],
6 validation_data = validation_generator,
----> 7 validation_steps = step_size_valid)
/anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in fit(self, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing, **kwargs)
725 max_queue_size=max_queue_size,
726 workers=workers,
--> 727 use_multiprocessing=use_multiprocessing)
728
729 def evaluate(self,
/anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_generator.py in fit(self, model, x, y, batch_size, epochs, verbose, callbacks, validation_split, validation_data, shuffle, class_weight, sample_weight, initial_epoch, steps_per_epoch, validation_steps, validation_freq, max_queue_size, workers, use_multiprocessing)
601 shuffle=shuffle,
602 initial_epoch=initial_epoch,
--> 603 steps_name='steps_per_epoch')
604
605 def evaluate(self,
/anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_generator.py in model_iteration(model, data, steps_per_epoch, epochs, verbose, callbacks, validation_data, validation_steps, validation_freq, class_weight, max_queue_size, workers, use_multiprocessing, shuffle, initial_epoch, mode, batch_size, steps_name, **kwargs)
263
264 is_deferred = not model._is_compiled
--> 265 batch_outs = batch_function(*batch_data)
266 if not isinstance(batch_outs, list):
267 batch_outs = [batch_outs]
/anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py in train_on_batch(self, x, y, sample_weight, class_weight, reset_metrics)
1015 self._update_sample_weight_modes(sample_weights=sample_weights)
1016 self._make_train_function()
-> 1017 outputs = self.train_function(ins) # pylint: disable=not-callable
1018
1019 if reset_metrics:
/anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow_core/python/keras/backend.py in call(self, inputs)
3474
3475 fetched = self._callable_fn(*array_vals,
-> 3476 run_metadata=self.run_metadata)
3477 self._call_fetch_callbacks(fetched[-len(self._fetches):])
3478 output_structure = nest.pack_sequence_as(
/anaconda/envs/azureml_py36/lib/python3.6/site-packages/tensorflow_core/python/client/session.py in call(self, *args, **kwargs)
1470 ret = tf_session.TF_SessionRunCallable(self._session._session,
1471 self._handle, args,
-> 1472 run_metadata_ptr)
1473 if run_metadata:
1474 proto_data = tf_session.TF_GetBuffer(run_metadata_ptr)
InvalidArgumentError: 2 root error(s) found.
(0) Invalid argument: Input to reshape is a tensor with 9 values, but the requested shape requires a multiple of 8
[[{{node training_12/Reshape}}]]
[[metrics_12/acc/Identity/_1029]]
(1) Invalid argument: Input to reshape is a tensor with 9 values, but the requested shape requires a multiple of 8
[[{{node training_12/Reshape}}]]
0 successful operations.
0 derived errors ignored.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the Keras model.fit call and the training and validation generator parameters shown in the traceback. Reproduce the failure with the stated sample count, batch sizes, steps, and microbatch count, then isolate which combination causes the reshape error. Done means identifying whether this is a usage issue or a TensorFlow Privacy defect and recording actionable guidance or a confirmed fix location.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100