Filter ignite stacktrace
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Python
- Stars
- 4.8k
- Forks
- 726
- Avg merge
- 5d 21h
- Merged PRs (30d)
- 5
Description
Should we consider filtering out the frames that come from ignite? Whenever there's a bug in my code I have to go through a lot of frames to find where ignite trace ends, and my code starts.
I think it is possible to do in Python.
E.g. I'd like to turn
/Mine/ml-mine/code.../lm/train.py in main(train, dev, nn_type, ninp, nhid, nlayers, dropout, dropouth, dropouti, dropoute, wdrop, tie_weights, bsz, bptt, max_epochs, lr, clip, adaptive, cutoffs, device, to_device, ckpt_embedding, lock_emb, kernel_size, num_levels, seed)
238
239 logger.debug(f'Invoking trainer.run for {max_epochs} epochs')
--> 240 return trainer.run(data=trn_dset, max_epochs=max_epochs)
.../ignite/ignite/engine/engine.py in run(self, data, max_epochs)
264 except BaseException as e:
265 self._logger.error("Engine run is terminating due to exception: %s", str(e))
--> 266 self._handle_exception(e)
267
268 return self.state
.../ignite/ignite/engine/engine.py in _handle_exception(self, e)
222 def _handle_exception(self, e):
223 if Events.EXCEPTION_RAISED in self._event_handlers:
--> 224 self._fire_event(Events.EXCEPTION_RAISED, e)
225 else:
226 raise e
.../ignite/ignite/engine/engine.py in _fire_event(self, event_name, *event_args)
182 self._logger.debug("firing handlers for event %s ", event_name)
183 for func, args, kwargs in self._event_handlers[event_name]:
--> 184 func(self, *(event_args + args), **kwargs)
185
186 def terminate(self):
.../lm/train.py in handle_exception(engine, e)
178
179 else:
--> 180 raise e
181
182 @validator.process_function()
.../ignite/ignite/engine/engine.py in run(self, data, max_epochs)
251 self.state.epoch += 1
252 self._fire_event(Events.EPOCH_STARTED)
--> 253 hours, mins, secs = self._run_once_on_dataset()
254 self._logger.info("Epoch[%s] Complete. Time taken: %02d:%02d:%02d", self.state.epoch, hours, mins, secs)
255 if self.should_terminate:
.../ignite/ignite/engine/engine.py in _run_once_on_dataset(self)
213 except BaseException as e:
214 self._logger.error("Current run is terminating due to exception: %s", str(e))
--> 215 self._handle_exception(e)
216
217 time_taken = time.time() - start_time
.../ignite/ignite/engine/engine.py in _handle_exception(self, e)
222 def _handle_exception(self, e):
223 if Events.EXCEPTION_RAISED in self._event_handlers:
--> 224 self._fire_event(Events.EXCEPTION_RAISED, e)
225 else:
226 raise e
.../ignite/ignite/engine/engine.py in _fire_event(self, event_name, *event_args)
182 self._logger.debug("firing handlers for event %s ", event_name)
183 for func, args, kwargs in self._event_handlers[event_name]:
--> 184 func(self, *(event_args + args), **kwargs)
185
186 def terminate(self):
.../lm/train.py in handle_exception(engine, e)
178
179 else:
--> 180 raise e
181
182 @validator.process_function()
.../ignite/ignite/engine/engine.py in _run_once_on_dataset(self)
205 self.state.iteration += 1
206 self._fire_event(Events.ITERATION_STARTED)
--> 207 self.state.output = self._process_function(self, batch)
208 self._fire_event(Events.ITERATION_COMPLETED)
209 if self.should_terminate or self.should_terminate_single_epoch:
.../lm/train.py in step(engine, batch)
107 output = output.view(-1, output.size(2))
108
--> 109 loss = loss_fn(output, y)
110 loss.backward()
111
into
/Mine/ml-mine/code.../lm/train.py in main(train, dev, nn_type, ninp, nhid, nlayers, dropout, dropouth, dropouti, dropoute, wdrop, tie_weights, bsz, bptt, max_epochs, lr, clip, adaptive, cutoffs, device, to_device, ckpt_embedding, lock_emb, kernel_size, num_levels, seed)
238
239 logger.debug(f'Invoking trainer.run for {max_epochs} epochs')
--> 240 return trainer.run(data=trn_dset, max_epochs=max_epochs)
<PYTORCH-IGNITE STACKTRACE SKIPPED. USE `ignite.set_stacktrace(True)`>
.../lm/train.py in step(engine, batch)
107 output = output.view(-1, output.size(2))
108
--> 109 loss = loss_fn(output, y)
110 loss.backward()
111
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the Python exception flow in ignite/engine/engine.py, especially run, _run_once_on_dataset, _handle_exception, and _fire_event. Use the example stacktrace to determine the desired filtering boundary, and verify that Ignite frames can be omitted while ignite.set_stacktrace(True) preserves the full trace.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- developer-experience, machine-learning
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100