tensorflow / tensorflow/tensorboard
Loading data on HDFS with reduced feature set (w/o TensorFlow installation)
Nobody has claimed this yet.
- Dominant language
- TypeScript
- Stars
- 7.2k
- Forks
- 1.7k
- Avg merge
- 4d 22h
- Merged PRs (30d)
- 1
Description
When loading TensorBoard data on HDFS without TensorFlow installation, I got the following error log (note: I modified HDFS data paths in the log) and could not load the data. Recently dependent module fsspec has updated (fsspec==2023.1.0) and supported seeking API
- https://filesystem-spec.readthedocs.io/en/latest/changelog.html#id1.
- https://github.com/fsspec/filesystem_spec/pull/1154
In response, I modified tensorboard/compat/tensorflow_stub/io/gfile.py as like
https://github.com/nyamashi/tensorboard/commit/ea9f6e6dd3c36b19d51563f2ab2f0ef7f87e3d40.
Then it looks working.
Is it possible to enhance the dependency (fsspec >= 2023.1.0) and add this modification in tensorboard/compat/tensorflow_stub/io/gfile.py?
Error message before the modification:
$ tensorboard --inspec --logdir {sample hdfs path}
TensorFlow installation not found - running with reduced feature set.
======================================================================
Processing event files... (this can take a few minutes)
======================================================================
Traceback (most recent call last):
File "/opt/myenv/python38/bin/tensorboard", line 8, in <module>
sys.exit(run_main())
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/main.py", line 46, in run_main
app.run(tensorboard.main, flags_parser=tensorboard.configure)
File "/opt/myenv/python38/lib/python3.8/site-packages/absl/app.py", line 308, in run
_run_main(main, args)
File "/opt/myenv/python38/lib/python3.8/site-packages/absl/app.py", line 254, in _run_main
sys.exit(main(argv))
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/program.py", line 276, in main
return runner(self.flags) or 0
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/program.py", line 289, in _run_serve_subcommand
efi.inspect(flags.logdir, event_file, flags.tag)
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/backend/event_processing/event_file_inspector.py", line 430, in inspect
inspection_units = get_inspection_units(logdir, event_file, tag)
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/backend/event_processing/event_file_inspector.py", line 384, in get_inspection_units
field_to_obs=get_field_to_observations_map(generator, tag),
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/backend/event_processing/event_file_inspector.py", line 191, in get_field_to_observations_map
for event in generator:
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/backend/event_processing/event_file_loader.py", line 244, in Load
for record in super(LegacyEventFileLoader, self).Load():
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/backend/event_processing/event_file_loader.py", line 178, in Load
yield next(self._iterator)
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/backend/event_processing/event_file_loader.py", line 109, in __next__
self._reader.GetNext()
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/pywrap_tensorflow.py", line 207, in GetNext
header_str = self._read(8)
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/pywrap_tensorflow.py", line 273, in _read
new_data = self.file_handle.read(n)
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/io/gfile.py", line 728, in read
(self.buff, self.continuation_token) = self.fs.read(
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/io/gfile.py", line 447, in func_wrapper
return func(self, *args, **kwargs)
File "/opt/myenv/python38/lib/python3.8/site-packages/tensorboard/compat/tensorflow_stub/io/gfile.py", line 526, in read
raise errors.InvalidArgumentError(
tensorboard.compat.tensorflow_stub.errors.InvalidArgumentError: b'{sample
hdfs path}/events.out.tfevents.xxxxx' is not seekable
With fsspec == 2023.1.0 and https://github.com/nyamashi/tensorboard/commit/ea9f6e6dd3c36b19d51563f2ab2f0ef7f87e3d40. Looks working:
$ tensorboard --inspec --logdir {sample hdfs path}
TensorFlow installation not found - running with reduced feature set.
======================================================================
Processing event files... (this can take a few minutes)
======================================================================
Found event files in:
{sample hdfs path}
These tags are in {sample hdfs path}
audio -
histograms -
images -
scalars
.....
tensor
....
======================================================================
Event statistics for {sample hdfs path}
audio -
graph -
histograms -
images -
scalars
....
sessionlog:checkpoint -
sessionlog:start -
sessionlog:stop -
tensor
....
======================================================================
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with tensorboard/compat/tensorflow_stub/io/gfile.py and the dependency configuration for fsspec. Reproduce the reduced-feature TensorBoard command against an HDFS logdir, then verify that event inspection completes without the not-seekable error after the dependency and gfile changes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100