huggingface:librispeech_asr cannot be loaded

Short description

tfds.load("huggingface:librispeech_asr")

Fails.

Exception:

EXCEPTION
Traceback (most recent call last):
  File "<frozen importlib._bootstrap>", line 978, in _find_and_load_unlocked
    -- code not available --
AttributeError: module 'datasets' has no attribute '__path__'

During handling of the above exception, another exception occurred:

EXCEPTION
Traceback (most recent call last):
...
  File "/home/az/i6/setups/2022-03-20--sis/recipe/i6_core/datasets/tf_datasets.py", line 41, in DownloadAndPrepareTfDatasetJob.run
    line: ds, info = tfds.load(self.dataset_name, data_dir=self.out_data_dir.get(), with_info=True)
    locals:
      ds = <not found>
      info = <not found>
      tfds = <local> <module 'tensorflow_datasets' from '/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/__init__.py'>
      tfds.load = <local> <function load at 0x7f428682b550>
      self = <local> Job<work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ>
      self.dataset_name = <local> 'huggingface:librispeech_asr', len = 27
      data_dir = <not found>
      self.out_data_dir = <local> <Path /home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir>
      self.out_data_dir.get = <local> <bound method AbstractPath.get of <Path /home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir>>
      with_info = <not found>
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/load.py", line 322, in load
    line: dbuilder = builder(name, data_dir=data_dir, try_gcs=try_gcs, **builder_kwargs)
    locals:
      dbuilder = <not found>
      builder = <global> <function builder at 0x7f428682b430>
      name = <local> 'huggingface:librispeech_asr', len = 27
      data_dir = <local> '/home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir', len = 128
      try_gcs = <local> False
      builder_kwargs = <local> {}
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/load.py", line 151, in builder
    line: cls = builder_cls(str(name))
    locals:
      cls = <not found>
      builder_cls = <global> <function builder_cls at 0x7f428682b3a0>
      str = <builtin> <class 'str'>
      name = <local> DatasetName('huggingface:librispeech_asr')
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/load.py", line 92, in builder_cls
    line: return community.community_register.builder_cls(ds_name)
    locals:
      community = <global> <module 'tensorflow_datasets.core.community' from '/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/community/__init__.py'>
      community.community_register = <global> <tensorflow_datasets.core.community.register_package.PackageRegister object at 0x7f4286851f40>
      community.community_register.builder_cls = <global> <bound method PackageRegister.builder_cls of <tensorflow_datasets.core.community.register_package.PackageRegister object at 0x7f4286851f40>>
      ds_name = <local> DatasetName('huggingface:librispeech_asr')
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/community/register_package.py", line 247, in PackageRegister.builder_cls
    line: return load.builder_cls_from_module(installed_dataset.module_name)
    locals:
      load = <global> <module 'tensorflow_datasets.core.community.load' from '/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/community/load.py'>
      load.builder_cls_from_module = <global> <function builder_cls_from_module at 0x7f4286859d30>
      installed_dataset = <local> _InstalledPackage(package=DatasetPackage(name=DatasetName('huggingface:librispeech_asr'), source=DatasetSource(root_path=GithubPath('github://huggingface/datasets/tree/master/datasets/librispeech_asr'), filenames=['dataset_infos.json', 'librispeech_asr.py'])), instalation_date=datetime.datetime(2...
      installed_dataset.module_name = <local> 'tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr', len = 123
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/community/load.py", line 46, in builder_cls_from_module
    line: importlib.import_module(module_name)
    locals:
      importlib = <global> <module 'importlib' from '/home/az/.linuxbrew/Cellar/python@3.9/3.9.9/lib/python3.9/importlib/__init__.py'>
      importlib.import_module = <global> <function import_module at 0x7f42c8d118b0>
      module_name = <local> 'tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr', len = 123
  File "/home/az/.linuxbrew/Cellar/python@3.9/3.9.9/lib/python3.9/importlib/__init__.py", line 127, in import_module
    line: return _bootstrap._gcd_import(name[level:], package, level)
    locals:
      _bootstrap = <global> <module 'importlib._bootstrap' (frozen)>
      _bootstrap._gcd_import = <global> <function _gcd_import at 0x7f42c8e1c310>
      name = <local> 'tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr', len = 123
      level = <local> 0
      package = <local> None
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
    -- code not available --
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
    -- code not available --
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
    -- code not available --
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
    -- code not available --
  File "<frozen importlib._bootstrap_external>", line 850, in _LoaderBasics.exec_module
    -- code not available --
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
    -- code not available --
  File "/home/az/.cache/tensorflow_datasets/modules/tfds_community/huggingface/librispeech_asr/a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871/librispeech_asr.py", line 21, in <module>
    line: from datasets.tasks import AutomaticSpeechRecognition
    locals:
      datasets = <local> <module 'datasets'>
      datasets.tasks = <local> !AttributeError: module 'datasets' has no attribute 'tasks'
      AutomaticSpeechRecognition = <not found>
ModuleNotFoundError: No module named 'datasets.tasks'; 'datasets' is not a package

Environment information

• Operating System: Ubuntu 20.04
• Python version: 3.9
• tensorflow-datasets/tfds-nightly version: tensorflow-datasets 4.5.2
• tensorflow/tf-nightly version: tensorflow 2.7.0

When I import datasets in advance, it looks slightly different:

  File "/home/az/i6/setups/2022-03-20--sis/recipe/i6_core/datasets/tf_datasets.py", line 41, in DownloadAndPrepareTfDatasetJob.run
    line: ds, info = tfds.load(self.dataset_name, data_dir=self.out_data_dir.get(), with_info=True)
    locals:
      ds = <not found>
      info = <not found>
      tfds = <local> <module 'tensorflow_datasets' from '/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/__init__.py'>
      tfds.load = <local> <function load at 0x7f162ccb8c10>
      self = <local> Job<work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ>
      self.dataset_name = <local> 'huggingface:librispeech_asr', len = 27
      data_dir = <not found>
      self.out_data_dir = <local> <Path /home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir>
      self.out_data_dir.get = <local> <bound method AbstractPath.get of <Path /home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir>>
      with_info = <not found>
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/load.py", line 322, in load
    line: dbuilder = builder(name, data_dir=data_dir, try_gcs=try_gcs, **builder_kwargs)
    locals:
      dbuilder = <not found>
      builder = <global> <function builder at 0x7f162ccb8af0>
      name = <local> 'huggingface:librispeech_asr', len = 27
      data_dir = <local> '/home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir', len = 128
      try_gcs = <local> False
      builder_kwargs = <local> {}
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/load.py", line 168, in builder
    line: return cls(**builder_kwargs)  # pytype: disable=not-instantiable
    locals:
      cls = <local> <class 'tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr.LibrispeechASR'>
      builder_kwargs = <local> {'data_dir': '/home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir'}
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/dataset_builder.py", line 941, in FileReaderBuilder.__init__
    line: super().__init__(**kwargs)
    locals:
      super = <builtin> <class 'super'>
      __init__ = <not found>
      kwargs = <local> {'data_dir': '/home/az/i6/setups/2022-03-20--sis/work/i6_core/datasets/tf_datasets/DownloadAndPrepareTfDatasetJob.VR7DkO3EFlRZ/output/data_dir'}
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/dataset_builder.py", line 202, in DatasetBuilder.__init__
    line: self.info.initialize_from_bucket()
    locals:
      self = <local> <tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr.LibrispeechASR object at 0x7f16dc354160>
      self.info = <local> !AttributeError: module 'datasets' has no attribute 'Audio'
      self.info.initialize_from_bucket = <local> !AttributeError: module 'datasets' has no attribute 'Audio'
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/utils/py_utils.py", line 152, in memoized_property.__get__
    line: cached = self.fget(obj)  # pytype: disable=attribute-error
    locals:
      cached = <local> None
      self = <local> <tensorflow_datasets.core.utils.py_utils.memoized_property object at 0x7f162cd69340>
      self.fget = <local> <function GeneratorBasedBuilder.info at 0x7f162cd6b790>
      obj = <local> <tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr.LibrispeechASR object at 0x7f16dc354160>
  File "/home/az/.local/lib/python3.9/site-packages/tensorflow_datasets/core/community/huggingface_wrapper.py", line 122, in GeneratorBasedBuilder.info
    line: info = self._info()
    locals:
      info = <not found>
      self = <local> <tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr.LibrispeechASR object at 0x7f16dc354160>
      self._info = <local> <bound method LibrispeechASR._info of <tfds_community.huggingface.librispeech_asr.a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871.librispeech_asr.LibrispeechASR object at 0x7f16dc354160>>
  File "/home/az/.cache/tensorflow_datasets/modules/tfds_community/huggingface/librispeech_asr/a64ac07051b81c34deab9a54594278ec23d03bbbcf10fd60b38412f466223871/librispeech_asr.py", line 105, in LibrispeechASR._info
    line: "audio": datasets.Audio(sampling_rate=16_000),
    locals:
      datasets = <global> <module 'datasets'>
      datasets.Audio = <global> !AttributeError: module 'datasets' has no attribute 'Audio'
      sampling_rate = <not found>
AttributeError: Failed to construct dataset huggingface:librispeech_asr: module 'datasets' has no attribute 'Audio'

The mock_huggingface_import function looks like it is very incomplete?

Looks related: #3712

Reproduce tfds.load("huggingface:librispeech_asr") and begin with the traceback's tensorflow_datasets/core/community/load.py, register_package.py, and generated librispeech_asr.py entry points. Trace the missing datasets.tasks and datasets.Audio attributes to determine the compatibility problem; done means the Librispeech ASR dataset loads successfully.