py-why / py-why/EconML

License Error Installing on Databricks

Open
#1,010 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Jupyter Notebook
Stars
4.8k
Forks
827
PR merge metrics
No merged PRs in 30d

Description

I'm trying to install econml on a databricks cluster with type r8g.large and I'm getting this error. The weird thing is i installed it on a different cluster with a different instance type, m5d.xlarge, and it worked. Has anyone seen this or know what to do about it?

Library installation attempted on the driver node of cluster 1009-205745-udzc9pvg and failed. Pip could not successfully build the wheel due to missing build dependencies or errors. Please check the wheel package contents and dependencies. Error code: ERROR_WHEEL_BUILD, error message: org.apache.spark.SparkException: Process List(/bin/su, libraries, -c, bash /local_disk0/.ephemeral_nfs/cluster_libraries/python/python_start_clusterwide.sh /local_disk0/.ephemeral_nfs/cluster_libraries/python/bin/pip install 'econml==0.16.0' --disable-pip-version-check) exited with code 1. error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [95 lines of output]
configuration error: project.license must be valid exactly by one definition (2 matches found):

      - keys:
          'file': {type: string}
        required: ['file']
      - keys:
          'text': {type: string}
        required: ['text']
  
  DESCRIPTION:
      `Project license <https://peps.python.org/pep-0621/#license>`_.
  
  GIVEN VALUE:
      "MIT"
  
  OFFENDING RULE: 'oneOf'
  
  DEFINITION:
      {
          "oneOf": [
              {
                  "properties": {
                      "file": {
                          "type": "string",
                          "$$description": [
                              "Relative path to the file (UTF-8) which contains the license for the",
                              "project."
                          ]
                      }
                  },
                  "required": [
                      "file"
                  ]
              },
              {
                  "properties": {
                      "text": {
                          "type": "string",
                          "$$description": [
                              "The license of the project whose meaning is that of the",
                              "`License field from the core metadata",
                              "<https://packaging.python.org/specifications/core-metadata/#license>`_."
                          ]
                      }
                  },
                  "required": [
                      "text"
                  ]
              }
          ]
      }
  Traceback (most recent call last):
    File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 332, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=[])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 302, in _get_build_requires
      self.run_setup()
    File "/usr/local/lib/python3.12/dist-packages/setuptools/build_meta.py", line 318, in run_setup
      exec(code, locals())
    File "<string>", line 37, in <module>
    File "/usr/local/lib/python3.12/dist-packages/setuptools/__init__.py", line 117, in setup
      return distutils.core.setup(**attrs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/dist-packages/setuptools/_distutils/core.py", line 158, in setup
      dist.parse_config_files()
    File "/local_disk0/.ephemeral_nfs/cluster_libraries/python/lib/python3.12/site-packages/_virtualenv.py", line 22, in parse_config_files
      result = old_parse_config_files(self, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/dist-packages/setuptools/dist.py", line 608, in parse_config_files
      pyprojecttoml.apply_configuration(self, filename, ignore_option_errors)
    File "/usr/local/lib/python3.12/dist-packages/setuptools/config/pyprojecttoml.py", line 71, in apply_configuration
      config = read_configuration(filepath, True, ignore_option_errors, dist)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/usr/local/lib/python3.12/dist-packages/setuptools/config/pyprojecttoml.py", line 136, in read_configuration
      validate(subset, filepath)
    File "/usr/local/lib/python3.12/dist-packages/setuptools/config/pyprojecttoml.py", line 60, in validate
      raise ValueError(f"{error}\n{summary}") from None
  ValueError: invalid pyproject.toml config: `project.license`.
  configuration error: `project.license` must be valid exactly by one definition (2 matches found):
  
      - keys:
          'file': {type: string}
        required: ['file']
      - keys:
          'text': {type: string}
        required: ['text']
  
  [end of output]

note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error
× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by inspecting pyproject.toml, especially the project.license metadata, and the setup.py packaging path shown in the traceback. Reproduce installation of econml==0.16.0 on the reported Python 3.12 Databricks environment; done means the wheel builds and installation succeeds without the license configuration error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.