fsspec_wrapper.core.AzureBlobFileSystem(...).exists("x") raises Value Error "name length is not within the permissible limits"

Open
#362 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Assessment

Difficulty
3/5
Estimated time
1-2 days
Newbie friendliness
38/100
Issue type
Bug
Clarity
Needs clarification
Activity status
Stale
Tech stack
azure, python
Domain
cloud

Research direction

Reproduce the abfss case with Azure Synapse and exists("x") first. Then read adlfs/spec.py around _exists and _container_exists, along with the fsspec_wrapper/core.py hook shown in the traceback. Done means a missing short path returns False instead of raising the resource-name ValueError.

Written by the indexing model from the issue text.

Description

3.8.10 | packaged by conda-forge | (default, May 11 2021, 07:01:05) \n[GCC 9.3.0]
Name: fsspec Version: 2021.6.1

  1. Open Azure Synapse,
  2. Create python notebook
  3. Execute the following code cell below
  4. Expect result to be False
  5. instead ValueError is raised
import fsspec
filesystem = fsspec.filesystem(
    protocol    = f"abfss",
    account_url = f"abfss://{STORAGE_ACCOUNT}.dfs.core.windows.net/"
)
filesystem.exists("x")
---------------------------------------------------------------------------
HttpResponseError                         Traceback (most recent call last)
~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/storage/blob/aio/_container_client_async.py in get_container_properties(self, **kwargs)
    361         try:
--> 362             response = await self._client.container.get_properties(
    363                 timeout=timeout,

~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/storage/blob/_generated/aio/operations/_container_operations.py in get_properties(self, timeout, request_id_parameter, lease_access_conditions, **kwargs)
    208             error = self._deserialize(_models.StorageError, response)
--> 209             raise HttpResponseError(response=response, model=error)
    210 

HttpResponseError: Operation returned an invalid status 'The specified resource name length is not within the permissible limits.'
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>OutOfRangeInput</Code><Message>The specified resource name length is not within the permissible limits.
RequestId:9feb8000-101e-0032-3c1b-e85f50000000
Time:2022-10-25T02:40:41.9122824Z</Message></Error>

During handling of the above exception, another exception occurred:

HttpResponseError                         Traceback (most recent call last)
~/cluster-env/clonedenv/lib/python3.8/site-packages/adlfs/spec.py in _container_exists(self, container_name)
   1084             ) as client:
-> 1085                 await client.get_container_properties()
   1086         except ResourceNotFoundError:

~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/core/tracing/decorator_async.py in wrapper_use_tracer(*args, **kwargs)
     73             if span_impl_type is None:
---> 74                 return await func(*args, **kwargs)
     75 

~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/storage/blob/aio/_container_client_async.py in get_container_properties(self, **kwargs)
    367         except HttpResponseError as error:
--> 368             process_storage_error(error)
    369         response.name = self.container_name

~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/storage/blob/_shared/response_handlers.py in process_storage_error(storage_error)
    149     error.additional_info = additional_data
--> 150     error.raise_with_traceback()
    151 

~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/core/exceptions.py in raise_with_traceback(self)
    246         try:
--> 247             raise super(AzureError, self).with_traceback(self.exc_traceback)
    248         except AttributeError:

~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/storage/blob/aio/_container_client_async.py in get_container_properties(self, **kwargs)
    361         try:
--> 362             response = await self._client.container.get_properties(
    363                 timeout=timeout,

~/cluster-env/clonedenv/lib/python3.8/site-packages/azure/storage/blob/_generated/aio/operations/_container_operations.py in get_properties(self, timeout, request_id_parameter, lease_access_conditions, **kwargs)
    208             error = self._deserialize(_models.StorageError, response)
--> 209             raise HttpResponseError(response=response, model=error)
    210 

HttpResponseError: The specified resource name length is not within the permissible limits.
RequestId:9feb8000-101e-0032-3c1b-e85f50000000
Time:2022-10-25T02:40:41.9122824Z
ErrorCode:OutOfRangeInput
Error:None
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>OutOfRangeInput</Code><Message>The specified resource name length is not within the permissible limits.
RequestId:9feb8000-101e-0032-3c1b-e85f50000000
Time:2022-10-25T02:40:41.9122824Z</Message></Error>

The above exception was the direct cause of the following exception:

ValueError                                Traceback (most recent call last)
<ipython-input-38-375a1a5b> in <module>
----> 1 filesystem.exists("d")

~/cluster-env/clonedenv/lib/python3.8/site-packages/fsspec_wrapper/core.py in hooked(*args, **kwargs)
     86                     super().do_connect()
     87             def hooked(*args, **kwargs):
---> 88                 return orig_attr(*args, **kwargs)
     89             return hooked
     90         else:

~/cluster-env/clonedenv/lib/python3.8/site-packages/adlfs/spec.py in exists(self, path)
   1290 
   1291     def exists(self, path):
-> 1292         return sync(self.loop, self._exists, path)
   1293 
   1294     async def _exists(self, path):

~/cluster-env/clonedenv/lib/python3.8/site-packages/fsspec/asyn.py in sync(loop, func, timeout, *args, **kwargs)
     66         raise FSTimeoutError
     67     if isinstance(result[0], BaseException):
---> 68         raise result[0]
     69     return result[0]
     70 

~/cluster-env/clonedenv/lib/python3.8/site-packages/fsspec/asyn.py in _runner(event, coro, result, timeout)
     22         coro = asyncio.wait_for(coro, timeout=timeout)
     23     try:
---> 24         result[0] = await coro
     25     except Exception as ex:
     26         result[0] = ex

~/cluster-env/clonedenv/lib/python3.8/site-packages/adlfs/spec.py in _exists(self, path)
   1306         if not path:
   1307             if container_name:
-> 1308                 return await self._container_exists(container_name)
   1309             else:
   1310                 # Empty paths exist by definition

~/cluster-env/clonedenv/lib/python3.8/site-packages/adlfs/spec.py in _container_exists(self, container_name)
   1087             return False
   1088         except Exception as e:
-> 1089             raise ValueError(
   1090                 f"Failed to fetch container properties for {container_name} for {e}"
   1091             ) from e

ValueError: Failed to fetch container properties for d for The specified resource name length is not within the permissible limits.
RequestId:9feb8000-101e-0032-3c1b-e85f50000000
Time:2022-10-25T02:40:41.9122824Z
ErrorCode:OutOfRangeInput
Error:None
Content: <?xml version="1.0" encoding="utf-8"?><Error><Code>OutOfRangeInput</Code><Message>The specified resource name length is not within the permissible limits.
RequestId:9feb8000-101e-0032-3c1b-e85f50000000
Time:2022-10-25T02:40:41.9122824Z</Message></Error>
Dominant language
Python
Stars
211
Forks
120
PR merge metrics
No merged PRs in 30d

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

More from fsspec/adlfs

All issues in fsspec/adlfs

Similar issues

More Python issues

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.