fsspec / fsspec/s3fs

botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

Open
#754 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1k
Forks
305
Avg merge
22h 37m
Merged PRs (30d)
4

Description

I have the following code that counts words but I am getting an error on this line:
fhand = s3.open('lithops-data-yey/notsobigtextfile_small.txt')
I am not sure why it says forbidden and don't know where to start in solving it.

The error is this:
Exception has occurred: PermissionError
Forbidden
botocore.exceptions.ClientError: An error occurred (403) when calling the HeadObject operation: Forbidden

The above exception was the direct cause of the following exception:

File "/Users/oanamoc/Desktop/Work/lithops/task1 copy.py", line 20, in
fhand = s3.open('lithops-data-yey/notsobigtextfile_small.txt')
PermissionError: Forbidden

My code is here:

from lithops.multiprocessing import Pool
import time
import s3fs

s3 = s3fs.S3FileSystem(anon=True)

def count_words(lines):
count = 0
for line in lines:
line = line.rstrip()
pieces = line.split()
for initial_word in pieces:
word = ''.join(c for c in initial_word if c.isalpha())
if word: # Skip empty words
count = count + 1
return count

if name == 'main':
fhand = s3.open('lithops-data-yey/notsobigtextfile_small.txt')

start = time.perf_counter()

word_number = 0
lines_per_chunk = 1000  # Number of lines to process per chunk

with Pool() as p:
    chunks = []
    while True:
        chunk = []
        for _ in range(lines_per_chunk):
            line = fhand.readline()
            if not line:
                break
            chunk.append(line)
        if not chunk:
            break
        chunks.append(chunk)

    line_numbers = p.map(count_words, chunks)

word_number = sum(line_numbers)

print('Word Count:', word_number)

finish = time.perf_counter()

print(f'Finished in {round(finish - start, 2)} second(s)')

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the failing s3.open call and the S3FileSystem(anon=True) setup in the provided Python script. Check the referenced bucket and object access configuration and compare the behavior with the s3fs documentation; done would require a reproducible, project-level cause and a confirmed fix for the 403 response.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
15/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.