aws / aws/sagemaker-spark

while trying to read .csv to a spark dataframe using conda_python3 kernel in sagemaker AWS. getting an error

Open
#132 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
301
Forks
129
PR merge metrics
No merged PRs in 30d

Description

Please fill out the form below.

### System Information
- **Spark or PySpark**:
- **SDK Version**:
- **Spark Version**:
- **Algorithm (e.g. KMeans)**:

### Describe the problem
while trying to read .csv to a spark dataframe using conda_python3 kernel in sagemaker AWS. getting an error. How t read a file from s3. I have tried using boto3, still getting the same error. If any changes has to be done using shell please specify the steps to do it in AWS an I am new to AWS.

### Minimal repo / logs
Py4JavaError: An error occured while calling o138.csv
: java.io.IOException: No FileSystem for scheme: s3
.
.
.
- **Exact command to reproduce**:

Contributor guide

Open the contributing guide

Research direction

No repository file, test, version, or exact command is provided. Start by reproducing the CSV read in the conda_python3 kernel on SageMaker with the reported S3 path, capturing the complete Py4JavaError and Spark configuration; done means a minimal reproducible case and a confirmed way to read the S3 CSV without the “No FileSystem for scheme: s3” error.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python, spark
Domain
cloud, data-engineering
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
18/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.