apache / apache/opendal

iCloud write behavior success random

Open
#4,073 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
5.4k
Forks
825
Avg merge
1d 14m
Merged PRs (30d)
127

Description

I did some control tests, divided into three groups, the first is the combination integration, the second is a single test control, the third is a single test cycle.
1. combination integration
```
failures:
behavior::test_read_full
behavior::test_read_with_invalid_seek
behavior::test_stat_nested_parent_dir
behavior::test_stat_file
behavior::test_stat_with_special_chars
behavior::test_stat_not_cleaned_path
behavior::test_blocking_create_dir
behavior::test_blocking_create_dir_existing
behavior::test_blocking_stat_file
behavior::test_blocking_write_file
behavior::test_blocking_read_full
behavior::test_blocking_stat_dir
behavior::test_blocking_stat_with_special_chars
behavior::test_blocking_write_with_special_chars

test result: FAILED. 51 passed; 14 failed; 0 ignored; 0 measured; 0 filtered out; finished in 1058.02s
```
2. take one write close timeout. example : `behavior::test_stat_with_special_chars` and `behavior::test_stat_not_cleaned_path`
```
running 1 test
fold_name:Downloads/opendal/ | folder_id:FOLDER::com.apple.CloudDocs::E6020E4E-B7E5-4EC9-BE6D-B6F1F1A76B41 | flod_id:FOLDER::com.apple.CloudDocs::E6020E4E-B7E5-4EC9-BE6D-B6F1F1A76B41
name:0344405b-3cfe-4d64-929c-96ae1ed6e8f3 !@#$%^&()_+-=;',.txt | size:3861011 | id:DE1D1828-79ED-487D-9589-0DC0F4D8166E
get_upload_url path:0344405b-3cfe-4d64-929c-96ae1ed6e8f3 !@#$%^&()_+-=;',.txt | contenws size:3861011 | docwsid:E6020E4E-B7E5-4EC9-BE6D-B6F1F1A76B41
test behavior::test_stat_with_special_chars ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 64 filtered out; finished in 30.41s

running 1 test
fold_name:Downloads/opendal/ | folder_id:FOLDER::com.apple.CloudDocs::E6020E4E-B7E5-4EC9-BE6D-B6F1F1A76B41 | flod_id:FOLDER::com.apple.CloudDocs::E6020E4E-B7E5-4EC9-BE6D-B6F1F1A76B41
name:cc266ad4-275d-4ef4-9450-20f0506ab859 | size:2366360 | id:EC6E454E-9821-4BC0-9971-C8A6FAEEA294
get_upload_url path:cc266ad4-275d-4ef4-9450-20f0506ab859 | contenws size:2366360 | docwsid:E6020E4E-B7E5-4EC9-BE6D-B6F1F1A76B41
test behavior::test_stat_not_cleaned_path ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 64 filtered out; finished in 23.18s
```
3. take one write close timeout.example : `behavior::test_stat_nested_parent_dir`.cycle.
```
running 1 test
fold_name:Downloads/opendal/e825c6ba-bba2-4168-b78d-59d3c00a9618/ | folder_id:FOLDER::com.apple.CloudDocs::7D633672-552D-42F8-9E94-31B65CB9EFE6 | flod_id:FOLDER::com.apple.CloudDocs::7D633672-552D-42F8-9E94-31B65CB9EFE6
name:2e38623e-e0c1-4275-82fb-ae147011d7dd | size:415617 | id:0D6AC306-F3B2-493A-859C-2365D10EB04A
get_upload_url path:2e38623e-e0c1-4275-82fb-ae147011d7dd | contenws size:415617 | docwsid:7D633672-552D-42F8-9E94-31B65CB9EFE6
test behavior::test_stat_nested_parent_dir ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 measured; 64 filtered out; finished in 17.48s

running 1 test
fold_name:Downloads/opendal/ddb80d80-82c6-4895-943a-ea20f9ab9c9d/ | folder_id:FOLDER::com.apple.CloudDocs::B0887CCF-9457-45B8-A598-C385414EB59F | flod_id:FOLDER::com.apple.CloudDocs::B0887CCF-9457-45B8-A598-C385414EB59F
name:c67b26b8-f256-48c0-b3b0-6b7bf545143b | size:3328180 | id:9D0A2C6C-0A87-423B-BED6-0883E2FB6305
2024-01-25T08:20:01.342297Z ERROR opendal::services: service=icloud operation=Writer::close path=ddb80d80-82c6-4895-943a-ea20f9ab9c9d/c67b26b8-f256-48c0-b3b0-6b7bf545143b written=3328180B -> data close failed: Unexpected (permanent) at Writer::close
Context:
uri: https://cws.icloud-content.com.cn:443/8084835989/singleFileUpload?tk=CAogdaPwUVodhNgk-OgEtYPhPPfh0o8YKWY795ZFcgXeCYwSbBCc6sn90zEYnNr8z9QxIgEKWgTucAASYLSRywFqJtUcl7VAfAu9lraM46G-A2N_7DwYrJEE2YAAFCBtHcwBRsaCdkZScib66UcoYLKAZhNOmhXEx4VkI_5nZ0xgo6ZOmrOqMeINt-bZ-hgcfA&ref=017e2eadd450dd676c48a7bcff3ca07ba0ee700012&c=com.apple.clouddocs&z=com.apple.CloudDocs&uuid=740c8846-22db-467b-a75e-e69a02ea2a12&dataclass=com.apple.Dataclass.CloudKit&p=219&e=AlvGt9bUbcTOlFxX50-SJU6jwRZ-Jr4ruXkwSwxGxFbwc9_dEd-QEiYHykyOb_yjGw
response: Parts { status: 408, version: HTTP/1.1, headers: {"connection": "close", "content-length": "0", "via": "icloudedge-china:wu31p00ic-hykt01030801:7401:2403RC93:xawuh3", "x-apple-request-uuid": "5a66ee98-9663-4446-9571-b3b38639cd0d", "x-apple-edge-response-time": "120000"} }
service: icloud
path: ddb80d80-82c6-4895-943a-ea20f9ab9c9d/c67b26b8-f256-48c0-b3b0-6b7bf545143b

failures:
behavior::test_stat_nested_parent_dir

test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 64 filtered out; finished in 138.52s
```
I think it is not opendal problem.Do we only need to ensure that `Read` is stable?

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the listed behavior tests, especially behavior::test_stat_nested_parent_dir, and inspect the iCloud Writer::close failure and HTTP 408 response. Compare the passing single-test runs with the combined integration run. Done means the source of the intermittent write failures is identified and the expected Read or write behavior is documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend, cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.