pimutils / pimutils/vdirsyncer
An "unsafe character" (+) seems to be derailing the sync process when syncing with Google.
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.9k
- Forks
- 185
- Avg merge
- 11h 49m
- Merged PRs (30d)
- 1
Description
Before you submit bug reports: https://vdirsyncer.pimutils.org/en/stable/contributing.html
Things to include in your bugreport:
- Your vdirsyncer version
0.16.8 - Your Python version
Python 3.7.9 - Your operating system
alpine docker container on ubuntu - Your config file
[general]
status_path = "/calendardata/.vdirsyncer/status/"
[pair calendar_test]
a = "my_calendar_local"
b = "my_calendar_remote"
collections = ["from a", "from b"]
[storage my_calendar_local]
type = "filesystem"
path = "/calendardata/test_calendar/"
fileext= ".ics"
[storage my_calendar_remote]
type = "google_calendar"
token_file = "/calendardata/.google_auth/"
client_id = "supersecretclientid"
client_secret = "supersercretsecret"
- Use
vdirsyncer -vdebugfor debug output. The output is sensitive, but
please attach at least the last few lines before the error (if applicable),
censored as necessary. This is almost always the most useful information.
There was a debugging note right before the error regarding an "unsafe character" which is the + symbol translated into %2b. I included snippets of the debug output surrounding the offending character. Ellipses show gaps (that I would think are fairly irrelevant) cut out.
debug: Using 1 maximal workers.
Syncing calendar_test/calendar_name
debug: PROPFIND https://apidata.googleusercontent.com/caldav/v2/calendar_name/events/
debug: {'User-Agent': 'randomnumbers.apps.googleusercontent.com', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n <D:propfind xmlns:D="DAV:">\n <D:prop>\n <D:resourcetype/>\n <D:getcontenttype/>\n <D:getetag/>\n </D:prop>\n </D:propfind>\n '
debug: Sending request...
debug: 207
debug: {'Date': 'Mon, 01 Feb 2021 23:05:42 GMT', 'Pragma': 'no-cache', 'Content-Type': 'text/xml; charset=UTF-8', 'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'Vary': 'Origin, X-Origin, Referer', 'Content-Encoding': 'gzip', 'Server': 'ESF', 'X-XSS-Protection': '0', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Alt-Svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'}
...
debug: b'<?xml version="1.0" encoding="UTF-8"?>\n<D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">\n ... ... <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0686402-108582152.ics</D:href>\n <D:propstat>\n <D:status>HTTP/1.1 200 OK</D:status>\n <D:prop>\n <D:getcontenttype>text/calendar; component=vevent</D:getcontenttype>\n <D:getetag>"63518914751"</D:getetag>\n <D:resourcetype/>\n </D:prop>\n </D:propstat>\n </D:response>\n ... ...D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0680272-245422177.ics</D:href>\n <D:propstat>\n <D:status>HTTP/1.1 200 OK</D:status>\n <D:prop>\n <D:getcontenttype>text/calendar; component=vevent</D:getcontenttype>\n <D:getetag>"63531356273"</D:getetag>\n <D:resourcetype/>\n </D:prop>\n </D:propstat>\n </D:response>\n ... ... /D:multistatus>\n'
...
debug: Normalized URL from '/caldav/v2/calendar_name/events/m6icp53ietetp7l1gqe15540lk%40google.com.ics' to '/caldav/v2/calendar_name/events/m6icp53ietetp7l1gqe15540lk@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/6pftiq1k4nm3dt9866thuo3br0%40google.com.ics' to '/caldav/v2/calendar_name/events/6pftiq1k4nm3dt9866thuo3br0@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0686402-108582152.ics' to '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/248h42p0kesa2tvhsrj4c04n3k%40google.com.ics' to '/caldav/v2/calendar_name/events/248h42p0kesa2tvhsrj4c04n3k@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/6pj3gcj36dgm4b9n6phm2b9k74s3ebb275im6bb46hj68chh64s66p9ic8%40google.com.ics' to '/caldav/v2/calendar_name/events/6pj3gcj36dgm4b9n6phm2b9k74s3ebb275im6bb46hj68chh64s66p9ic8@google.com.ics'
...
debug: Normalized URL from '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E029A129CFACD0017700001706C698DFBD5A36141BDE96A1B1058C02B.ics' to '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E029A129CFACD0017700001706C698DFBD5A36141BDE96A1B1058C02B.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/tooubg46aa4rfimr0up83370h8%40google.com.ics' to '/caldav/v2/calendar_name/events/tooubg46aa4rfimr0up83370h8@google.com.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/WEBEX-MEETING+CENTER-6.0680272-245422177.ics' to '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E07E35C50B92D4017700001708697308B07EEDA4F95AE7E466CF804AD.ics' to '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E008700E07E35C50B92D4017700001708697308B07EEDA4F95AE7E466CF804AD.ics'
debug: Normalized URL from '/caldav/v2/calendar_name/events/cdh3ae3464p3abb66hhj4b9k64p30b9pc9j3cb9h6so66d35c9i38p9kco%40google.com.ics' to '/caldav/v2/calendar_name/events/cdh3ae3464p3abb66hhj4b9k64p30b9pc9j3cb9h6so66d35c9i38p9kco@google.com.ics'
...
debug: Already normalized: '/caldav/v2/calendar_name/events/0478200E00074C5B7101A82E0087001013269558E0D201770000170B92F3FD560EB8F48B07E8F4C3FCA120E.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/apaqr9f9rpo4pppefe9h9cp1ks@google.com.ics'
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/ik7bg09ub8cjs2p1i3nkdk3114@google.com.ics'
...
debug: Already normalized: '/caldav/v2/calendar_name/events/9s46nk3pb4nho69g592b5fd65g@google.com.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/3ffviaqu42ev1e2lj1proijedc@google.com.ics'
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/5608r6i8nq19msdp1anr100s3v@google.com.ics'
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n <C:calendar-multiget xmlns:D="DAV:"\n xmlns:C="urn:ietf:params:xml:ns:caldav">\n <D:prop>\n <D:getetag/>\n <C:calendar-data/>\n </D:prop>\n <D:href>/caldav/v2/calendar_name/events/or57be33sferu9juai0vn13u20@google.com.ics</D:href>... ...\n<D:href>/caldav/v2/calendar_name/events/9s46nk3pb4nho69g592b5fd65g@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/3ffviaqu42ev1e2lj1proijedc@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/5608r6i8nq19msdp1anr100s3v@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/0aln4lncn4oml6n60ubesepo24@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/b4i7bp6ialrv59cv1cga2mgqao@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/02nmc16u3nvrbcu7drajeut5a9@google.com.ics</D:href>\n<D:href>/caldav/v2/calendar_name/events/C38E9895-D6BA-4C30-9FFF-270FF248EA7A.ics</D:href> ... ... </C:calendar-multiget>'
...
debug: Sending request...
debug: 207
debug: {'Cache-Control': 'no-cache, no-store, max-age=0, must-revalidate', 'Content-Type': 'text/xml; charset=UTF-8', 'Date': 'Mon, 01 Feb 2021 23:06:00 GMT', 'Expires': 'Mon, 01 Jan 1990 00:00:00 GMT', 'Pragma': 'no-cache', 'Vary': 'Origin, X-Origin, Referer', 'Content-Encoding': 'gzip', 'Server': 'ESF', 'X-XSS-Protection': '0', 'X-Frame-Options': 'SAMEORIGIN', 'X-Content-Type-Options': 'nosniff', 'Alt-Svc': 'h3-29=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'Transfer-Encoding': 'chunked'}
...
debug: b'<?xml version="1.0" encoding="UTF-8"?>\n<D:multistatus xmlns:D="DAV:" xmlns:caldav="urn:ietf:params:xml:ns:caldav" xmlns:cs="http://calendarserver.org/ns/" xmlns:ical="http://apple.com/ns/ical/">\n ... ...<D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics</D:href>\n <D:status>HTTP/1.1 404 Not Found</D:status>\n </D:response>\n <D:response xmlns:carddav="urn:ietf:params:xml:ns:carddav" xmlns:cm="http://cal.me.com/_namespace/" xmlns:md="urn:mobileme:davservices">\n <D:href>/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics</D:href>\n <D:status>HTTP/1.1 404 Not Found</D:status>\n </D:response>\n</D:multistatus>\n'
...
debug: Already normalized: '/caldav/v2/calendar_name/events/uec002tktnj5if16b4lvb7tsa8@google.com.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/tlp35ob9j4rl3fi8l2o4vjbp0g@google.com.ics'
debug: Already normalized: '/caldav/v2/calendar_name/events/ts4p2oqivvt4i2abj7qhsanhuk@google.com.ics'
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics'
debug: Skipping '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics', properties are missing.
debug: Unsafe character: '%2B'
debug: Already normalized: '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics'
debug: Skipping '/caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0680272-245422177.ics', properties are missing.
error: Unknown error occurred for calendar_test/calendar_name: /caldav/v2/calendar_name/events/WEBEX-MEETING%2BCENTER-6.0686402-108582152.ics
error: Use `-vdebug` to see the full traceback.
debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/cli/tasks.py", line 67, in sync_collection
debug: partial_sync=pair.partial_sync
debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/sync/__init__.py", line 135, in sync
debug: b_nonempty = b_info.prepare_new_status()
debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/sync/__init__.py", line 58, in prepare_new_status
debug: if prefetch else ()):
debug: File "/usr/local/lib/python3.7/site-packages/vdirsyncer/storage/dav.py", line 503, in get_multi
debug: raise exceptions.NotFoundError(href)
error: 1 out of 12 tasks failed.
~
Reading the output, it seems the href cant be found. Is that because the href is using the encoded character? Any ideas on how to resolve the issue?
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with storage/dav.py, especially get_multi, and trace how sync/init.py prepares status and handles the encoded href. Reproduce the failure using a Google Calendar event whose href contains '+', then inspect the URL normalization and 404 handling shown in the traceback. Done means syncing those events no longer fails with NotFoundError while preserving normal href handling.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100