Too many resources error

• Your vdirsyncer version: vdirsyncer, version 0.16.7
• If applicable, which server software (and which version) you're using: Apple Calendar Server
• Your Python version: Python 3.6.5
• Your operating system: MacOS 10.13.6

Hi there,

I am trying to utilize vdirsyncer to one-way sync some calendars from Apple Calendar Server into a singlefile ics. Unfortunately, the server has an internal limit, it cannot export more then 5k items per request. When trying to export > 5k items this leads to a 403 error:

debug: {'Date': 'Fri, 27 Jul 2018 08:16:45 GMT', 'Server': 'Twisted/15.4.0 TwistedWeb/9.0.0', 'Content-Length': '220', 'Strict-Transport-Security': 'max-age=604800', 'Content-Type': 'text/xml', 'Keep-Alive': 'timeout=5, max=99', 'Connection': 'Keep-Alive'}
debug: b"<?xml version='1.0' encoding='UTF-8'?>\n<error xmlns='DAV:'>\r\n  <number-of-matches-within-limits/>\r\n  <error-description xmlns='http://twistedmatrix.com/xml_namespace/dav/'>Too many resources</error-description>\r\n</error>"
error: Unknown error occured for username/calendar: 403 Client Error: Forbidden for url: https://server.example/calendars/__uids__/461533AC-B6F4-43A9-B2DE-CF019C2DB5C7/calendar/
error: Use `-vdebug` to see the full traceback.
debug:   File "/Users/jan/.venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/cli/tasks.py", line 69, in sync_collection
debug:     partial_sync=pair.partial_sync
debug:   File "/Users/jan/.venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/sync/__init__.py", line 136, in sync
debug:     b_nonempty = b_info.prepare_new_status()
debug:   File "/Users/jan/.venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/sync/__init__.py", line 59, in prepare_new_status
debug:     if prefetch else ()):
debug:   File "/Users/jan/.venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/storage/dav.py", line 475, in get_multi
debug:     headers=self.session.get_default_headers()
debug:   File "/Users/jan/.venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/storage/dav.py", line 390, in request
debug:     return http.request(method, url, session=self._session, **more)
debug:   File "/Users/jan/.venvs/vdirsyncer/lib/python3.6/site-packages/vdirsyncer/http.py", line 163, in request
debug:     r.raise_for_status()
debug:   File "/Users/jan/.venvs/vdirsyncer/lib/python3.6/site-packages/requests/models.py", line 939, in raise_for_status
debug:     raise HTTPError(http_error_msg, response=self)

Is there any chance to limit the amount of items that get requested per run?

Thank you & best regards,
Jan

Start by tracing storage/dav.py, especially get_multi and request, from the sync flow in sync/init.py. Check how the CalDAV response is fetched and whether existing tests cover large collections. Done means a sync against a server that rejects more than 5,000 items can complete with requests bounded to a supported amount.