pimutils / pimutils/vdirsyncer

Sync attempt causes 401 auth required after 0.19.2 -> 0.19.3 update

Open
#1,138 13 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
185
Avg merge
11h 49m
Merged PRs (30d)
1

Description

After updating from 0.19.2 to 0.19.3 the server rejects the connection with 401 instead of 207 (-vdebug outputs below)

Nothing other than the update from 0.19.2 to 0.19.3 was changed. Same Python, same configuration, same server setup. Downgrading to 0.19.2 again fixes the issue and the sync works again

Server: Radicale 3.2.3 behind Apache 2.4.62 (mod_proxy)
OS: Gentoo Linux (Server and Client)
Python: 3.12

Hardcoded password instead of password.fetch was also tried without success.

-vdebug output of 0.19.3 (FAIL)

debug: Fetching value for password.fetch with command strategy.
debug: Found cached value for ['command', '~/.config/vdirsyncer/pass.sh'].
Syncing user_contacts/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX
debug: ====================
debug: PROPFIND https://www.domain.xxx/radicale/user/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/
debug: {'User-Agent': 'vdirsyncer/0.19.3', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n            <propfind xmlns="DAV:">\n                <prop>\n                    <resourcetype/>\n                    <getcontenttype/>\n                    <getetag/>\n                </prop>\n            </propfind>\n            '
debug: Sending request...
Syncing user_calendar/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX
debug: ====================
debug: PROPFIND https://www.domain.xxx/radicale/user/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/
debug: {'User-Agent': 'vdirsyncer/0.19.3', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n            <propfind xmlns="DAV:">\n                <prop>\n                    <resourcetype/>\n                    <getcontenttype/>\n                    <getetag/>\n                </prop>\n            </propfind>\n            '
debug: Sending request...
Syncing user_calendar/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX
debug: ====================
debug: PROPFIND https://www.domain.xxx/radicale/user/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/
debug: {'User-Agent': 'vdirsyncer/0.19.3', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n            <propfind xmlns="DAV:">\n                <prop>\n                    <resourcetype/>\n                    <getcontenttype/>\n                    <getetag/>\n                </prop>\n            </propfind>\n            '
debug: Sending request...
debug: 401
debug: <CIMultiDictProxy('Date': 'Thu, 12 Sep 2024 17:44:17 GMT', 'Server': 'Apache', 'WWW-Authenticate': 'Basic realm="Radicale - Password Required"', 'Content-Length': '448', 'Content-Type': 'text/html; charset=iso-8859-1')>
debug: <StreamReader 448 bytes eof>
error: Unknown error occurred for user_contacts/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX: 401, message='Unauthorized', url='https://www.domain.xxx/radicale/joker/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/'
error: Use `-vdebug` to see the full traceback.
debug:   File "/usr/lib/python3.12/site-packages/vdirsyncer/cli/tasks.py", line 74, in sync_collection
debug:     await sync.sync(
debug:   File "/usr/lib/python3.12/site-packages/vdirsyncer/sync/__init__.py", line 150, in sync
debug:     b_nonempty = await b_info.prepare_new_status()
debug:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
debug:   File "/usr/lib/python3.12/site-packages/vdirsyncer/sync/__init__.py", line 55, in prepare_new_status
debug:     async for href, etag in self.storage.list():  # type: ignore[attr-defined]
debug:   File "/usr/lib/python3.12/site-packages/vdirsyncer/storage/dav.py", line 672, in list
debug:     response = await self.session.request(
debug:                ^^^^^^^^^^^^^^^^^^^^^^^^^^^
debug:   File "/usr/lib/python3.12/site-packages/vdirsyncer/storage/dav.py", line 413, in request
debug:     return await http.request(method, url, session=session, **more)
debug:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
debug:   File "/usr/lib/python3.12/site-packages/vdirsyncer/http.py", line 216, in request
debug:     response.raise_for_status()
debug:   File "/usr/lib/python3.12/site-packages/aiohttp/client_reqrep.py", line 1093, in raise_for_status
debug:     raise ClientResponseError(
debug: 401
debug: <CIMultiDictProxy('Date': 'Thu, 12 Sep 2024 17:44:17 GMT', 'Server': 'Apache', 'WWW-Authenticate': 'Basic realm="Radicale - Password Required"', 'Content-Length': '448', 'Content-Type': 'text/html; charset=iso-8859-1')>
debug: <StreamReader 448 bytes eof>
error: Unknown error occurred for user_calendar/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX: 401, message='Unauthorized', url='https://www.domain.xxx/radicale/joker/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/'
error: Use `-vdebug` to see the full traceback.

-vdebug output of 0.19.2 (OK)

debug: Fetching value for password.fetch with command strategy.
debug: Found cached value for ['command', '~/.config/vdirsyncer/pass.sh'].
Syncing user_contacts/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX
debug: ====================
debug: PROPFIND https://www.domain.xxx/radicale/user/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/
debug: {'User-Agent': 'vdirsyncer/0.19.2', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n            <propfind xmlns="DAV:">\n                <prop>\n                    <resourcetype/>\n                    <getcontenttype/>\n                    <getetag/>\n                </prop>\n            </propfind>\n            '
debug: Sending request...
Syncing user_calendar/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX
debug: ====================
debug: PROPFIND https://www.domain.xxx/radicale/user/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/
debug: {'User-Agent': 'vdirsyncer/0.19.2', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n            <propfind xmlns="DAV:">\n                <prop>\n                    <resourcetype/>\n                    <getcontenttype/>\n                    <getetag/>\n                </prop>\n            </propfind>\n            '
debug: Sending request...
Syncing user_calendar/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX
debug: ====================
debug: PROPFIND https://www.domain.xxx/radicale/user/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/
debug: {'User-Agent': 'vdirsyncer/0.19.2', 'Content-Type': 'application/xml; charset=UTF-8', 'Depth': '1'}
debug: b'<?xml version="1.0" encoding="utf-8" ?>\n            <propfind xmlns="DAV:">\n                <prop>\n                    <resourcetype/>\n                    <getcontenttype/>\n                    <getetag/>\n                </prop>\n            </propfind>\n            '
debug: Sending request...
debug: 207
debug: <CIMultiDictProxy('Date': 'Thu, 12 Sep 2024 17:43:26 GMT', 'Server': 'WSGIServer/0.2 CPython/3.12.3', 'DAV': '1, 2, 3, calendar-access, addressbook, extended-mkcol', 'Content-Type': 'text/xml; charset=utf-8', 'Content-Encoding': 'gzip', 'Content-Length': '1132')>
debug: <StreamReader>
debug: Already normalized: '/radicale/user/XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX/'

config

[general]
status_path = "~/.local/share/vdirsyncer/"

# ---[ CardDav contacts sync ]--------------------------------------------------------
[pair user_contacts]
a = "user_contacts_local"
b = "user_contacts_remote"
collections = ["XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX"]
metadata = ["displayname"]

[storage user_contacts_local]
type = "filesystem"
path = "~/.contacts/"
fileext = ".vcf"

[storage user_contacts_remote]
type = "carddav"
url = "https://www.domain.xxx/radicale/user/"
username = "user"
password.fetch = ["command", "~/.config/vdirsyncer/pass.sh"]
# -------------------------------------------------------------------------------------

# ---[ CalDav calendar sync ]---------------------------------------------------------
[pair user_calendar]
a = "user_calendar_local"
b = "user_calendar_remote"
collections = ["XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX", "XXXXXXXXXXXXX-XXXXXXXXXXXXXX-XXXXXXXXXXXXXX"]
metadata = ["displayname", "color"]

[storage user_calendar_local]
type = "filesystem"
path = "~/.calendars/"
fileext = ".ics"

[storage user_calendar_remote]
type = "caldav"
url = "https://www.domain.xxx/radicale/user/"
username = "user"
password.fetch = ["command", "~/.config/vdirsyncer/pass.sh"]
# -------------------------------------------------------------------------------------

Apache vhost

RewriteEngine On
RewriteRule ^/radicale$ /radicale/ [R,L]

<Location "/radicale/">
    <IfModule security2_module>
        SecAuditEngine Off
        SecRuleEngine Off
    </IfModule>

    AuthType     Basic
    AuthName     "Radicale - Password Required"
    AuthUserFile "/etc/radicale/users"
    Require      valid-user

    ProxyPass        http://localhost:5232/ retry=0
    ProxyPassReverse http://localhost:5232/
    RequestHeader    set X-Script-Name /radicale
    RequestHeader    set X-Remote-User expr=%{REMOTE_USER}
</Location>

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by comparing the HTTP request and authentication handling around vdirsyncer/storage/dav.py:672 and vdirsyncer/http.py:216 between 0.19.2 and 0.19.3. Reproduce the sync against the described Radicale 3.2.3 and Apache setup, then verify that the 0.19.3 path authenticates successfully and returns the expected 207 response.

Written by the indexing model from the issue text.

Assessment

Tech stack
apache, python
Domain
backend, cli
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.