diff --git a/CHANGELOG.md b/CHANGELOG.md index dde4a28..d88cee7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -18,14 +18,19 @@ This release works with caldav 3.7.0: the tester carries the definitions of the - **`save-load.todo.status`**, **`save-load.todo.status.rfc10051`** and **`save-load.todo.priority`** - which STATUS values (RFC 5545's four; PENDING and FAILED from RFC 10051 apart) and which PRIORITY values (0 to 9) a task keeps, with the extra keys `values`, `map` and `refused`. Ox drops PENDING and FAILED and keeps priorities 1, 5 and 9 only; Zimbra stores PENDING and FAILED as NEEDS-ACTION and refuses CANCELLED. PRIORITY:0 dropped is `full`: no PRIORITY means the same. - **`supported-task-modes`** - whether the server advertises RFC 10051 with `CALDAV:supported-task-mode-set` on the calendar home or a calendar. Neither Ox nor Zimbra does, and the RFC 10051 status verdict says so. - **`save-load.icalendar.x-parameter`** - whether an X- parameter on SUMMARY is stored, in an event and then in a task. Ox and Zimbra drop it from the event. +- **`save-load.todo.alarm`** - whether a task's VALARM is stored as sent: its ACTION, its relative TRIGGER and its DESCRIPTION. Ox keeps it, but makes the TRIGGER absolute and replaces the DESCRIPTION with "Open-XChange" (`broken`, the behaviour saying what changed); it keeps an event's VALARM as sent. caldav does not define this feature yet, so the tester adds its definition itself until a caldav release has it. +- **`save-load.event.recurrences.rdate`** - whether an event's RDATE is stored. Zimbra drops every RDATE, and keeps an RRULE. caldav does not define this feature yet, so the tester adds its definition itself until a caldav release has it. - **`multiget.deleted`** - whether a calendar-multiget answers a deleted object as missing. Zimbra still returns its data, while a GET answers 404. - **`save-load.stable-url`** also stores a task under a name that is not its UID, and records `uid_ics_components: [VTODO]` where only the task is renamed `.ics` (Ox). -- **`save-load.icalendar.olson-tzid`** - whether an event with `TZID=Asia/Tokyo` and no VTIMEZONE is placed in Tokyo time, judged by time-range searches in UTC; the behaviour says whether the server adds a VTIMEZONE of its own. Where `full`, a client may leave the VTIMEZONE out. caldav does not define this feature yet, so the tester adds its definition itself until a caldav release has it. See https://github.com/python-caldav/caldav/issues/849. +- **`save-load.icalendar.olson-tzid`** - whether an event with `TZID=Asia/Tokyo` and no VTIMEZONE is placed in Tokyo time, judged by time-range searches in UTC; the behaviour says whether the server adds a VTIMEZONE of its own. Where `full`, a client may leave the VTIMEZONE out. The event is compared with the same event carrying its VTIMEZONE and with one at 12:00 UTC, so a search that misplaces zoned times or only matches start times does not stop the probe from reaching a verdict. caldav does not define this feature yet, so the tester adds its definition itself until a caldav release has it. See https://github.com/python-caldav/caldav/issues/849. +- **`search.time-range.event.overlap`** and **`search.time-range.todo.overlap`** - whether a time-range search finds an event or task that overlaps the range without starting inside it (RFC 4791 section 9.9): one range inside the object, one straddling its end. Robur only finds events whose DTSTART is in the range, Cyrus and GMX only tasks whose DTSTART is. +- **`search.time-range.event.tzid`** - whether a time-range search places an event with `TZID=Asia/Tokyo` and its VTIMEZONE at its time in UTC. Infomaniak finds it only by a range that also reaches 12:00 UTC, its local time read as UTC. caldav does not define these three features yet, so the tester adds their definitions itself until a caldav release has them. ### Fixed - A probe for a feature the installed caldav does not know no longer aborts the whole run. The check logs a warning and loses that feature only, and a check reading it gets "unknown"; a check with no other feature is not run. - **`save.duplicate-uid.cross-calendar`** - whether saving the copy moved the original is read from a listing of the original calendar, not from a search by UID, which a stale search index answers wrongly. +- **`save.if-none-match`** also puts an object with another UID under a taken name, as a new object saved there would have one: Bedework (3 and 5) refuses the same UID with 412, but another one with 500, which the library does not read as "exists". It is now `ungraceful` there. ## 1.8.0 - 2026-10-30 diff --git a/src/caldav_server_tester/checks.py b/src/caldav_server_tester/checks.py index 4319cf5..5fc8855 100644 --- a/src/caldav_server_tester/checks.py +++ b/src/caldav_server_tester/checks.py @@ -10,7 +10,7 @@ from urllib.parse import quote, unquote, urlsplit, urlunsplit from xml.etree import ElementTree from xml.sax.saxutils import escape as xml_escape -from zoneinfo import ZoneInfo +from zoneinfo import ZoneInfo, ZoneInfoNotFoundError from caldav.calendarobjectresource import Event, Journal, Todo, _quote_uid from caldav.collection import Principal @@ -4782,7 +4782,10 @@ class CheckPutEtag(_RawPut, Check): If-Match backwards: 412 on the existing object, 201 on the missing one. Zimbra compares ``*`` as a literal etag, so If-Match: * never holds and If-None-Match: * always does - it overwrites, and so does SOGo, which - ignores both. The library sends If-None-Match: * on save(no_overwrite=True) + ignores both. Bedework refuses If-None-Match: * on an existing object + with 412 for the same UID, but with 500 for another one, which is what a + new object saved under a taken name carries; so where the same UID got + 412, another UID is tried too. The library sends If-None-Match: * on save(no_overwrite=True) where save.if-none-match is full, so refusing to create is worse than overwriting: every such save of a new object would fail. """ @@ -4924,6 +4927,23 @@ def _grade_if_none_match(self, url, ical): return self._graded( "ungraceful", f"If-None-Match: * on an existing object answered {existing} rather than 412" ) + ## A new object saved under a taken name has another UID. Bedework (3 + ## and 5) refuses the same UID with 412, and another one with 500. + other = self._status(self._put(url, self._ical("csc_put_etag_other"), if_none_match="*")) + if other is not None and 200 <= other < 300: + return self._graded("unsupported", "If-None-Match: * overwrote an existing object with another UID") + if other is not None and other != 412: + return self._graded( + "ungraceful", + f"If-None-Match: * on an existing object answered {other} rather than 412, for another UID", + ) + if other is None: + ## Graded full, one dropped answer would hide Bedework's 500 + return self._graded( + "unknown", + "If-None-Match: * on an existing object got no answer for another UID, " + "so refusing it otherwise than with 412 (ungraceful) is not ruled out", + ) if absent is None: ## Graded full, the library would send it on every new object return self._graded( @@ -5614,6 +5634,82 @@ def _grade(self, raw): return self._kept(raw, "COMPLETED") +## An RFC 5545 section 3.3.6 duration, e.g. -PT15M or -P0DT0H15M0S +DURATION = re.compile(r"([+-]?)P(?:(\d+)W)?(?:(\d+)D)?(?:T(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?)?", re.IGNORECASE) + + +def _duration_seconds(text): + """The seconds an RFC 5545 duration spans, signed; None where ``text`` is none.""" + match = DURATION.fullmatch(text.strip()) + if not match or text.strip().upper().rstrip("T").endswith("P"): + return None + sign, weeks, days, hours, minutes, seconds = match.groups() + total = sum(int(n or 0) * f for n, f in zip((weeks, days, hours, minutes, seconds), (604800, 86400, 3600, 60, 1))) + return -total if sign == "-" else total + + +class CheckTodoAlarm(_RoundTrip, Check): + """ + Checks whether a task's VALARM is stored as sent: its ACTION, its + relative TRIGGER and its DESCRIPTION. Ox makes the TRIGGER absolute, + so the alarm no longer follows the task when it is moved, and puts + "Open-XChange" in the DESCRIPTION; it keeps an event's VALARM as sent + (seen by hand, not probed here). A property the server adds to the + VALARM, such as a UID, is fine, and so is the same TRIGGER written + another way: -P0DT0H15M0S, or with RELATED=START or VALUE=DURATION, + the defaults. + """ + + depends_on = {PrepareCalendar} + LABEL = "todo-alarm" + FEATURE = "save-load.todo.alarm" + features_to_be_checked = {FEATURE} + UID = "csc_alarm_task" + WHAT = "a task with a VALARM" + COMPONENTS = ("VTODO",) + SENT = {"ACTION": "DISPLAY", "TRIGGER": "-PT15M", "DESCRIPTION": "caldav-server-tester alarm"} + ## Parameters that change nothing: RFC 5545's defaults, and a language + HARMLESS = {"TRIGGER": {"RELATED=START", "VALUE=DURATION"}, "DESCRIPTION": {"LANGUAGE"}} + EXTRA = ( + "BEGIN:VALARM\r\n" + f"ACTION:{SENT['ACTION']}\r\n" + f"DESCRIPTION:{SENT['DESCRIPTION']}\r\n" + f"TRIGGER:{SENT['TRIGGER']}\r\n" + "END:VALARM\r\n" + ) + + def _kept(self, name, params, value): + harmless = self.HARMLESS.get(name, set()) + if any(p.upper() not in harmless and p.split("=")[0].upper() not in harmless for p in params.split(";") if p): + return False + if name != "TRIGGER": + return value == self.SENT[name] + ## An absolute TRIGGER is no duration: None, and so not the one sent + return _duration_seconds(value) == _duration_seconds(self.SENT[name]) + + def _grade(self, raw): + alarm = component_lines(raw, "VALARM") + if not alarm: + return self._graded("unsupported", "the VALARM is dropped") + changes = [] + for name, sent in self.SENT.items(): + stored = next( + (m for m in (re.fullmatch(rf"{name}(?:;([^:]*))?:(.*)", line, re.IGNORECASE) for line in alarm) if m), + None, + ) + params = stored and (stored[1] or "") + if stored and self._kept(name, params, stored[2]): + continue + whose = "its" if changes else "the alarm's" + if stored is None: + changes.append(f"{whose} {name} is dropped") + else: + text = f"{name};{params}:{stored[2]}" if params else stored[2] + was = f" {sent}" if name == "TRIGGER" else "" + changes.append(f"{whose} {name}{was} is stored as {text}") + return self._graded("broken", "; ".join(changes)) if changes else "full" + + class _ValueSweep(_RoundTrip): """Stores a task with each of ``VALUES`` for property ``NAME`` in turn, and records which are kept, which are stored as another, and which are @@ -5837,6 +5933,68 @@ def _grade(self, raw): return self._graded("unknown", "cannot test, the object came back without SUMMARY") +## An RDATE value's start: a DATE-TIME, or a PERIOD by its first half +RDATE_VALUE = re.compile(r"(\d{8}T\d{6})(Z?)(?:/.*)?") + + +def _rdate_instants(line): + """The occurrences an RDATE line gives, in UTC as YYYYMMDDTHHMMSSZ; a value + that is no DATE-TIME (a DATE, or a TZID this machine does not know) is + kept as written, so it compares unequal.""" + head, _, values = line.partition(":") + tzid = re.search(r";TZID=\"?([^;:\"]+)", head, re.IGNORECASE) + try: + zone = ZoneInfo(tzid[1]) if tzid else None + except (ZoneInfoNotFoundError, ValueError): + zone = None + tzid = None + out = [] + for value in values.split(","): + match = RDATE_VALUE.fullmatch(value.strip().upper()) + if not match or (not match[2] and zone is None): + out.append(value.strip().upper()) + continue + moment = datetime.strptime(match[1], "%Y%m%dT%H%M%S") + moment = moment.replace(tzinfo=utc) if match[2] else moment.replace(tzinfo=zone).astimezone(utc) + out.append(moment.strftime("%Y%m%dT%H%M%SZ")) + return out + + +class CheckRdate(_RoundTrip, Check): + """ + Checks whether an event's RDATE (RFC 5545 section 3.8.5.2), two more + occurrences besides DTSTART, is stored. The dates may come back on one + line or several, in a time zone, as periods, or with DTSTART among them. + Zimbra drops every RDATE, while it keeps an RRULE. + """ + + depends_on = {PrepareCalendar} + LABEL = "rdate" + FEATURE = "save-load.event.recurrences.rdate" + features_to_be_checked = {FEATURE} + UID = "csc_rdate" + WHAT = "an event with an RDATE" + + @property + def _sent(self): + year = self.checker.fixture_base_year + return [f"{year}0127T120000Z", f"{year}0203T120000Z"] + + @property + def EXTRA(self): + return f"RDATE:{','.join(self._sent)}\r\n" + + def _grade(self, raw): + lines = [line for line in component_lines(raw, "VEVENT") if re.match(r"RDATE[;:]", line, re.IGNORECASE)] + if not lines: + return self._graded("unsupported", "the RDATE is dropped") + stored = {instant for line in lines for instant in _rdate_instants(line)} + stored.discard(f"{self.checker.fixture_base_year}0120T120000Z") # DTSTART, which RFC 5545 lets an RDATE repeat + if stored == set(self._sent): + return "full" + return self._graded("broken", f"the RDATE {','.join(self._sent)} is stored as {'; '.join(lines)}") + + class CheckAttendeePartstat(Check): """ Checks whether an attendee's PARTSTAT can be modified via a direct PUT. @@ -6920,26 +7078,93 @@ def _run_check(self): self.set_feature("search.is-not-defined", False) -class CheckOlsonTzid(_RoundTrip, Check): - """ - Checks whether an event with a TZID from the tz database and no VTIMEZONE, - which RFC 5545 section 3.6.5 requires, is placed in that time zone. The - server's reading is judged by time-range searches in UTC: one for the - event's hour as Tokyo time, one for it as UTC, and, where neither finds - it, one for noon in every zone, telling a time read in yet another zone - from a search that lags. Whether the server adds a VTIMEZONE of its own goes in - the behaviour. See https://github.com/python-caldav/caldav/issues/849 +## A VTIMEZONE for Asia/Tokyo, which has no daylight saving time and is far +## from the zones servers tend to run in +TOKYO_VTIMEZONE = ( + "BEGIN:VTIMEZONE\r\n" + "TZID:Asia/Tokyo\r\n" + "BEGIN:STANDARD\r\n" + "TZOFFSETFROM:+0900\r\n" + "TZOFFSETTO:+0900\r\n" + "TZNAME:JST\r\n" + "DTSTART:19700101T000000\r\n" + "END:STANDARD\r\n" + "END:VTIMEZONE\r\n" +) + + +class _TimeRangeProbe: + """Time-range searches on the fixture year's 20 January, as the server + answers them: no post-filter, no workarounds. The probe objects are at + 12:00 to 13:00 there (``_ical``). A window reading where one is placed + (TOKYO, UTC and the overlap probe's) contains its DTSTART, since Robur + only finds what starts inside the range, and stays clear of its end, + since DAViCal and SOGo find an object in a range starting there. The + others are there to tell placements apart: an EARLY one finds an object + placed an hour early, MIDDLE one between Tokyo time and UTC, BOTH and + ANYWHERE span several placements. A window is a start after 00:00 UTC + and a length.""" + + ## 12:00 in Tokyo is 03:00 UTC + TOKYO = (timedelta(hours=2, minutes=45), timedelta(minutes=30)) + TOKYO_EARLY = (timedelta(hours=2, minutes=15), timedelta(minutes=30)) + UTC = (timedelta(hours=11, minutes=45), timedelta(minutes=30)) + UTC_EARLY = (timedelta(hours=11, minutes=15), timedelta(minutes=30)) + MIDDLE = (timedelta(hours=4, minutes=15), timedelta(hours=7, minutes=30)) + BOTH = (timedelta(hours=2, minutes=45), timedelta(hours=9, minutes=30)) + ## Noon anywhere from UTC-12 to UTC+14 + ANYWHERE = (timedelta(hours=-12), timedelta(days=2)) + + REFERENCE_UID = "csc_time_range_reference" + MISPLACED = "cannot test, the time-range search does not place an event at 12:00 UTC right" + + def _placed_in_utc(self, hits, windows): + """Whether ``hits``, which of ``windows`` found the reference event at + 12:00 UTC, are what a search placing it right answers. Bedework 3 + answers with other objects than the ones in the range.""" + found = dict(zip(windows, hits)) + return found[self.UTC] and not found[self.TOKYO] + + def _uids_found(self, window, todo=False): + """The UIDs a search over ``window`` finds, among the events, or the + tasks with ``todo``""" + start = datetime(self.checker.fixture_base_year, 1, 20, tzinfo=utc) + window[0] + calendar = self.checker.tasklist if todo else self.checker.calendar + kind = {"todo": True, "include_completed": True} if todo else {"event": True} + objects = calendar.search( + start=start, + end=start + window[1], + post_filter=False, + compatibility_workarounds=False, + **kind, + ) + uids = set() + for obj in objects: + try: + uids.add(str(obj.component.get("UID"))) + except Exception: + ## Another object, one the library cannot parse + pass + return uids - Asia/Tokyo has no daylight saving time and is far from the zones servers - tend to run in, where a TZID read as floating would land. The windows - take the middle of the hour: DAViCal and SOGo find an event in a range - starting at its end. - Where the bare TZID is not placed in Tokyo, the same event carrying a - VTIMEZONE is stored too, as a control: if the search misplaces that one as - well, the search is at fault, not the reading of the TZID, and nothing can - be told. Xandikos' index drops the TZID once it answers a query, so a - time lands in the server's own zone either way. +class CheckOlsonTzid(_TimeRangeProbe, _RoundTrip, Check): + """ + Checks whether an event with a TZID from the tz database and no VTIMEZONE, + which RFC 5545 section 3.6.5 requires, is placed in that time zone. See + https://github.com/python-caldav/caldav/issues/849 + + The server's reading is judged by time-range searches in UTC, comparing + the event with two more: the same one carrying its VTIMEZONE, and one at + 12:00 UTC. Found by the same searches as the first, the TZID is + understood; as the second, it is read as UTC. Comparing rather than + expecting fixed answers copes with a search placing zoned times wrong + (Infomaniak, search.time-range.event.tzid) or finding only what starts + inside the range (Robur, search.time-range.event.overlap). Where the + two cannot be told apart, nothing can: Xandikos' index drops the TZID + once it answers a query, so a time lands in the server's own zone either + way. Whether the server adds a VTIMEZONE of its own goes in the + behaviour. """ depends_on = {PrepareCalendar, CheckSearch} @@ -6950,19 +7175,16 @@ class CheckOlsonTzid(_RoundTrip, Check): WHAT = "an event with a TZID but no VTIMEZONE" TZID = "Asia/Tokyo" CONTROL_UID = "csc_olson_tzid_control" - VTIMEZONE = ( - "BEGIN:VTIMEZONE\r\n" - "TZID:Asia/Tokyo\r\n" - "BEGIN:STANDARD\r\n" - "TZOFFSETFROM:+0900\r\n" - "TZOFFSETTO:+0900\r\n" - "TZNAME:JST\r\n" - "DTSTART:19700101T000000\r\n" - "END:STANDARD\r\n" - "END:VTIMEZONE\r\n" + REFERENCE_UID = "csc_olson_tzid_reference" + VTIMEZONE = TOKYO_VTIMEZONE + WINDOWS = ( + _TimeRangeProbe.TOKYO_EARLY, + _TimeRangeProbe.TOKYO, + _TimeRangeProbe.UTC_EARLY, + _TimeRangeProbe.UTC, + _TimeRangeProbe.BOTH, + _TimeRangeProbe.ANYWHERE, ) - ## 12:00 in Tokyo is 03:00 UTC; the middle of that hour - TOKYO = timedelta(hours=3, minutes=15) def _run_check(self) -> None: if not self.feature_check_result("search.time-range.event"): @@ -6973,67 +7195,170 @@ def _run_check(self) -> None: def _body(self): return self._ical(self.uid, tzid=self.TZID) - def _found(self, start, length=timedelta(minutes=30), uid=None): - """Whether a search for ``length`` from ``start``, a UTC time on the - event's day, finds the event, or the one with ``uid``""" - start = datetime(self.checker.fixture_base_year, 1, 20, tzinfo=utc) + start - objects = self.checker.calendar.search( - start=start, - end=start + length, - event=True, - post_filter=False, - compatibility_workarounds=False, - ) - for obj in objects: - try: - if str(obj.component.get("UID")) == (uid or self.uid): - return True - except Exception: - ## Another object, one the library cannot parse - pass - return False - def _grade(self, raw): added = any(line.upper() == "BEGIN:VTIMEZONE" for line in unfolded_lines(raw)) notes = [f"the server adds a VTIMEZONE for {self.TZID}"] if added else [] + others = ( + (self.CONTROL_UID, self._ical(self.CONTROL_UID, vtimezone=self.VTIMEZONE), "with its VTIMEZONE"), + (self.REFERENCE_UID, self._ical(self.REFERENCE_UID), "at 12:00 UTC"), + ) + urls = [] try: - tokyo, as_utc = self._found(self.TOKYO), self._found(timedelta(hours=12, minutes=15)) - ## Noon anywhere from UTC-12 to UTC+14 - elsewhere = tokyo or as_utc or self._found(timedelta(hours=-12), timedelta(days=2)) - control = self._control_placed() if not tokyo and elsewhere else True - except (DAVError, OSError) as exc: - return self._graded("unknown", f"cannot test, the time-range search fails ({exc})") - if tokyo and not as_utc: + for uid, body, what in others: + urls.append(self._url(uid)) + if not self._ok(self._put(urls[-1], body)): + return self._graded("unknown", f"cannot test, the same event {what} could not be stored") + try: + found = [self._uids_found(window) for window in self.WINDOWS] + except (DAVError, OSError) as exc: + return self._graded("unknown", f"cannot test, the time-range search fails ({exc})") + finally: + for url in urls: + self._delete(url) + bare, control, reference = ( + tuple(uid in uids for uids in found) for uid in (self.uid, self.CONTROL_UID, self.REFERENCE_UID) + ) + if not any(bare): + ## May be a search index lagging behind the PUT + return self._graded("unknown", "cannot test, the time-range search does not find the event") + if not self._placed_in_utc(reference, self.WINDOWS): + return self._graded("unknown", self.MISPLACED) + if control == reference: + return self._graded( + "unknown", + "cannot test, the time-range search does not tell the event at Tokyo time, VTIMEZONE and all, " + "from one at 12:00 UTC", + ) + if bare == control: return self._graded("full", *notes) - if tokyo: + if bare == reference: + return self._graded("unsupported", f"TZID={self.TZID} without a VTIMEZONE is read as UTC", *notes) + reading = f"TZID={self.TZID} without a VTIMEZONE is read as neither Tokyo time nor UTC" + return self._graded("unsupported", reading, *notes) + + +class CheckTimeRangeTzid(_TimeRangeProbe, _RawPut, Check): + """ + Checks whether a time-range search places an event with TZID=Asia/Tokyo, + and the VTIMEZONE for it, at its time in UTC. Infomaniak finds one only + by a range also reaching its local time read as UTC, as if the search + preselected by the time with the TZID dropped and then checked it + properly. An event at 12:00 UTC goes along, and where the search does + not place that one right, nothing can be told. save-load.icalendar.olson-tzid + is the same without the VTIMEZONE. + """ + + depends_on = {PrepareCalendar, CheckSearch} + LABEL = "time-range-tzid" + FEATURE = "search.time-range.event.tzid" + features_to_be_checked = {FEATURE} + UID = "csc_time_range_tzid_event" + + def _run_check(self) -> None: + self.set_feature(self.FEATURE, self._placed()) + + def _placed(self): + if not self.feature_check_result("search.time-range.event"): + return self._graded("unknown", "cannot test, time-range search does not work") + windows = (self.TOKYO_EARLY, self.TOKYO, self.UTC_EARLY, self.UTC, self.MIDDLE, self.BOTH, self.ANYWHERE) + objects = ( + (self.UID, self._ical(self.UID, vtimezone=TOKYO_VTIMEZONE), "the event"), + (self.REFERENCE_UID, self._ical(self.REFERENCE_UID), "the same event at 12:00 UTC"), + ) + urls = [] + try: + for uid, body, what in objects: + urls.append(self._url(uid)) + if not self._ok(self._put(urls[-1], body)): + return self._graded("unknown", f"cannot test, {what} could not be stored") + try: + found = [self._uids_found(window) for window in windows] + except (DAVError, OSError) as exc: + return self._graded("unknown", f"cannot test, the time-range search fails ({exc})") + finally: + for url in urls: + self._delete(url) + if not self._placed_in_utc([self.REFERENCE_UID in uids for uids in found], windows): + return self._graded("unknown", self.MISPLACED) + tokyo_early, tokyo, utc_early, as_utc, middle, both, anywhere = (self.UID in uids for uids in found) + if tokyo and as_utc: return self._graded( "unknown", "cannot test, the time-range search finds the event both as Tokyo time and as UTC" ) - if control is None: - return self._graded("unknown", "cannot test, the same event with its VTIMEZONE could not be stored") - if control is False: + ## An hour off either way is found by the window next to the right one + if tokyo and not tokyo_early: + return "full" + if as_utc and not utc_early: + return self._graded("unsupported", "an event at 12:00 Asia/Tokyo is read as 12:00 UTC") + if both and not (tokyo or as_utc or middle): return self._graded( - "unknown", - "cannot test, the time-range search does not place the event at Tokyo time even with its VTIMEZONE", + "broken", + "an event at 12:00 Asia/Tokyo is found only by a range also reaching 12:00 UTC, " + "its local time read as UTC", ) - if as_utc: - return self._graded("unsupported", f"TZID={self.TZID} without a VTIMEZONE is read as UTC", *notes) - if elsewhere: - reading = f"TZID={self.TZID} without a VTIMEZONE is read as neither Tokyo time nor UTC" - return self._graded("unsupported", reading, *notes) - ## Not found at all may be a search index lagging behind the PUT + if anywhere: + return self._graded("unsupported", "an event at 12:00 Asia/Tokyo is read as neither Tokyo time nor UTC") return self._graded("unknown", "cannot test, the time-range search does not find the event") - def _control_placed(self): - """Whether the search places the same event, carrying its VTIMEZONE, - at Tokyo time; None where it cannot be stored""" - url = self._url(self.CONTROL_UID) + +class CheckTimeRangeOverlap(_TimeRangeProbe, _RawPut, Check): + """ + Checks whether a time-range search finds an event, and a task with + DTSTART and DUE, overlapping the range without starting inside it: a + range inside the object, and one straddling its end. RFC 4791 section + 9.9 has an object found where it overlaps the range; Robur only finds + the ones whose DTSTART is in it. Journals have no end, and are not + probed. + """ + + depends_on = {PrepareCalendar, CheckSearch} + LABEL = "time-range-overlap" + EVENT = "search.time-range.event.overlap" + TODO = "search.time-range.todo.overlap" + features_to_be_checked = {EVENT, TODO} + ## The objects are at 12:00 to 13:00 UTC + AT_START = (timedelta(hours=11, minutes=45), timedelta(minutes=30)) + INSIDE = (timedelta(hours=12, minutes=15), timedelta(minutes=30)) + AT_END = (timedelta(hours=12, minutes=45), timedelta(minutes=30)) + + def _run_check(self) -> None: + self.set_feature(self.EVENT, self._overlap(todo=False)) + self.set_feature(self.TODO, self._overlap(todo=True)) + + def _overlap(self, todo): + kind = "task" if todo else "event" + if todo and getattr(self.checker, "tasklist", None) is None: + return self._graded("unknown", "cannot test, there is no task calendar") + if not self.feature_check_result(f"search.time-range.{'todo' if todo else 'event'}"): + return self._graded("unknown", f"cannot test, time-range search for {kind}s does not work") + self.CALENDAR = "tasklist" if todo else "calendar" + uid = f"csc_time_range_overlap_{kind}" + url = self._url(uid) try: - if not self._ok(self._put(url, self._ical(self.CONTROL_UID, vtimezone=self.VTIMEZONE))): - return None - return self._found(self.TOKYO, uid=self.CONTROL_UID) + if not self._ok(self._put(url, self._ical(uid, todo=todo))): + return self._graded("unknown", f"cannot test, the {kind} could not be stored") + try: + at_start, inside, at_end = ( + uid in self._uids_found(window, todo) for window in (self.AT_START, self.INSIDE, self.AT_END) + ) + except (DAVError, OSError) as exc: + return self._graded("unknown", f"cannot test, the time-range search fails ({exc})") finally: self._delete(url) + if not at_start: + return self._graded( + "unknown", + f"cannot test, the time-range search does not find the {kind} by a range containing its start", + ) + if inside and at_end: + return "full" + if not inside and not at_end: + return self._graded("unsupported", f"only {kind}s starting inside the range are found") + if not inside: + return self._graded("unsupported", f"a {kind} is not found by a range inside it") + return self._graded( + "unsupported", f"a {kind} is not found by a range straddling its {'DUE' if todo else 'DTEND'}" + ) class CheckAlarmSearch(Check): diff --git a/src/caldav_server_tester/future_features.py b/src/caldav_server_tester/future_features.py index 945c8e3..6ccaf22 100644 --- a/src/caldav_server_tester/future_features.py +++ b/src/caldav_server_tester/future_features.py @@ -26,6 +26,10 @@ "move.if-match": { "description": "A MOVE with an If-Match the object no longer has fails with 412 and moves nothing (RFC 9110 section 13.1.1). Where unsupported, a MOVE cannot guard against an object changed since it was read." }, + "save-load.event.recurrences.rdate": { + "description": "An event's RDATE (RFC 5545 section 3.8.5.2), occurrences given by date rather than by an RRULE, is stored. 'unsupported' when it is dropped (Zimbra drops every RDATE, and keeps an RRULE), 'broken' when it is stored with other dates.", + "links": ["https://datatracker.ietf.org/doc/html/rfc5545#section-3.8.5.2"], + }, ## caldav PR 848 (https://github.com/python-caldav/caldav/pull/848) "save-load.todo.status": { "description": "Each STATUS RFC 5545 section 3.8.1.11 gives a task (NEEDS-ACTION, IN-PROCESS, COMPLETED, CANCELLED) is stored as sent. 'unsupported' when one is dropped or stored as another, 'ungraceful' when a task with one is refused, the worse of the two where both apply; the extra keys say which. Zimbra refuses CANCELLED. The statuses RFC 10051 adds are save-load.todo.status.rfc10051.", @@ -68,16 +72,40 @@ "default": {"support": "full"}, "links": ["https://datatracker.ietf.org/doc/html/rfc5545#section-3.2"], }, + "save-load.todo.alarm": { + "description": "A task's VALARM is stored as sent: its TRIGGER, relative to the task's DTSTART, and its DESCRIPTION. A property the server adds to the VALARM, such as a UID, is fine. 'unsupported' when the VALARM is dropped. 'broken' when it is kept but rewritten (Ox makes the TRIGGER absolute, so the alarm no longer follows the task when it is moved), the behaviour saying what was changed: the alarm is still there, but not the one sent, which is worse than a missing one.", + "default": {"support": "full"}, + "links": [ + "https://datatracker.ietf.org/doc/html/rfc5545#section-3.6.6", + "https://datatracker.ietf.org/doc/html/rfc5545#section-3.8.6.3", + ], + }, "multiget.deleted": { "description": "A calendar-multiget naming an object that was deleted answers it as missing, as a GET does. 'broken' when it still returns the object's data after the DELETE succeeded and a GET answers 404, so a client fetching by multiget sees deleted objects.", "default": {"support": "full"}, }, ## caldav PR 850 (https://github.com/python-caldav/caldav/pull/850) "save-load.icalendar.olson-tzid": { - "description": "An object whose TZID names a time zone of the tz database (\"Olson\" names, such as Europe/Berlin) is placed in that time zone without a VTIMEZONE for it. RFC 5545 section 3.6.5 requires the VTIMEZONE, so the default is 'unsupported', and a client should send one; where 'full', it may leave the VTIMEZONE out and keep the object small. The probe stores an event with TZID=Asia/Tokyo and no VTIMEZONE and judges the server's reading by time-range searches in UTC. 'unsupported' when the time is read as UTC or another zone, 'ungraceful' when the event is refused. 'fragile' where the reading depends on circumstances: the search path (Xandikos) or the client (Synology's web UI). A server reading the TZID as floating, in a time zone of its own, is caught unless it runs in Tokyo time (then 'full') or in UTC (then the behaviour says 'read as UTC'). The behaviour notes whether the server adds a VTIMEZONE of its own. A client of the server may still misread a bare TZID where the server does not: Synology's web UI was reported to read it as UTC (https://github.com/python-caldav/caldav/issues/849). See also save-load.icalendar.vtimezone, whether a VTIMEZONE sent is kept.", + "description": "An object whose TZID names a time zone of the tz database (\"Olson\" names, such as Europe/Berlin) is placed in that time zone without a VTIMEZONE for it. RFC 5545 section 3.6.5 requires the VTIMEZONE, so the default is 'unsupported', and a client should send one; where 'full', it may leave the VTIMEZONE out and keep the object small. The probe stores an event with TZID=Asia/Tokyo and no VTIMEZONE, and compares where time-range searches in UTC find it with where they find the same event carrying its VTIMEZONE and one at 12:00 UTC. 'full' when it is placed as the event with the VTIMEZONE is, so the verdict says whether the bare TZID is read as the zoned one; whether a zoned time is placed right is search.time-range.event.tzid. 'unsupported' when it is placed as UTC or elsewhere, 'ungraceful' when the event is refused. 'fragile' where the reading depends on circumstances, such as the client (Synology's web UI). The behaviour notes whether the server adds a VTIMEZONE of its own. A client of the server may still misread a bare TZID where the server does not: Synology's web UI was reported to read it as UTC (https://github.com/python-caldav/caldav/issues/849). See also save-load.icalendar.vtimezone, whether a VTIMEZONE sent is kept.", "default": {"support": "unsupported"}, "links": ["https://datatracker.ietf.org/doc/html/rfc5545#section-3.6.5"], }, + ## caldav PR (search.time-range.*.overlap, search.time-range.event.tzid) + "search.time-range.event.overlap": { + "description": "An event is found by a time-range search overlapping it without containing its DTSTART: a range inside the event, and one straddling its DTEND. RFC 4791 section 9.9 has a VEVENT overlap a range when (start < DTEND AND end > DTSTART). 'unsupported' when only events starting inside the range are found (Robur). The library widens every time-range search by a day and filters the result client-side by default, which hides this for events shorter than about a day.", + "default": {"support": "full"}, + "links": ["https://datatracker.ietf.org/doc/html/rfc4791#section-9.9"], + }, + "search.time-range.todo.overlap": { + "description": "A task with DTSTART and DUE is found by a time-range search overlapping it without containing its DTSTART: a range inside the task, and one straddling its DUE. RFC 4791 section 9.9 has such a VTODO overlap a range when ((start <= DTSTART OR start < DUE) AND (end > DTSTART OR end >= DUE)). 'unsupported' when only tasks starting inside the range are found. The task counterpart of search.time-range.event.overlap; journals have no end and are not probed.", + "default": {"support": "full"}, + "links": ["https://datatracker.ietf.org/doc/html/rfc4791#section-9.9"], + }, + "search.time-range.event.tzid": { + "description": "An event whose times carry a TZID, with the VTIMEZONE RFC 5545 requires, is found by a time-range search around its time in UTC. The probe stores one at 12:00 Asia/Tokyo (03:00 UTC). 'unsupported' when it is placed at another time, the behaviour saying where (read as UTC, say); 'broken' when it is found only by a range that also reaches its local time read as UTC (Infomaniak: the search seems to preselect by the time with its TZID dropped, then check it properly). The library widens every time-range search by a day and filters the result client-side by default, which hides this. See also save-load.icalendar.olson-tzid, the same without the VTIMEZONE.", + "default": {"support": "full"}, + "links": ["https://datatracker.ietf.org/doc/html/rfc4791#section-9.9"], + }, } diff --git a/tests/fake_time_range.py b/tests/fake_time_range.py new file mode 100644 index 0000000..37ae6ec --- /dev/null +++ b/tests/fake_time_range.py @@ -0,0 +1,46 @@ +"""A time-range search over what a RewritingServer stores, answered as +different servers answer it.""" + +import re +from datetime import datetime, timedelta, timezone +from unittest.mock import Mock + + +def _time(block, name, hours): + """The UTC time and the wall time, read as UTC, of property ``name``, + ``hours`` ahead of UTC unless it ends in Z""" + match = re.search(rf"\r\n{name}[^:\r\n]*:(\d{{8}}T\d{{6}})(Z?)\r\n", block) + wall = datetime.strptime(match[1], "%Y%m%dT%H%M%S").replace(tzinfo=timezone.utc) + return wall - timedelta(hours=0 if match[2] else hours), wall + + +def fake_search(server, offset=9, zoned=9, mode="overlap", inclusive=False): + """A search reading a bare TZID as ``offset`` hours ahead of UTC, and one + with its VTIMEZONE as ``zoned`` hours. ``mode`` is how the range is + matched: "overlap" as RFC 4791 section 9.9 has it, "dtstart" with only the + start inside it (Robur), "index" overlapping both the time and the wall + time read as UTC (Infomaniak). ``inclusive`` finds an object in a range + starting at its end (DAViCal, SOGo).""" + + def search(start, end, event=False, todo=False, **kwargs): + component = "VTODO" if todo else "VEVENT" + found = [] + for body in server.objects.values(): + if f"BEGIN:{component}" not in body: + continue + block = body[body.index(f"BEGIN:{component}") :] + hours = zoned if "BEGIN:VTIMEZONE" in body else offset + begin, wall_begin = _time(block, "DTSTART", hours) + stop, wall_stop = _time(block, "DUE" if todo else "DTEND", hours) + overlap = (start <= stop if inclusive else start < stop) and end > begin + if mode == "dtstart": + matched = start <= begin < end + elif mode == "index": + matched = overlap and end > wall_begin and start < wall_stop + else: + matched = overlap + if matched: + found.append(Mock(component={"UID": re.search(r"\r\nUID:(.*)\r\n", body)[1]})) + return found + + return search diff --git a/tests/test_icalendar_roundtrip.py b/tests/test_icalendar_roundtrip.py index 32f8b8f..0380e04 100644 --- a/tests/test_icalendar_roundtrip.py +++ b/tests/test_icalendar_roundtrip.py @@ -12,7 +12,7 @@ from caldav.compatibility_hints import FeatureSet from caldav.lib.error import AuthorizationError -from caldav_server_tester.checks import CheckParticipant, CheckVtimezone, CheckXParameter, CheckXProperty +from caldav_server_tester.checks import CheckParticipant, CheckRdate, CheckVtimezone, CheckXParameter, CheckXProperty from .test_floating_event import RewritingServer @@ -162,3 +162,46 @@ def test_x_parameter_dropped_is_unsupported(): verdict = _run(CheckXParameter, RewritingServer(rewrite=lambda b: b.replace(";X-CALDAV-SERVER-TESTER=kept", ""))) assert verdict["support"] == "unsupported" assert "in an event" in verdict["behaviour"] + + +def test_rdate_kept_is_full_and_sent_as_two_dates(): + sent = [] + assert _run(CheckRdate, RewritingServer(rewrite=lambda body: sent.append(body) or body)) == {"support": "full"} + assert len(sent) == 1, "an event only" + assert "\r\nRDATE:20270127T120000Z,20270203T120000Z\r\n" in sent[0] + + +def test_rdate_split_over_lines_is_full(): + def split(body): + return body.replace( + "RDATE:20270127T120000Z,20270203T120000Z", "RDATE:20270127T120000Z\r\nRDATE:20270203T120000Z" + ) + + assert _run(CheckRdate, RewritingServer(rewrite=split)) == {"support": "full"} + + +def test_rdate_dropped_is_unsupported(): + """Zimbra 8.8 and 10.1 drop every RDATE, and keep an RRULE""" + verdict = _run(CheckRdate, RewritingServer(rewrite=lambda b: re.sub(r"\r\nRDATE:[^\r]*", "", b))) + assert verdict == {"support": "unsupported", "behaviour": "the RDATE is dropped"} + + +def test_rdate_cut_short_is_broken(): + verdict = _run(CheckRdate, RewritingServer(rewrite=lambda b: b.replace(",20270203T120000Z", ""))) + assert verdict["support"] == "broken" + assert "20270203T120000Z" in verdict["behaviour"] + + +@pytest.mark.parametrize( + "stored", + [ + "RDATE;TZID=Europe/Oslo:20270127T130000,20270203T130000", + "RDATE;VALUE=PERIOD:20270127T120000Z/PT1H,20270203T120000Z/20270203T130000Z", + "RDATE:20270120T120000Z,20270127T120000Z,20270203T120000Z", + ], + ids=["in-a-zone", "as-periods", "with-dtstart"], +) +def test_rdate_written_another_way_is_full(stored): + """The same occurrences, whatever form the server writes them in""" + rewrite = lambda b: b.replace("RDATE:20270127T120000Z,20270203T120000Z", stored) # noqa: E731 + assert _run(CheckRdate, RewritingServer(rewrite=rewrite)) == {"support": "full"} diff --git a/tests/test_olson_tzid.py b/tests/test_olson_tzid.py index 7eb2fd6..ead2c92 100644 --- a/tests/test_olson_tzid.py +++ b/tests/test_olson_tzid.py @@ -3,11 +3,12 @@ RFC 5545 section 3.6.5 requires a VTIMEZONE for every TZID, but most servers place a bare TZID=Asia/Tokyo right; Synology's web UI was reported to read it as UTC (https://github.com/python-caldav/caldav/issues/849). The probe -judges the server's reading by time-range searches. +judges the server's reading by time-range searches, comparing the +event with the same one carrying its VTIMEZONE and with one at the same +time in UTC. """ import re -from datetime import datetime, timedelta, timezone from unittest.mock import Mock import pytest @@ -16,6 +17,7 @@ from caldav_server_tester.checks import CheckOlsonTzid +from .fake_time_range import fake_search from .test_floating_event import RewritingServer BASE = "http://example.com/cal/" @@ -23,27 +25,6 @@ VTIMEZONE = "BEGIN:VTIMEZONE\r\nTZID:Asia/Tokyo\r\nEND:VTIMEZONE\r\n" -def _search(server, offset, inclusive=False, zoned=9): - """A time-range search reading each stored DTSTART as ``offset`` hours - ahead of UTC, or ``zoned`` hours where the object carries a VTIMEZONE, the - event lasting an hour; ``inclusive`` finds it in a range starting at its - end""" - - def search(start, end, **kwargs): - found = [] - for body in server.objects.values(): - event = body[body.index("BEGIN:VEVENT") :] - stamp = re.search(r"\r\nDTSTART[^:]*:(\d{8}T\d{6})", event)[1] - hours = zoned if "BEGIN:VTIMEZONE" in body else offset - begin = datetime.strptime(stamp, "%Y%m%dT%H%M%S").replace(tzinfo=timezone.utc) - timedelta(hours=hours) - stop = begin + timedelta(hours=1) - if (start <= stop if inclusive else start < stop) and end > begin: - found.append(Mock(component={"UID": re.search(r"\r\nUID:(.*)\r\n", body)[1]})) - return found - - return search - - def _run(server, offset=9, time_range="full", search=None): checker = Mock() checker._features_checked = FeatureSet({"search.time-range.event": {"support": time_range}}) @@ -52,7 +33,7 @@ def _run(server, offset=9, time_range="full", search=None): checker._client_obj = server checker.fixture_base_year = 2027 checker.calendar.url.join.side_effect = lambda name: BASE + name - checker.calendar.search.side_effect = search or _search(server, offset) + checker.calendar.search.side_effect = search or fake_search(server, offset) CheckOlsonTzid(checker)._run_check() return checker._features_checked.is_supported(FEATURE, return_type=dict) @@ -79,7 +60,7 @@ def test_a_vtimezone_added_by_the_server_is_noted(): def test_a_range_starting_at_the_end_does_not_confuse_it(): """DAViCal and SOGo find an event in a range starting at its end""" server = RewritingServer() - assert _run(server, search=_search(server, 9, inclusive=True))["support"] == "full" + assert _run(server, search=fake_search(server, 9, inclusive=True))["support"] == "full" def test_read_in_the_servers_own_zone_is_unsupported(): @@ -89,7 +70,7 @@ def test_read_in_the_servers_own_zone_is_unsupported(): assert "neither" in verdict["behaviour"] -@pytest.mark.parametrize("offset", [13, 14, -12]) +@pytest.mark.parametrize("offset", [10, 13, 14, -12]) def test_read_in_a_zone_far_from_utc_is_unsupported(offset): """Read as floating in Auckland in January, say: the day before in UTC""" verdict = _run(RewritingServer(), offset=offset) @@ -104,7 +85,7 @@ def component(self): raise ValueError("no data") server = RewritingServer() - search = _search(server, 9) + search = fake_search(server, 9) assert _run(server, search=lambda **kwargs: [Broken(), *search(**kwargs)])["support"] == "full" @@ -153,7 +134,7 @@ def test_a_search_misplacing_zoned_times_too_is_unknown(): """Xandikos: once its index answers, a time lands in the server's zone, VTIMEZONE or not, so the search cannot tell what the TZID is read as""" server = RewritingServer() - verdict = _run(server, search=_search(server, 0, zoned=0)) + verdict = _run(server, search=fake_search(server, 0, zoned=0)) assert verdict["support"] == "unknown" assert "VTIMEZONE" in verdict["behaviour"] assert not server.objects @@ -170,10 +151,31 @@ def test_the_control_event_carries_a_vtimezone_and_is_removed(): assert not server.objects -def test_no_control_event_where_the_bare_tzid_is_placed_right(): +def test_a_reference_in_utc_goes_along_and_is_removed(): sent = [] - _run(RewritingServer(rewrite=lambda body: sent.append(body) or body)) - assert len(sent) == 1 + server = RewritingServer(rewrite=lambda body: sent.append(body) or body) + assert _run(server)["support"] == "full" + assert any(re.search(r"\r\nDTSTART:\d{8}T120000Z\r\n", body) for body in sent) + assert not server.objects + + +def test_a_search_matching_starts_only_reads_it(): + """Robur only finds an event whose DTSTART is in the range""" + server = RewritingServer() + assert _run(server, search=fake_search(server, 9, mode="dtstart"))["support"] == "full" + verdict = _run(server, search=fake_search(server, 0, mode="dtstart")) + assert verdict["support"] == "unsupported" + assert "UTC" in verdict["behaviour"] + + +def test_a_search_dropping_the_tzid_in_its_index_reads_it(): + """Infomaniak finds a zoned event only by a range reaching its local + time read as UTC, VTIMEZONE or not""" + server = RewritingServer() + assert _run(server, search=fake_search(server, 9, mode="index"))["support"] == "full" + verdict = _run(server, search=fake_search(server, 0, mode="index")) + assert verdict["support"] == "unsupported" + assert "UTC" in verdict["behaviour"] def test_a_control_that_cannot_be_stored_is_unknown(): @@ -194,7 +196,7 @@ def put(self, url, body, headers=None): def test_the_control_is_removed_when_its_search_fails(): server = RewritingServer() - search = _search(server, 0) + search = fake_search(server, 0) def failing(**kwargs): if any(CheckOlsonTzid.CONTROL_UID in url for url in server.objects): @@ -203,3 +205,33 @@ def failing(**kwargs): assert _run(server, search=failing)["support"] == "unknown" assert not server.objects + + +def test_a_reference_that_cannot_be_stored_is_unknown(): + class RefusingReference(RewritingServer): + def put(self, url, body, headers=None): + if CheckOlsonTzid.REFERENCE_UID in url: + return Mock(status=403, headers={}) + return super().put(url, body, headers) + + server = RefusingReference() + verdict = _run(server) + assert verdict["support"] == "unknown" + assert "could not be stored" in verdict["behaviour"] + assert not server.objects + + +def test_a_search_misplacing_utc_times_is_unknown(): + """Bedework 3: a search answering with other objects than the ones in + the range says nothing about the TZID""" + server = RewritingServer() + search = fake_search(server, 9) + + def misplacing(start, end, **kwargs): + found = search(start=start, end=end, **kwargs) + return [obj for obj in found if obj.component["UID"] != CheckOlsonTzid.REFERENCE_UID] + + verdict = _run(server, search=misplacing) + assert verdict["support"] == "unknown" + assert "12:00 UTC" in verdict["behaviour"] + assert not server.objects diff --git a/tests/test_put_etag.py b/tests/test_put_etag.py index 2817f68..ebe054d 100644 --- a/tests/test_put_etag.py +++ b/tests/test_put_etag.py @@ -13,6 +13,7 @@ If-None-Match: * always does, overwriting an existing object. """ +import re from unittest.mock import Mock from urllib.parse import unquote @@ -44,7 +45,9 @@ class EtagServer: ``accepts`` the If-Match etags a conditional PUT succeeds with, ``if_match_star``/``if_none_match_star`` a key of ``STAR``. ``answers`` overrides the reply to a ``(header, object exists)`` pair with a status, or - raises it where it is an exception. + raises it where it is an exception. ``other_uid_status`` answers a PUT of + another UID to an existing object, whatever its header: Bedework (3 and 5) + answers 500 with If-None-Match: * and without it alike. """ def __init__( @@ -60,7 +63,9 @@ def __init__( rewrite=None, rewrite_updates=None, raw_body=True, + other_uid_status=None, ): + self.other_uid_status = other_uid_status self.rewrite = rewrite self.rewrite_updates = rewrite_updates self.raw_body = raw_body @@ -88,6 +93,8 @@ def put(self, url, body, headers=None): raise answer if answer is not None: return self._response(answer) + if exists and self.other_uid_status and _uid(body) != _uid(self.bodies.get(url, body)): + return self._response(self.other_uid_status) if header is None: if self.create_raises: raise self.create_raises @@ -129,6 +136,10 @@ def delete(self, url): return Mock(status=204) +def _uid(ical): + return re.search(r"^UID:(.*)$", ical, re.MULTILINE)[1].strip() + + def _run_all(server): checker = Mock() checker._features_checked = FeatureSet() @@ -318,6 +329,38 @@ def test_if_none_match_refusing_an_existing_object_otherwise_than_412_is_ungrace assert str(status) in inm["behaviour"] +def test_if_none_match_refusing_another_uid_otherwise_than_412_is_ungraceful(): + """Bedework refuses the same UID with 412, and a new object at a taken + name, the case save(no_overwrite=True) guards, with 500.""" + inm = _if_none_match(EtagServer(other_uid_status=500)) + assert inm["support"] == "ungraceful" + assert "500 rather than 412, for another UID" in inm["behaviour"] + + +@pytest.mark.parametrize("status", [503, 504]) +def test_if_none_match_without_an_answer_for_another_uid_is_not_graded(status): + """Graded full, Bedework's 500 for another UID would come back with one dropped answer.""" + inm = _if_none_match(EtagServer(other_uid_status=status)) + assert inm["support"] == "unknown" + assert "another UID" in inm["behaviour"] + + +def test_if_none_match_overwriting_with_another_uid_is_unsupported(): + inm = _if_none_match(EtagServer(other_uid_status=204)) + assert inm["support"] == "unsupported" + assert "another UID" in inm["behaviour"] + + +def test_if_none_match_overwriting_is_not_tried_again_with_another_uid(): + """Where the same UID already overwrote, that is the verdict. Zimbra + redirects (302) an object PUT under another name to .ics (caldav's + save-load.stable-url hint for it), which another UID would only obscure.""" + server = EtagServer(if_none_match_star="literal", other_uid_status=302) + inm = _if_none_match(server) + assert inm == {"support": "unsupported", "behaviour": "If-None-Match: * overwrote an existing object"} + assert server.requests.count(("If-None-Match", "*")) == 2 # the existing object and the missing one + + def test_if_none_match_failing_on_a_missing_object_is_ungraceful(): inm = _if_none_match(EtagServer(answers={("If-None-Match", False): 500})) assert inm["support"] == "ungraceful" diff --git a/tests/test_time_range_overlap.py b/tests/test_time_range_overlap.py new file mode 100644 index 0000000..b629a3a --- /dev/null +++ b/tests/test_time_range_overlap.py @@ -0,0 +1,145 @@ +"""Whether a time-range search finds an event or task overlapping the range +without starting inside it. + +RFC 4791 section 9.9 has an object overlap a range; Robur only finds the +ones whose DTSTART is in it. +""" + +from unittest.mock import Mock + +import pytest +from caldav.compatibility_hints import FeatureSet +from caldav.lib.error import DAVError + +from caldav_server_tester.checks import CheckTimeRangeOverlap + +from .fake_time_range import fake_search +from .test_floating_event import RewritingServer + +BASE = "http://example.com/cal/" +TASKS = "http://example.com/tasks/" +EVENT = "search.time-range.event.overlap" +TODO = "search.time-range.todo.overlap" + + +def _run(server, mode="overlap", search=None, tasklist=True, event_range="full", todo_range="full"): + checker = Mock() + checker._features_checked = FeatureSet( + {"search.time-range.event": {"support": event_range}, "search.time-range.todo": {"support": todo_range}} + ) + checker.features_checked = checker._features_checked + checker.debug_mode = None + checker._client_obj = server + checker.fixture_base_year = 2027 + checker.calendar.url.join.side_effect = lambda name: BASE + name + checker.calendar.search.side_effect = search or fake_search(server, mode=mode) + if tasklist: + checker.tasklist.url.join.side_effect = lambda name: TASKS + name + checker.tasklist.search.side_effect = search or fake_search(server, mode=mode) + else: + checker.tasklist = None + CheckTimeRangeOverlap(checker)._run_check() + return tuple(checker._features_checked.is_supported(f, return_type=dict) for f in (EVENT, TODO)) + + +def test_found_overlapping_is_full(): + server = RewritingServer() + event, todo = _run(server) + assert event["support"] == "full" + assert todo["support"] == "full" + assert not server.objects + + +def test_the_task_goes_to_the_task_calendar(): + sent = {} + + class Recording(RewritingServer): + def put(self, url, body, headers=None): + sent[url] = body + return super().put(url, body, headers) + + _run(Recording()) + assert any(url.startswith(TASKS) and "BEGIN:VTODO" in body for url, body in sent.items()) + assert any(url.startswith(BASE) and "BEGIN:VEVENT" in body for url, body in sent.items()) + + +def test_a_range_starting_at_the_end_does_not_confuse_it(): + server = RewritingServer() + assert _run(server, search=fake_search(server, inclusive=True))[0]["support"] == "full" + + +def test_found_only_by_its_start_is_unsupported(): + """Robur""" + server = RewritingServer() + event, todo = _run(server, mode="dtstart") + assert event["support"] == "unsupported" + assert "start" in event["behaviour"] + assert todo["support"] == "unsupported" + assert not server.objects + + +def test_missed_by_a_range_straddling_its_end_is_unsupported(): + server = RewritingServer() + search = fake_search(server) + + def no_straddle(start, end, **kwargs): + return [] if (start.hour, start.minute) == (12, 45) else search(start=start, end=end, **kwargs) + + event, todo = _run(server, search=no_straddle) + assert event["support"] == "unsupported" + assert "DTEND" in event["behaviour"] + assert todo["support"] == "unsupported" + assert "DUE" in todo["behaviour"] + + +def test_missed_by_a_range_inside_it_is_unsupported(): + server = RewritingServer() + search = fake_search(server) + + def not_inside(start, end, **kwargs): + return [] if (start.hour, start.minute) == (12, 15) else search(start=start, end=end, **kwargs) + + event, _ = _run(server, search=not_inside) + assert event["support"] == "unsupported" + assert "inside it" in event["behaviour"] + + +def test_not_found_at_its_start_is_unknown(): + server = RewritingServer() + event, todo = _run(server, search=lambda **kwargs: []) + assert event["support"] == "unknown" + assert todo["support"] == "unknown" + assert not server.objects + + +def test_a_failing_search_is_unknown(): + server = RewritingServer() + event, _ = _run(server, search=DAVError("500 Internal Server Error")) + assert event["support"] == "unknown" + assert not server.objects + + +@pytest.mark.parametrize("answer", [403, DAVError("500")]) +def test_an_object_that_cannot_be_stored_is_unknown(answer): + event, _ = _run(RewritingServer(put_answer=answer)) + assert event["support"] == "unknown" + + +def test_without_a_task_calendar_the_task_is_unknown(): + event, todo = _run(RewritingServer(), tasklist=False) + assert event["support"] == "full" + assert todo["support"] == "unknown" + + +def test_without_time_range_search_for_tasks_the_task_is_unknown(): + server = RewritingServer() + event, todo = _run(server, todo_range="unsupported") + assert event["support"] == "full" + assert todo["support"] == "unknown" + assert not server.objects + + +def test_without_time_range_search_for_events_the_event_is_unknown(): + event, todo = _run(RewritingServer(), event_range="unsupported") + assert event["support"] == "unknown" + assert todo["support"] == "full" diff --git a/tests/test_time_range_tzid.py b/tests/test_time_range_tzid.py new file mode 100644 index 0000000..74b4381 --- /dev/null +++ b/tests/test_time_range_tzid.py @@ -0,0 +1,137 @@ +"""Whether a time-range search places an event with a TZID, and the +VTIMEZONE for it, at its time in UTC. + +Infomaniak finds it only by a range also reaching its local time read as +UTC. +""" + +import re +from unittest.mock import Mock + +import pytest +from caldav.compatibility_hints import FeatureSet +from caldav.lib.error import DAVError + +from caldav_server_tester.checks import CheckTimeRangeTzid + +from .fake_time_range import fake_search +from .test_floating_event import RewritingServer + +BASE = "http://example.com/cal/" +FEATURE = CheckTimeRangeTzid.FEATURE + + +def _run(server, zoned=9, mode="overlap", search=None, time_range="full"): + checker = Mock() + checker._features_checked = FeatureSet({"search.time-range.event": {"support": time_range}}) + checker.features_checked = checker._features_checked + checker.debug_mode = None + checker._client_obj = server + checker.fixture_base_year = 2027 + checker.calendar.url.join.side_effect = lambda name: BASE + name + checker.calendar.search.side_effect = search or fake_search(server, zoned=zoned, mode=mode) + CheckTimeRangeTzid(checker)._run_check() + return checker._features_checked.is_supported(FEATURE, return_type=dict) + + +def test_the_event_carries_its_vtimezone(): + sent = [] + _run(RewritingServer(rewrite=lambda body: sent.append(body) or body)) + assert "\r\nTZID:Asia/Tokyo\r\n" in sent[0] + assert "DTSTART;TZID=Asia/Tokyo:" in sent[0] + + +def test_placed_right_is_full(): + server = RewritingServer() + assert _run(server)["support"] == "full" + assert not server.objects + + +def test_placed_right_by_its_start_only_is_full(): + """Robur matches the start only, which the windows contain""" + assert _run(RewritingServer(), mode="dtstart")["support"] == "full" + + +def test_read_as_utc_is_unsupported(): + verdict = _run(RewritingServer(), zoned=0) + assert verdict["support"] == "unsupported" + assert "UTC" in verdict["behaviour"] + + +def test_placed_elsewhere_is_unsupported(): + verdict = _run(RewritingServer(), zoned=-5) + assert verdict["support"] == "unsupported" + assert "neither" in verdict["behaviour"] + + +def test_found_only_reaching_the_wall_time_is_broken(): + """Infomaniak""" + server = RewritingServer() + verdict = _run(server, mode="index") + assert verdict["support"] == "broken" + assert "read as UTC" in verdict["behaviour"] + assert not server.objects + + +def test_not_found_at_all_is_unknown(): + assert _run(RewritingServer(), search=lambda **kwargs: [])["support"] == "unknown" + + +def test_a_search_ignoring_the_range_is_unknown(): + found = [Mock(component={"UID": CheckTimeRangeTzid.UID})] + assert _run(RewritingServer(), search=lambda **kwargs: found)["support"] == "unknown" + + +def test_a_failing_search_is_unknown(): + server = RewritingServer() + assert _run(server, search=DAVError("500 Internal Server Error"))["support"] == "unknown" + assert not server.objects + + +def test_refused_is_unknown(): + assert _run(RewritingServer(put_answer=403))["support"] == "unknown" + + +def test_without_time_range_search_it_is_unknown(): + server = RewritingServer() + assert _run(server, time_range="unsupported")["support"] == "unknown" + assert not server.objects + + +def test_a_reference_in_utc_goes_along_and_is_removed(): + sent = [] + server = RewritingServer(rewrite=lambda body: sent.append(body) or body) + assert _run(server)["support"] == "full" + assert any(re.search(r"\r\nDTSTART:\d{8}T120000Z\r\n", body) for body in sent) + assert not server.objects + + +def test_a_search_misplacing_utc_times_is_unknown(): + """Bedework 3: a search answering with other objects than the ones in + the range says nothing about the TZID""" + server = RewritingServer() + search = fake_search(server) + + def misplacing(start, end, **kwargs): + found = search(start=start, end=end, **kwargs) + return [obj for obj in found if obj.component["UID"] != CheckTimeRangeTzid.REFERENCE_UID] + + verdict = _run(server, search=misplacing) + assert verdict["support"] == "unknown" + assert "12:00 UTC" in verdict["behaviour"] + assert not server.objects + + +@pytest.mark.parametrize("zoned", [1, 5, 8, 10, -5]) +def test_placed_in_another_zone_is_unsupported(zoned): + """Read in another zone, as floating in the server's own, say: not 'full', + nor Infomaniak's 'broken', however close to Tokyo or UTC""" + verdict = _run(RewritingServer(), zoned=zoned) + assert verdict["support"] == "unsupported" + assert "neither" in verdict["behaviour"] + + +@pytest.mark.parametrize("zoned", [1, 5, 10]) +def test_placed_in_another_zone_by_its_start_only_is_unsupported(zoned): + verdict = _run(RewritingServer(), zoned=zoned, mode="dtstart") + assert verdict["support"] == "unsupported" diff --git a/tests/test_todo_roundtrip.py b/tests/test_todo_roundtrip.py index 0a64819..21dae28 100644 --- a/tests/test_todo_roundtrip.py +++ b/tests/test_todo_roundtrip.py @@ -1,7 +1,8 @@ """Whether a task keeps what it was stored with. Ox answers a task carrying an ATTENDEE with 201 and then 404 on GET, drops -ESTIMATED-DURATION, and drops COMPLETED where the task has no STATUS. +ESTIMATED-DURATION, drops COMPLETED where the task has no STATUS, and +rewrites a task's VALARM. """ import re @@ -12,6 +13,7 @@ from caldav_server_tester.checks import ( CheckCompletedWithoutStatus, + CheckTodoAlarm, CheckTodoAttendee, CheckTodoPriority, CheckTodoStatus, @@ -28,6 +30,7 @@ CheckTodoAttendee, CheckUnknownProperty, CheckCompletedWithoutStatus, + CheckTodoAlarm, CheckTodoStatus, CheckTodoStatusRfc10051, CheckTodoPriority, @@ -137,6 +140,82 @@ def add_status(body): assert _run(CheckCompletedWithoutStatus, RewritingServer(rewrite=add_status))["support"] == "full" +def test_an_alarm_dropped_is_unsupported(): + def drop_alarm(body): + return re.sub(r"BEGIN:VALARM\r\n.*END:VALARM\r\n", "", body, flags=re.DOTALL) + + verdict = _run(CheckTodoAlarm, RewritingServer(rewrite=drop_alarm)) + assert verdict == {"support": "unsupported", "behaviour": "the VALARM is dropped"} + + +def test_an_alarm_given_a_uid_is_full(): + def add_uid(body): + return body.replace("BEGIN:VALARM\r\n", "BEGIN:VALARM\r\nUID:csc-alarm-uid\r\n") + + assert _run(CheckTodoAlarm, RewritingServer(rewrite=add_uid))["support"] == "full" + + +def test_an_alarm_rewritten_is_broken(): + """Ox: a relative TRIGGER made absolute, and the DESCRIPTION its own""" + + def rewrite(body): + body = re.sub(r"\r\nTRIGGER[;:][^\r]*", "\r\nTRIGGER;VALUE=DATE-TIME:20270120T114500Z", body) + return re.sub(r"(BEGIN:VALARM.*\r\nDESCRIPTION:)[^\r]*", r"\1Open-XChange", body, flags=re.DOTALL) + + verdict = _run(CheckTodoAlarm, RewritingServer(rewrite=rewrite)) + assert verdict["support"] == "broken" + assert verdict["behaviour"] == ( + "the alarm's TRIGGER -PT15M is stored as TRIGGER;VALUE=DATE-TIME:20270120T114500Z; " + "its DESCRIPTION is stored as Open-XChange" + ) + + +@pytest.mark.parametrize( + "trigger", + ["TRIGGER:-P0DT0H15M0S", "TRIGGER;VALUE=DURATION:-PT15M", "TRIGGER;RELATED=START;VALUE=DURATION:-PT15M"], + ids=["spelled-out", "value-default", "both-defaults"], +) +def test_an_alarm_trigger_written_another_way_is_full(trigger): + """The same duration, or a parameter at its RFC 5545 default, changes nothing""" + + def rewrite(body): + return re.sub(r"\r\nTRIGGER[;:][^\r]*", "\r\n" + trigger, body) + + assert _run(CheckTodoAlarm, RewritingServer(rewrite=rewrite))["support"] == "full" + + +def test_an_alarm_description_with_a_language_is_full(): + def rewrite(body): + return re.sub(r"(BEGIN:VALARM.*\r\n)DESCRIPTION:", r"\1DESCRIPTION;LANGUAGE=en:", body, flags=re.DOTALL) + + assert _run(CheckTodoAlarm, RewritingServer(rewrite=rewrite))["support"] == "full" + + +def test_an_alarm_kept_without_its_trigger_is_broken(): + def rewrite(body): + return re.sub(r"\r\nTRIGGER[;:][^\r]*", "", body) + + verdict = _run(CheckTodoAlarm, RewritingServer(rewrite=rewrite)) + assert verdict == {"support": "broken", "behaviour": "the alarm's TRIGGER is dropped"} + + +def test_an_alarm_action_rewritten_is_broken(): + def rewrite(body): + return body.replace("ACTION:DISPLAY", "ACTION:AUDIO") + + verdict = _run(CheckTodoAlarm, RewritingServer(rewrite=rewrite)) + assert verdict == {"support": "broken", "behaviour": "the alarm's ACTION is stored as AUDIO"} + + +def test_an_alarm_with_the_task_description_unchanged_is_not_confused_with_it(): + """Only the VALARM's own DESCRIPTION counts""" + + def add_task_description(body): + return body.replace("\r\nBEGIN:VALARM", "\r\nDESCRIPTION:the task's own\r\nBEGIN:VALARM") + + assert _run(CheckTodoAlarm, RewritingServer(rewrite=add_task_description))["support"] == "full" + + class ValueServer(RewritingServer): """Stores ``name:value`` as ``stored[value]`` (None drops it), and refuses the values in ``refused``."""