V 3.0-dev - async support and misc #610
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
3 tasks
tobixen
force-pushed
the
v3.0-dev
branch
3 times, most recently
from
f7c9243 to
3d1b79b
Compare
Member
Author
|
@tevepiercy - quite a lot of API changes have been done in this pull request / git branch. I've asked the AI to update the docs and examples, but with one exception I haven't done any QA of the doc updates done in this branch. The exception is the "tutorial", the AI destroyed the whole "tutorial"-concept and spewed lots of irrelevant information to it. Definitively the documentation needs some human hands to become useful for human consumption. (I've particularly seen that the Claude AI seem to have no concepts of what is important for the end user and what is not - it's just dumping down whatever it has in its current "context") Please have a look and see if you can work with some of this. The docs and examples are lliving under |
Add comprehensive design documentation for the Sans-I/O architecture: - SANS_IO_IMPLEMENTATION_PLAN.md: Overall implementation strategy - SYNC_ASYNC_OVERVIEW.md: How sync/async code sharing works - PROTOCOL_LAYER_USAGE.md: Guide to using the protocol layer - CODE_REVIEW.md: Architecture review and decisions Also add AI-POLICY.md documenting AI assistant usage guidelines. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implement the foundation of the Sans-I/O architecture with a protocol layer that separates HTTP I/O from CalDAV/WebDAV logic: Protocol layer (caldav/protocol/): - types.py: Data classes for requests/responses (PropfindRequest, etc.) - xml_builders.py: Pure functions to build XML request bodies - xml_parsers.py: Pure functions to parse XML responses Response handling (caldav/response.py): - BaseDAVResponse: Common response interface for sync/async - Parsed results accessible via response.results property Tests (tests/test_protocol.py): - Comprehensive unit tests for XML building and parsing - Tests for various CalDAV operations (PROPFIND, REPORT, etc.) This layer has no I/O dependencies and can be used with any HTTP client. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implement high-level CalDAV operations that build on the protocol layer: Operations (caldav/operations/): - base.py: Base operation classes and utilities - davobject.py: Generic DAV object operations (get_properties, etc.) - calendarobject.py: Calendar object operations (save, load, delete) - calendarset.py: Calendar set operations (calendars, make_calendar) - principal.py: Principal operations (calendar_home_set, etc.) - calendar.py: Calendar operations (search, events, todos) Each operation: - Uses protocol layer for XML building/parsing - Returns typed request/response data classes - Has no I/O - caller provides HTTP transport Tests (tests/test_operations_*.py): - Unit tests with mocked responses for each operation type - Tests for error handling and edge cases Co-Authored-By: Claude Opus 4.5 <[email protected]>
Refactor test infrastructure with a unified server abstraction: Test server framework (tests/test_servers/): - base.py: Abstract TestServer base class with common interface - embedded.py: In-process servers (Radicale, Xandikos) - docker.py: Docker-based servers (Baikal, Nextcloud, etc.) - config_loader.py: Load server configs from YAML/environment - registry.py: Server discovery and registration Shared test utilities (tests/fixture_helpers.py): - Common fixtures for calendar creation/cleanup - Helpers that work with both sync and async tests Docker test server improvements: - Fixed Nextcloud tmpfs permissions race condition - Fixed Baikal ephemeral storage configuration - Fixed SOGo and Cyrus credential configuration - Added DAViCal server configuration Updated tests/conf.py: - Integrate with new test server framework - Support both legacy and new configuration methods Co-Authored-By: Claude Opus 4.5 <[email protected]>
Implement full async support using httpx (with niquests fallback): Async client (caldav/async_davclient.py): - AsyncDAVClient: Full async HTTP client with connection pooling - Support for HTTP/2 when h2 package is available - Async context manager for proper resource cleanup - Auth negotiation (Basic, Digest, Bearer) Public API (caldav/aio.py): - AsyncPrincipal, AsyncCalendar, AsyncEvent, AsyncTodo, etc. - Factory methods: AsyncPrincipal.create(), etc. - Async-compatible get_davclient() function Auth utilities (caldav/lib/auth.py): - Shared authentication logic for sync/async clients Tests: - test_async_davclient.py: Unit tests for async client - test_async_integration.py: Integration tests against real servers Documentation: - docs/source/async.rst: Async usage guide - examples/async_usage_examples.py: Example code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Refactor domain objects to work with both sync and async clients: Client consolidation (caldav/base_client.py): - BaseDAVClient: Shared logic for sync/async clients - Unified get_davclient() implementation - Common configuration handling Domain object updates: - caldav/davobject.py: Detect client type, delegate to async when needed - caldav/collection.py: Calendar/CalendarSet with async support - caldav/calendarobjectresource.py: Event/Todo/Journal async support The same domain object classes work with both sync and async clients: - With DAVClient: Methods return results directly - With AsyncDAVClient: Methods return coroutines to await Other updates: - caldav/davclient.py: Use BaseDAVClient, simplified - caldav/config.py: Support test server configuration - caldav/search.py: Python 3.9 compatibility fixes - caldav/__init__.py: Export async classes Co-Authored-By: Claude Opus 4.5 <[email protected]>
CI/Build improvements: - .github/workflows/tests.yaml: Add async tests, fix Nextcloud password - .github/workflows/linkcheck.yml: Add documentation link checker - pyproject.toml: Add pytest-asyncio, httpx deps, warning filters - tox.ini: Configure async test environments - .pre-commit-config.yaml: Update hook versions Test improvements: - tests/test_caldav.py: Fix async/sync test isolation - tests/test_examples.py: Use get_davclient() context manager - Filter Radicale shutdown warnings in pytest config Bug fixes: - Don't send Depth header for calendar-multiget (RFC 4791 §7.9) - Fix HTTP/2 when h2 package not installed - Fix Python 3.9 compatibility in search.py Documentation: - README.md: Add async usage examples - docs/source/index.rst: Link to async documentation - CONTRIBUTING.md: Update development guidelines Co-Authored-By: Claude Opus 4.5 <[email protected]>
Document all changes for the v3.0 release: - Full async API with AsyncDAVClient - Sans-I/O architecture (protocol and operations layers) - Unified test server framework - HTTP/2 support - Various bug fixes and improvements Co-Authored-By: Claude Opus 4.5 <[email protected]>
Review changes: - Set minimum Python version to 3.10 (remove 3.9 from CI and classifiers) - Make httpx optional (install with `pip install caldav[async]`) - Add CI job to test sync client with requests fallback - Add HTTP library documentation (docs/source/http-libraries.rst) - Update changelog to reflect final niquests decision - Add _USE_NIQUESTS/_USE_REQUESTS flags to davclient.py for testing The sync API remains fully backward-compatible. Only niquests is a required dependency; httpx is optional for async support. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Prefix all internal functions in the protocol and operations layers with underscore to indicate they are private implementation details: - caldav/protocol/xml_builders.py: _build_* functions - caldav/protocol/xml_parsers.py: _parse_* functions - caldav/operations/*.py: All utility functions now prefixed with _ The __init__.py files now only export data types (QuerySpec, CalendarInfo, SearchStrategy, etc.) rather than implementation functions. All call sites updated to import private functions directly from submodules with local aliases for backward compatibility within the codebase. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Link to local file instead of ReadTheDocs URL since the page doesn't exist on RTD yet (only in v3.0-dev branch). Co-Authored-By: Claude Opus 4.5 <[email protected]>
With Python 3.10 as the minimum version, we can simplify imports: - Use collections.abc for Callable, Container, Iterable, Iterator, Sequence - Use typing.DefaultDict and typing.Literal directly - Remove redundant sys.version_info < (3, 9) checks - Remove unused import sys from collection.py Also update tests to use new expand parameter format: - expand="client" → expand=True - expand="server" → server_expand=True The backward-compatible support for string values was removed as part of the caldav 3.0 changes. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Change all imports from `from caldav.davclient import get_davclient` to `from caldav import get_davclient` - Update documentation references to use `caldav.get_davclient` - Remove TestExpandRRule tests (deprecated methods will be removed in 4.0) - Update deprecation message in tests/conf.py Co-Authored-By: Claude Opus 4.5 <[email protected]>
Remove 9 design documents specific to the abandoned async-first-with-sync-wrapper approach: - ASYNC_REFACTORING_PLAN.md (original async-first plan) - PHASE_1_IMPLEMENTATION.md, PHASE_1_TESTING.md (old phases) - PLAYGROUND_BRANCH_ANALYSIS.md, CODE_REVIEW.md (old branch analysis) - SYNC_WRAPPER_DEMONSTRATION.md, SYNC_ASYNC_OVERVIEW.md (old approach) - PERFORMANCE_ANALYSIS.md (event loop overhead analysis) - SYNC_ASYNC_PATTERNS.md (general patterns survey) Keep API analysis documents that contain design rationale still relevant to current implementation: - API_ANALYSIS.md (parameter naming, URL handling) - URL_AND_METHOD_RESEARCH.md (URL semantics for methods) - ELIMINATE_METHOD_WRAPPERS_ANALYSIS.md (keep wrappers decision) - METHOD_GENERATION_ANALYSIS.md (manual implementation decision) Update README.md to reflect current structure. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add pytest.mark.filterwarnings to tests that intentionally use the deprecated date_search method for backward compatibility testing: - testTodoDatesearch - testDateSearchAndFreeBusy - testRecurringDateSearch Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add alias methods to DAVClient for API consistency with AsyncDAVClient: - supports_dav() → check_dav_support() - supports_caldav() → check_cdav_support() - supports_scheduling() → check_scheduling_support() This allows sync users to use the same cleaner API as async users. Note: get_principal(), get_calendars(), get_events(), get_todos(), and search_calendar() are already available in the sync client. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add API_NAMING_CONVENTIONS.md documenting: - Recommended vs legacy method names - Migration guide from date_search to search - Deprecation timeline for 4.0 Update docstrings in DAVClient: - Mark principal(), check_dav_support(), check_cdav_support(), check_scheduling_support() as legacy - Add detailed docs for recommended methods: get_principal(), supports_dav(), supports_caldav(), supports_scheduling() Update date_search docstring in collection.py: - Add Sphinx deprecated directive - Include migration example Update tutorial.rst: - Use get_principal() instead of principal() in all examples Update docs/design/README.md: - Add API_NAMING_CONVENTIONS.md to index - Reorganize API Design section Co-Authored-By: Claude Opus 4.5 <[email protected]>
The `name` parameter (for displayname) is only meaningful for Calendar objects, not for all DAVObject subclasses like Principal, CalendarSet, or CalendarObjectResource. Changes: - Remove `name` parameter from DAVObject.__init__() - Add Calendar.__init__() that accepts `name` parameter - Keep `name` as a class attribute on DAVObject (defaults to None) This is a minor breaking change for anyone who was passing `name` to non-Calendar DAVObject subclasses, but such usage was never meaningful. Fixes: #128 Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Run black/ruff formatting on affected files - Reorder imports per pre-commit hooks - Add h2 to DEP001 ignore (optional HTTP/2 dependency) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Document the internal flow through the layered architecture for: - Fetching calendars (sync and async) - Creating events - Searching for events - Sync token synchronization - Creating calendars Explains the Protocol layer, Operations layer, and dual-mode domain objects that enable both sync and async usage. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Documents findings from the pre-release code review: - Duplicated code between sync/async clients (~240 lines) - Dead code (auto_calendars, auto_calendar, unused imports) - Test coverage assessment by module - Architecture strengths and weaknesses - GitHub issues #71 and #613 analysis - Recommendations for v3.0, v3.1, and v4.0 Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add entries for: - API consistency aliases (supports_dav, supports_caldav, supports_scheduling) - Calendar class name parameter (issue #128) Co-Authored-By: Claude Opus 4.5 <[email protected]>
The _started_by_us tracking makes sense for Docker servers (which can be started externally), but not for embedded servers (Radicale, Xandikos) which always run in-process. Embedded servers cannot be "externally started" in a meaningful way - if they're accessible, it's because we started them in this process. Keeping the original behavior for embedded servers. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Two issues fixed: 1. Xandikos shutdown: Changed to properly cleanup the aiohttp runner BEFORE stopping the event loop. The old code stopped the loop first, which caused "cannot schedule new futures after shutdown" errors because the executor was shut down while requests were still in flight. 2. Server restart after stop: Added _was_stopped flag to prevent using is_accessible() to detect running servers after a stop. After stop() is called, the port might still respond briefly before fully closing, so subsequent start() calls would incorrectly think the server was running and skip starting a new one. These fixes prevent test failures when running multiple tests that start/stop the same embedded server (Radicale or Xandikos). Co-Authored-By: Claude Opus 4.5 <[email protected]>
Documents the Strategy pattern approach for handling multiple data representations (string, icalendar, vobject) in CalendarObjectResource. Key concepts: - Explicit ownership transfer via edit_*() methods - Safe read-only access via get_*() methods (returns copies) - Explicit write access via set_*() methods - Backward compatible legacy properties See #613 Co-Authored-By: Claude Opus 4.5 <[email protected]>
Incorporated feedback from @niccokunzmann in issue #613: - Added Null Object Pattern (NoDataStrategy) to eliminate None checks - Added borrowing pattern with context managers (Rust-inspired) - Added state machine diagram for edit states - Clarified this is more of a State pattern than Strategy pattern - Added comparison table of edit methods vs borrowing approach Co-Authored-By: Claude Opus 4.5 <[email protected]>
Co-Authored-By: Claude Opus 4.5 <[email protected]>
The test class is TestForServerRadicale not TestForServerLocalRadicale. The -k filter needs to match the actual class name. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Documents all usages of obj.data, obj.icalendar_instance, obj.icalendar_component, obj.vobject_instance and their aliases throughout the codebase. Related to issue #613. Co-Authored-By: Claude Opus 4.5 <[email protected]>
…ated_call These methods are deprecated and tests should verify they emit deprecation warnings while still testing their functionality. Co-Authored-By: Claude Opus 4.5 <[email protected]>
This adds a safer API for accessing and modifying calendar data: New read-only methods (return copies, no side effects): - get_data() - returns iCalendar string - get_icalendar_instance() - returns copy of icalendar object - get_vobject_instance() - returns copy of vobject object New edit context managers (explicit ownership): - edit_icalendar_instance() - borrow icalendar for editing - edit_vobject_instance() - borrow vobject for editing The context managers prevent concurrent modification of different representations by raising RuntimeError if already borrowed. Also adds DataState classes (Strategy/State pattern) for internal data management, which will enable future optimizations. Backward compatibility is maintained - existing properties still work. Fixes #613 Co-Authored-By: Claude Opus 4.5 <[email protected]>
This helps catch issues early. Exceptions are made for: - niquests asyncio.iscoroutinefunction deprecation (upstream fix pending) - radicale resource warnings (upstream issue) Co-Authored-By: Claude Opus 4.5 <[email protected]>
Adds optimized methods for internal use that avoid unnecessary parsing: - get_component_type() - determine VEVENT/VTODO/VJOURNAL without full parse - Optimized implementations for RawDataState using string search/regex Also adds internal helper methods to CalendarObjectResource: - _get_uid_cheap() - get UID without state changes - _get_component_type_cheap() - get type without parsing - _has_data() - check for data without conversions These will be used to optimize internal code paths. Co-Authored-By: Claude Opus 4.5 <[email protected]>
has_component() previously converted to string to count components, which caused a side effect of decoupling icalendar instances. Now uses the cheap _get_component_type_cheap() accessor which uses simple string search or direct object inspection without triggering format conversions. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Remove invalid WARNING category filters (logging != warnings) - Re-enable ResourceWarning and thread exception filters for Radicale Co-Authored-By: Claude Opus 4.5 <[email protected]>
Document the upstream Radicale bug (#1972) that causes ResourceWarning and PytestUnraisableExceptionWarning during test server shutdown. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Mark issue #613 design as implemented with summary of new API methods. Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add a Strategy/State pattern for managing calendar data representations: - RawDataState, IcalendarState, VobjectState, NoDataState - Safe context managers for editing: edit_icalendar_instance(), edit_vobject_instance() - Cheap accessors: _get_uid_cheap(), _get_component_type_cheap() - New public API: get_data(), get_icalendar_instance(), get_vobject_instance() Also includes: - Unit tests for the new data API - Optimizations to internal methods (is_loaded, has_component, etc.) - Tutorial simplification and new howtos documentation - Updated examples to use the new API Co-Authored-By: Claude Opus 4.5 <[email protected]>
The id property now reads the UID directly from calendar data using cheap accessors, rather than storing it as a separate attribute that could get out of sync. Changes: - id property getter extracts UID from data via _get_uid_cheap() - Falls back to direct icalendar instance lookup (without triggering load) - id setter is a no-op for parent class compatibility - __init__ modifies UID in data when id parameter is provided - _set_icalendar_instance and _set_vobject_instance keep _state in sync Co-Authored-By: Claude Opus 4.5 <[email protected]>
Add a new pattern for building search queries:
searcher = calendar.searcher(event=True, start=..., end=...)
searcher.add_property_filter("SUMMARY", "meeting")
results = searcher.search()
This avoids requiring users to import CalDAVSearcher directly.
Changes:
- Add _calendar field to CalDAVSearcher to store bound calendar
- Make calendar parameter optional in search()/async_search()
- Add Calendar.searcher() method that creates a bound CalDAVSearcher
- Add tests for the new API pattern
Also fixes a bug where copy(keep_uid=False) didn't properly update the
UID. The issue was that _state was cached with the original data before
the icalendar component was modified.
Co-Authored-By: Claude Opus 4.5 <[email protected]>
The cdav module was imported inside TYPE_CHECKING block but used in runtime type hints (props: Optional[List[cdav.CalendarData]]). This caused NameError at runtime. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Update pyproject.toml with comprehensive ruff configuration - Replace black/reorder_python_imports with ruff in pre-commit config - Reformat all Python files with ruff format - Fix duplicate dictionary keys in compatibility_hints.py (actual bugs) - Configure ignore rules for existing code patterns that would require substantial changes (bare except, %-formatting, etc.) 🤖 Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Planning document for work after issue #599 completion, covering: - RFC compliance gaps (ACL, scheduling, availability) - Enhanced search and sync features - Advanced features (managed attachments, sharing) - Robustness improvements (collision avoidance, recurrence) - Testing and documentation expansion Based on analysis of CalDAV RFCs and open issues. Co-Authored-By: Claude Opus 4.5 <[email protected]>
- Fix lychee pre-commit hook rev format (v0.18.0 not lycheeverse/[email protected]) - Ignore Radicale's importlib.abc.Traversable deprecation warning (Python 3.14) 🤖 Generated with Claude Code Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've been "vibing" with Claude Code in a "playground" branch for a long time now, but the end result starts looking like something it's possible to continue with.
I still have a TODO-list before I can make a release candidate for 3.0, so this is still a draft PR.