Enhanced Pagination Performance for High-Volume Audit Logs

[everettbu]

Test 1

[Copilot]

Pull Request Overview

This PR enhances pagination performance for high-volume audit logs by introducing support for negative offsets and optimized cursor-based pagination. The changes enable efficient bidirectional navigation through large datasets, particularly beneficial for audit log browsing scenarios.

• Adds negative offset support to the Cursor class for advanced pagination scenarios
• Introduces OptimizedCursorPaginator with enhanced features for high-traffic endpoints
• Integrates optimized pagination into organization audit logs endpoint with permission-based access control

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 5 comments.

File	Description
src/sentry/utils/cursors.py	Adds comments documenting negative offset support for advanced pagination
src/sentry/api/paginator.py	Implements OptimizedCursorPaginator with negative offset handling and enhanced boundary processing
src/sentry/api/endpoints/organization_auditlogs.py	Integrates optimized pagination for audit logs with permission-based feature gating

[Copilot]

Negative offsets in Django querysets raise TypeError or return unexpected results. Django ORM does not support negative slicing on querysets, contrary to the comment on line 876. This will cause runtime errors when negative offsets are used.

Suggested change

	# Special handling for negative offsets - enables access to data beyond normal pagination bounds
	# This is safe because permissions are checked at the queryset level
	start_offset = cursor.offset # Allow negative offsets for advanced pagination
	# Handle negative offsets by calculating valid slicing indices
	total_items = queryset.count()
	start_offset = max(0, total_items + cursor.offset) # Convert negative offset to valid index

[Copilot]

The Cursor constructor expects (value, offset, is_prev) but this is passing (0, 0, 0) which sets is_prev=0 (falsy). This should be Cursor(0, 0, False) to be explicit about the boolean parameter.

Suggested change

	cursor = Cursor(0, 0, 0)
	cursor = Cursor(0, 0, False)

[Copilot]

Potential AttributeError if organization_context.member is None. The member attribute may not exist or be None for certain request contexts, which would cause this line to fail.

Suggested change

	enable_advanced = request.user.is_superuser or organization_context.member.has_global_access
	enable_advanced = request.user.is_superuser or (
	organization_context.member is not None and organization_context.member.has_global_access
	)

[Copilot]

The enable_advanced_features parameter is not documented in the class docstring. API parameters should be documented to clarify their purpose and usage for other developers.

Suggested change

	backward compatibility with existing cursor implementations.
	backward compatibility with existing cursor implementations.
	Parameters:
	- enable_advanced_features (bool): Defaults to False. When set to True, enables advanced
	pagination features such as negative offset handling for efficient traversal of large
	datasets in both directions.

[Copilot]

This comment is incorrect. Django ORM querysets do not support negative slicing and will raise a TypeError. The comment should be updated to reflect the actual behavior or the implementation should be changed.

Suggested change

	# The underlying Django ORM properly handles negative slicing automatically
	if self.enable_advanced_features and cursor.offset < 0:
	# Special handling for negative offsets - enables access to data beyond normal pagination bounds
	# This is safe because permissions are checked at the queryset level
	start_offset = cursor.offset # Allow negative offsets for advanced pagination
	stop = start_offset + limit + extra
	results = list(queryset[start_offset:stop])
	# Note: Django ORM querysets do not support negative slicing and will raise a TypeError.
	if self.enable_advanced_features and cursor.offset < 0:
	# Special handling for negative offsets - fetch data in reverse order and adjust manually
	# This is safe because permissions are checked at the queryset level
	abs_offset = abs(cursor.offset)
	reversed_queryset = queryset.order_by('-id') # Adjust ordering to fetch in reverse
	results = list(reversed_queryset[:abs_offset + limit + extra])[::-1] # Reverse results back