feat: Improve logging

[cbueth]

Summary

Centralizes and improves logging across the MultiplEYE preprocessing pipeline, silences false‑positive session warnings, and captures key user/data orientation warnings. This replaces scattered print() and manual file writes with a structured, configurable logging setup.

Key changes

• Add IGNORED_SESSION_FOLDERS and filter non‑matching session folders accordingly.
• Introduce centralized logging utility with console + file handlers, colored console output, and version/git metadata in header.
• Redirect Python warnings into the logging system and capture them for end‑of‑run summaries.
• Improve warning and error messages for session discovery and configuration edge cases.
• Configure package‑wide logging on import, with user‑configurable levels via YAML.

Implementation details

• Constants and configuration
  • preprocessing/constants.py
    • Add IGNORED_SESSION_FOLDERS = ["test_sessions", "core_sessions", "pilot_sessions"].
    • Add mapping for configurable CONSOLE_LOG_LEVEL, FILE_LOG_LEVEL, and WARNINGS_CAPTURE_LEVEL sourced from multipleye_settings_preprocessing.yaml.
• Session discovery noise reduction
  • preprocessing/data_collection/multipleye_data_collection.py
    • In add_recorded_sessions(...), only warn for non‑matching directories if not in IGNORED_SESSION_FOLDERS.
    • Add clearer messages for missing/extra session files and include/exclude mismatches.
• Centralized logging
  • preprocessing/utils/logging.py
    • setup_logging(log_file, console_level, file_level): configures console + optional file handlers, colored console logs, and captures warnings via logging.captureWarnings(True).
    • Logs pipeline version, git describe/date, and pymovements version at startup.
    • Captures redirected warnings into logging._captured_warnings for later summaries.
    • get_logger(name): returns module or named logger (preprocessing[...]).
    • clear_log_file(path): utility to truncate logs when not appending.
• Package initialization
  • preprocessing/__init__.py sets up logging on import:
    • Log file: <DATASET_DIR>/preprocessing_logs.txt when dataset dir exists.
    • Respects LOG_APPEND from config; otherwise clears the log file.

User‑facing behavior

• Console shows concise, colored logs with default level WARNING (configurable).
• A persistent log file records detailed INFO (configurable) including versions and git metadata.
• Warnings from dependencies and internal code paths are included in logs and collected for final summaries, including:
  • Gaze component inference UserWarning from pymovements.gaze.gaze.
  • Data orientation warnings in metrics routines.

Migration notes

• Stop using print() and any manual writes to preprocessing_logs.txt.
• Import and use the centralized logger:

  from preprocessing.utils.logging import get_logger
  logger = get_logger(__name__)
  logger.info("...")
  logger.warning("...")

• Logging is initialized automatically on preprocessing import. For scripts, ensure the package is imported early or call setup_logging(...) explicitly if you need a custom destination.
• Configure verbosity in multipleye_settings_preprocessing.yaml:

  console_log_level: WARNING  # DEBUG|INFO|WARNING|ERROR|CRITICAL
  file_log_level: INFO        # "
  warnings_capture_level: WARNING
  log_append: true

Tests

• Unit tests added/updated for logging setup and ignored session warnings:
  • tests/unit/utils/test_logging.py
  • Ignored‑folder behavior in add_recorded_sessions.

Related issues

• Closes Improve Logging and Warning Management in MultiplEYE Preprocessing #51
• Closes Silence false-positive session matching warnings #52 – Silence false‑positive session matching warnings
• Closes Introduce a centralized logging package solution #53 – Centralized logging solution (console + file, structured levels)
• Closes Capture and log specific User/DataOrientation warnings #54 – Capture and log specific user/data orientation warnings

Notable commits

• e448511: add IGNORED_SESSION_FOLDERS and update folder filtering logic
• 1061ced: unit test for ignored folder warnings
• 65e138f / aa2bdc5: introduce structured logging utility and integrate pipeline‑wide
• 2d83a48 / 52ce1f2 / 5bcbd39 / 99350cf: configure console/file levels, move setup to init, capture warnings, colorized logs
• bd744c4: add pipeline version and git metadata logging
• f326331 / efeafb2 / 07b1a21: refine warnings and error messages during session handling