Project Structure

CALYPSO_DEV/
├── utils/              # Shared utilities (dependencies for other tools)
│   ├── video-reader/  # Video frame extraction
│   ├── face-detector/ # Face detection and tracking
│   └── calypso-visualizer/ # Visualization utilities
│
├── extractors/         # Feature extraction tools (from raw media)
│   ├── face-landmarks/ # Facial landmark extraction (MediaPipe backend)
│   └── gaze-estimator/ # Eye gaze estimation (L2CS-Net)
│
├── analyzers/          # Analysis tools (process extracted features)
│   └── gait-analyzer/  # Gait analysis from SMPL motion capture data
│
└── notebooks_demos/    # Example notebooks

Getting Started

Installation

Each tool can be installed independently:

# Install shared utilities
pip install -e utils/video-reader
pip install -e utils/face-detector

# Install extractors
pip install -e extractors/gaze-estimator

# Install analyzers
pip install -e analyzers/gait-analyzer

Tools

Utilities

• video-reader: Extract video frames with temporal metadata
• face-detector: Face detection with tracking (YOLO, MediaPipe)

Extractors

• gaze-estimator: Eye gaze direction estimation (L2CS-Net)

Analyzers

• gait-analyzer: Biomechanical gait analysis from SMPL data
  • TUG phase segmentation
  • Gait event detection (Heel-Strike/Toe-Off)
  • Spatiotemporal gait features
  • Arm swing analysis

TODO

Extractors to Add

• MediaPipe Landmarks - Facial landmark detection

  • Added extractors/face-landmarks/
  • Uses MediaPipe Tasks with modular backend architecture

• Pose Estimator - Body pose extraction

  • Add to extractors/pose-estimator/
  • General pose estimation tool

• Walking Pose Estimator - Gait analysis specific

  • Add to extractors/walking-pose/
  • Specialized for walking/gait analysis

Code Improvements

• Unified Model Download - Create common download utility

  • Add utils/model-downloader/ or similar
  • Standardize model management across all tools

• Notebooks for Each Tool - Add demo notebooks

  • notebooks_demos/video_reader_demo.ipynb
  • notebooks_demos/face_detector_demo.ipynb ✓ (exists)
  • notebooks_demos/gaze_estimator_demo.ipynb
  • Add for each new extractor

Development Guidelines

See Python_guidlines.md for coding standards.

License

GPL-3.0-or-later

---

Made with love by Interaction Team