Projectaria-tools

**[Features]**
- **[Core - Python]**
- Sophus python binding
- Add [SO3, SE3 interface](https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/core_code_snippets/calibration#python-binding-for-sophus-library) in python based on Sophus library. Example code is provided in [sophus_quickstart_tutorial notebook](https://github.com/facebookresearch/projectaria_tools/blob/main/core/examples/sophus_quickstart_tutorial.ipynb)

- [Python type annotation](https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/installation/type_hinting)
- Python type hinting/ stubs are automatically generated as part of the pypi package when installing projectaria_tools with pip install. Users can also generate them on their own using the `generate_stubs.py` script.

- [Google Colab runnable notebooks](https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/getting_started#running-the-jupyter-notebooks-on-google-colab)
- Python notebooks can now be run in Google Colab -> [Dataprovider Quickstart Tutorial](https://colab.research.google.com/github/facebookresearch/projectaria_tools/blob/main/core/examples/dataprovider_quickstart_tutorial.ipynb) | [Machine Perception Services Tutorial](https://colab.research.google.com/github/facebookresearch/projectaria_tools/blob/main/core/examples/mps_quickstart_tutorial.ipynb)
- No installation on local machine required to test and play with projectaria_tools

- **[Core]**
- Add `cameraId` to `ImageDataRecord`
- Allow the `ImageDataRecord` to list from which camera the data came from

- Continuous integration
- GitHub Actions runs Python Unit test

- Dependencies
- Update to use VRS v1.1.0
- Remove cereal dependency and use directly rapidjson

- **[MPS]**
- Calibrated and generalized EyeGaze
- Support of calibrated eye gaze via [in-session calibration](https://facebookresearch.github.io/projectaria_tools/docs/ARK/mps/eye_gaze_calibration)
- Support for multiple wearers in a single Aria capture. The eye gaze output will contain a `session_uid` field that will help distinguish between different wearers.

- Python type format
- `print(X)` will now display object content

- **[Tools]**
- [MPS Replay Viewer {C++}](https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/visualization_guide#mps-3d-replay-visualizer)
- Renders static scene and dynamic elements: 2D/3D observations rays + eye gaze data

**[BugFix]**
- [Core]
- [{bug fix} update crop and rescale to SensorCalibration](https://github.com/facebookresearch/projectaria_tools/commit/feaa489303ddf7fee3e74ed1fdb3001df142f541)
- Update the API to make calibration data to match from the sensor and device access point: `get_sensor_calibration(stream_id).camera_calibration()` and `provider.get_device_calibration().get_camera_calib(name)` to match.

**[Known Issues]**
- [Core]
- The Sophus API has been updated, if you encounter issues, please update to v1.2 of Project Aria Tools
- Here is how to update your existing code following the API change for SO3/SE3:
- `.matrix()` to `.to_matrix()`
- `.quaternion()` -> `.rotation().to_quat()[0]` or `to_quat_and_translation()[0]`

**[Documentation]**
- [Core]
- [VRS to MP4](https://facebookresearch.github.io/projectaria_tools/docs/data_utilities/advanced_code_snippets/vrs_to_mp4) Tutorial showing how to export VRS RGB images to a MP4 video.
- Additional information added to [3D Coordinate Frame Conventions](https://facebookresearch.github.io/projectaria_tools/docs/data_formats/coordinate_convention/3d_coordinate_frame_convention)
- [MPS]
- [Eye Gaze Data Formats](https://facebookresearch.github.io/projectaria_tools/docs/data_formats/mps/mps_eye_gaze) updated to include `calibrated_eye_gaze.csv` and `summary.json`
- [Eye Gaze Calibration](https://facebookresearch.github.io/projectaria_tools/docs/ARK/mps/eye_gaze_calibration)

**[Thank you to our new contributors]**
brentyi
[Seanwarren-meta](https://github.com/facebookresearch/projectaria_tools/commits?author=seanwarren-meta)
Selcuk Karakas
Przemyslaw Szczepanski
[Guru Somasundaram](https://github.com/facebookresearch/projectaria_tools/commits?author=gsomas)

**Full Changelog**: https://github.com/facebookresearch/projectaria_tools/compare/1.1.0...1.2.0

- The ADT library has been updated to support dataset versioning.
- Data schema update
- Fix quaternion order in ‘aria_trajectory.csv’
Corrected to `qw, qx, qy, qz` from `qx, qy, qz, qw`
- Fix gravity field names are now called `gravity_x/y/z_world` to align with MPS layout
- Change `SkeletonMetaData.json` to `skeleton_aria_association.json` to better reflect the file content
- Change `gt-metadata.json` to `metadata.json`
- Users are STRONGLY ADVISED to pull from the release branch and [follow ADT download instructions](https://facebookresearch.github.io/projectaria_tools/docs/open_datasets/aria_digital_twin_dataset/dataset_download) to update their ADT datasets to v1.1.
- [ASE]
- Released a more accurate set of camera FishEye model calibration parameter

- [Documentation]
- Minor updates

[BugFix]
- [Core]
- AriaViewer (reset line plots when new timestamp is requested)
- [ADT]

Initial release (https://ariatutorial2023.github.io/)

[Core]
- Provide C++/Python VRS data provider (sensor data and configuration) and utilities (camera poses and intrinsics manipulation)

[Tools]
- Aria VRS and MPS visualizers

[Projects]
- ADT - Aria Digital Twin
- A real-world dataset, with hyper-accurate digital counterpart & comprehensive ground-truth annotation
- ASE - Aria Synthetic Environments
- A procedurally generated synthetic Aria dataset for large-scale ML research.

[Documentation]
- Project Aria Documentation (Aria Research Kit, Open Dataset and Project Aria Tools)