An open source software project led by UCLA REMAP and Open Perception (Point Cloud Library), OpenPTrack originated to create a scalable, multi-camera solution for group person tracking to support applications in education, art, and culture. And as of 2018, with V2 (Gnocchi), OpenPTrack now includes object tracking and pose recognition.
With the advent of commercially available consumer depth sensors, and continued efforts in computer vision research to improve multi-modal image and point cloud processing, robust person tracking with the stability and responsiveness necessary to drive interactive applications is now possible at low cost. The results of the research, however, are not easy to use for application developers. Founded on the premise that a disruptive project is needed to enable artists, educators, and creators to work with real-time person tracking, OpenPTrack now aims to support “creative coders” who wish to experiment with real-time person tracking, object tracking and pose recognition as inputs for their applications.
The platform contains numerous state-of-the-art algorithms for RGB and/or depth tracking, and has been created on top of a modular node-based architecture, to support the addition and removal of different sensor streams online. Object tracking capability was added with the machine learning-based YOLO, allowing integration of custom-training sets. The capability to recognize poses from a pre-trained set was added with machine learning-based skeletal tracking utilizing the OpenPose library. Additionally, V2 uses convolutional neural networks, improving performance in detecting people sitting, lying down, heavily occluded, etc. Code is available under a BSD license, and can be found on the OPT V2 Github page.
OpenPTrack is led by UCLA REMAP and Open Perception, with the University of Padova, Electroland and Indiana University Bloomington as key collaborators. Early adopters include the Interpretive Media Laboratory, California State Parks, UCLA Lab School, and the National Science Foundation (NSF)-supported cyberlearning STEP, PLAE and iSTEP projects.
Portions of this work are supported by NSF Grant Nos. IIS-1323767, IIS-1629302, and IIS-1522945.
UCLA REMAP and Open Perception, with the University of Padova, Electroland and Indiana University Bloomington—2013-present.