PHANES AI has released SeePhys Hub, a comprehensive system for collecting multimodal data designed to convert everyday human activity into training data for robots. According to QbitAI, the launch on October 14th was the result of collaboration with Tinno Mobile, which contributed its expertise in cost control, supply chain, and quality assurance, while PHANES AI focused on algorithm development and product design.
The kit includes four cameras mounted on a head strap, two wristbands, three motion trackers attached to the waist and shins, and a pair of tactile gloves. These components collectively capture first-person, 360-degree panoramic vision, hand posture, high-density tactile feedback, as well as full-body posture and force.
The company's main argument is that egocentric data obtained from head-mounted cameras is an important source for robot training due to its scalability. However, such data often does not reflect all body actions. A camera positioned at eye level sees hands but frequently misses elbows or legs, and it does not record the force with which a person presses against an object. Tasks requiring walking, bending, and two-handed coordination require this missing information.
SeePhys Hub utilizes a sparse node approach. The three trackers combine IMU motion signals with range data obtained via UWB between nodes, and algorithms integrate this data with visual information from the head and wrists to estimate full-body posture. The head strap weighs no more than 220 grams, and each wristband weighs no more than 75 grams; both devices can operate continuously for 24 hours. The synchronization error between the head strap cameras, IMUs, and UWB is less than 0.5 milliseconds.
The gloves are an optional element. Founder and CEO Yang Xiao reported that data collection operators asked the team not to equip workers with gloves because they might interfere with delicate tasks, such as washing dishes, and introduce noise from sensors. Customers can choose configurations based on the specific task, while the team researches methods for estimating tactile impact based on video data.
The accompanying platform handles data parsing, cleaning, segmentation, and semantic labeling, and also provides connectivity for scene reconstruction, pose estimation, hand tracking, and motion transfer onto robot bodies. Yang, who became a professor at Harbin Institute of Technology (Shenzhen) at the age of 26, described the device as a tool for working with mid-level data situated between video internet and high-precision motion capture systems or real robot data.