🤖ICRA 2025

Sikai Li, Samanta Rodriguez, Yiming Dou, Andrew Owens, Nima Fazelii, “Tactile Functasets: Neural Implicit Representations of Tactile Datasets'', in ICRA, 2025, in Submission.

https://arxiv.org/abs/2409.14592

Abstract:

Modern incarnations of tactile sensors produce high-dimensional raw sensory feedback such as images, making it challenging to efficiently store, process, and generalize across sensors. To address these concerns, we introduce a novel implicit function representation for tactile sensor feedback. Rather than directly using raw tactile images, we propose neural implicit functions trained to reconstruct the tactile dataset, producing compact representations that capture the underlying structure of the sensory inputs. These representations offer several advantages over their raw counterparts: they are compact, enable probabilistically interpretable inference, and facilitate generalization across different sensors. We demonstrate the efficacy of this representation on the downstream task of in-hand object pose estimation, achieving improved performance over image-based methods while simplifying downstream models. We release code, demos and datasets at this https URL.

Modern tactile sensors are capable of streaming highresolution feedback from the contact between the robot and objects. These high-resolution data streams pose significant challenges for processing and storage. We ask whether there is a more compact and efficient method to store data and perform inference for tactile data? This question is inspired by the observation that tactile signals are lower in information density when compared to natural image.

Above is the architecture of our tactile functaset framework. Orange layers represent the trunk model, with blue layers being the modulations mapped from red latent vectors. Each red latent vector is stored as one tactile functa point for a specific data point. The entire model takes pixel locations of tactile images as input and output corresponding pixel values.