Classification of Household Materials via Spectroscopy

Paper | IEEE

Z. Erickson, N. Luskey, S. Chernova, and C. C. Kemp, “Classification of Household Materials via Spectroscopy”, IEEE Robotics and Automation Letters (RA-L), 2019. (presented at ICRA 2019) (Finalist for Best Paper Award in Service Robotics at IEEE Conference on Robotics and Automation (ICRA 2019))

Abstract

Recognizing an object’s material can inform a robot on the object’s fragility or appropriate use. To estimate an object’s material during manipulation, many prior works have explored the use of haptic sensing. In this paper, we explore a technique for robots to estimate the materials of objects using spectroscopy. We demonstrate that spectrometers provide several benefits for material recognition, including fast response times and accurate measurements with low noise. Furthermore, spectrometers do not require direct contact with an object. To explore this, we collected a dataset of spectral measurements from two commercially available spectrometers during which a robotic platform interacted with 50 flat material objects, and we show that a neural network model can accurately analyze these measurements. Due to the similarity between consecutive spectral measurements, our model achieved a material classification accuracy of 94.6% when given only one spectral sample per object. Similar to prior works with haptic sensors, we found that generalizing material recognition to new objects posed a greater challenge, for which we achieved an accuracy of 79.1% via leave-one-object-out cross-validation. Finally, we demonstrate how a PR2 robot can leverage spectrometers to estimate the materials of everyday objects found in the home. From this work, we find that spectroscopy poses a promising approach for material classification during robotic manipulation.

Code

https://github.com/Healthcare-Robotics/smm50

SMM50 Dataset

SMM50 dataset (16 MB): https://goo.gl/Xjh6x4

cd smm50/data
wget -O smm50.tar.gz https://goo.gl/Xjh6x4
tar -xvzf smm50.tar.gz
rm smm50.tar.gz

Object List

Metal Plastic Wood Paper Fabric
steel
aluminum
aluminum foil
copper
magnesium
titanium
brass
lead
zinc
iron
PP
PVC
HDPE
PET
blue polyethylene
green polyethylene
red polyethylene
yellow polyethylene
thermo polypropylene
thermo teflon
ash
cherry
curly maple
hard maple
hickory
red cedar
red elm
red oak
walnut
white oak
cardboard
green construction paper
orange construction paper
red construction paper
magazine paper
newspaper
notebook paper
printer paper
receipt paper
textbook paper
cotton canvas
cotton sweater
cotton towel
denim
felt
flannel
gauze
linen
satin
wool