Below are some project ideas for the class. You're also welcome
to propose your own project. We'll pick projects in the
class' first meeting.
Robotic Manipulation
We would like our robots to be able to perceive even novel objects that it
has never seen before, and be able to figure out how to grasp them or to
manipulate them. In this project, students will apply
machine learning to develop algorithms to enable STAIR to do these things.
Planned projects include:
(a) Using touch sensing (haptic grasping). The human sense of touch is
tremendously helpful for grasping objects and manipulating objects. STAIR
currently uses no touch (haptic) information, even though we already have touch
sensors on the robot. In this project, we'll develop algorithms to interpret
the touch sensor data, and apply this to the problems of grasping objects
and of opening doors.
(b) Grasping using 3d sensors. Unlike humans, our robots are
not limited to a few senses such as sight and touch. For example, one of
STAIR's sensors is a high-resolution 3d scanner, that lets us directly measure
the 3d position of every pixel seen in our camera. In this project, you'll
develop algorithms to enable the robot to use this scanner to grasp and
manipulate objects.
Robotic perception, computer vision, object recognition.
Most work in computer vision has focused on using a single camera (visible light) to
detect objects. This has proved to be very hard to do well. In contrast, STAIR is instrumented
with multiple cameras, range scanners, and other
sensors. Using these sensors, we believe you'll be able to make
object recognition dramatically better than the state-of-the-art in the literature,
and develop practical object recognition systems.
Planned projects include: (a) Object recognition from 3d. Using a high-resolution
3d scanner, develop learning approaches to object recognition that use 3d data
as well as visible light. (b) Hyperspectral imaging. Whereas most cameras
measure light energy in the red, blue and green channels (corresponding to the human
eye), there is no reason for robots not to use other light spectra--including ones
that humans cannot see. In this project, students will develop object recognition
algorithms that use hyperspectral imaging. (c) High resolution images. Most computer
vision algorithms are tuned to relatively low resolution (say 320x240, or 640x480) images.
But given a 10 megapixel camera, we will be able to dramatically better see small
object features better, and improve object recognition. In this project, students
will explore the use of high resolution images for object recognition.
Robot navigation using cameras.
Most robots today use very expensive (~$5K) laser scanners to measure distances
and detect obstacles. We envision using inexpensive cameras to develop new
navigation systems, as well as improve STAIR's laser scanner-based navigation.
The key trick will be using an upward facing camera and a map of the ceiling.
Specifically, students will use a camera (and possibly a laser) pointed at the
ceiling) to map out the ceiling of a building, and use that for navigation.
The principal advantage of this is that even though chairs may be moved, doors
may open and shut, etc. so that the environment can change significantlt from
week to week, the appearance of a building's ceiling hardly ever changes. So,
this provides robots a way to determine their absolute position even over long
timescales.
Miscellany: More specialized projects.
Here're a few other project ideas that are more specialized to specific
technical areas. I'd recommend you work on these only if you have particular
expertise and excitement in the topic described.
(a) Spoken dialog system/speech recognition. In this project,
students will develop a spoken dialog system for STAIR, to enable it to engage
in complex dialog and understand complex requests to carry out different tasks.
(b) Structure from motion for obstacle avoidance. In this projects, students will
use a camera to detect obstacles in the robot's environment, even ones that cannot
be sensed with a laser.
(c) Computer graphics data synthesis. In this computer graphics+computer vision project, students will
artificially synthesize many images of office scenes,
to provide training data to learning algorithms, to improve object recognition performance.
Propose your own.
You are also welcome to propose your own research project.