A mobile manipulation robot needs to be capable of opening doors to enter a building or to move between rooms. In an emergency situation, possibly the easiest method to gain entry is via “door breaching”. This means using a mechanical device to apply a large sudden force to the door, or rapid disassembly of the door using an explosive charge. However, for inspection or maintenance applications it is preferable to open a door via non-destructive means. It is assumed that any door the robot must enter can be unlocked electronically.

This report is about using a tracked drive robot to autonomously open doors via pulling. We discuss multiple perception algorithms for automatically detecting doors and associated features like the hinge and handle. We also outline some preliminary work on motion planning for the arm to grasp and pull the door open. Motion planning to open pull-type doors is more difficult than for push-type doors, particularly using a non-holonomic robot. It requires the coordinated motion of the robot’s base and two arms.

The methods discussed here were designed for the robot CHIMP (CMU Highly Intelligent Mobile Platform), a mobile manipulator designed to operate in hazardous environments [1]. The robot has multiple cameras, dual spinning LiDAR sensors, two 7-DOF arms and a differential-drive with tracks made from polyurethane. Theoretically CHIMP can make zero-radius turns, however its heavy mass means the rubber tracks don’t favor sharp turns on high-friction surfaces. For robots with a holonomic (omnidirectional) base, the motion planning problem will be easier to solve.

Research work on door opening by autonomous mobile robots can be categorized by the type of the robot’s mobile base, which results in a specific kinematic motion model, and whether the robot is pushing or pulling the door.

Various research work has looked at opening doors by pushing, using a robot arm on a differential-drive mobile base:

• The YAMABICO-10 robot had a 6-DOF arm and a differential-drive base [2].
• CARDEA at MIT had a Segway base and a 5-DOF arm using SEA (Series Elastic Actuators) [3].
• The STAIR (Stanford Artificial Intelligence Robot) had a 5-DOF Katana arm on a Segway [4].
• EL-E at Georgia Tech had a 6-DOF manipulator (Katana arm + linear actuator) and an Erratic-brand base [5].

Using a 5-DOF arm it’s possible to turn the door handle and push open the door without using coordinated, whole-arm motion planning [4,5]. It’s also possible to push open a door simply by driving forward, if the robot starts with its base in the correct initial position. However it’s relatively easy to make corrections to the base position because a differential-drive robot can rotate in-place. Also collision-checking is simple for robots that rotate on their center axis, within a circle. The drive wheels on the EL-E robot are offset to the front [5], with a 3rd caster wheel for balance, so a modified footprint is required for exact collision checking.

Other work has looked at pushing doors using a robot with a holonomic (omni-directional) drive:

• PSR-1 at KIST (Korea Institute of Science and Technology) had a 6-DOF arm and omnidirectional mobile base [6].
• The Milestone-2 experiment on the PR2 robot at Willow Garage, which has steerable caster wheels and a 7-DOF arm [7].

Using a robot capable of omni-directional motion makes it easy to plan movements for the base. A holonomic drive can rotate whilst driving through the door, equivalent to having an extra yaw joint in the arm. Steerable caster wheels are a bit more difficult to control and planning a smooth path is important to prevent rapid, instantaneous rotation changes by the wheels.

In this report we focus on the problem of opening doors by pulling, which is more difficult than pushing open doors.

The strategy or behavior used to pull open a door is dependent on if a robot has wheels or legs, and if motion of the base is restricted in any direction (Fig. 2).

Various work has looked at pulling open small doors like cupboards, drawers or fridges. This task can be achieved using a 7-DOF arm without moving the robot’s base, so we ignore that research in this report.

Two projects have demonstrated pulling open doors and driving the robot through:

• Later experiments using the PR2 robot at Willow Garage [8-11].
• Packbot 510 by iRobot, with tracked drive and 6-DOF total for manipulation [12].

The most complete demonstration of a fully autonomous robot capable of opening doors was with the PR2 robot [8-10]. At the time, other groups were able to download the source code and run the motion planning software of their own PR2 robot in their lab. Initially the robot opened pull-type doors using one arm [8], with the assumption that the door would not move after being released. The motion planning problem was decomposed into multiple sub-problems, to reduce the dimensionality. A graph-based planner was used to first solve for the pose of the robot’s base and door, then subsequently solve for the arm motion [8].

The PR2 can also open spring-loaded pull-type doors using a two-arm strategy [9,10]. The graph-based planning method from [8] was improved and can solve the entire motion planning problem, using a combined state space for the base and arm. The PR2 was also used to open heavy spring-loaded doors [11] that required too much opening force for the dual-arm hand-off approach used in [9,10]. The strategy employed in [11] is that after cracking the door ajar, the PR2 rotates around the door and uses its back to hold the door open, without need of the second arm.

Using a track drive robot to pull open a door is more challenging. If the arm(s) aren’t long enough then after pulling the door inwards, such that it collides with the base, the robot will need to be repositioned. The company iRobot has developed a semi-autonomous door opening system for the PackBot 510 robot using an application-specific underactuated manipulator [12]. A user-guided perception system detects the door knob or handle in the image from a 2D camera. The handle is grasped using a touch-sensitive gripper and 3-DOF in the arm. The complicated kinematics for pulling the door is simplified by using a flexible 2-DOF joint in the wrist that is electromechanically activated. A spring-loaded door can be held open using a 1-DOF flipper arm at the front of the robot, allowing the arm to regrasp the door on the inside face. A sequence of scripted motions is used to control the arm and base, similar to the approach used for the CHIMP robot in the DARPA Robotics Challenge [1].

This section describes our approach for detecting the door using camera and/or LiDAR, and constrained motion planning to turn the handle and pull the door open using a 7-DOF arm. Our work on combined arm-base motion planning is unfinished at this time.

We studied various examples of pull-type doors that a rescue robot might encounter in an industrial environment (Fig. 3). All the doors are a different color or texture to the surrounding wall, which suggests that using a camera is a good approach. However, the surface of the door is typically co-planar with the surrounding wall and a LiDAR sensor will fail to discriminate the geometry. Some doors do have a raised moulding or frame surrounding the door, but various design considerations mean that pull-type doors are usually flush with the wall surface.

For our experiments we used the most difficult test-case, being a pull door that was the same color and flush with the surrounding frame (see Fig. 4 below). The algorithm parameters for the various modules were found using grid-search and a dataset of camera and LiDAR data.

Detecting Door Candidates:

This means detecting the position and outline of the door.
Four algorithms were tested:

1. Camera-only:
This method detects the best-fitting quadrilateral in the image (Fig. 4e). Without a 3D sensor, this does not output the pose of the door.

2. Camera + LiDAR:
This method finds patches in the image and uses these to check sets of co-planar points in the point cloud (Fig. 5).

3. LiDAR-only:
An iterative method to detect all vertical planes that satisfy dimensions specified by the ADA (Americans with Disabilities Act), similar to that used on the PR2 robot in [7]. This algorithm fails to correctly detect the door outline and hinge when the door surface is coplanar with the surrounding frame (Fig. 6b).

4. User-guided:
This method is seeded by input from the human operator:

Door Pose Refinement:

This means determining the exact location of the hinge (door leaf), the handle, and the width of the door. Three algorithms were tested, numbered here to match the method of door detection.

1 or 2. Camera image:
The door handle must be located in either of two regions (Fig. 8.a), so these pixels are analyzed.

3. LiDAR point cloud:
This looks for a cluster of LiDAR points that are offset from the door (Fig. 8.b). Unfortunately the point cloud produced by the spinning LiDAR was too sparse to get reliable detections of a reflective metal door handle. The authors of [7] had the same problem with the nodding LiDAR on the PR2 robot.

4. Via user-input:
The human operator has already clicked on 2 points in the pointcloud. Instead of just selecting the plane, these points are used to define the door position. The first point is used as the hinge axis and the second point is used to determine the door width and handle height.

To summarize, algorithms 1 and 2 can reliably detect various types of door. Algorithm 2 (camera + LiDAR) can detect the pose of the door relative to the robot. Algorithm 3 (LiDAR-only) works for most types of door, where the door surface is offset from the door frame, however detecting all planes in an unstructured 3D point cloud took a long time to compute. Algorithm 4 (user-guided) enables almost instant detection of doors for a teleoperated robot.

The Hokuyo LiDAR has a depth accuracy of +/- 30mm, or closer to 35mm when using the dual scanner LiDAR on the CHIMP robot. We were able to learn an approximate LiDAR depth error model when the robot is facing front-on towards a wood door, but there is still a risk that the robot is too close or too far away from the door. Therefore touch-based manipulation is required to check the depth to the door and/or location of the handle. The results described in the literature show the same conclusion, that touch-based manipulation is critical for door opening.

The allowable motion of a door, also known as its “affordance”, is constrained to 90° rotations of the handle lever and 90° rotations of the door hinge. We represent these two constraints using a TSR (Task Space Region) chain [13] and solve the constrained motion planning problem using the CBiRRT2 algorithm [13]. Planning the motion to open a door means finding a viable meta-plan that combines collision-free trajectories for the gripper along these two constraints.

During motion planning, collision-checking is performed between the robot’s geometry and the environment (Fig. 9). The door is modeled using a cuboid shape, with its pose as found by the door detector. Any other objects near the door are modeled using a Voxel Grid generated from the latest LiDAR scan. The robot’s geometry is represented using multiple intersecting bounding boxes (cuboids), which is more efficient than a polygon mesh.

There are various strategies for opening a pull door using a robot with tracked drive (Fig. 10 below):

• A. Arms are long enough to full open the door without moving the base.
• B. The base is driven along an arc-shaped path while opening the door.
• C. Arm and base motions are interleaved.

In this work we focused on motion planning for the arm(s) which meant we could only test strategy ‘A’ in Fig. 10.

In the lab we demonstrated automatic door opening for pull-type doors. The perception system using fused camera + LiDAR can correctly detect the door pose and the handle location. The motion planner was able to generate a plan to push down on the door handle and pull it in towards the robot. However, the base motion planner was not implemented and the robot could not automatically drive through the doorway.

At the DARPA Robotics Challenge Finals we used a simpler approach, for speed and reliability. There was some concern that using the camera image could be unreliable in an outdoor environment, with varying levels of brightness, therefore we used only the LiDAR sensor for perception. Planning the constrained trajectories for the arms was computationally-intensive and time-consuming, therefore we developed a library of scripted motion primitives. The final system comprised of a user-guided system to detect the door hinge and handle, and a scripted sequence of arm motions to manipulate the door [1]. The CHIMP robot successfully opened a push-type door, with a human operator driving the base and initiating arm motion scripts.

It is risky to use open-loop scripted motions because the robot could be too close to the door surface or too far away from the door handle. These situations mean that the robot’s arm could collide with the door or its gripper could fail to touch the handle. On day 1 of the competition, the robot was driven too close to the door and at a slight angle. When turning the door handle the robot’s forearm was pushing hard against the door surface before the latch was released. This caused the door to ‘stick’ in the frame, and when the door ‘popped’ open the robot was momentarily destabilized and rolled backwards. On day 2 of the competition, the robot was positioned too far away from the door surface. The operator tried to turn the handle 7 times, with slight adjustments in-between, until the door finally pushed opened.

Future work will be to:

• Investigate why the constrained motion planner and/or collision checking is so slow.
• Develop a combined arm-and-base motion planner.
• Compare the planning complexity and success-rate of opening pull-type doors using robots with various base types.
• Implement a more reliable method to detect the position of the door and handle, such as using force (touch) sensing or an I.R. distance sensor.