This paper explores articulated-pose estimation, assuming that video-rate depth information is available, from either stereo cameras or other sensors. We use these depth measurements in the traditional linear brightness constraint equation, as well as in a similar constraint equation on depth, called the depth constraint equation. We introduce "shifted" constraint equations that allow for larger motions without requiring iterative estimation. The resulting constraint equations are linear on a modified parameter set. After solving these linear constraints, there is a single closed-form non-linear transformation to return the updates to the original pose parameters.

Our tracking results, both on synthetic data and on real data (see below), argue strongly for video-rate depth information, from either stereo cameras or other sensors. Without the true depth information, when we estimated depths, the tracking behavior became unstable and failed catastrophically within duration of our test sequences.

Constraint equations | |||

BCCE without depth | BCCE (with depth) | ZCCE | BCCE + ZCCE |

BCCE wo/depth | BCCE w/depth | ZCCE | BCCE+ZCCE |

Img. Seq. | FOE | Constraint equations | |||

BCCE without depth | BCCE (with depth) | ZCCE | BCCE + ZCCE | ||

unshifted | slow jump | slow jump, unshifted BCCE wo/depth | slow jump,
unshifted BCCE w/depth | slow jump, unshifted ZCCE |
slow jump, unshifted BCCE+ZCCE |

slow walk | slow walk, unshifted BCCE wo/depth | slow walk,
unshifted BCCE w/depth | slow walk, unshifted ZCCE |
slow walk, unshifted BCCE+ZCCE | |

fast jump | fast jump,
unshifted BCCE w/depth | fast jump, unshifted ZCCE |
fast jump, unshifted BCCE+ZCCE | ||

fast walk | fast walk,
unshifted BCCE w/depth | fast walk, unshifted ZCCE |
fast walk, unshifted BCCE+ZCCE | ||

shifted | fast jump | fast jump,
shifted BCCE w/depth | fast jump, shifted ZCCE |
fast jump, shifted BCCE+ZCCE | |

fast walk | fast walk,
shifted BCCE w/depth | fast walk, shifted ZCCE |
fast walk, shifted BCCE+ZCCE |

B wo/Z = BCCE without true depth; B w/Z = BCCE with true depth; Z = ZCCE only; B+Z = BCCE and ZCCE

Img. Seq. | FOE | Eqs | Body rotation error | Body position error | Right-wrist position error | Left-wrist position error | Right-ankle position error | Left-ankle position error | ||||||

mean | max | mean | max | mean | max | mean | max | mean | max | mean | max | |||

slow jump | unshifted | B wo/Z | 131 | 197 | 44 | 73 | 68 | 90 | 72 | 92 | 64 | 91 | 12 | 19 |

B w/Z | 5 | 15 | 2 | 15 | 4 | 15 | 4 | 11 | 4 | 16 | 2 | 5 | ||

Z | 4 | 7 | 3 | 14 | 14 | 22 | 7 | 11 | 6 | 12 | 1 | 2 | ||

B+Z | 2 | 4 | 0.2 | 0.5 | 7 | 14 | 4 | 5 | 4 | 9 | 0.4 | 1 | ||

slow walk | unshifted | B wo/Z | 71 | 102 | 73 | 135 | 25 | 42 | 27 | 41 | 44 | 76 | 22 | 34 |

B w/Z | 2 | 5 | 3 | 8 | 2 | 7 | 37 | 57 | 6 | 10 | 3 | 7 | ||

Z | 2 | 5 | 1 | 2 | 4 | 8 | 17 | 36 | 4 | 10 | 27 | 32 | ||

B+Z | 0.3 | 1 | 0.2 | 0.4 | 3 | 6 | 5 | 10 | 6 | 13 | 7 | 11 | ||

fast jump | unshifted | B w/Z | 23 | 57 | 9 | 20 | 35 | 61 | 27 | 43 | 30 | 56 | 10 | 23 |

Z | 14 | 27 | 7 | 18 | 36 | 52 | 34 | 45 | 28 | 46 | 4 | 13 | ||

B+Z | 12 | 27 | 6 | 14 | 37 | 58 | 33 | 53 | 31 | 48 | 5 | 13 | ||

shifted | B w/Z | 11 | 23 | 11 | 29 | 11 | 26 | 11 | 20 | 17 | 39 | 6 | 13 | |

Z | 2 | 4 | 5 | 13 | 14 | 28 | 8 | 16 | 14 | 27 | 2 | 9 | ||

B+Z | 2 | 6 | 2 | 5 | 10 | 20 | 7 | 13 | 18 | 33 | 2 | 4 | ||

fast walk | unshifted | B w/Z | 17 | 41 | 20 | 48 | 10 | 19 | 46 | 67 | 39 | 63 | 27 | 37 |

Z | 2 | 6 | 2 | 5 | 32 | 57 | 35 | 60 | 31 | 54 | 33 | 45 | ||

B+Z | 2 | 4 | 3 | 5 | 28 | 51 | 32 | 58 | 37 | 57 | 38 | 55 | ||

shifted | B w/Z | 5 | 13 | 13 | 28 | 6 | 15 | 15 | 29 | 13 | 29 | 26 | 34 | |

Z | 1 | 3 | 6 | 11 | 1 | 6 | 5 | 13 | 3 | 9 | 6 | 13 | ||

B+Z | 2 | 4 | 2 | 3 | 1 | 2 | 4 | 11 | 3 | 7 | 10 | 15 |