zhu_unsupervised_2018: Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

Contributions

The authors propose a new input representation that captures the spatiotemporal distribution of events, and a set of unsupervised loss functions that allows for learning of motion information only from the event stream.

Input Representation

Given a set of $N$ input events ${(x_{i}, y_{i}, t_{i}, p_{i})}_{i \in [1, \infty^{n}}$ , and a set of $B$ bins to discretize the time dimension, the timestamps are scaled to the range $[0, B - 1]$ , and the event volume is generated as:

$\begin{aligned} t_{i}^{*} & = (B - 1) (t_{i} - t_{0}) / (t_{N} - t_{1}) \\ V (x, y, t) & = \sum_{i} p_{i} k_{b} (x - x_{i}) k_{b} (y - y_{i}) k_{b} (t - t_{i}^{*}) \\ k_{b} (a) & = max (0, 1 - | a |) \end{aligned}$

where $k_{b} (a)$ is the bilinear sampling kernel.

<biblio.bib>