Jethro's Braindump

zhu_unsupervised_2018: Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion

Contributions

The authors propose a new input representation that captures the spatiotemporal distribution of events, and a set of unsupervised loss functions that allows for learning of motion information only from the event stream.

Input Representation

Given a set of \(N\) input events \(\left\{\left(x_{i}, y_{i}, t_{i}, p_{i}\right)\right\}_{i \in\left[1, \infty^{n}\right.}\), and a set of \(B\) bins to discretize the time dimension, the timestamps are scaled to the range \([0, B-1]\), and the event volume is generated as:

\begin{aligned} t_{i}^{*} &=(B-1)\left(t_{i}-t_{0}\right) /\left(t_{N}-t_{1}\right) \\\
V(x, y, t) &=\sum_{i} p_{i} k_{b}\left(x-x_{i}\right) k_{b}\left(y-y_{i}\right) k_{b}\left(t-t_{i}^{*}\right) \\\
k_{b}(a) &=\max (0,1-|a|) \end{aligned}

where \(k_{b}(a)\) is the bilinear sampling kernel.

<../biblio.bib>

Links to this note