zhu_unsupervised_2018: Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion
Unsupervised Event-based Learning of Optical Flow, Depth, and Egomotion
Contributions
The authors propose a new input representation that captures the spatiotemporal distribution of events, and a set of unsupervised loss functions that allows for learning of motion information only from the event stream.
Input Representation
Given a set of \(N\) input events \(\left\{\left(x_{i}, y_{i}, t_{i}, p_{i}\right)\right\}_{i \in\left[1, \infty^{n}\right.}\), and a set of \(B\) bins to discretize the time dimension, the timestamps are scaled to the range \([0, B-1]\), and the event volume is generated as:
\begin{aligned}
t_{i}^{*} &=(B-1)\left(t_{i}-t_{0}\right) /\left(t_{N}-t_{1}\right) \\\
V(x, y, t) &=\sum_{i} p_{i} k_{b}\left(x-x_{i}\right) k_{b}\left(y-y_{i}\right) k_{b}\left(t-t_{i}^{*}\right) \\\
k_{b}(a) &=\max (0,1-|a|)
\end{aligned}
where \(k_{b}(a)\) is the bilinear sampling kernel.
<../biblio.bib>