Abstract

Predicting scene dynamics from visual observations is challenging. Existing methods capture dynamics only within observed boundaries failing to extrapolate far beyond the training sequence. Node-RF (Neural ODE-based NeRF) overcomes this limitation by integrating Neural Ordinary Differential Equations (NODEs) with dynamic Neural Radiance Fields (NeRFs), enabling a continuous-time, spatiotemporal representation that generalizes beyond observed trajectories at constant memory cost. From visual input, Node-RF learns an implicit scene state that evolves over time via an ODE solver, propagating feature embeddings via differential calculus. A NeRF-based renderer interprets calculated embeddings to synthesize arbitrary views for long-range extrapolation. Training on multiple motion sequences with shared dynamics allows for generalization to unseen conditions. Our experiments demonstrate that Node-RF can characterize abstract system behavior without explicit model to identify critical points for future predictions.

overview

Long-term extrapolation of single sequence

Node-RF learns from a single dynamic scene and predicts the motion evolution beyond the training window. By applying a neural ODE to the full state, the model inherits an inductive bias toward physically plausible dynamics.

D-NeRF
4DGS
HexPlane
MotionGS
TiNeuVox
Ours
Node-RF


Generalization to novel motion sequences

Node-RF learns the underlying dynamics of a scene from multiple motion sequences starting from different initial conditions (position and velocity) under shared dynamics. The learned dynamics enable the model to generalize to unseen initial conditions and generate novel motion sequences for those configurations.

Oscillating Ball

D-NeRF
Ours
Node-RF
GT


Bifurcating Hill

Vid-ODE
SimVP
Ours
Node-RF
GT

Citation

Acknowledgements

The website template was borrowed from Michaël Gharbi.