Amazing work Usman M. Khan pushing our research forward in world models for the tough outdoors!
Excited to share my new preprint on depth-regularized JEPA World Models! This is our (humble) first piece of AI research at Aigen that we're sharing publicly, with hopefully bigger and better things to come. In industry, a lot of what you work on is by nature proprietary so it's pretty nice to be able to put something out there! Hopefully the community benefits and finds it interesting. This work represents a pretty simple idea: using 3D structure (depth) as a prior seems to help JEPA world models learn better physics, at least in some scenarios. The age-old debate in deep learning about inductive bias vs pure deduction is these days mostly leaning towards simply scaling dataset and model size. Here we show that sometimes (especially when you need a compact model) this type of prior actually helps generalization. My favorite result here is that providing depth as a 3D geometric prior seems to help even outside of things that have to do with geometric structure, like the model's conception of how lighting should behave. I suspect that the JEPA formulation lends itself well to this type of prior, pushing the model to more sensible representations.. maybe more evidence that JEPA is the right way to truly teach a model to understand physics? Link: https://lnkd.in/e3eWzDW8 I probably will not have the time to do the last-mile work to turn this into a conference-worthy submission but willing to collaborate if someone wants to! Thank you to Royden Wagner for the endorsement to post in cs.LG