Radiant: Post CVPR
We're now officially past CVPR, with SIGGRAPH rapidly approaching. The amount of NeRF research has been staggering.
For those in the US, hope everyone has enjoyed their long weekend and July 4th.
On the newsletter front, I have been busily building several resources for the NeRF community, which are all expected to launch in the immediate future. I can’t wait to share more!
NeRF Research
The amount of progress and attention NeRFs have been generating within the academic community is plain to see. It’s also incredibly tough to read everything. With that in mind, here are some of my favorite papers from the first half of this year.
Language Embedded Radiance Fields (LERF)
I truly don’t think I can state enough about this paper. The amount of potential use cases this unlocks is nearly unfathomable and while I don’t believe by itself will bring large scale disruption, incorporating it with another method will supercharge any NeRF.
It also brings me great excitement that semantics are just one avenue that people can pursue with NeRFs. Matt Tancik gives an example in his lecture at MIT here.
Neuralangelo
The great photogrammetry killer. Not exactly, as Jonathan Stephens and I took a look at, but it does represent a major step forwards. If you want to try out Neuralangelo for yourself, SDF Studio (through nerfstudio) now supports it.
DynIBaR
Just look at this video.
I had to watch it several times over to make sure I wasn’t looking a duplicate video of the Groundtruth. It was originally published earlier this year, but has attracted a bunch more attention recently through the help of some well timed Twitter threads.
HOSNeRF
Similar to Dynibar, dynamic NeRFs (NeRFs with motion) are coming and with them will unlock amazing new possibilities. Keep in mind these are coming from just one input video. The code hasn’t been released yet, but this is one right at the top of my list to try.
HumanRF
This paper from Synthesia, who recently achieved unicorn status has rippling downstream effects. I believe there is a future where technology similar or built off of HumanRF is consumed in people’s living rooms, stadiums, or at events. Think of watching a photorealistic 3D fashion catwalk, enjoying the latest Hasan Minhaj interview with former President, Barack Obama, or a MKBHD tech review right in your living room.
Binary Radiance Fields
Compression is finally starting! BiRF achieves the significantly smaller file sizes using binarization to ascribe a +1 or -1 to encode local features. As you can see, the resulting file is minuscule, despite a higher PSNR score.
View Dependent Podcast
For those that prefer audio based content, I have started a NeRF podcast called View Dependent with Jonathan Stephens.
You might also have watched Jonathan Stephens’ tutorials on how to install Instant-NGP, back in the early days of NeRF.
We have recorded two episodes thus far and have some amazing episodes and guests lined up!
episode 1:
We looked at Neuralangelo and what is it/isn’t.
episode 2:
Coming very soon! We spoke to Fernando Rivas, co-founder of Volinga AI, about how NeRFs are affecting Virtual Production and the entertainment industry right now.
Feel free to send me any feedback or if you’re building something amazing with NeRFs and want to be a guest on the show.
SIGGRAPH
I am working on putting together a NeRF exhibit and panel at SIGGRAPH this year. We are currently looking for corporate sponsors for the event. If you are interested in learning more or would like to help offset the costs, please shoot me a message. All funds will directly go towards putting the event on and showing the amazing work people have done with NeRFs.
I will also be attending SIGGRAPH this year and would love to meet up and talk NeRFs. Feel free to send me a note!
Closing Out
Always feel free to reach out to me with any comments, questions, or feedback. If you hate something, let me know! If you love something, please consider subscribing if you haven’t already or sharing Radiant with someone you think might love it as well.
There are some truly amazing things coming very shortly and I cannot wait to be able to share them with you. I hope you all have an amazing rest of your week!
Michael Rubloff