Just found out Stable Diffusion can generate 3D models from 2D images now

I was reading a blog post from Hugging Face yesterday and stumbled on this new technique where you feed a regular photo into an AI and it creates a full 3D mesh you can rotate around. I tested it with a picture of my dog and the result was rough but honestly impressive for a free tool. The article said it uses something called neural radiance fields under the hood. Has anyone else played around with converting pictures to 3D objects yet?

2 comments

2 Comments

grant.jason16d ago

Tried it with a landscape photo I took on vacation and the depth was super wonky in some spots but the overall shape was surprisingly solid for something running on my laptop. I think the tech is still early but once they iron out the kinks its gonna be huge for indie game devs who dont have a 3D modeling budget. The nerf stuff is interesting but the real breakthrough will be when they can do this with video frames instead of just single photos.

charles_coleman16d ago

That part you said about it being rough on a laptop really hit home for me. I tried the same thing with an old photo of my grandparents' house and the edges were all blurry and weird in a bunch of spots. It's frustrating because you can see what it's trying to do but the execution just isn't there yet for everyday use. I feel like we're all beta testers right now and that's okay but it still stings a little when you get something that looks like a melted toy instead of a real model. You're right that once they fix those kinks it's going to change things for people who can't afford ZBrush or Blender training. I hope they figure out the video part soon because that would be a game changer for sure.