Metaphor3D

Text-to-3D side project exploring generative AI before Stable Diffusion existed. Built with Three.js and fusion models. Pre-PMF but the right instinct.

My close friend from high school, Zhizhuo, was deep in research on generating 3D assets with AI. He called me up with that energy you can’t ignore. He had the research, he needed someone to build the product layer. I said yes because the idea was genuinely cool and I wanted to see what AI could do outside of text.

This was the tail end of senior year at NYU Shanghai. Coopsight was winding down, Tildenn hadn’t started yet. A good window for a side project.

We explored different interface patterns first. A Discord bot? An email system where you send queries and get 3D assets back? We racked our brains before landing on a web app. The target was indie game developers and Roblox creators who spent disproportionate money on custom 3D assets or settled for generic stock.

We built an MVP using text-to-3D fusion models, wrapped in a Three.js viewer where you describe what you want and see generated meshes rendered in the browser. Zhizhuo handled the ML pipeline, I owned the web interface, the viewer, and the overall design.

Sometimes the most honest outcome of a project is realizing you don’t have product-market fit and moving on before you burn out trying to force it.

The tech was fascinating but the product-market fit wasn’t there. We couldn’t clearly identify who needed this urgently enough to pay for it. The models were too early. The output quality wasn’t consistent enough. We were building on capabilities that barely worked yet, and the “yet” part wasn’t moving fast enough for a real product.

Zhizhuo continued his research. I moved on to build Tildenn. But what I took from this: AI isn’t just text. There are entirely different modalities (3D, image, spatial) each with their own constraints. This was pre-Stable Diffusion, pre-everything. And learning to prototype fast in ambiguous technical territory, where the underlying tech might not even be ready? That skill carried into everything after.