Retopography and AI

Is anyone familiar with attempts/products/research, commercial or otherwise, to automate at least some portions of retopography using AI?

AFAIK ZBrush has an auto retopology but its not something you would use for like a main character, for example.

My background is in software development (including some light AI work) and from my limited understanding of retopography it seems like there are some tasks that neural networks in particular would be a good candidate for.

My first though is to take many medium-detail head models and train a network on their retopologized counterparts and see if its capable of creating anything like a reasonable head topology.

Now I don't think its quite this easy or we would have these tools already, likely it will take a couple layers of different networks working together to get anything reasonable.

Thinking aloud, a good place to start may be on fill topology, ie you go into retopoflow and set up your main edge loops and then get auto-fill on the rest.

I've been considering returning to university and am thinking AI retopology may be a good research topic. Any thoughts?

crew

Jonathan Williamson April 12, 2018 6:28pm

I suspect we will see a lot more research coming out on AI-drive retopology methods in the coming years.
My hunch is that machine learning will take over much of the heavy lifting, utilizing only initial input from the artist to define constraints and starting points.
One of the challenging with retopology, particularly from an automated standpoint, is intuitively understanding the deformation requirements the mesh must meet during animation.
Helping a machine understand somewhat standardized model parameters, like a human biped, is reasonable, but the moment you start introducing additional limbs and muscle structures it gets far more complicated. This is where I believe retopology will always need at least some artist-input in order to get animation-ready results.
- Report
Catherine Irkalla April 12, 2018 9:14pm

Thanks for the input! Yes I agree with you - this is why I'm thinking a good place to start is automated fill-in.
I do think it would take some trial and error to learn exactly which parts can be easily automated, but I tend to think a partially-automated system where the artist and AI work together is a good solution.
I'm thinking something like a system where the artist defines certain features up front and then an AI can offer several different options on how to connect those features with varying levels of completeness (ie perhaps it can fill in only 'obvious' parts, leaving the junctions to the artist as one option, or it can guess at where to place junctions in another)
I certainly don't think this is an easy weekend project, but I do think that this is an area I'm interested in exploring. Just what I need another project LOL well maybe this one can get me some scholarship money for college anyway ;-)
- Report