Make an MMD from scratch without any skills

 Overview

Even if you don't have the skills to create MMD dance motions and don't have much artistic interest in creating 3D models, your goal is to make your favorite character dance with your own choreography.

Although it is labeled as having no skills, the dance mmd motion is my own dance, and the 3D model creation is created with an editor (Vroid Studio) that specializes in character creation.

Basically, I collect articles that explain the technology.

create mmd  motion 

• 3D model creation :

1. Create 3D model in Vroid Studio
2. Convert Vroid model so that it can be moved with MMD

• mmd motion creation : 

1. film your dance
2. Extract MMD motions from dance videos on Google Colaboratory

• Create MMD :

1. Load 3D model, motion, music (wav file) to MMD
2. Convert to AVI file and you're done!

FFmpeg is used as an auxiliary tool when creating motion and MMD.

Create 3D model in Vroid Studio

A tool that allows you to create a character by drawing a picture for each part necessary for a character, such as hair, eyes, and body. As long as the color and shape match, you can look like your favorite character, so the threshold is quite low. There are people who have released 3D models and costumes, so there is also a diversion.

Convert Vroid model so that it can be moved with MMD

Vroid model cannot be moved with MMD in its original format, so it must be converted.

film your dance

You have to dance yourself to create the dance mmd motion you want to move. let's do our best.

If you get permission, you can use an existing dance video.

Extract MMD motions from dance videos on Google Colaboratory

It is possible to extract MMD motions from dance videos by machine learning on Google Colaboratory provided by Google, which is the core tool for creating MMD this time.

As long as you have a Chrome browser, you can run MMD automatic tracing on a computer without a GPU or a laptop, all on the cloud (collab)!

All data necessary for execution, such as source code, is stored in the cloud. MMD motion data, etc. of execution results are saved in Google Drive, so you can download them to your local environment.

The required time is 30 minutes for the introductory part, about 20 minutes for the preparation of the execution part, and about 120 minutes for the execution part. Please allow approximately 3 hours. It is also possible to divert the joint position data (JSON) to other applications.

Load 3D mmd model, mmd motion, music (wav file) to MMD

Start MMD, load the 3D model and motion created in the previous work by dragging and dropping, and play it to make the model dance.

Music (wav files) can also be read by drag and drop.