Sunday, December 22, 2019

Machine Vision mashup Just Dance and Fortnite into one dance game (Article)

GeekCon makers' conference is a place where anything can and should happen. its a place for geek dreams to come true.

At Feng-GUI, we love reshaping the box and see how machine vision can be applied to more interactive implementations.
So, We've created a Just Dance game with Fortnite dances and emotes




Fortnite Battle Royale is an online multi-player shooting game. Fortnite player are capable of dancing, and their dance moves are spreading worldwide as a plague. The dances are also referred as Emotes, Dances or Dance Emotes in Fortnite. see http://orcz.com/Category:Fortnite_Battle_Royale_Emotes

Just Dance Fortnite mashup both games into a single game.
In this rhythm game, the player must mimic the on-screen Fortnite dancer's choreography. The background song and feedback is a chosen Just Dance song. The body pose analysis is done in realtime using OpenPose MobileNet deep neural network.

The player stands with a green screen background and has to mimic the dancers of Fortnite.
In front of the player, a screen contain 3 layers.
At the right side, the fortnite dancer. At the left side, the player.
The background animation and music is taken from Just Dance game video songs.


The game created as a plugin inside Open Broadcaster Software (OBS).
Fortnite videos are loaded to right pane.
Just Dance videos are loaded as background.
Player camera is loaded to left pane with Chroma Key (green screen) effect.


photos by ilan levi http://www.ilanlevi.com



OBS filter pose detection and matching implemented using OpenCV and SIFT matching of OpenPose MobileNet deep neural network.
OBS Filter Source code at https://github.com/Feng-GUI/experience/tree/master/JustDanceFortnite