VMC, LIV and Mixed Reality

Note: this video shows a dynamic tracker for the camera and not a static one as shown in the screenshot below

VirtualMotionCapture can be used to track your VRM avatar (If your model is not in VRM format, import it in Blender, fix it with CATS Blender plugin and export it as VRM). To create a mixed reality recording of a game another software needs to be used. The software I use for compositing is LIV and the software for recording I use is OBS.

At first I used LIV and VMC to record and stream Beat Saber. The reason was that I didn't want to mod the game. So in theory using VMC and LIV is an upside because you don't need to mod Beat Saber and you don't need to create an avatar that works with CustomAvatar. If your game is already modded it's better to use CameraPlus + CustomAvatar or VMC because that combination is more friendly for the CPU. If you want to use LIV + VMC in Beat Saber note that you cannot have CameraPlus and CustomAvatar installed. You can either use just LIV, VMC + CameraPlus, VMC + LIV or CustomAvatar + CameraPlus.

Start SteamVR, then run LIV and lastly run VMC. I suppose that you use SteamVR already if not install it and run it. Then close it. Download LIV from the Steam store and run it. Click on Install LIV SteamVR plugin. Close LIV. Run SteamVR. Make your trackers active and then run LIV.

Download VMC from booth.pm or its GitHub page and unzip it somewhere. Run the program. A control panel will open. Click on Open VRM. There you can either open a file you have on your hard drive or open a model uploaded to VRoid Hub. Click on Agree/Import and then your avatar should show up in a separate window.

The camera view is being prepared in VMC so click on camera on the control panel. Click on Free and then move your avatar to the desired location (or the camera rather). In order to move your avatar you need to press and hold the middle mouse button (you can also hold Alt + Shift, right mouse button to rotate around the center of the avatar).

When you are done with your camera view click on settings. In the Controller Number dropdown list choose Tracker (LIV Virtual Camera) and then click on Export externalcamerqa.cfg. While in the settings window check Enable VirtualCam. You will have to install it first and register the dlls using regsvr32 command. (in order to do that hit Start, and type cmd, right-click the shown cmd and run as administrator. Then type regsvr32 "[path to the dll]". Register both dlls. Then you should be able to enable the VirtualCam.

In LIV click on Camera and add an camera using the first button. Choose VMC_Camera as the device. Choose the output resolution. If you don't see the correct resolution change the color mode. Set it to static. Save it and then click on the edit button (second button from the left). There click on Calibrate. If you don't see import/export option tick Details in the upper right corner of the window. Import the file you exported in VMC and click on Save.

Now you are almost all set. What you need to do in order to make your avatar move is to calibrate it. Click on Calibration on the control panel. Choose one of the options and then make a t-pose so that the white spheres (trackers) match the places where they should be placed on your avatar's body (HMD in your head, hand controllers in your hands etc).

When you are finished with your pose press the trigger and stand still for five seconds (for some reason when I t-pose I need to have the trackers slightly in front of the avatar inorder to get it tracked correctly). Your avatar should now move.

Go to Background settings again and then click on greenback background (if you haven't done that already).

If you want your avatar to move lips click on LipSync and choose your microphone from the device list.

In LIV click on Sync and Launch if you haven't done that already. Then open OBS and add a new source - the LIV output window - to your scene (this should be now showing the VMC avatar and Beat Saber launched by LIV). You should see that the background is green. In order to get rid of it you need to add Chroma Key filter to the window capture. Right-click the Window Capture in question and click on Filters. Add Chroma Key and choose Green in the Key Color Type dropdown list.

Note that LIV outputs only video so if you want sound in your recording you will have to add an audio source as well. I wrote about OBS settings in more detail in Setting up LIV for Mixed Reality.

Now you should be all set!

Thank you for reading
S'Tsung (stsungjp @ Twitter)