|My avatar in-game and me slashing some flying cubes!||Here's a FBT avatar video tutorial|
In order to use an avatar with CustomAvatar plugin though you need your avatar in a specific format - .avatar. You can't simply convert a VRM into .avatar though so in this guide I'll show you what needs to be done in order to get .avatar ready for full body tracking. You can have your avatar in another format, it needs to be importable into Blender. I create VRM avatars so I'll be using that as an example.
First you need to export your model from VRoid Studio. In order to be able to work with it I had to choose that anyone can act as this avatar.
Now what you need is program named Blender. You can download it from https://www.blender.org/download/. Then you will also need CATS Blender plugin which will do all the work needed. Make sure to download the correct version depending on your version of Blender. The plugin can be downloaded from https://github.com/michaeldegroot/cats-blender-plugin/releases.
Launch Blender and navigate to Edit -> Preferences. Click on Istall button and find the zip archive you just downloaded. Install it. After the plugin is tested it will show in the plugin list, active (ticked). Now if you go to File -> Import, you should be able to see a new import option for .vrm (or you can click on Import in the CATS plugin).
Delete the three objects in the scene since you won't need them (just select them with your mouse and hit delete) and import your previously exported VRM model.
You should be able to see your model but with an armature that looks broken. Access the CATS blender plugin. Press N and then find CATS tab and click on it. Click on Fix model. The model should show textures and you should be able to see correctly looking bones.
At this point you can export the model, but if you want your avatar to actually hold the sabers in game you should Start pose mode and create closed fists. Rotate bones on one of the avatars hands. What I do is tick the x-axis option under options to work on both hands at once. If you forget to do that, simply select the bones you rotated, CTRL+C. Then select the same bones on the other hand and press CTRL+SHIFT+V. After you create closed fists (note that the fingers should be closed around imaginary saber) you can save pose to a shape key. Then Clear all transformation (right-click the view) and Stop the pose mode (both of these are in the CATS plugin tab). The shape keys can be accessed in Unity by clicking on your prefab's body in the hierarchy window under Skinned mesh renderer in the inspector. (If you are creating an avatar for VRChat you can go to Eye tracking and Visemes and create more shape keys if you want your avatar to be able to blink, have eyes tracked and be able to lip sync. You need to choose closed eye visemes for the eyes and choose 3 corresponding mouth visemes (A, O and CH) out of which 15 will be generated).
Export your model through the CATS plugin options or File->Export. The plugin will tell you that you should have max 4 textures if you use more (I don't know if this is a standard but VRChat SDK will show you an error if you have more materials than 4, my avatars with 14ish textures worked just fine but their performance was lower than when using just 1 texture. Anyway it's easier to work with less textures so I suggest you do this).
Under Optimization in the CATS plugin there you can atlas your materials if need be (another plugin needed). Under Material tab you can combine the same materials (which should have already happened during 'Fix Model') into one and convert them to png if need be. Under Atlas you can atlas all or just selected materials to one single texture. First you need to click on generate list and then you can select which textures to atlas. Then click on Save atlas and choose the location where to save it. It won't be part of the fbx you will be exporting so you will have to manually copy it to the Materials folder in your Unity project. Creating an atlas saves a lot of work in Unity where you have to set all shaders to Beat Saber ones and add the textures (note: if your model looks dim even with unlit shaders it is most probably because Unity defaults to gray color for base color - change it to white).
If you model has more triangles than 70 000 you should decimate your model. That can be found under Decimation. If your model exceeds the number of triangles you should do it before creating any shape keys. There are several options 'Safe' which won't affect any shape keys, 'Half' which will break some and 'Full' which will completely get rid of all shape keys. Custom allows you to decimate parts of your model. The optimal number of triangles should be below 32 000.
In order to be able to create avatars for Beat Saber you need to download a specific build of Unity. Install Unity Hub and create an account and choose a license. After that navigate to https://unity3d.com/get-unity/download/archive and download Unity 2018 1.6. via the Unity Hub.
Then download the avatar template project from https://bs.assistant.moe/Avatars/resources/UnityProject.zip. Create Unity projects folder somewhere and copy the project there. In Unity Hub click on ADD and choose the folder with the project. Then you can click on the project and it will open (make sure that it opens in the correct Unity if you have more versions installed).
Right-click in the Assets folder (or choose Asset->Import New Asset) and import your exported fbx via Import New Asset. Then click on your model in the Assets and check the Inspector.
Click on a rig tab and change it from Generic to Humanoid. Keep the 'Create from this Avatar' and click on apply. It should be already configured, if you click on Configure you should see all the bones mapped correctly and showing green. If they are in red you'd have to manually drag the bones to the correct places (or try Automap from the Mapping options). It's also possible that some bones will be misplaced so delete those and assign them to the correct places. Unity will show you a red error in the console window and won't let you proceed until you fix this.
Then click on Materials and link your materials. Click on the little arrow next to your model and it should show you the armature and body and whatever else is in the fbx file. Click on the mesh of the body. In the inspector tab if you scroll down you should see a list of materials. Here you will need to change the shaders to BeatSaber shaders otherwise your avatar will be just pure white in the game. You'll be using two shaders UnlitGlow for textures without transparent parts and UnlitGlow Cutout Dithered for those that have transparent parts. If the texture doesn't show after you change the shader you will need to load the texture (if you created an atlas you will need to do that). If the shader from your model doesn't have a base color set, it will default to light gray in Unity. Usually what you want is to set it to white (if your avatar looks dim, that's the reason why it happens).
After this you can start the process of creating your game object. Create an empty game object (right click in the Hierarchy window) and set its position to 0,0,0. Rename it to the model's name. In my case aurora. Then if you don't need full body tracking add 4 empty game objects in the game object you created. Name them Head, Body, LeftHand, RightHand. (FBT needs Pelvis, LeftLeg, RightLeg in addition to these)
Create a cube and set its height to a human height. In my case I set it to 1.8. Set the position so the bottom of the cube is at 0,0,0. That means you'd have to position it to y=0.9 in my case. Drag your model into the scene and position it to 0,0,0. Then scale it to desired height. In my case it's anything between 1.6 and 1.8. My model's size is 1.6 but I scaled it a little bit (to 1.07) - my flatmate is higher than me. You can delete or hide the cube (hiding is done by unchecking a box in front of its name in the inspector tab).
You will need to place and rotate the created objects. Body object can stay where it is. Click on Head and move it to where your HMD would be. Somewhere around the eyes/nose with Z axis going forward, Y axis going upwards. Right and left hand objects should be slightly in front of your hands. With axis Z going forward (along the saber blade) and Y towards the wrist. Pelvis and feet trackers are placed and oriented depending on the trackers you use. For feet trackers it usually means that they are placed on your instep, with Z going upwards, Y going forward leaning down towards your toes. If your pelvis tracker is on the back like mine, the Z axis goes behind you, Y goes up. Note that for full body tracking working correctly you may need to edit your rig first so the bones are at correct places and correctly rotated. (I had a hard time figuring out what is correct since the original rig I had didn't work, then I decided to see if I can make it work in VRChat. What works in VRChat works usually in Beat Saber. Often setting roll on bones to 0 fixes leg problems. In armature edit mode, click on a bone, select everything pressing A and press ALT+R) (You can check this video out Full Body Tracking Fixes)
Then drag your model in the Hierarchy window to your game object with its name. Now for each of your objects you need to find the corresponding bone and create an empty game object there. You then drag the object into the respective object you created before. More specifically. You will navigate in your armature to head and create an empty object there. You will drag this object into head and rename it to HeadTarget. You'll do the same for the remaining objects you have naming them LeftHand, RightHand, PelvisTarget, LeftLegTarget and RightLegTarget.
When you have all objects and targets in place you will need to add two components to your model. Click on the root of your model in the Hierarchy window and add VR IK and IK Manager Advanced components. Then link the Targets in the IK Manager Advanced with the ones you created. You can either drag them there or choose them from a list after clicking on the dot next to the empty field. You can leave the VR IK empty because it will populate on its own. Sometimes it doesn't and you have to add the bones yourself. If you are creating an avatar for full body tracking uncheck 'Plant Feet' in the VR IK component and set Leg_Position Weight and Leg_Rotation Weight for each leg to some values in the IK Manager Advanced. They should be closer to 1 so when you move the legs they would follow or rotate accordingly.
At this point you can see if what you did actually works. Click on Play and move the avatar's head around to see if it moves the avatar.
If everything works click on the root object and add two components to it - Avatar Descriptor and Event Manager. Fill out the info in the descriptor. In order to get a thumbnail there you need to have a picture in your assets with the texture set to sprite (2d and UI) and then you can drag it into the Descriptor component.
Then all you need to do is click on Avatar Exporter and click Export. If you'd want to reexport the avatar into the same file name note that you can't. It won't rewrite the file so either delete the previous file or export with a different name.
In order to get your avatar in the game you need CustomAvatar plugin that can be downloaded from https://github.com/nicoco007/BeatSaberCustomAvatars/releases. I also had to install DynamicOpenVR from https://github.com/nicoco007/DynamicOpenVR/releases in order for the plugin to work. If by a chance the plugin won't show up just copy it several times. Don't ask me why it doesn't work the first time, it works, it just may need several attempts at installing. You will need to copy your avatar into CustomAvatars folder in the Beat Saber folder. After that you run Beat Saber and choose the Custom Avatar mod settings. There you can choose your avatar and see if it works. Resize the model so it fits your height or arm span otherwise it may not work properly. If you want to use your avatar in first person view your avatar should also have First Person Exclusion component added. There you set size to 1 and choose the element to exclude which would be the head. For this to work the head mesh needs to be separated from the body (you can do this in Blender in Edit mode. Select the head mesh, press P, click on separate by selection).
If you download an avatar that is ready for full body tracking and you have it activated you may see your avatar be twisted when you choose it. This is something that seems to happen many avatars you download and does not seem to get fixed with the plugin's calibration. It can be fixed via Open VR Input-Emulator that can be downloaded from its GitHub page - https://github.com/matzman666/OpenVR-InputEmulator/releases. Close SteamVR before installing OpenVRIE. After it is installed it can be accessed from the SteamVR dashboard. You need to click on a grayed out cogged wheel. In the OpenVRIE settings you will need to change the Device Offset of the hip tracker. It is possible that you may need to fix other offsets too. The avatars may not be properly set up for FBT (usually pitch for the feet trackers) or they are set for different trackers.
If you want to record or stream so people can view your avatar you can use CameraPlus mod that is available from ModAssistant (but you can manually copy it too, for some reason the modassistant is giving me a hard time too). CameraPlus is operated from the Beat Saber window. You right click on the running Beat Saber window and it should show up. There you can add cameras, move them around, change their angles and position etc. I recommend saving it to a profile. Unfortunately once in a while it has to be re-reloaded.
This page wouldn't exist if chachakoala wouldn't help me with setting up full body tracking and then telling me how to get the mods working since it seemed I was running into a wall at every single step. I'd like to thank Tachrayonic for answering my questions about targets and objects.
Thanks the community for creating these mods and everything else that makes Beat Saber enjoyable for players and viewers!
Ren (stsungjp @ twitter)