My name is Stevie Vu and I'm focused on Virtual Reality. Follow this blog if you want updates on how my projects are going as well as what I'm learning.
Here is a prototype I made using VR to design a space.
Inspired from the Global VR/AR summit I attended last week. It was mentioned how 3D models will be used more in online shopping because it gives you a better sense of what you are buying than a 2d image.
If providing 3d models becomes the standard and widely available it would make it easy to use Augmented Reality or Virtual Reality to help consumers decide whether they want it or not.
VR Interactions and Save/Load System
Created the prototype using Microsoft MRTK for the VR interactions in Unity. The scrolling menu specifically was a very cool tool that's available through MRTK.
The save/load system was something I implemented for the first time. It was implemented through storing the furniture rotation, position and scale information in a game save data class. Using the JSON utility you can then transform the save data class into a JSON file as long as all the data information is serializable.
This is a collection of take-aways, thoughts, and notes from attending the 2021 VR/AR Global Summit panel on fundraising for your VR/AR initiative.
XR Job Series - Raising money for your project or company tips
Don't forget to look into government and corporate financing programs. This isn't a comprehensive list but I did some research and here are programs that I think local VR/AR businesses might be interested in.
Research and Development Tax credits. For example SR&ED.
Innovate has programs that can be beneficial
For this fall you might want to pay attention to The Canada Foundation for Innovation. They will be hosting a competition in fall 2021 for the 2023 Innovation Fund .
If your AR/VR project has a community focus or social benefit you should also consider corporate grants
For example Rogers Wireless provides grants to organizations who have a focus on helping youth.
Reach out to organizations who are aligned with your vision and could be strategic partners.
They might not have funding they can provide at the moment but there may be other benefits that you could leverage. Establishing these relationships before going to angel investors or VC would also be advantageous.
Have a 6-12 month plan for what you would do with funding. A panelist mentioned that he expects the businesses that he invest in to have a commercially viable product within 9 months. It's important to get gauge and get an initial reaction to the VR/AR product or service.
What makes your VR/AR initiative unique?
Your chances of receiving funding will be low if a quick google search of your idea brings up competitors who are similar and you don't have a differentiation factor.
Founder Dynamics. If you are not a solo-founder, you should be aware that they will try to evaluate your relationship and dynamics with your co-founders. This was stressed as an important aspect for several panelist.
A "No" might not always be a "No". One of the panelist who has raised a lot of money mentioned that for a lot of their deals the reason why they got it was because they didn't give up. Situations change and new developments for your VR/AR initiative may change investors mind and convince them to invest with you.
How COVID has impacted the way they look at investing in VR/AR
There is now a lot less resistance to investing in teams who are distributed.
If you were operating prior to COVID they will be expecting to hear how you utilize the increased interest in VR/AR.
Recently completed two VR projects since the last update. One was the Desert Climbing VR project. Here is a video along with my key learnings:
Desert Climbing VR Project
Key Learnings
Faster Prototyping - Use free assets
I made the 3D models myself and realized that it took up a lot more time then I thought it would in the end. Ultimately this slowed down the time it took to make a prototype of some of the features. It made me realize that it would be better to use free assets in the Unity store or other websites to add some basic visuals and move through the building process faster.
Dialogue system
I made the dialogue system myself and its triggered through listening to button inputs on the Quest 2 controller. The Dialogue uses scriptable objects which store or reference the text and images needed which are inserted into a basic UI Canvas. It was only after making it that I realized that the system I built makes it difficult to do things like display multiple images and etc. If I were to try it a second time I would try and use Unity's timeline instead which might provided a lot more flexibility and faster implementation. I enjoyed using Scriptable Objects and can see how they are a great tool for any game developer.
Biggest Challenge
It was my first time implementing a VR climbing mechanic and there were challenges dealing with character body collisions with rock protrusions as well as the hand poses themselves when grabbing the rock. Perhaps I'll find a solution in the future but I realized that automated hand poses for grabbing objects would be pretty useful in increasing realism. Turning off the hand doesn't really help since seeing your hands is a big part of climbing.
Adding Story
Adding story was a fun creative challenge. It made me think more deeply about the mechanics of the game and helped with setting up scenes.
VR Archery - Chinese New Year Inspired
I was thinking about what I should do for my next project and my wife gave me some inspiration. I decided to do a VR Archery game inspired by Chinese New Years. I'm pretty happy with it since I applied what I learned previously and it allowed me to gain more time on the game design itself. There's a lot more I could do to improve it as well.
Key Learnings
Decoupling systems
I used scriptable objects to connect systems and components but even now as I look back I realized I could have done a better job of that. This was the first time I set up a spawn system with enemies and used coroutines to do it. An unexpected challenge would be overlapping spawns which I could solve by having the spawn points have their own spawn manager instead of using just one.
I was excited that I was able to load levels using scriptable objects. This made making new levels with different parameters really easy.
XR Interaction Toolkit
I feel that I learned a lot about the XR interaction toolkit with this project when it comes to extending the features through overriding the base functions. Really useful and I can see how I could have avoided some spaghetti code in previous projects through using this.
Challenges
The stumbling blocks were challenges that I didn't realize would be there during the initial planning process. One for example was destroying game objects with XR grab interactables. Turns out that the current XR Toolkit throws errors when you do this and I found a solution on a forum post to clear the colliders which helped. Unique specifically to XR toolkit.
I tried Object pooling as well but because I used coroutines, when you turn the enemy inactive they ended up being respawned. I solved this through removing the enemy from the list after they spawn so that they don't return to the list as an inactive object.
This is to showcase my current process for approaching VR projects. Currently I want to create a simple short story driven VR experience while implementing a few new systems that I haven't done before in the past.
My first step is to decide on a project concept and then create a VR project scope.
Project Concept
For this project my concept is focused on a small adventure experience about a child who wants to help their grandma who is sick. I've developed a small storyboard about how I want the player to experience the story.
I do want to establish limits for the project which is that I want to finish it in 1-1.5 weeks. I try to keep that in mind as I work on the storyboard.
Concept/Story
The setting only has the child and the grandma. It'll be a lonely house in the desert. The grandma is the only family for the child.
I'm choosing a Desert because it removes they are naturally empty and reduce the number of art assets I'll need to make. It also puts visual emphasis on loneliness and hopefully strengthens the focus on family in the story.
A simple character design inspiration for the grandma character.
The grandma is sick and that is the motivation for the story. There is a magic necklace that the grandma owns which is supposed to help those who are lost. Reveals that there is a cure high up in the desert mountains.
Child climbs the mountain to get into a cave. The cave has a plot twist for the child.
Gaming Systems Needed
As a part of the project scope I then try to think of all the systems I need to facilitate the experience. I try to divide this into two parts after I make the list with a rough time estimate. *Usually wrong since I don't have a lot of experience lol*
The first half is core systems to create a minimum viable VR experience that can be completed in the shortest time period and then the additional nice to haves if there is time.
Core System
Scene Loading System(1 day)
Dialogue/Story Sequence/ Input system(2 days)
primary button A to quit game
primary button B to progress to next dialogue and story section
Story tracker
Climbing System (2 days)
Basic Art(1 day)
Sick Grandma in bed
Necklace
Mountain
Reward
Total Time : 6 days
Additional
Better Art Assets
Skybox - (2 hours)
Hand models
House in the Dessert Scene - Main Menu (2 hours )
Haptic feedback - (2 hours)
Sandstorm / Wind Effects (3 hours)
Medicine Interaction ( 2 hours)
Background music & Voice over (3 hours)
Action Scene
I really want to try creating a VR action scene. Might try this if I can complete everything else quickly. Doesn't add a ton to the story other then making it cool.
Last month I got a Oculus Quest 2 and I've been working on VR projects since. This is a quick summary of those projects.
VR Flower Explainer
Updated and recreated the Flower Explainer Project to work with the Quest 2 Input system. I used Unity's XR Input system to make game objects interactable.
VR Gun Mechanics
I wanted to create a VR Fire Extinguisher Training experience and thought that the mechanics for using a VR Gun could translate over pretty well. There are a ton of tutorials for how to create VR guns so these tutorials were easy to find.
VR Fire Extinguisher Training
Built using my skills from previous projects to create this. Main challenge became state tracking for the fire extinguisher as I wanted it to only work if you are holding it up in your left hand, unlatched the key holding it and holding the nozzle. I also used spatial sound for this one along with a video that's played in game which I haven't done before.
Unity Shader Graph - Toon Shading and other effects
Visuals are definitely an area of weakness in my skillset. I took a week to explore Unity's Shader graph in URP. Did some tutorials on toon shading which I think might be a way to improve the visual appeal of my projects moving forward. I've always loved cell shading and the art style of games like wind waker but I'm not sure I'll use toon shading for all my projects.
Regardless, after doing several tutorials I've found that the outline effect might be something that I'll keep as it can highlight what objects are interactable in a VR experience.
Been busy with moving homes soon and helping with renovations to try and get everything done soon. Hopefully all of that will be out of the way in a week or two.
Started this practice piece on trying to bring to life a warrior concept art by Satoshi Matsuura. My practice with anatomy sculpting gives me confidence with roughing out the shape for the character but a challenge is figuring out how to do all the armor pieces in a way that makes sense.
My learning outcome from the project so far has been using a lattice modifier to create the chest armor piece to bend the model non-destructively.
Previously I wanted to test out Camera projection mapping in Blender but it didn't work out because of the photo I was using. This week I tried taking my own photos and trying to replicate them in 3D and it worked out pretty well. I didn't spend too much time on modeling every tiny detail so you'll have to ignore the minor inaccuracies.
Regardless, camera projection mapping is a very useful tool as a shortcut to trying to do PBR texturing. I've still have a lot to learn when it comes to capturing details regarding roughness, metallic and normal mapping.
Character Sculpting - Learning human head anatomy Practice
