External program crashing unity

We have been investigating a problem where our open source audio engine (Pure Data) fails to start correctly, and unity entirely crashes (not the editor). Once PD has failed to load correctly, starting vinyl will only result in a crash. So we are working on getting our game playable on more computers.
For a lot of computers manually killing PD in the task manager and then restarting vinyl will make the game work. If we fail to connect with PD we kill it, starting vinyl when PD is broken doesn't kill it though. So we believed that we were unable to end the process.
However running the kill command (Process.kill() in C#) does seem to work. So now we believe unity is crashing before it gets to the point where it kills PD and tries to restart it.

There is code that could cause an exception early in the initialization process. The biggest thing we are wondering about is why Unity would entirely crash if anything there went wrong. For most of unity if an unhanded exception is raised it will just stop the thread and continue the rest of the game. However for some reason an error in the initialization of PD is cause the entire system to crash.
So we are going to look at making the code safer (try catch blocks) and hopefully we can make it more recoverable from an error.

Game Graphics Assignment 13

Last assignment of the class and we were able to choose something cool to do.

I chose holograms as the cool thing I would try to make. Since I started learning game graphics I have been interested every time I have seen a hologram in a game. I often will stay there in the game and watch it, trying to figure out what they are doing.

So for this assignment I decided to look into it and see if I could make something like it. After a short time goggling hologram shader I found a nice post about how to implement one, step by step. Although the post was not about how to write it as a shader but was instead how to implement it in some specific GUI application. It still showed some specific steps that gave the effect I was looking for. From there I would look at it and try to see how to implement that effect in code.

For each of the parts I was able to figure out a way to implement it.

Link to post about a hologram shader

There are 5 major effects that I implemented to get the final look.

1. a very simple one, I used a solid color. So I started with setting the color to float3(0.05f, 0.84f, 0.33f). This was a simple first step.

2. I calculated the fresnel effect and used that to do transparency. This makes the edges not transparent and the parts facing you directly are transparent. You can see the effect doing this gives.

Before Fresnel transparency

After Fresnel transparency

3. I added static to the color to add in the randomness. I wasn’t sure whether I wanted to add the static to the color of the alpha. But I think it looked a bit better adding it to the color instead of the alpha. However as a programmer I wanted it to really show, if an actual artist had this they may want it to be using for the alpha because it was more subtle with the alpha.

For static I used code sent to me by JP, it worked well. Although I think if I were doing this for longer I would look at trying to get static with spatial dependency. This static is completely random. Where I think the static could look better if there were lines of static and areas of static rather than per pixel random static.

Here is the function I used for static.

0.5f + (0.5f * frac(sin(dot((i_texcoord + g_secondsElapsed), float2(12.9898, 78.233))) * 43758.5453));

I multiplied that output with the color.

4. For this I added lines that move through the hologram. For a long time I was multiplying the line by the color, so that the line was a dark line going through the mesh. However I realized that didn't make as much sense as doing it with the alpha. Once I tried that I liked it better. So these lines are lines of 0.0f alpha that move through the mesh.

I did a lot of things to get these lines. One thing that I did is that the line is based off of the pixel's world position. I weight each of the position values (x, y and z) and use the sum of them in the calculation. This way it is based off of the position, so the line will move along the mesh. I can also control how the line moves, although the x and z are fairly arbitrary, I can make sure lines are correctly moving up and down the mesh. And the x and z are then used for more random lines.

I ended up making a function to create these lines that would take in all the parameters needed. This way I was able to have multiple lines easily (I ended up with 5).

Here is the function

float CreateRandomLine(in float3 i_worldPosition, in float3 i_positionWeights, in float i_secondsElapsed,

  in float i_speed1, in float i_speed2, in float i_linePercent, in float i_thickness, in float i_randomNumber = 1.0f, in float i_percentOfTime = 1.0f)

{

  float l_randomLine = saturate(frac((abs(((i_worldPosition.y * i_positionWeights.y) + (i_worldPosition.x * i_positionWeights.x) + (i_worldPosition.z * i_positionWeights.z)) - ((i_secondsElapsed - 100000.0f) / i_speed1))) * i_speed2) / i_linePercent);

  l_randomLine = saturate(l_randomLine * i_thickness);

  return saturate(l_randomLine + PercentOfTime(i_secondsElapsed * i_randomNumber, i_percentOfTime));

}

I built the function up slowly. It ended up being quite complicated. But it wasn't too hard to make because I was just working with one part of it at a time.

Here is an example of me using the function.

l_positionWeights.x = -0.5f;

l_positionWeights.y = 0.0f;

l_positionWeights.z = 0.2f;

l_finalAlpha *=  CreateRandomLine(i_position, l_positionWeights, g_secondsElapsed, 0.5f, 0.2f, 0.2f, 20.0f, 1.66f, 0.4f);

5. This was the final thing that I did. I wasn't sure if I was going to try it, but I figured out a way to do it while at home and so I wanted to try it. This was adding position distortion lines to the hologram. So this would be in the vertex shader. This is much the same as the alpha lines, but instead they will move the position of the vertices. This gives the look of the hologram distorting and I think it really helped.

I used the same CreateRandomLine as before. But instead of multiplying it by the alpha I multiplied it by an offset and then added that to the position.

l_positionWeights.x = 1.3f;

l_positionWeights.y = 0.4f;

l_positionWeights.z = 0.2f;

l_positionOffset.x = 0.1f;

_positionOffset.y = -0.1f;

l_positionOffset.z = 0.0f;

l_positionOffset *= (1.0f - CreateRandomLine(i_position, l_positionWeights, g_secondsElapsed, 0.45f, 0.35f, 0.5f, 50.0f, 1.378f, 0.6f));

l_position += float4(l_positionOffset, 0.0f);

Again with the function I was able to more easily make more of these. I ended up with 4 of these distortion lines.

Overall I like how it turned out. I can tell a programmer did this because all of the effects are overemphasized. I feel like an artist would turn down a lot of what I did. But it looks good and it was fairly easy to manipulate.

Link To Code

Finally work on vinyl

The game keeps getting changed, although I think it has all been for the better. However we need to very quickly decided what we are trying to make, and although some specifics may change as we implement prototype them. We need to at least know what we are trying to put in the game. This weekend Mike is going to put together a design wiki so we can all have a reference for what we are building.
I have never had a design doc or wiki that the team has actually followed, but I hope that was because everything else I have worked on was a prototype.

A lot of the changes to vinyl come from GDC where we learned some ways that we can manipulate the audio in positive ways. So then we wanted to change the game such that it had positive audio effects built into the gameplay.

Another inspiration for change was a talk given by Gordon Durity of EA Sports. SSX does a nice job of taking the user's music and integration audio manipulation with gameplay. Another thing they did was they had an inspiration video made that was given to the production team. The video was of real snowboarding and had music added on. The development team used the video as the base for the feel they wanted the game to have.
I quickly put together a video similar to it. Although it was not just showing off the feel but also showing some mechanics I was proposing. Specifically a pause with a handstand, remixing when grinding and lowering the volume in the air.
Link to Video

The video needed a lot more time put into it. But I think it helped smooth the transition from running down a pipe dodging balls, to the more interactive mechanics with audio manipulation tied to each.

Game Graphics Assignment 12

I hadn't done the directional lighting before this assignment, so that was the first thing I had to do. With that I wanted to use functions for my shaders. Although I did not do this for every part of my shaders, specifically the vertex shaders, I did convert my fragment shaders to use functions.

This caused some problems with lighting, but eventually I was able to refactor my lighting to such a way that I could use modular functions to do the lighting. The one problem that I found after doing this was that if I made a change to one of the function files, it would not recompile the shader. So when I was playing with the epsilon of my shadows function I would have to go into a shader and add/remove a space, save and then run the game. So if I were to do this again (presumably for something more than class) I would find a way to get a dependency graph working for the shaders.

There were plenty of minor mistakes I made with the code, but overall I had no major problem with the assignment.

One thing that helped with the assignment was the GUI. I changed my GUI item to show the shadowmap texture which allowed me to see the lights perspective at all times. This wasn't a life saver, but it was really easy to do and it was helpful.

Here is a screeshot of pix for the shadow map texture.

Everything should be working.

Link to code

Game Graphics Assignment 11

Hey,

This was a really easy assignment overall. it only took like 2:30 hours. But i had already drawn on screen space from game engineering.

I made a GUIQuad that had all the information needed to draw a quad on screen then made some helper functions to get the transform matrix i needed. Then i made a function that can draw a quad given that struct.
At that point i made a few super simple shaders and switch between them based on some info in the struct.

So with that i could draw the post processing effect which was just a quad the size of the screen with the screen texture, and then i could draw the gui texture which was a smaller quad with the alpha test shader.

I didn't do the directional lighting because i had done it in game engineering and i am crazy busy, so any time i can save i am taking.

Evecything should be working
Link To Code

Game Graphics Assignment 10

This assignment ended up taking longer then it should have, but i am not too surprised because part of it was done on the flight home from montreal and the rest was still before my brain had recovered from Ubisoft and GDC.

The one mistake that took a lot of time was that when i was getting the index of a sampler for the shader, i was then just calling set texture even if it returned (uint)-1. This was causing pix to crash and eventually i just had to go in search of why pix was crashing to continue working on the assignment. I am not sure why it was causing problems now, i had that code in there for a few assignments. But now pix works again.
I had talked with people who were using the projected position instead of the view position, which i implemented first. However after an email from JP, i changed it over to use the view position. It makes no visual difference (that i can see) but it should be cleaner in all situations, where the projected position was a non linear depth.

Everything should be working.
Link to code

pix view of the depth texture, it is only showing the red color (because the texture is just R16F).

Data Visualization Tool

For Vinyl we will be doing a lot of audio data parsing and manipulation. This means we will be looking at large arrays of values and making changes for them. So to do this I created an array visualization tool that can take an array and visualize it along a graph. I also wanted to be able to scroll across the data, zoom in/out and mouse over a specific value. I also wanted to be able to do this for multiple arrays either on different graphs or the same graphs.
For this I first looked for tools that already did this. The main one I looked at was Excel. But i found it would have been a challenge to dynamically make the graphs based off of the data, and different graph combinations of arrays.
Another tool i found was a C# project called ZedGraph that did a nice display of an array, and it was open source. So I used that to display the graphs. With that my tool can now display one graph with N number of arrays on it. It also can display 4 graphs at the same time.

It overall works well, but I am running into one problem I will need to look into the ZedGraph code to fix. The problem is that I am trying to draw hundreds of thousands of values, and it is slow. So i need to change the code so that when you are zoomed out far enough that you are trying to draw too much data, it will only draw some of it. The dission i have is how to bring down the data. I could display every X values. so if i want to show 50 data points and have 170. i could just display every 4th point. The other way is to average the values around it.
I think I will take the every X values so that i don't end up averaging the Waveform into 0. But I will see how it works.