Monday, September 22, 2014

Expression Interpilation - The Mouth

One of the hardest things I'll need to tackle is figuring out a way to do expressions. Now, I could just make a whole bunch of presets, but that's not the way I like to do things. Instead, I'd like to come up with an algorithmic way to create expressions.

Focusing on the mouth, two things immediately come to mind: the first is that the muzzle changes shape depending on if the mouth is opened or not.
The second thing is that the mouth seems to work independently from the muzzle, especially while the mouth is close - it almost appears to be superimposed when the mouth isn't opened.

Alright, well let's look at the curves needed to create both mouths, starting with the closed mouth:
You can see that the curve that makes the mouth is completely independent to the curve that makes the muzzle. It's also worth pointing out that the mouth only needs a 3 point bézier curve (two end points and 1 control point) instead of the standard 4 point bézier (two end points and 2 control points). 

Now let's compare this to the curves of the open mouth:
Instead of having two separate curves, we have one line made of several curves - 4 to be exact. This is 2 more than when the mouth is closed. The good news is that some things haven't changed at all - the end points and their respective control points for the curve that makes the muzzle are unchanged. Also, the end point of the curve that's at the corner of the mouth still doesn't have any control points. However, it would seem that there's now two curves that make up the mouth instead of just one, and one of them has an end point that has collinear control points (which is why the top of the mouth is curved instead of a sharp corner like the bottom).

Ok, so in order to create both shapes, we have to satisfy the minimum requirements of either. This means we'll likely end up with a less optimal shape. That would be obvious in the case when the mouth is closed, since it usually requires 2 curves total. So let's come up with a case where we can create the shape of the closed mouth using 4 curves:
In a sense, this is the same as the example with only 2 curves - the curves that make up the mouth are identical - they might as well be one curve. Everything else pretty much remains unchanged between when the mouth is close and when the mouth is opened... EXCEPT the corner that makes up the top-front of the mouth!

Look at the curves for when the mouth is opened... 
Do you see how the control points are collinear for that one corner, making the top part of the mouth a smooth curve? Compare that to what we just made and you'll see that corner's control points AREN'T collinear when the mouth is closed, which makes it a sharp corner. Do you know what that means? It means we have to interpolate that corner when transitioning between a closed mouth and an open one!

Let's play pretend for a moment - let's say we have a slider called "muzzle-mouth curve" that is used to go between a closed mouth and an open one. When the slider is set to 0 we get the closed mouth shape, and when it's set to 1 we get the open mouth shape. Between the two shapes it would create, we already know what it would look like when it's set to 0. In order to get the other shape, we'd have to make the control point of that one corner (the one that determines the curve of the top-most part of the muzzle) move around until it's collinear with the other control point for that corner. That would look like this:
(I admit, I accidentally moved some other control points too, but ignore that.)
You can see that the top part of the mouth, despite the mouth being closed, now looks rounded. The two identical curves on the inside of the mouth remain unchanged, but we've still managed to get that smooth upper-mouth we need when a character has their mouth opened!

There! We've taken the first steps needed for doing interpolated expressions!

I'm Not Dead

Hi!

It's been a while since I've posted, so here's my way of letting you know two things:
  1. I'm not dead.
  2. Neither is this project.
Right now I'm working on another project which is taking up a large portion of my time, as well as a few others... So many projects, ah! So, yeah, I haven't had a lot of time to focus on this project up until now.

Instead of doing more with layering, since layers are just a huge pain in the ass, I'll be making a post on my thoughts about how one would use my system to interpolate between all the many complicated facial expressions that the characters of the show express. To start off with, I'd like to go back to the mouth and muzzle - instead of having pre-made shapes, I'd like to come up with an algorithmic way to create the mouth which would involve sliders.

I'll be posting more about it soon!

Tuesday, August 12, 2014

Eyes Test 3

Alright, I've been struggling with figuring out how I'm going to do layers... I decided I'd take a break from that and focus on 3D stuff. By applying transformations to a 4x4 Matrix (and then casting it to a 3x3 Matrix), I can use it to give the impression that a shape exists in a 3D space.


Oh yeah, I also added in some code for doing gradients. Anyways, I've at least got those things out of the way, so I'll be focusing again on doing layers....ugh.

Friday, August 8, 2014

Layering - a Work In Progress

Hi!

Today I decided I needed to focus on doing proper, optimized layering, but unfortunately it's complicated enough that I feel I need to make sure I have everything written in stone before actually trying to do layering myself. For that reason, I've started a document describing my thoughts and ideas for how I might (and will) go through getting layering to work. It is a WIP, so you might actually catch me working on it in real-time.

Click here to view the document.

I also suspect this will take several days to get in order, which is why I'm posting now as apposed to later when I've finished everything. That way, you can keep an eye on my progress.

Thursday, August 7, 2014

Putting it all together - Eye Test 2

This is the result of everything I've done today. Hot dog, I got a LOT of stuff done today.


I still have a lot of work to do in order for this to be efficient and working properly. It might be necessary for me to do 3D vectors and 4x4 Matrices as well to get things like this eye test to work right, but THAT'LL BE FOR ANOTHER DAY, MKAY?

Parenting

Wow, I never thought I'd post this much stuff in one day.

I've whipped up a class called Entity that serves to contain shapes and other entities. It manages it's own attributes, such as rotation, shear, scale, etc... and can "recursively" draw to the screen in a layered manner similar to how Flash might.

I did a simple test where there's only two entities.
  1. The first entity only contains a red shape
  2. The second entity contains a hollow box, the first entity, and a green shape
I then performed the same test I did the other day with the matrices, but instead I did it on the second entity. The first entity does it's own thing, but is always affected by any changes done to it's parent - the second entity.


Masking - Take 2

Ok, I found a solution. My solution is to type-cast the masked layer's alpha array to contain values of type <type 'numpy.uint16'> instead of <type 'numpy.uint8'>, which allows me to multiply the values of the two alpha arrays without worrying about modulation since 255*255 (the largest value this could possibly produce) isn't bigger than nor equal to 2^16. I then divide the resulting product-array by the value 255, and then finally assign it to the alpha values in the masked layer.

        masked_alpha = pygame.surfarray.pixels_alpha(self.output_layer)
        mask_alpha = pygame.surfarray.pixels_alpha(self.mask_layer)

        new_masked_alpha = numpy.uint16(masked_alpha)
        new_masked_alpha *= mask_alpha
        new_masked_alpha /= 255
        masked_alpha *= 0
        masked_alpha += new_masked_alpha

        del new_masked_alpha
        del masked_alpha
        del mask_alpha


The result is beautiful.


The only complaint that I have about using this method is that since I have to type-cast the one array, the values have to be copied over instead of referenced, which can be SLOW. Even so, I'm very happy with this result since it performs perfectly and is relatively fast.