How to make a snapchat filter in 5 steps
Here are 5 steps how to do this by yourself:
1. Use computer vision to find faces on video Using the OpenCV library you are able to detect all the faces visible on a video feed. You can use webcams the get a live video feed or prerecorded video can be used.
2. Find landmarks on the detected faces On the detected face landmarks are defined. Landmarks are important spots on a face, like the corner of a mouth or eye. These landmarks are used to detect changes in facial expressions. You can define multiple classes of facial expression, e.g. mouth open, mouth closed. These classes are used as trigger points to add filters.
3. Collect training data For each class training data has to be collected to train the machine learning algorithm. This is done by filming your own face and performing the right expression for each class. For the example of mouth open and closed you need two sets of training data, one for each facial expression you want to detect. More training data is always better but everything around a few minutes is sufficient to make a working filter.
4. Add your filter Now that you can detect facial expression it is time to add a filter when performing one of the expressions. Filters can be anything you can find on the internet. To place the filter on a certain part of the face the landmarks can be used again. Two points on the image are mapped to two landmarks, that way the image will move around when you move. Finally, you make the filter appear when a certain face expression is detected. In the example a rainbow appears when the person opens his mouth.
5. Have fun! Have fun using your self made snapchat filter and start experimenting with all the possibilities!
By Eva Bouton (Applied Computer Sciences at Erasmushogeschool Brussel) & Louis Lavens (Computer Sciences at UGent).