The technology used behind Instagram and Snapchat filters is something which we couldn’t have thought a decade ago. Creating a whole new picture of the user by adding such cool features like dog face or flower crowns is a prove of innovative excellence in the field of technology. Computer Vision and Machine Learning Algorithms are the two major concepts behind these features. Nowadays Instagram and Snapchat features are the most used features in a smartphone. So in this article we would unfold all the little tricks and technical mysteries which lie behind our Snapchat and Instagram filters.
Lets scratch the surface with bare facts:
The concept of bringing different selfie lenses was first done by a
Ukrainian startup Looksery. Snapchat got hold of this Odesa-based face changing startup in September 2015 worth $150 million dollars, and according to a survey,currently it has doubled up its net worth value.
Now a days, programmers have started making their own version of filters.The transparency level of these technology allows any technically sound person to make their own rendition of Snapchat filters. For doing so one needs to understand the MSQRD app development phenomenon.
Wanna know more about the insights of Instagram, then click here.
The underlying principles behind these filters:
For a feature that exists as bit more than a gimmick, there’s lies a surprising amount of underlying technology empowering it. Computer Vision and Image Recognition features are the key focus to these filters.
1.) Face detection:
Once we input any image,the HOG/SVM detector starts working. It automatically divides the image in pyramidal forms . Entering each new inputs into the pyramidal section, makes use of a sliding window . The sliding window concept is quite simple. By looping over an image with a constant step size, small image patches which are typically of size 64 x 128 pixels are extracted at different scales within the picture. Depending on each entry on the sliding windows,the face recognition algorithm decides whether its a face or not. The HOG detector functions on computing the current window and then passes to the SVM classifier (Linear or not) for making decisions. Know more about HOG/SVM detectors here.
2.) Facial Landmarks:
Facial Land-marking means demarcating the face in different smaller sections like landmarks. Extracting facial landmarks for image recognition is a relatively cheap operation for the CPU. However for the programmers end it is a Hercules task because of its complexity . Once the face gets a bounded by virtual line around it, it automatically functions by estimating the most obvious positions and features to occurs in a human face.This feature allows “flower-crown”to be on your head even on practically moving your head quite many times.One Millisecond Face Alignment with an Ensemble of Regression Trees provides detailed analysis about facial alignment and segmentation.
3.) Image Processing:
Active Shape Model serves as a key starting point in this image recognition process. Machine Learning scientists train the data set for recognizing the human face efficiently. It serves as a basic training kit for comparing the inserted face or images. On comparing with the basic skeleton of the facial image,it keeps shading the areas which seemed different from the Active Shape Model. All these creates a mesh work of integrated lines along the virtual image.This results in a 3D virtual image on which Snapchat and Instagram features can be easily added. Some filters demands for raising eyebrows or opening mouth,for these the basic procedures are just the same with bit more machine learning algorithms and training the dataset with much vivid libraries.
Conclusion to all the discussion:
So far we have discussed from basic to pretty advance concepts about the technology behind Snapchat and Instagram filter. Snapchat and Instagram filters are very consistent in terms of innovation. . The aforesaid principles serves as the platform to many technologies like Virtual makeover, face morphing. It was an experimental venture which proved to be one of the highest grossing application on today’s date.
Happy exploring readers!