Final Report

So finally it’s over. Today is the last date for submission of the GSoC project. This entire ride was a lot informative as well as an experience filled one. I thank Peragro organisation for accepting my GSoC project and my mentors Matthew Lagoe, Branko Majic and sueastside for helping me out fully throughout this project.

Below are the modules of the peragro to which i have contributed in. I will give a brief description of each module and what all work has been done during this summers.

Peragro-at : Peragro’s analyzer and transcoder framework, which is the standalone heart of Peragro. It has two main components.

• Analyzer - analyzes file contents for potential assets and extract features or metadata. command line to analyze :

  pt -v a file.blend

  Analyzer has a plug-in system that searches for an appropriate plugin for analyzing the given file type (blender file in our case) and prints the assets such as mesh, images , textures and their metadata.

• Transcoder - It converts a file type to usable asset. Transcoder also have a plugin system same as that of analyzer that allows to convert one file type to another and can be used to change the resolution, rotation, size etc.
  
  
  pt t file.blend  "Cube1-data(application/x-blender.mesh)" "image/png" --size=64,64

  The above command will transcode the 'Cube1-data' asset to 'png' format with size 64x64

My Contribution to peragro-at:

1. Audio feature extraction: My project started with creating a plugin that can extract audio features like tempo, global key, mood, gender etc. Essentia is an open-source library for audio analysis and audio-based music information retrieval which was used to extract features metadata from an audio file. Essentia’s pre-build command line feature extractor (streaming_extractor_music) was used to extract the low level features such as tempo, global key, Average loudness etc. For the high level features (such as mood, gender, genre etc) extraction, Essentia need to be build with gaia (which implement similarity measures and classification) to be able to run high-level models. For each high level feature like for say gender, there is a pre-trained model (gender.history file). Each model is basically a transformation history that maps a pool (a feature vector) of such lower-level descriptors produced by low-level extractor into probability values of classes on which the model was trained. For high level, "essentia_streaming_extractor_music_svm" was used as command line extractor which need to be configured with the models. Code:https://github.com/peragro/peragro-at/blob/master/src/damn_at/analyzers/audio/feature_extraction.py
   Conf_file:https://github.com/peragro/peragro-at/blob/master/src/damn_at/analyzers/audio/profile.conf
2. Docker : The next thing that need to be done was to make the project easy to install and use. So we decided to dockerize the project. The Docker image can be easily pulled from the docker hub as shown below:
   docker pull peragro/peragro-at

   and voila we are good to go. Now to analyze a file just use the command line:
   

   docker run --rm -v `pwd`:/peragro -ti peragro/peragro-at pt a file.blend

   Code: https://github.com/peragro/peragro-at/blob/master/Dockerfile
3. Added .fbx file support for 3D models Code: https://github.com/peragro/peragro-at/blob/master/src/damn_at/analyzers/mesh/analyzer_assimp.py
4. Resolved many pylint and other errors and added travis support. 

All of my Commits :https://github.com/peragro/peragro-at/commits/master?author=sagar-kohli

Peragro-index : ElasticSearch based indexing for the extracted meta-data. Indexing is done through pipeline command and directly through stored analyzed data.

My Contribution to peragro-index:

1. Command line indexing: Previous version of command line was too lengthy so I made it short. Command line for indexing analyzed data:
   pt a file.blend | pt index elastic
   

   Code: https://github.com/peragro/peragro-index/commit/c65cb2411305ccff49f655859337c9f165a3d1bf

   
   
2. Docker: Similar to peragro-at, I have also dockerized the peragro-index which will help the user not to install the dependencies along with peragro-at.
   
   Code : https://github.com/peragro/peragro-index/blob/master/Dockerfile

All of my commits : https://github.com/peragro/peragro-index/commits/master?author=sagar-kohli

Final Words:

I would like to thank my mentors Matthew Lagoe (Botanic), Branko Majic and sueastside for guiding me. They were really encouraging and helpful. I’ve had a wonderful journey so far. Ofcourse, this is not the end. My work for GSoC introduces so much more possibility and I would like to explore and continue contributing :)