Under the coding hood: Hacktory 2018
It was around 3 in the noon that Kurian Benoy messaged me telling about a Hackathon organized by a couple of companies working on disruptive technology, and asked if I was interested. Having gone for one hackathon already, I was eager to try once again. On checking their website, I came to understand that around 4 start-up companies were hiring students at the hackathon for job positions and internships. This did sound like a golden opportunity and so considered applying for the hackathon.
The hackathon would shortlist ideas and only top 20 would get to participate. It just so happened that I always had a wish to make an app that would scan a musical script/sheet and generate the related music automatically by using machine learning to understand the notations, similar to how Google lens can scan words and read it out using the camera. Though it sounds easy on paper, making the machine learning model, a working back-end and a good application to display was challenging. Kurian felt it was worth a shot submitting this idea. Jithin James and Tino T Max were willing to help us out with the project if we made it through. We realized that the idea had already been researched upon previously, but Jithin said trying to implement it under 24 hours was the challenge we were taking. Infact Soundslice entire business revolves about this core idea.
We submitted the entry by Tuesday, and it was on Saturday afternoon (7th July) that I received a confirmation of selection. I was overjoyed, because the idea was shortlisted in the top 20 :) The rest of the team were equally excited and we got everything, including our resumes (because they recommended to do so) for the journey to Trivandrum
Introduction
We all made it to the venue by separate means, and met up at b’hub at Trivandrum. The organizers were very friendly and attentive to our needs. We were introduced to the various sponsors of the event, and were reminded the motto behind having this hackathon. There was no winner for the hackathon, but rather an opportunity to get internships and Job offers in disruptive technology companies.
The even began at 10:30 AM and we were hyped for it!
Day 1
We decided to modify our idea a bit. Now, this was our goal from the product:
- Scan a musical sheet
- Use Machine Learning to identify characters and send appropriate sound back to the device
- Generate an augmented reality Piano of one octave to hover over the musical transcript and show how the keys are pressed up on when played in real life
Jithin was well versed in Machine learning and computer vision, and so decided to invest his time focusing on the ML model using Tensorflow. Tino used his knowledge in Augmented Reality to create the AR Piano and generate the music when the required notes were played. Kurian was the node between the two, focusing on the back-end to join the ML model to the AR Piano created by Unity 3D. I did most of the menial task as finding the audio and researching solutions.
The volunteers kept providing us with snacks, food and technical support all the time. The disruptive technology start-ups were also there, and they interacted with us. Representatives from i-Boson Innovations , an AR based company, were impressed with the work that Tino did and helped us improve our work. The openness made it easier for us to approach them with doubts anytime.
They allowed us to apply to talk to officials from any 3 companies that were there over there. They would call us accordingly and set up an opportunity to interact and if worthy, a detailed interview that can lead to internship or job offers.
Day 2
Even in the night, when we started loosing focus and drift into sleep, around 3 AM volunteers started playing soft music on the guitar and piano to set a peaceful ambiance. It helped us relax and ease ourselves to better focus later that day.
We started facing problems later in the morning — Tino’s renders disappeared because an update, Jithin’s model was not able to predict character’s properly due to random size and dimensions of the page image captured. Kurian was struggling to integrate Tensorflow.js to Unity system. Eventually, we could not make the product and were feeling slightly gloomy. But then a volunteer, Nayas came and said that it was alright if it failed. The fact that we tried to achieve it and got parts working was itself commendable. Like how Mr.Vishnu, CEO of I-Boson Innovations says, it is alright if the project is small, but do it as neatly as possible. All these positive support helped us feel better and confident about the progress we made.
Presentation
I prepared a simple presentation to explain the product. I started explaining by comparing it with Google “text to speech” featured, and questioned why not the same for Music. Many were impressed to see how a simple image could make a piano play, thus act as a virtual teacher.
Along with our idea, there were several other wonderful ideas such as predicting tomorrow price of gasoline, decentralized voting, chatbot that understands your health conditions, and many more.
Overall
The highlight feature of this hackathon was the fact that everyone was friendly — fellow participants, volunteers and recruiters. It felt absolutely calm and the organizers worked very hard to keep it so smooth. Even when Tino had to download a heavy data file and the internet was slow, they set up a better connection just for him at another room. Hospitality was extremely wonderful. I eagerly look forward to many more of their events in the future.
