Why we Killed our Android Program (and what we learnt)

With successful programs like entrepreneurship, digital marketing, product management and data analytics already running, it was time for us at UpGrad to think about the launch of a new program. After doing some preliminary market research, we decided that this would be in the area of developing Android Apps.

Reconnaissance

So, first things first! We know you are probably expecting to hear that we didn’t gauge the market well enough. But the truth is, we started the process with extensive market research.
In order to identify the challenges which are faced by recruiters while recruiting Android developers from the existing talent pool, we interviewed a number of senior leaders. We knew that getting their point of view on this talent pool and what they felt was lacking in the industry; was a crucial first step for us to understand the market we were about to deep-dive into.
Here are some of the responses we got from these leaders:

“Android developers must appreciate the full spectrum of android OS and not focus on only building UI heavy apps; which is the problem with current android developers” – Aurobinda Nanda, Head of Product Engineering, Happiest Minds

“Finding Android developers who can build professional, Android apps that are optimized and function well at scale is a massive challenge” – Akash Sureka, VP Mobility, Persistent Systems

 

“Android developers are as responsible for testing their apps as for developing them. Most mobile teams are lean and follow the continuous integration approach which doesn’t leave too much scope for full-fledged Quality Assurance teams. Not everyone in the current talent pool understands and appreciates this” – Vishwesh Jirgale, R&D Head, Microsoft

From the horse’s mouth

Once we knew what recruiters thought, it was time for us to move to the next step. We interviewed present-day Android developers to learn about challenges in their jobs, how they transitioned to Android, what was the motivation to do so, what does a typical work day look like, how satisfied they were with their jobs; etc:

• We realized that the demand for skilled Android developers is extremely high in the market, and increasingly so.
• Android developers typically switch jobs every one to two years.
• They have the option to work remotely.
• They see their code materializing in the form of a tangible app which has a direct impact on their customers and hence, developers’ job satisfaction level is high.
• There is great variance in the skill-sets of Android developers across different organisations.

A picture starts emerging

Based on interviews with recruiters, current Android developers and competition analysis, we came up with the broad features for our program. It was now time for us to move on to interviewing the most important set of people, our TG, (target group or audience i.e., direct, potential consumers of the program) and testing out these features/program.
There were and are several reports available online which establish the need for an Android program, featuring the Android job market as a booming one by 2020 and beyond, the trends for which are already beginning to show. Given this, we just wanted to talk to our TG and know if they were willing to learn Android through the online medium and pay for it? What were the pain points, if any? Which of our features did they appreciate?

Meet our buyers

We spoke to two different sets of potential TG.
While talking to professionals, we realized that people who are not into actual coding or development were bored with their jobs and wanted to move to active coding. Android, being a program with a very low entry barrier, was one of the fields they were looking to enter. They also wanted to learn only for a certain amount of time in the week, rather than full-time.
After talking to such professionals, and pitching to them the features of this program, we realized that they were willing to pay for it. They were ready to register for the program and pay for it after two months – when the program would be launched.

We also spoke to our next set of TG, which was mainly students. While talking to them, we realized that:

• They were aware of the latest happenings in Android,
• Wanted to learn Android,
• Were too confused to learn via the free courses available online,
• Some preferred workshop formats whereas others understood the importance of more rigor and quality,
• Some preferred offline whereas others were comfortable with the online way of learning.

We told them about the program, mentioned the features and they expressed interest to register and pay for the program. However, in this entire process, what we completely neglected was:
There is a huge difference in committing for a later date of payment vs willingness/committing to pay immediately.”

Roadblocks begin

We realize now that we should have instead asked them whether they would commit for immediate payment. That would have given us a clearer, more accurate picture.
Another thing we missed here, and should have questioned, was that even though students expressed interest to learn, did they have the time to learn?
Would they focus more on campus-based job placements, tests/examinations and other activities, rather than invest extra time in learning a new technology?
Do they realize the importance of learning a new technology while still in college?
These were a few questions, the answer to each of which we assumed was a yes, since the students said that they would register and pay for the course. However, students themselves did not take into account the time commitment required and ended up committing to a later date since it was an easy fix.

Other successes can be distracting

Since we got a good response for the program in the interviews, we decided to evaluate the competition and proceed.
We spoke to offline players in Mumbai and got the total number of people they cater to, projected the national offline numbers basis that and combined it with a few online competitor numbers to come up with a business case for the program.
In order to ensure the best possible experience for students, we partnered with the best in the industry and roped in mentors from various companies like ClearTax, SnapDeal, BookMyShow, Swiggy and many more.
We were also named Google’s official training partner for Android and also brought on board one of Google’s developer expert, to help develop our curriculum.

Marketing – a big slice of the pie

It was now time to begin marketing our product and figure out the channels that work.
The first channel we tried were ‘webinars.’ We organised webinars around ‘Careers in Android’ and called the leads (those viewing/visiting the platform) to ask if they were interested in the program. We briefed them about all the features exactly as you would do while you pitch the program and informed them of the price-point; along with the program commencement date – a month later.

Here’s the response we got:

Here, interested means that a person has exclusively said ‘Yes, I am interested in registering for this paid program on Android Development.’ With this overwhelming response, we concluded that this channel is bound to work however we were not able to get any conversions (get leads to transition to a paid customer) from the webinars after the program was actually launched.
At this point of time, we did not have our program page up and running, along with the payment gateway to monitor whether people actually paid.
What we could have done was create a simple landing page with all the features and monitored applications/inbound inquiries to re-validate the data.
Next, we came up with a plan of organizing seminars-cum-entrance test in colleges to make students aware of ‘Android as a Career’ and get applications from students based on this activity.
For this, we waited for the program page to be up and running so we could validate this channel. We pursued this channel once the content production had already begun and online marketing was ongoing.
What we realized after reaching 900 students, was that majority of the students are interested in Android but have no plans of learning it now.
They were too casual about their career to think about investing their free time after college into learning a new technology rather than enjoying the college life.
This conclusion could have been reached earlier had we done either a quantitative research or just created a simple landing page and monitored inbound student inquiries through conducting seminars, as a part of market research itself.
In parallel, we had online marketing campaigns running for both sets of TGs. Soon after analyzing the data, we realized that while the cost per application submitted was within our budget, the conversion was too low, which led to a higher customer acquisition cost (CAC).

The decision is made

This was the point when we realized that the two biggest channels of customer acquisition, which we were relying on, had failed to yield results.

We decided to tweak online marketing and try it out again. This time the cost per application submitted shot up so high that we required a 25% conversion rate to achieve the required CAC.
This was when we checked for offline partnerships, email databases, offline workshops for professionals; but no channel yielded a satisfactory result and we decided to scrap the Android Program.
Even though we were able to create a great user experience and features that users loved, we failed to create a sustainable business case out of it – crucial for any Minimum Viable Product to live and succeed.
So, here are a few learnings that we have had, and would like to pass on to product managers or CEO’s of startups or any organisation trying to create and build a product. Hopefully, they can derive some value out of these as mistakes to avoid:
Learning #1: While validating your idea, make sure you understand the difference between ‘I am willing to enroll and pay for the service now’ vs ‘I will surely enroll and pay for it, in the future’
Learning #2: While qualitative analysis is important, you must perform a quantitative analysis as well for validating hypotheses and building a business case.
Learning #3: You do not need the final product to be ready for testing a particular customer acquisition channel.
Learning #4: If you are thinking of using online marketing channels for customer acquisition, you need to know that Cost Per Lead is highly affected by your brand awareness and SEO (Search Engine Optimization) presence.