How I transformed from Software Quality Engineering to Data Science World
Background — About Me
I am Pratik Barjatiya, an Electronics and Communication Engineer, with more than 7 years of experience in Quality Assurance [Test Automation & Performance Testing] and an aspiring Data Science professional. I started my career as a Software Test Engineer where my primary role involved web application and mobile application testing, which further evolved and I was deeply involved in architecture of E2E Functional and Performance Test Strategy, Test Plan and Test Approaches and Test Reporting artifacts for API, WEB and Mobile based Applications in OTT, Fin-Tech, e-Commerce & Telecom Domain. I was highly motivated with continuous technical contributions in Quality Engineering Space. I had made my hands dirty over multiple startup applications like [ Hotstar (Media — OTT), Udio (Fintech, PG, Wallet & Remittance), Ridlr (Logistics — Commute), Limetray (Food Ordering & Delivery), Twin (Messenger), AhaLife (Luxury e-Commerce), VUClip (Media — OTT),etc.] right from bootstrap phase to a fairly matured niche stage
Unfortunately or fortunately, I had to move to Bangalore due to some personal reason and there I got an opportunity to work with Zoomcar, a leading self-drive rental cars marketplace where I was exposed to IoT based software solutions. When I joined Zoomcar as a Sr. SDET, I never imagined that under ~2 years I would be working into Data Science — Data Engineering world. Surreal as it sounds, that is my reality now.
In this post, I would like to share some insights I learned throughout this journey in hopes that it will help any reader who is contemplating making a career transition. For me, that transition was from Software Engineering to Data Science, but I believe that most of these insights apply to any kind of career transformation.
So here it goes…
First, find your passion!
“There is no passion to be found playing small — in settling for a life that is less than the one you are capable of living.” — Nelson Mandela
I didn’t give any thought to career planning before I begin my IT Career. What helped me a lot in that sense was my curiosity to find and track evolving trends in technology while performing my duties in the associated organizations, that encouraged me to ask myself the right questions and pushed me to take charge of my career path.
So I guess my first tip is to take time and think about things like:
- Where do I see myself in ? 1 year/3 years/5 years
- Target role I will be excited to do
- Type of work I’ll be doing and Type of work I’ll be avoiding
- Last but not the least Salary/Earning expectations
More blessed are you, if you can find a mentor to accompany you during this self-discovery process
First Baby Steps
“It is better to take many small steps in the right direction than to make a great leap forward only to stumble backward.” — Louis Sachar
OK, let’s assume the passion discovery process brought you to the realization that you want to make a career transformation.
The next thing you need to know is that this transformations doesn’t happen overnight. It’s a process that is comprised of many small steps. In my case, the transition took more than 2 years.
Let’s start at the beginning, The curiosity towards Data Science was initiated when I was working at Hotstar, I was deeply noticing and acquiring the data skills with the in-house Data Science folks, while testing app personalization, recommendation and A/B events. I was practically first exposed to the world of Data Science and Machine Learning when I started working as Sr Quality Assurance Engineer at Western Union Pune TEC (Core R&D) Product Center. We had newly established facility, first offshore Engineering Center, a massive inauguration with CEO Hikmet Ersek along with Onsite Leadership was planned. We had a hackathon conducted during this time within our Core R&D group showcasing use of trending technologies viz. blockchain and ML based solutions for various pain areas across cross-border cross-currency remittance system. A collaborative preemptive money transfer system based on decision trees powered with sentiment analysis over voice based chat-bot was the highlight of hackathon built in pieces by multiple teams. We had built a sentiment based preemptive customer — agent interaction reducer.
Later my effort for sentiment analysis system was shaped to a project in itself and I made my hands dirty in building sentiment analysis engine using Standford Core NLP and NLTK library for customer chats, emails as well as agent feeds. This engine was designed to set priority order for live customer issues and extract new feature roadmap for the entire product and closely aligning with CSAT medallia scores by providing polarity to each message. I slowly started to pick up more and more concepts as I progressed with the project. I had this opportunity to read a lot of interesting articles and go though research papers.
After a few months, I had few internal presentations over the solutions. I quickly learned that this wasn’t such an easy task as the information was plenty with widespread approaches and rapidly evolving and mediocre documented, but this was a great learning experience for me. It was my very first experience with machine learning code and I got a chance to understand how a lot of basic concepts (for example, tokenization, words to sentences annotation, POS Tagger, MorphaAnnotator,etc)are implemented. Another important lesson that this experience taught me, is the importance of reproducibility in academic research and that just because something is written in a paper doesn’t automatically mean that it is true.
The next small step I took was volunteering to give an internal introductory talk on Machine Learning and Sentiment Analysis (this helped). I believe that there is no better way to learn than to teach, so this was a perfect opportunity for me to learn about the topic.
Be persistent!
“Success is stumbling from failure to failure with no loss of enthusiasm.”― Winston S. Churchill
“Keep Going, Your hardest times often lead to the greatest moments of your life. Keep going. Tough situations build strong people in the end.”― Roy T. Bennett, The Light in the Heart
At this point, I already realized that this is the path I wanted to pursue, so I started to go into high gear in terms of my self-learning. This included taking Andrew Ng’s amazing Machine Learning course followed by reading blog posts and papers, following influential researchers and practitioners on Twitter and LinkedIn, subscribing to Machine Learning and Deep Learning DL’s, attending relevant meetups and even listening to ML podcasts.
I was determined to do everything in my power to put myself in a position in which when an opportunity presented itself, I would be ready for it.
But this isn’t enough. To make any career transformation, I enrolled INSAID GCDAI program to make it more measurable and concrete. Regular Classes, assignments, quizzes, fire-side chats with industry leaders, etc kept me engaging and kept my persistence excited. I followed Kaggle, Analytics Vidhya, TowardsDataScience through this transition and it was very helpful to learn from other professionals.
Very often, we have the “desire” to achieve something, but we don’t have the “determination” to go to the end. To do this, we need “dedication” and this means, to give our time systematically. To do this takes one fundamental thing: ‘DISCIPLINE’!
DESIRE | DETERMINATION | DEDICATION | DISCIPLINE
I would like to call these 4 words, the 4 D’s of success and if you can put them together, make them work together, no target in the world will be excluded during your lifetime!
They are strictly connected and just taking out one of the four, the other three can’t work anymore. It is like an Ireland dolmen. If you take away a stone, all the others collapse on the ground. The “magic equilibrium” is gone!
Be explicit regarding your career aspirations!
“Remember that wherever your heart is, there you will find your treasure. You’ve got to find the treasure, so that everything you have learned along the way can make sense.”
–Paulo Coelho, The Alchemist
I am fortunate to be working for a company that highly encourages personal growth and personal development. This is not something true always. Once people around you are aware of your aspirations, opportunities are likely to start presenting themselves.You have to believe people to provide opportunity to help you.
Embrace every opportunity!
Especially if you feel that it’s a challenge and you’re not yet ready for it. These are exactly the experiences that will help you grow the most.
“If somebody offers you an amazing opportunity but you are not sure you can do it, say yes — then learn how to do it later!” ~ Richard Branson
I was lucky enough to have the support of my managers as well as the encouragement and guidance of friends and colleagues, which led me to participate in several projects with the team on top of my daily work as an Automation Engineer. For example, I was given the chance to re-build churn model for associates over Spark ML which was ~100x faster with similar accuracy
Another meaningful experience I had during that time is fetching city based demand requests with Geo-ip based on the application network traffic sessions data. There is no chance I would have gotten these amazing opportunities if I hadn’t let my managers know of my ambitions!
Get your hands dirty — There is no shortcut
“Knowing is not enough; we must apply. Willing is not enough; we must do.” ~ Johann Wolfgang von Goethe
Once you feel you have the basics done, you should get as much hands-on experience as possible. True expertise is achieved through practice. Every time you learn a new concept, try to actually “get your hands dirty” and play around with some code. Kaggle is a great platform for getting hands-on experience, and I highly recommend it. For those of you who don’t know it, Kaggle is a very active online community of data scientists and machine learners. The community is very supportive of newcomers to the field.
A tip that can go a long way in terms of getting practical experience is finding a partner in crime — someone who is equally (or at least somewhat) passionate about the subject that can accompany you through your learning journey. Once you decide on learning goals or milestones together, you are both less likely to cut corners or decide to skip the practical aspect of the learning experience. It also always helps to have someone to bounce ideas off of or consult with in case you are struggling with a problem. The INSAID friends were my partners in this phase to brainstorm and make the hands dirty. We also had the community support from our mentors. We started practice over Open Source practice data-sets, kaggle data-sets
At first, we tackled past competitions and did lot of EDA and build ML models. A curated INSAID curriculum exercises are shared at git repo - INSAID-Assignment. The nice thing about Kaggle is that there is a variety of competitions in various domains which means that you can always find a competition that is interesting for you or relevant to your current work so that you can utilize domain expertise that can help you put yourself ahead of the pack.
Don’t wait for the perfect moment!
“If we wait until we’re ready, we’ll be waiting for the rest of our lives.” ~ Daniel Handler, The Ersatz Elevator
There will always be more to learn. Don’t wait for a moment in which you feel you are ready to make the transition, because that moment might never come. There will always be new algorithms to learn, new MOOCs you can take, new libraries to know, more skills to acquire. We live in an era in which technologies are constantly changing and evolving, natural curiosity and the ability to self-learn are more important than any specific skill.
For me, what helped me come to this realization is my mentor who simply asked me during one of our sessions: “What are you waiting for? What’s keeping you from applying to Data Science positions?” He was right. At that point, I was already learning intensively for about more than an year and had sufficient hands-on experience to rely on in any technical interview. I decided to take his advice and start interviewing internally (with the support of my manager) to data science positions.
Finally —
These are the things that happened to me. There is no right or wrong way when it comes to career transformations and I believe that every person should try and find their own path.
There is a common conception that when you make a significant career change, you are basically starting from scratch. I don’t believe that this is true. You might be entering a new field, but that doesn’t mean that all of your past experience becomes void. My Software Engineering background has made me proficient in things like writing modular and reusable code, version control and continues integration. My Software Automation experience also greatly contributes to my day to day work and provides me with a unique perspective that a “traditionally trained” data scientist might be missing. For example, I am very passionate about applying the same quality standards that are practically inherent in “traditional” Software Development into the world of Machine Learning (Unit Testing, Data Validation, etc..). All that to say that you should be aware of your strengths and the unique perspective and capabilities that you bring to the table.
Never underestimate your Self Belief
You are interesting enough for that callback. You are driven enough to meet the deadline, grown-up enough to balance work and learning, strong enough to turn down or walk away from a job that doesn’t respect what you do.If all of that’s the case, then someone, somewhere will notice you’ll get a call that makes your day. Someone will say, “Congratulations! You’re hired! And we’re so glad to have you.”
Good Luck!
