The Interview with the Hiring Manager series from DataMatrix aims to provide aspiring data science, engineering & analyst candidates a glimpse of how potential hiring managers view the field. In this series, Hiring Managers share best practices, tips & feedback for prospective candidates.
This week, we speak to Mahesh Pasupuleti1, Senior Director of Data Engineering at Poshmark. Mahesh began his career as a software engineer at Collabria, followed by a stint as Systems Architect at Elance. He then went onto a long stint at MobiTV, rising to Director of Platform Engineering, before joining Poshmark to manage its data infrastructure and machine learning pipelines.
Could you tell us a little bit about your career journey and what brought you into data science?
What have been some of the key learnings and shifts in the machine learning field as you’ve journeyed through it?
When I first came in touch with the field, the focus was primarily on data warehouse solutions and understanding users and user behavior through transactional data. We have since seen a lot of growth in the behavioral data collected by companies: how much time users spend on a platform, what interactions they perform, what products they are interested in, how they can be kept engaged and so on. All of these questions can be answered using machine learning.
A little over a decade ago, companies like Netflix were creating recommendation challenges to draw people in to learn about the field. There has since been a big shift in terms of algorithms and how they are being adapted. Now, we’ve reached a point where machine learning is nearly at the point of being commoditized. Earlier, you needed team members with a deep understanding of the models. Now, the models themselves are being developed as a product. The industry is evolving such that you don’t need a deep understanding of models, but can develop them so long as you have the ability to play with the data.
The lesson I have learnt in this journey is that you have to pick the right problem and make sure you have the right dataset for it. One must ensure that one has good quality data that one can rely on to develop a model. But not everything is driven by a model, not every problem needs a data science solution. So, it’s important to understand for each problem how far you must go to solve it.
In your experience, what are the most exciting aspects of building and managing data teams?
What are the key challenges you face in building and managing machine learning teams?
What are some of the biggest mistakes that companies make during the hiring process that either result in bad hires or drive away good candidates?
Many of the graduates wanting to do data science that I have seen come in with the sole intention of developing models. But before you develop the models, you have to do all the groundwork necessary to get there. And some people may not be interested in doing that. Not establishing that requirement is one mistake companies make.
Further, with any hire, you want to make sure that they are culturally fit for your organization. For instance, some organizations may be engineering-driven or have a data-science-first methodology. Candidates with a similar predominant focus would be considered as the best fit. Some candidates come from a research background with much longer time frames for projects, while what we call practical data requires models to be developed and put into production within typical time frames of six months to one year. So, you want to see candidates show qualities that fit those requirements. Finally, for a startup like us, we need to know if prospective team members can wear many different hats and do everything needed to make their project a success.
When you bring a new team member onboard, you’ve already spent significant effort and time in hiring them. So, you want to make sure that they are successful. Therefore, it’s important that you properly vet them during the interview process to know whether they are going to fit culturally or not.
How do data problems typically get defined in your organization? Are they defined by the business or do they arise from the data itself?
We have tried to do this in both ways. Initially, we began with one of the best collection frameworks for high quality data. To use that data, we developed what we call data applications, which aren’t actual models but provide some insights into the data.
At the second level, we looked at what models we could develop on the strength of this data. This was somewhat driven by engineering, by the desire to design and develop models. There has also been the demand from business, from growth and marketing teams, for the data science needed to run efficient campaigns.
Further, engineering also developed a recommendations platform which has been in production for a year, and which has begun delivering value to the company.
Could you give us a sense of the tool stack deployed in your organization as well as the way your machine learning teams are structured?
What can candidates expect in terms of the hiring process at your organization?
We receive applications from candidates from a variety of backgrounds. So, we first review resumes for technical fit, based on their backgrounds and the work they have done in the world of data science and machine learning. Those candidates that we feel are a potential fit receive a take-home assignment involving a business problem to be solved using a modelling algorithm. This is followed by a phone screen with a software engineering or machine learning algorithmic focus, depending on the candidate’s strengths.
Then, in the new normal, we follow up with virtual onsites. These involve four to five hours of interviews with the head of the data science team, a machine learning engineer, a hiring manager, and a big data engineer. For machine learning engineers the focus is on their ability to design, architect, scale and integrate a model – basically the overall lifecycle. Finally, we also have a cultural fit interview with a different panel.
What are some of the things you would look for in a resume that would make you sit up and take notice of a candidate?
Looking at a resume, we want to get a sense of whether candidates list projects associated with a business or ones they explored as part of learning how to model data. Next, we look at whether the resume gives a sense of the details of problems solved, algorithms used, and so on. Further, while we don’t expect our candidates or potential hires to have the same tools as what we use in the organization, we do check if they have something comparable. Finally, we look for signs of whether they can scale up, whether they have developed models in a distributed environment or have only created models on a single stack. Those are the things I look for in a candidate with experience.
With fresh graduates we look for potential to learn. We look for whether candidates have done a little bit of work to align their applications with the job requirement, are able to grasp what the position describes, and highlight what they have learned. We are generally very flexible with resumes because our take-home assignments are appropriately difficult, and we are confident of interviewing candidates who are able to put in the effort and come through that stage.
What are some things you would like to hear during interviews that would make you want to hire a candidate?
In interviews, we look for an ability to deep-dive on past projects. We generally work with whatever candidates know – we ask them to explain their projects; the challenges they faced; how well they know their business problems, data sets, and algorithmic work; and how they plan on revising their models.
We are also big on cultural fit. We expect individuals to show us that they are not afraid to fail. Also, they should be willing to work with different organizations because, as machine learning engineers, they have to work with product teams, data engineering teams, users to get user feedback, and so on. We generally look for people who have a can-do attitude and are willing to do whatever it takes.
What tips and strategies would you give to a prospective candidate on how best to apply and interview with you?
We put a lot of thought into creating job descriptions and our job descriptions will include six-month and one-year accomplishments. When candidates review, understand and internalize the accomplishments listed, the interview conversation will generally be very organic. I would say, spend time on understanding the job description and what the company does and speak to that. We generally look for
- Software engineers that can work with algorithms
- Overall experience with building and deploying models (not simply deploy only)
- Ability to monitor and improve model performance
- Ability to understand the business problem and communicate with business users
- Ability to learn and take initiative
- Culture and team fit
We are open to training candidates who bring in strong skills in one area but may be less skilled in other areas. We are hiring not only to solve current challenges, but also future challenges that we have not yet discovered.
What vital skills do you think young data scientists should focus on developing? How can candidates who are early in their data science careers and don’t have a lot of conventional work experience stand out from the crowd?
I personally think they should learn and understand different toolsets, as well as try to play with a variety of sample projects and datasets available on public sites to continue to improve their skills. Some of the Big Data technologies & Machine Learning frameworks we are typically involved with include SparkML, TensorFlow, PyTorch, and Keras.I would say that they should become familiar with all open source tools and networks consistently.
To stand out from the crowd, candidates should do some research into the organizations that they are applying to and see what kind of tool sets they deploy. One practical way of doing this is to log onto job sites and compile the various job descriptions to highlight the most common tools, and to familiarize oneself with them. When applying for jobs, candidates should also highlight what they can bring to the table. They can do so by finding ways to understand what the organization is developing.