Meet Michael
What is your current role?
I am currently a Senior Engineer on the Machine Learning Platforms team
What does your team do? How does it fit into the world of Wayfair?
Data science and machine learning are incredibly complex domains which require expertise in the fields of mathematics, engineering, and business. We wanted to avoid adding “Dev-Ops” to that list of requirements. In practice, this meant building a solution for all of these “Dev-Ops” challenges which all data scientists can leverage. That’s where my team comes in.
We work closely with data scientists, engineers, and product managers in two distinct capacities. Primarily, we support machine learning application development and deployment. Secondarily, we research the challenges our data scientists face in their workflows in order to find commonalities and build solutions to enable future projects.
How long have you been at Wayfair?
I’ve been at Wayfair since May 2016.
What was the first 90 days like starting here?
Prior to my work at Wayfair, I studied Bioinformatics (DNA sequencing, genetics, etc.) which led me to discover my passion for programming. With a Bioinformatics degree and only a few (looking back, embarrassing) pet projects, I needed a lot of refinement to feel content calling myself a software engineer. Luckily for me, Wayfair offers an amazing program for people who are looking to launch careers as software engineers: Wayfair Labs.
Labs is a two-ish month crash course on everything you need to succeed as an engineer at Wayfair. From day one you’re immersed, learning about the tech stack, the company, working on actual projects, and building relationships. For anyone interested, come with an open mind and be ready to learn.
In the first two months, I completed two onboarding projects which required me to learn PHP and JavaScript—languages I’d never touched before. But at no point did I feel like I had been thrown to the wolves, as I was surrounded by my cohort of thirty other engineers. Everyone helped one another and we all learned to be successful engineers. I still learn from this crew.
The third month of Labs is when I got matched to the Sorting Hat team within the Data Science organization. I got to work with some phenomenal data scientists on several machine learning applications, including the optimization of email notification and email content My role was to support the data scientists I worked with regarding all things engineering. What did this mean in practice? Writing Flask web services to serve machine learning model predictions, building ETL pipelines in Airflow, re-implementing old jobs in PySpark, converting old python code into production quality projects, and building Python libraries to standardize common copy/paste code.
How has your role changed since you started here?
Over time my role has changed greatly.
When I first started working with Wayfair Data Science, it was a relatively small team supported by only a few engineers. Since then, the data science organization (notice I didn’t use “team” there) and the engineering support staff have grown considerably.
Along that growth journey, we somewhat lost the ability to meaningfully support data scientists in their engineering efforts in the one-on-one capacity we had started with. We had to change our thinking. If we couldn’t always assist them individually, how could we empower them instead?
We’ve since become a platform team. Whereas previously I was heavily involved in the launch of numerous ML applications, I am now building tooling and infrastructure to pull more of the nitty-gritty of production systems away from data scientists and jam it under the hood. By abstracting the operational work (requisitioning VMs, setting up a new repo, getting build/deploy jobs written, etc.) data scientists can now spend more time thinking about data science and less about operational engineering challenges.
Michael’s Day-to-Day
Every day is different, but can you outline what a typical day looks like for you?
As a technical contributor, I’m getting about 60% true “hands on keyboard” time. I’ve elected to spend some of my time on organizational challenges (recruiting, interviewing, working with our Wayfair Labs engineers)—let’s call that 20%. The last 20% of my time is spent fulfilling a product management style role.
Are there particular seasons of the year that are busier than others? If so why?
For our team, the work pace is fairly consistent. The closest we get to a “busy season” is the time spent in preparation for Cyber 5, when much of Wayfair is crushing. As our team owns the platform which currently runs thirty or so machine learning applications, serving approximately a combined 200-500 million requests per day, we are on alert during Cyber 5 to make sure everything runs smoothly.
Collaboration Across Wayfair
What are some of the different teams that you work with?
I’m super grateful that I get to work with so many different and talented teams on many amazing products. Due to our team’s unique position, we often act as the interface between the engineering and data science organizations. As a result, I’ve gotten to work with teams (engineers and product managers) from Data Science, Transportation, Storefront, B2B, Ad Tech, Supply Chain, and more.
Our machine learning platform also wouldn’t be possible without the help of our amazing Kubernetes team. We work with them a lot.
What is your role in your relationship to product?
For the majority of my time at Wayfair, I haven’t had a dedicated product partner. This means that I had the opportunity to raise my hand and do much of that work myself. Today, as a platform team, much of that work is selling and educating internal Wayfair teams on our platform’s offerings. In the past that was interfacing with data scientists, product, and engineers to help manage project roadmap and success.
Michael’s Favorite Parts of the Job
What are some tech tools that you have enjoyed using at Wayfair?
Docker and Kubernetes are my favorite tools, hands down. Both of these are super approachable and wildly flexible. Not too long ago, Wayfair decided to cut over to containers as a company, and it was amazing to see some of the first applications running on Kubernetes at Wayfair. Now, we are working to get all of our machine learning applications containerized and deployed on our platform “Sweetwater.” Right now we’re at about 30 applications with probably another 15 in development.
Having a large engineering organization means that there are experts on everything and learning new tech is both easy and realistic.
What is the most interesting problem you are currently trying to solve?
How do we make machine learning at Wayfair more approachable, efficient, and reliable?
What recent project are you most proud of?
Our "Sweetwater" platform is leveraged by data scientists and engineers alike to rapidly deploy new machine learning applications.
Prior to Sweetwater, deploying a new machine learning application could take months after the models were finalized. Each application typically required refactoring code, building a new service, acquiring infrastructure, adding logging/monitoring, setting up build/deploy etc, so we abstracted away as much of that as possible. Now, applications are often built and deployed on the same day! We’re aiming to get this process down to minutes through some really cool custom code generation scripts, code defined contracts, and pipelines.
