How we built a resilient solution inspired by the design of seafaring vessels

At Microsoft, we define resiliency as the ability of a system to gracefully handle and recover from failure. In the cloud — where applications run on commodity hardware, communicate over the internet, and compete for bandwidth and resources — failure is inevitable. The ability to detect that failure and recover quickly are the hallmarks of resilient data products. In this article, the second of two in a series (see part 1 here), I describe how we adapted a simple resiliency design pattern to ship a resilient data product as fast as we could.

My team recently took dependencies on over…


Anomaly detection through the lens of online dating

Author Matt Wright is joined for this article by co-author Deepsha Menghani

Data scientists on the Customer Growth Analytics team at Microsoft are spoiled. For years we’ve enjoyed the privilege of working alongside a world-class data engineering team, whose charter includes maintaining our high-quality data platform. They bring innovative solutions to tough problems, but with limited capacity, they aren’t always able to provide immediate support, and sometimes we have to get creative.

My team recently had to ingest and integrate new data from over 90 independent sources. We were designing a model that required this data, and we had to…


What I’ve learned about family and work relationships during mandatory quarantine

Since the beginning of the quarantine, my wife and I have gone out for a jog with our son every evening and returned home to the smell of our neighbors smoking pot. They’re an older couple — retired with two adult children — and they’re clearly enjoying their empty nest. They have matching silver ponytails. They wear the same long-sleeve t-shirts under the same Patagonia vests. They wear the same blue jeans and the same running shoes. They’re a great example of how two people can become one after decades of cohabitation. Their difference in height is their only distinguishing…


Because expertise in data science may never be enough.

Ten people from sales and marketing have gathered in a conference room to hear you speak. You’re the only data scientist, you’re nervous, and you’re reading directly from the slides on the screen. You’re competing for the attention of everyone in the room, but their smartphones are winning that fight. It seems like no one cares about what you’re saying. “Random forest this. Confusion matrix that. Next slide.” You catch someone yawning, and another finally speaks up. “While this is all very interesting, I don’t know what to do with it.”

I’ve been witness to this cringe-making scene on a…


How Hollywood horror inspires us to focus on problems that yield measurable results

Data scientists are curious, creative, and resourceful. They come from disparate backgrounds, and they’re capable of finding wonder in the mundane. They’re constantly asking why and what if and finding pleasure in the process. Sometimes these questions yield interesting results, but interesting does not imply impactful. If data scientists want to prove the value of their work, they’ll need to occasionally highlight its impact. They must apply their curiosity to scenarios that will yield measurable results. They must focus on scenarios that scream for data science.

What do these scenarios look like?

A good way to start identifying them is…


Three months ago, I told my boss, “That’s it. I’m not coming into work anymore.” And I haven’t, except on one occasion to pick up the plant on my desk. But in a few days I’ll be headed back to work because today my son turns three months old, and I’ve exhausted the generous parental leave policy at Microsoft. Three months without (normal) work! I can hardly believe it. The days have been long, and often blurred together over consecutive sleepless nights. But the weeks have gone by quickly, and I am so grateful to have had the opportunity to…


The late nineties at Lewis & Clark High School in Spokane, WA

When I was a freshman in high school, my second-period math teacher gave extra-credit points and full-size candy bars to students who purchased a TI-83 graphing calculators and brought them to school. He wrote their names on the whiteboard at front of the classroom so that we could all identify the most privileged among us. The rest of in class who couldn’t afford a $100 calculator shared older models owned by the high school. But they didn’t align with the exercise in our text book and we had to share them, which made learning difficult. We didn’t get the extra…


I live in a story, and that story is a sub narrative of countless other stories where dramatic plot twists and routine outcomes equally abound.

  1. In my first year at UW, I had a part-time job working for a Near Eastern Languages and Civilization professor, assisting in the translation and digital transcription of an Arabic language travelogue written by 19-year-old Alexander Svoboda in late 19th-Century Iraq. We were extracting data from the story, joining it with external data, wrapping HTML tags around proper nouns, and linking them to supplementary information. For example, there is a tag on the in the…

Matt Wright

Principal Data Scientist at Microsoft, U.S. Army veteran, and aspiring polymath.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store