Escape from the “data request” backlog: balancing competing priorities @ Blue Apron

When I first joined Blue Apron, data support at the company was in dire straits. Data requests were communicated by sending an email to a group alias, which was copied into Asana. Then, once a week, the team would sit in a cramped conference room to stack-rank new requests in comparison to ongoing or previously requested projects. Immediately after, the team would meet with the entire company — yes, you read that right, the entire company —to discuss prioritization of data requests, and ostensibly, timelines.

As you would guess, there were a few drawbacks to this process.

First, projects took longer than the team expected because data quality was unpredictable, so our velocity was slow. Second, the backlog of requests grew exponentially as the company scaled. More people = more questions = more emails to “data-requests@.” Third, leaders learned that the best way to get their questions answered quickly was to roll deep to the company-wide prioritization. It was not uncommon to see an entire department walk in to the room, presumably just to make a point. Fourth, the data analysts/modelers/scientists were demoralized because it was impossible to make headway, the list only grew and grew, and they had no agency to direct their own work streams.

So what did we do? The first step was to make our prioritization process more opaque — yes, more opaque — by shutting the company-wide meeting down and ingesting requests through a form that included a business justification for their question. The added effort required to defend requests eliminated casual questions, and reducing transparency about our overall pipeline created a state of information asymmetry that improved our team’s ability to negotiate with business leaders. Finally, I had some hard conversations with members of the business about the realities of our team’s resourcing and throughput, which resulted in about 40% of requests being deleted from the backlog.

Eventually, the Analytics team migrated away from this reactive data request pipeline to strategic quarterly roadmaps, which I’ll expand upon in another post. But it was amazing to see the change in the team’s momentum and morale after we reduced distractions, established air cover, and provided them with agency to proactively address items coming in through the queue.

Although every company varies in its data and analytics infrastructure, resourcing, and talent, it seems that balancing short- and long-term constraints are a constant point of tension for service-oriented functions such as an Analytics team, and at Blue Apron we are constantly working to strike a balance (and, only occasionally, running around like our hair is on fire). Here are some tips that we’ve identified to make things work as we have evolved as a team:

• Find scalable solutions for repeated problems. For example, our consumer analytics team is extremely short-staffed right now, and dozens of A/B tests for our digital platform are piling up as we analyze initiatives related to our physical product (which have not been A/B tested, and thus require more time/lift). Instead of spending time quickly churning out individual analyses, an analyst on the team is spending the next few weeks building an A/B testing tool that ingests ongoing test groups and then surfaces comparisons between test/control across key metrics such as LTV, churn, order rate, conversion, etc. Because many of our analyses also segment results based on key attributes, this tool will also allow for filtering across those commonly occurring segments to surface interaction effects. The benefit of prioritizing this scalable solution is that it will allow our digital PM’s to self-serve while maintaining an acceptable degree of rigor, while allowing the analyst to dig in to a bigger project that is more engaging that repetition of the same analysis over and over again.
• Gain cross-functional alignment on the team’s priorities by surfacing trade-offs and being transparent about priorities. When our team was very small (4 analysts), we were faced with extreme resource constraints in terms of what we could work on. One quarter, we were so constrained that we were forced to put together a simple list of 5 projects that we would work on, the estimated headcount for this effort, and timelines for completion. Everything on the list was a big project with (1) a high degree of urgency, where (2) analytical rigor was incredibly important — and, in fact, “analytical support for our S1 filing” was actually not #1 on the list. We then circulated our list to leadership at the company, and asked them to weigh in to the group if they disputed our prioritization so we could have a conversation about trade-offs.
• Say no — with a smile. In the example above, we had to make some hard trade-offs, and to save us from having the same conversation over and over again, we provided transparent answers to our partners so they could make plans rather than kicking the can down the road. In general, I have found that our business partners appreciate candor around constraints, especially if you offer an explanation about the priorities that are taking precedence so they can escalate to their leadership if they challenge the broader decision.
• Highlight that the team is a servant to the business, not an individual group. Given that we are a centralized team with embedded partnerships, sometimes it helps to remind folks that our team serves a broader set of needs and that the rising tide lifts all boats.