The Great Grocery Game

It was about 9 o'clock in the morning, early Saturday, with the sun shining and my refrigerator totally empty.

I had to buy groceries. Luckily I live in San Francisco, a city filled with startups who can’t wait to outsource the minutiae of everyday life. At the tap of a button a complete stranger will come over to my house and live life on my behalf. It’s both disturbing and beautiful.

With so many choices it can be hard to know which one to pick. So instead of take a gamble, I decided to put them to the test. Three delivery services, head-to-head, in a race to purchase and deliver an identical order of groceries on a Saturday afternoon. Game on.

The Experiment

I started by compiling my grocery list. I took my last 12 trips to Safeway and built a comprehensive database of every item I had purchased. From that I crafted a standard basket - the order that I felt was representative of a normal trip. After a few adjustments to make sure I didn’t get too much food, I ended up with this.

While this list may seem simple, it includes some hidden landmines:

With my cart assembled it was time to order.

Ordering Process

When ordering on Postmates or Instacart, you start by choosing the store. You then go through a loop of picking a category, scrolling through the list to find your item, and then adding it to your cart.

The merchandising in these apps is terrible. Sometimes the item you want is right at the top of the list. Other times you have to scroll through 15 screens of junk to find what you want.

Search in these apps can be painful. Typing in a brand will bring up every single item under that brand name across the store. When you get in to store brands there’s no way you’ll find what you need.

The biggest value to Instacart over Postmates is the ability to schedule ahead and plan out substitutes. Instacart let’s you take any order and pick the day it will show up where Postmates restricts you to just “Get It Now”. You can also pick and choose what alternative items you want on Instacart in the event your favorites are out of stock.

Ordering on Exec was much simpler in comparison. Getting an Exec Errand is just a simple text box to fill out. My typical grocery store process involves my wife texting me a list and me going to the store so it was easy to just copy & paste the list in. Having simple text is great for fuzzy search. For example, I asked Exec for the cheapest bacon on sale and I ended up with a much higher quality product at the same price as the others. The big issue is specificity. For certain items we struggled with how to word our request such that it was perfectly clear what we wanted.

I took my standard cart and entered it into each app without triggering delivery, double checking each to make sure they were at parity.

Cost

Each service has a very different cost structure. No one service is the best at any pricepoint.

Postmates has a dynamic pricing model based on time, demand, distance, and order size. I paid $12 for delivery on this order, though I’ve seen the price be as low as $7.50. They also charge a flat 5% fee on top of the purchase price.

Exec has a model that’s purely labor based. The more time your Exec spends working, the more you pay. Their rate is $25/hour with a 45 minute minimum. On top of that, you pay 50¢/mile for travel and a 3% fee on any purchases.

Instacart’s pricing is the most ambiguous. They charge a flat delivery fee based on order size: $7.99 for under $35 and $3.99 otherwise. Often on the weekends they will offer free delivery for orders over $60. Where it gets tricky is that you pay the retail price for grocery items plus an unspecified fee on top. So if you’re shopping at Safeway where discounting is rampant, it’s not clear just how much more you’re paying. For my order, the Instacart cost for any food item was ~20% over what I would pay at the checkout.

My Costs

Items Food Cost Fees Delivery Total
Postmates 13 $44.34 $2.22 $12.00 $58.56
Exec14 $50.71 $19.50 $0$70.21
Instacart11 $33.71 $6.44 $3.99$44.14

Cost By Order Size

This graph helps show how the costs change with order size.

cost-graph.png

Timeline

When noon on Saturday rolled around, I started the race. I triggered each app simultaneously to delivery my groceries as soon as possible. I kept a record of every call, text, and push notification I received and compiled them together. This timeline tracks all communication I received along the way. grocery-timeline.png Each event is an SMS or Push Notification unless otherwise specified

Results

As the orders came in I went through each item and cataloged them along the following criteria

So how did they do? As expected, each service had no problem fulfilling the easy items on my list. Though, for about half of my grocery list, there were problems.

Exec

exec.JPG

Exec was the first order to arrive with a total delivery time of 65 minutes. Exec makes no promises on delivery time and if they run over, it costs you directly. Of all three services, they were the only ones able to fill my 14 item order. Overall the order quality was top notch.

Postmates

postmates.JPG

Postmates was the second order to arrive with a total delivery time of 75 minutes. Postmates promises a 60 minute delivery, making them 15 minutes late. My postmate was the only competitor who was a woman, a detail that weighed heavily in their favor. She did a fantastic job on the produce, blowing away everyone on her handling of my avocado request.

Instacart

instacart.JPG

Instacart was the third order to arrive with a total delivery time of 111 minutes. They promise delivery within a 3 hour window, so this delivery was well within the normal range.

Quality Accuracy
Item Postmates Exec Instacart              Postmates Exec Instacart
16oz Cinnamon Toast Crunch
1 Quart, 1% Lucerne Milk
1lb Strawberries
1x Avocado
McCormick Guacamole Mix
2 Liter Coke Zero
Safeway Wheat Crackers
Eat Right Sesame Chicken Frozen Dinner ~
Chobani Non-fat Strawberry
Chobani Non-fat Raspberry
1lb Chicken Breast, Boneless Skinless ~ ~
1 Bottle Cupcake Sauvignon Blanc
3x Boston Crème Bakery Donuts
1 Package Bacon
13/13 14/14 9/11 11/14 12/14 8/13
Postmates Exec Instacart              Postmates Exec Instacart

Summary

Postmates was hands down the best value per dollar. With their inexpensive delivery fees and great accuracy, I feel like I had a great experience relative to the amount I paid. It’s hard to say whether this was due to the startup or because the competitor was a female and thus had more attention to detail.

Exec excelled at speed and accuracy. I was stunned by how fast they put my order together and how spot on the items were. My Exec communicated promptly; I never felt like I was out of the loop. However, the one thing they messed up on they got majorly wrong, partially due to me not being specific enough.

Instacart wins at convenience and price. My wife liked the fact that Instacart “got the job done without getting in your way”. They try to frontload all of the decision making so you won’t be bothered when the job is happening. And despite their cloudy pricing model they came out cheapest for small-to-medium shopping trips.

Going forward I’ll use Exec for all my same-day trips and in the event I need complicated shopping orders. I’ll continue to use Instacart for small, simple orders that I want to plan in advance.


If you’d like to use Postmates, click here to sign up

If you’d like to use Exec, click here to sign up

If you’d like to use Instacart, click here to sign up

Special thanks to my wife for helping me organize and coordinate this experiment and to Kim MacVaugh for playing along.

 
222
Kudos
 
222
Kudos

Read this next

Building a Data-Informed Startup

At MinoMonsters, we use data every single day. We spend a lot of time looking at it and we believe it’s important. Today we will monitor and record over 10M points of data. But what are we doing with all that information?... Continue →