As we finally head towards the end of 2020, people are thinking ahead to a new year that is hopefully full of positive changes.  For many people, the stresses of 2020 led to weight gain (not to mention that people working from home suddenly discovered the down-side to having access to their fridges 24/7).  But even before 2020 overweight and obesity have been rampant in most developed countries. 

Every New Year, millions of people resolve to diet and hit the gym, but despite these periodic efforts, obesity rates have steadily climbed over the past decades.  Clearly it is hard to stick to a diet, and there is the ever constant search for the “best diet”.  In this post, I’ll describe our recent paper that examined which diets are easiest to stick to by looking at trends in Internet searches for diet recipes. 

If you want the TL;DR summary: the Paleo diet appeared to be easiest to stick to throughout the year, with Weight Watchers and Low Carb being close seconds… during the pandemic lockdown, the Paleo diet appeared easiest to stick to (more people actually started it than dropped out, in fact!), and second place prize went to Low Carb. Weight Watchers fared particularly poorly during the pandemic, perhaps because in-person support groups that some WW dieters attend were curtailed.

Knowing which diets are easiest to stick to could potentially help people to have better success at losing weight, but the problem is that clinical diet studies are very hard to do.  Researchers have to recruit participants and follow them for weeks, months, or even years, and the dropout rate of participants is very high, so in many cases the researchers don’t know why the participants dropped out or what happened to their weight afterwards.  And because of the intense monitoring associated with the studies, the results tend not to be reflective of people “in the wild” who decide to diet on their own without any medical monitoring. It is unclear whether wild dieters find it easier or harder to stick to a diet compared to clinical dieters.  Also, because of the difficulties in doing clinical studies, it is almost unheard of to compare two different diets side-by-side in the same study to see how well people lost weight and were able to maintain the diet.

In 2019, myself and a group of collaborators took a data science look at which diets appear easiest for people to stick to, and published the results in our paper “How long do people stick to a diet resolution? A digital epidemiological estimation of weight loss diet persistence“.  Using US Internet search data for diet recipes, we looked at a variety of popular diets, including Weight Watchers, South Beach, Paleo, Low Carb, and Low Fat diets, and examined how long new January dieters appeared to stick to them (sadly, the Keto diet hadn’t been around long enough by then to give us enough data to include in the study).

Here is what several years of Google search trend data look like for various diets (note that some diets have longer span of data than others):

There is a sharp peak in searches for all diet recipes every January, and by using a mathematical model to examine how quickly the recipe searches dropped off our team was able to estimate how long people appeared to stick to each kind of diet (assuming of course that some cross-section of people on those diets at any one time are searching for diet recipes). In addition to the large spikes in new dieters each January, the Internet search data also show a distinct dip in the number of dieters during the US winter holiday season (from late November to the end of December… you can see that in the plot above there is a dip in diet recipe searches for a month or two right before every January peak).  Clearly for many people the lure of yummy holiday food is too much during that period. Assessing how easy a diet is to stick to is thus not just about New Year’s resolution diets, but also how easy it is for longer-term dieters to stick to a plan even during difficult times of year when we’re awash in delectable food.

Assessing which diets perform best

With our mathematical model that we fit to the trend data (see our paper for details), we estimated how long new January dieters appeared to stick to their diets before ceasing to search for recipes.  We found that the easiest diets to stick to appeared to be Weight Watchers, Paleo, and Low Carb, all of which had compliance times of around 5 to 6 weeks.  The worst were Low Fat and South Beach, with only 4 week and 3 week compliance, respectively.  As far as December dieting goes, Paleo had by far the fewest people dropping out (with only around 15% of dieters dropping out), and the rest had between 20% to around 40% dropping out.  Here is a chart summarising the results:

Characteristics of the best performing diets

What are the characteristics of our best performing diets?  The Paleo diet allows a wide range of unprocessed foods with a focus on foods that hunter gatherers would have found familiar.  The Paleo diet thus excludes grains, legumes, dairy, most alcohol (although some people feel red wine is OK), and refined sugars.  Meat, eggs, veggies, fruits and honey are all allowed on the diet, so it doesn’t restrict carbs or fat.  In general, the diet encourages a fairly wide range of food, which is perhaps what makes it easier to stick to.  Studies have also shown that the Paleo diet is also better for weight loss compared to some other popular diets.

Similarly, the Weight Watchers diet also encourages a wide range of foods, with a focus on moderation in quantities and encouraging healthy choices over processed food.  The Low Carb diet doesn’t encourage as wide a range of food, however some people find Low Carb diets fairly easy to stick to, at least for limited periods.  As we see in our data though, the times of the year when there are lots of cakes and cookies can be the downfall of Low Carb dieters.

While using this data mining approach meant that we could compare diets side-by-side, it did have the drawback that we didn’t know how much people lost on each of the diets, and we don’t know their reasons for ceasing to search for recipes (was it really because they had stopped the diet?).  We also didn’t know if people switched diets.

Which diet was easiest to stick to during the pandemic lockdown?

We now have over a full year more of recipe search data since we wrote our paper.  We might wonder… did people find it harder to stick to their diets during the pandemic lockdowns in various US states that occurred between March and April of 2020?  And, if so, which diets had the worst dropout rates?

To examine this, I did what is called “trend-correcting” the data.  Diets wax and wane in popularity over the years, and first I had to divide out the long term trends.  Here is what the data for 2020 look like when I do that, overlaid on the trend-corrected data from 2019:

It’s pretty clear that dieters (except those on the Paleo diet!) seemed to have a lot more trouble sticking to their diets during March and April of 2020 when social distancing from their refrigerator became difficult.  Then for most diets there was a rebound effect later in 2020 where more people than usual appeared to be dieting (likely in a bid to lose their pandemic lockdown pounds).

If we look at the percentage difference between the pink 2020 curve and the prediction from the year before, here is what we get:

The Paleo diet actually gained followers during the pandemic, with up to 50% more people following that diet than we would have expected compared to the year before!  The next easiest diet for people to adhere to appeared to be the Low Carb diet, which only lost around 20% more dieters than usual during the lock down.  The hardest diets for people to follow during the lockdown were Weight Watchers and South Beach.  For Weight Watchers, part of the issue might have been that some people following it choose to attend in-person support meetings, and during the lockdown these were likely sharply curtailed.  This also likely explains why it really didn’t rebound as much as other diets after the lockdown… people were still avoiding group settings.

Summary

For new January dieters, holiday dieters, and lockdown dieters it was Paleo diet for the win!

Second place: Low Carb.