The Math Says Yes, But Human Behavior Says No

Data scientists are busy writing algorithms to optimize employee productivity, improve trucking routes, and update retail prices on the fly. But those pesky humans and their demands for a reasonable schedule and consistent pricing keep getting in the way. Which then proves when it comes to algorithmic model development, “real world” human behavior is the hard part.

Courtesy of Flickr. By Nathan Gibbs
Courtesy of Flickr. By Nathan Gibbs

The traveling sales person is still one of the most interesting math problems in terms of optimization. The problem can be summarized this way: take a sales person and their accounts in various cities. Now, optimize the shortest possible route for that sales person to visit each account once, and then come back home, all within a defined time period. What may sound like an easy problem to solve is easy is actually one bedeviling planners to this day—and it ultimately involves a lot more than math.

While the math in the traveling sales person has been painstakingly improved over the years, the human element is still a very large factor in real world implementations. That’s because while the most mathematically optimized route for a sales person might be visiting three accounts in one day, it doesn’t take into account the schedules of those customers he/she intends to visit, necessary employee bathroom breaks, hotel availability and the fact the sales person also wants to visit their ailing grandmother after stop two.

The traveling salesperson problem also applies to transportation optimization. And again, sometimes the math doesn’t add up for human beings. For example, at particular shipping company, optimization software showed the best route combination for delivering packages. However, there was one small catch: the most optimized route ignored Teamster and Federal safety rules of drivers needing to take pre-defined breaks, and even naps after a certain amount of hours on the road.

Modeling is getting better though. An article in Nautilus shows how transportation models are now incorporating not only the most mathematically optimized route, but also human variables such as the “happiness” of drivers. For instance, did the driver have a life event such as death in the family? Do they prefer a certain route? How reliable are they in terms of delivering the goods on time? And plenty of other softer variables.

Sometimes optimization software just flat out misses the mark. I’m reminded of a big chain retail store that tried to use software to schedule employee shifts. The algorithm looked at variables such as store busyness, employee sales figures, weather conditions, and employee preferences and then mapped out an “ideal” schedule.

Too bad the human element was missing though as some employees were scheduled 9a-1pm and then 5p-9pm the same day, essentially swallowing their mornings and evenings whole. The algorithm essentially ignored the costs of employees having to travel back and forth to work, much less the softer side of quality of life for employees struggling to balance their day around two shifts with a four hour gap in between. Rest assured that while the store employee schedule was “optimized,” employee job satisfaction took a tumble.

Lastly, online retailers are experimenting with pricing optimization in near real time. You’ve undoubtedly seen such pricing models in action; you place an item in your shopping cart, but don’t buy it. Then, a couple hours later you come back to your shopping cart and the price has jumped a few dollars. This dynamic pricing has caused some customers to cry foul, especially because to some, it feels a lot like “bait and switch.” And while dynamic online pricing is becoming more commonplace, it doesn’t mean that consumers are going to like it—especially because humans have a preference for consistency.

Thus, from pricing, employee scheduling, to trucking route optimization, the computers say one thing, but sometimes humans beg to differ. Indeed, there’s a constant push-pull between mathematics and the human element of what’s practical and reasonable. As our society becomes more numbers and computer driven and thereby “optimized,” expect such battles to continue until a comfortable equilibrium can be achieved.  That is, until the computers don’t need us anymore. Then all bets are off.



  1. This is similar to the problem with economics and the argument for behavioral economics. Data doesn’t always reflect how people behave. The data reflects their “digital persona”. The data appears logical and rational after the fact. But the data doesn’t reflect the random thoughts and “out of system” meanderings that went on in discussion and conversations.

    Algorithms assume logical, rational behavior. The algorithms are written by people who are governed by behavior; experiences and biases. Imposing these behaviors on data in the algorithm can result in correlations that reflect the algorithm creator’s behavioral characteristics based on digital personas but not necessarily those of the “real people”.

    • Richard, thanks so much for commenting. You are very right, real world behavior is often different than what we tell others we do, or how we respond to surveys. And the Financial Crisis of 2008 (and others) has taught us time and again that in times of “normality” we actually do behave much like the models say we should, but add in a dash of panic and all heck breaks loose (so do the models,and so do the bank accounts of many a hedge fund!).

      I really appreciate that you took time to comment!

  2. Appreciating the time and energy you put into your blog and detailed information you provide.
    It’s awesome to come across a blog every once in a while that isn’t the same old rehashed information.
    Wonderful read! I’ve saved your site and I’m adding your RSS feeds to my Google account.

    • Jeff thank you for the nice compliment. It’s always nice to be appreciated. Let me know if there are specific topics you want me to cover. If I am able I will give it a whirl!

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s