The Wall Street Journal Takes a Deep Dive Into Why Golfers Overestimate Their Ability
(Click here to read the article on wsj.com)
By Brian Costa
Consider the recreational golfer eyeing the green, armed with better technology than the sport has ever seen. Laser-guided binoculars report the precise yardage to the flagstick, while GPS devices instantly give the distances to the front, middle and back of the green.
Yet new data suggests that even players who know with increasing ease how far they must hit the ball are still dogged by a persistent psychological flaw: They think they will hit the ball farther than they actually do.
Forty percent of approach shots land short of the green, eight times the percentage that land behind the green, according to a study of more than six million such shots recorded by Arccos, a shot-tracking system based on sensors attached to clubs.
The data isn’t merely an indication of golfers’ collective ability. It reveals a persistent—and tough to overcome—cognitive bias that causes most players to regularly choose the wrong club.
“You end up making decisions based on your best shots,” said Arccos chief executive Sal Syed. “That’s not smart.”
Jeff Beglane, a 31-year-old insurance adjuster in Westford, Mass., with a 12 handicap, said if he is 150 yards out, he will likely pull out his 9-iron. His reasoning: That’s about how far the ball travels when he hits a 9-iron flush. “Then I leave it 20 yards short and I’m like, hmm, why did it land short?” he said.
The answers can include variables such as wind and elevation changes. Mostly, though, the disappointing result is due to the inability of most players to hit their best shot consistently. “In reality, how many times have I hit that shot, and how many times have I attempted it?” Beglane said. “But in the moment, you’re not thinking about that.”
David Dunning, a psychology professor at the University of Michigan, said the tendency reflected in the Arccos data is consistent with a phenomenon he has studied for two decades: why people overestimate their ability.
In 1999, he co-authored a study that found that people tend to hold overly favorable views of their abilities in a wide range of social and intellectual disciplines. The study attributed incompetence, in part, to the inability of the incompetent to recognize their lack of ability. But the tendency isn’t limited to the deeply flawed.
In a separate 1992 study, engineers at two Silicon Valley software firms were asked how they ranked relative to their co-workers. Forty-two percent of engineers at one firm and 32% of engineers at the other firm rated themselves in the top 5% of performers. Similar mathematical impossibilities have been found in informal surveys of students at business schools.
“When we think about ourselves and how good we are, we think about our potential—the person we’re going to be, the golfer we’re going to be after we take enough lessons,” Dunning said. “People tend to give a little bit too much weight to the positive side of their history and discount too much the negative side.”
RELATED: Join the Big Data Revolution and Get Your Arccos Caddie Smart Sensors Today!
Even depth perception can be skewed by wishful thinking. Dunning co-authored a 2009 study that found that people tend to perceive desirable objects as being closer to them than undesirable objects placed at the same distance. What confuses some golfers is that, in many respects, confidence is essential. Players are told to visualize their best shot and swing free of self-doubt. But choosing a club requires a more pragmatic assessment of potential outcomes.
“You’d like to swing always with total confidence,” said golf psychologist Bob Rotella. “But with your strategy decisions, you’re probably better off erring on the side of being more conservative.”
Technological advancements only recently enabled this to be easily quantified.
Syed, 36, launched Arccos shortly after getting his MBA from Yale in 2012. The company’s sensors work in tandem with GPS and Bluetooth technology to log the start and end point of every shot in relation to the hole.
The company declined to say how many users it has, but said the shots in its database were hit by male and female players of all age and ability levels, and included only those where the player had an unobstructed shot at the green.
Initially, the system was designed merely to give golfers better data. By knowing, for instance, the actual average distance hit with an 8-iron over time, players could theoretically make better club choices. The company says the average user has improved his or her handicap by more than three strokes.
But a new version of Arccos to be released later this month goes a step further. In collaboration with Microsoft , it will essentially look to do the job that was once the domain of the caddy—making recommendations on strategy—using machine learning.
Syed said Arccos will offer real-time club recommendations on nearly every golf hole in the world based on a range of factors. Among them: data on how a player has fared with each club, how similar players have played similar holes in the past, the dimensions of a given hole and even weather conditions.
“This is the equivalent of a play being called in the NFL by a computer,” Syed said.
Replacing human decision-making to such an extent is likely to be viewed with some initial skepticism. When asked about Arccos’s data on approach shots, Rotella noted that not all of them that came up short represented a mistake. At times, he said, if the hole is near the front of the green, it can be better to miss short than long.
But clearly, golfers are not plagued by underestimating their ability. In the Arccos study, only 5% of approach shots landed past the back edge of green, while 37% landed on the green, 9% landed left and 10% landed right.
“One advantage of that cognitive bias is you feel good about it,” Syed said. “When we tell you that you actually don’t hit a 7-iron 150 yards, you hit it 138, it might crush you. But once you internalize that information, you’re going to make better decisions.”
