Fraser Institute's wait-time survey: Does it still count if most doctors ignored it?

Fill out this survey and have a chance to win $2,000 — that's the annual enticement from the Fraser Institute, an offer made to thousands of doctors whose names appeared on a mailing list.

But it wasn't tempting enough to get doctors to participate.

No medical oncologists in Saskatchewan, Manitoba or New Brunswick took the bait.  

Zero responses came back from radiation oncologists in New Brunswick or from cardiovascular surgeons in Manitoba. 

Not a single plastic surgeon in Prince Edward Island or Newfoundland answered the questionnaire.

Across Canada, just seven per cent of psychiatrists on the list bothered to answer the short survey asking them to estimate how long their patients are waiting for care.

The Fraser Institute is a think-tank that has long advocated for more private-sector options in the Canadian health-care system.

Every year for more than two decades it has published a gloomy report about wait times for health care. This year's came out on Wednesday. 

1 in 5 doctors respond

And every year only around one in five doctors participate, despite that offer of a $2,000 cash draw. In Ontario, less than 15 per cent of all specialists on the mailing list weighed in on the issue of wait times.

The survey — just six questions — doesn't ask the busy specialists to check their patient records or submit any hard patient data. Doctors are asked only to estimate how long their patients wait to see them, and then wait for diagnostic tests and surgeries. 

We would love it if we had more responses.- Bacchus Barua, Fraser Institute

"We're absolutely clear about the fact that this is a survey," said Bacchus Barua, one of the authors of the report. "This is not something we can control. We would love it if we had more responses."

As a way to measure wait times, it's "preposterous," said Steven Lewis, a health policy consultant based in Saskatoon. "Why not use a thermometer rather than asking people for their opinion about the weather?"

'Participation bias' skews results 

"Physicians are inundated with surveys, so they pick and choose," Lewis said. "It's also plausible that the most frustrated physicians respond, representing the worst of wait-time experiences."

It's called "participation bias"— a well-established fact in statistical science that people who take the time to answer a survey are different than the ones who ignore it.

"Individuals sometimes complete surveys when they are having particular difficulties with the issue being studied," said Monique Gignac, a professor at the Dalla Lana School of Public Health at the University of Toronto. In other words, doctors who don't think wait times are unreasonable might not be motivated to fill out the survey.

"Doctors were also asked to mentally average wait times for what might have been very different conditions and experiences among their patients," she said. "As a result, the questions may have introduced a number of biases into the study."

'To inform patients'

For the Fraser Institute, wait times have held a historic fascination. Its founder, Michael Walker, is cited in this year's report as the person "responsible for helping navigate the beginnings of wait-time measurement in Canada."

Back in the late 1980s, Walker was a vocal critic of the Canadian health-care system.

"High-income Canadians effectively are prevented from using their incomes to buy a higher standard of health-care equipment and service," he wrote in a 1989 report for the Heritage Foundation, a U.S. conservative think-tank. 

Three years later, Walker started releasing the annual Canadian wait-time surveys. In the 1992 report, he stated that the Fraser Institute's objective was "the redirection of public attention to the role of competitive markets in providing for the well-being of Canadians."

In more recent versions of the report, there is no longer any discussion of policy alternatives, just the facts — as reported by a fraction of the country's specialists.

Barua said he only has one agenda: "to inform patients about wait times in Canada."

However, the 2013 survey has been entered as evidence in a constitutional challenge against medicare. The plaintiff, investor-owned Cambie Surgeries Corporation in Vancouver and its CEO, Dr. Brian Day, aim to change the law to allow private payment for medically necessary hospital and physician care. The attorneys general of B.C. and Canada are intervening on behalf of the Canada Health Act and the B.C. Medicare Protection Act.

• Landmark private health care lawsuit heads to court
• Should Canada have a hybrid public-private health care system?

Health policy analyst Karen Palmer was sitting in on the trial when one of the Fraser Institute's wait-time study researchers was called as an expert witness for the private clinic. 

Palmer, an adjunct professor at Simon Fraser University, said the flaws in the reports methodology were exposed under cross-examination from the lawyers representing the B.C. attorney general.

Provincial data available

"The Fraser Institute methodology is — and I use the word carefully — an abomination," Lewis said, adding that there's a better way: use real data.  

"If you tag a referral to the billing code for an office visit, you can easily calculate the time between seeing the GP and getting referred, and the visit to the specialist," he said.

'The Fraser Institute methodology is — and I use the word carefully — an abomination.'- Steven Lewis, health policy analyst 

Barua said the Fraser Institute doesn't have the resources to do a more detailed survey, and sticking to this format allows them to compare results with the older surveys.

There is another source of data about wait times in Canada. Since 2006, the Canadian Institute for Health Information has used provincial data to track wait times in five priority areas: cancer, heart surgery, joint replacement, sight restoration and diagnostic imaging.

The 2016 report, released in March, concluded that: "Wait times for urgent procedures were at or approaching benchmark targets." 

It found mixed results for some elective procedures, but overall painted a much brighter wait-time picture for Canadians.

"Despite this lack of consistent improvement in wait times for joint replacement and cataract surgery, Canada continues to perform well on median wait times for these procedures when compared internationally," the report said.

But what about all of those anecdotes, people everyone knows who've had horrific waits? Palmer said it's important to take a close look at those examples.

In most cases, Palmer believes there's been a miscommunication.

"There's sometimes a disconnect between the patient's perceived need and the physician's professional judgment of the urgency," Palmer said.

Sometimes patients forgot to book appointments, or family doctors didn't advocate hard enough to get them seen faster. And Palmer said regional specialists could work together to triage patients into the treatment stream they need, because not everyone needs surgery.

"For those who are legitimately waiting too long, we need to improve how we organize and deliver care," she said. "Changing how we pay for care to allow private duplicative insurance is not the solution."  

Palmer said the widespread media coverage of the Fraser Institute's survey undermines public confidence in the health-care system. "I think it's unfortunate that it goes out unchallenged every year."