Once again the Lone Dog has been mulling over the topic of lies, damned lies, and statistics as they are utilized by the refractive surgery industry (RSI).  What happens when a credible source puts some spin on the ball?  This article is late breaking news, so I guess we'll find out.

"Pupil Size and Quality of Vision after LASIK"

Ophthalmology*
Volume 110 (August 2003)
Pages 1606-1614

Brought to you by Steven Schallhorn MD (of the United States Navy) and five co-authors (also from the United States Navy except for one from the University of California).

*The official journal of the American Academy of Ophthalmology.

The LD takes exception to two aspects of this article: 

1. Garbage in garbage out (GIGO) on the pupil size measurements
   
   
2. Accepting the pupil measurements as valid, you conclude what?

Introduction to the Laboratory of LASIK Experiments

Schallhorn et al performed LASIK on both eyes of 100 individuals.  Unlike the more typical both-eyes-one-day approach, patients waited one month between eyes.  Before surgery, and at 1 month, 3 months, and 6 months after surgery, the patients completed a standardized questionnaire that was designed to elicit information about glare, haze and halos under low-light conditions.  The authors then looked for a correlation between the type and severity of self-reported problems and the pupil diameter of individual patients.  They decided there wasn't any such correlation.

Like most studies of this nature, not all patients completed the survey at all time points.  Despite a fairly large initial enrollment, this can make for sample size problems.  For example, if 25 patients completed the 3 month survey and 35 completed the 6 month survey, but only 10 completed both surveys, then you have a relatively small number of patients providing information comparing 3 and 6 month symptoms.  To get decent numbers you have to be fanatic about getting a survey back from almost every patient still living.  The importance of this issue is exacerbated when you start out with a small number of a certain sub-type of patient to begin with.  But I'm spoiling the surprise.  Onward to GIGO.

GIGO

Let's start with the title.  The first two words are Pupil Size.  Without knowing thing one about the background or purpose of the article, Lou "Big Pupil" Schmo can correctly deduce that some people measured some other people's pupils and that this was an important measurement in the experiment.

So what do we know about the method of pupil size measurement?  Page 1607 right column paragraph 2 tells us that "Pupil diameter was measured under low light conditions (< 5 lux) using the Colvard Pupillometer.  Patients were seated and asked to fixate on an object at the opposite end of the room... The operator then aligned the device to read the pupil diameter on the scale to the nearest 0.5 mm."

Harking back to Accurate Pupil Measurement in LVC, we understand that the goal here is to measure the patient's maximum dark-adapted pupil diameter (MDPD).  It is insufficient to measure the patient's pupils in the dark.  To get from "in the dark" to "maximum" you need a plan, which must be followed consistently for all subjects or the pupil size data set is a glorious hash and any conclusions you might reach based upon it are unsubstantiated.  However, nothing is said about the period of dark adaptation.  Although "fixate on an object at the end of the room" sounds good for controlling accommodation, what if the end of the room was about 7 feet away?  What if the projector screen was a strong light source at the end of the room?  What if any of the myriad other ways to screw this measurement up were not attended to?  If you claim a precision of 0.5 mm, you need to demonstrate an accurate technique.  In this case the technique rests just as heavily on the method of dark adaptation as it does on the correct deployment of the Colvard pupillometer.

The acid test of a Methods section is this:  can other researchers, using the information presented in the article, duplicate the experiment?  Nope.  A researcher with good basic knowledge of the pupil dark response needs to email Schallhorn and ask him the following questions:

1] What was the duration of dark adaptation?

What steps were taken to:

2] control patient alertness?
3] control accommodation?
4] control exposure to stray light sources?
5] ensure that the individuals performing the measurements did so accurately?

This may just be obnoxious nit-picking from a cowardly anonymous writer who has a bone to chew.  You tell me.  The article is about pupil size.  It was necessary to measure pupil size, and not just any pupil size, but the MDPD.  Conclusions were drawn to the 0.5 mm (or 500 microns for Procyon digital infrared video-pupillographer advocates).  Based on what the authors say about their own experiment, do you believe they attended to details?  If they did not, do you believe that their conclusions are still more-or-less valid because hey, 0.5 mm here and there doesn't matter that much?

When in doubt, check the data out.

Page 1610 left column Figure 1 gives the distribution of mesopic (low light) pupil sizes for 99 subjects. 

[Let me deliver a sidebar gripe on the term "mesopic" (also photopic and scotopic).  In this case the term refers to the ambient illumination of the room the patient was sitting in.  This illumination level is sensed by the retina, which adapts to it by turning on rods (night time photoreceptors).  The retina also tells the brain to tell the pupil to get bigger.  It is really the retina that is experiencing the mesopic illumination.  The pupil is just a hole in the iris that is enlarging as the retina reaches a steady state of function under mesopic illumination.  So if you want to get really compulsive about the terminology, the pupil should be described as being dark-adapted to a certain degree, not as being mesopic.  Like, "The dark-adapted pupil diameter (5 minutes at 5 lux) was measured."  Personally the LD doesn't consider this as being foolishly compulsive, but rather as being precise.  Others would disagree.]

Anyway, Figure 1 gives us the distribution of 99 pupil diameters measured in the mostly-dark.  Pupil size is definitely correlated with age.  The average age of the patients in this study was 36 years (ranging between 23 and 66 years).  Since the ages of a modest number of much older subjects can influence the average value substantially, we would also like to know the median age (the age that divides the subjects into half-younger and half-older).  I'm going to make an assumption that — given that the patients probably were in the Navy — there were more young ones than old ones, so that the median age was less than 36 years (perhaps much less).  Something else to email Schallhorn about.

Why should we care about the median subject age?  The average pupil diameter was 5.7 mm, and (statistically) 97.5% of patients had pupils smaller than 7.5 mm.  If we consult various other references on the MDPD vs age, we find that Schallhorn et al's numbers are on the low side.  If we were able to look at the age distribution and discovered that over half of the patients were under, say, 30 years old, then this average would be on the very low side.  Unless these 99 patients were a really unusual group of humans, the correct suspicion is that most of the recorded pupil diameters were not MDPDs.

GIGO!

We need this data for later:

Oh yes, the GASH info was collected by survey.  For the LD's thoughts on surveys see True Lies in LVC Clinical Research (Part 1).

You Conclude What?

Page 1613 left column second paragraph.  They conclude:

1. Pupil size is a significant predictor of glare, haze, and halo reports after LASIK, especially in the first post-operative month.
   
   
2. Glare and halo reports at 3 months were only weakly predicted by pupil size.
   
   
3. Pupil size was not found to be a significant variable 6 months after treatment.

For purposes of discussion we are going to accept that their pupil measurements were MDPD measurements.  They were using a VISX Star2 laser with an optical zone of 6.0 mm and no blend zone.  If a patient with astigmatism is treated on this laser he doesn't get a circular optical zone that is 6.0 mm in diameter, but rather an oval zone where the short axis of the oval is less than 6.0 mm.  So some people with 6.0 mm pupils will still have negative clearance (gaps where the treated area does not reach to the edge of the pupil).  The degree of negative clearance will depend on the level of astigmatism.  It gets fussy.  So let's not worry about the 5.5 — 6.0 mm pupils for now.  Just look at Lou's compatriots, the individuals with pupils larger than 6.0 mm.

The study started with 15 such patients.  The study ended with 8 such patients.  Correct.  Only 8 patients with large pupils had 6-month data on glare, haze and halos.  Only 8 patients who definitely had negative clearance for 360 degrees (untreated area sending light through the pupil, all the way around) were included in the final analysis.

Here comes the spin...

Remember lies, damned lies, and statistics?  Well, Schallhorn et al's statistics (probably provided by co-author Linda Borque PhD of UC Los Angeles, a fine institution) support conclusion #3 above.  They do.  Really.  A Kruskal-Wallis ANOVA showed H<5 and P>0.05.  This is what statistics are for, to transform some amusingly small amount of data into a conclusion that withstands statistical criticism.  But how about Dog criticism?

Let's take the human approach.  You are Lou.  Lou has huuuuge pupils (pick some number, 8.0 mm?) and does not have any glare, haze, or halos (the full set is glare, arcs, starbursts and halos — abbreviated GASH).  Lou is told that, based on a study published just this year with very up-to-date data that used a Kruskal-Wallis ANOVA analysis, there is no association between pupil size and the risk of GASH afterwards.  Some really fine researchers with excellent reputations figured this out.  Sign here.

If you were Lou and were thinking that in your personal statistical universe it just doesn't add up that 1) light gets in through the pupil, 2) the laser treatment won't correct the focus of all the light that is going through your pupil, and 3) there is no earthly reason* why you should NOT see this unfocused light as well as seeing the focused light... would you be willing to go against your gut on the basis of 8 patients?

Me neither.

*Little known fact:  the Stiles Crawford Effect of the First Kind does not apply to cones.