Mac, Linux see up to five percent variance in data

Neurological imaging tool FreeSurfer, curated at the Martinos Center for Biomedical Imaging, is a commonly-used open-source software tool employed by researchers to measure cortical thickness and volume of varying brain structures from the MRI scan of a patient. Recent research has shown that the calculations, given the same input, can vary up to 15 percent between different versions of the software, and up to five percent between Linux and OSX. No explanation has been given by the curators for the phenomenon.

To test the software, a study pool of 274 participants was separated into two groups plus a control group. Thirty scans were picked at random, 10 out of each group. The FreeSurfer application was installed on two different iMacs, each with a different processor and operating system, along with three different Xeon-based Mac Pro systems running OS X 10.5 and 10.6, and a single HP workstation under CentOS 5.3, also with a Xeon processor.

Three different FreeSurfer versions were used to count 190 subcortical volumes, and 68 cortical thicknesses. The same brain re-calculated on the same workstation didn't see any variance. All the Macintosh workstations on the same operating system and software revision had identical results, regardless of processor and RAM configuration. Differences, however, were revealed between the HP and any Macintosh station, including the similar architecture Xeon-based Mac Pro. The individual FreeSurfer versions also calculated brain volumes differently. OS X 10.5 also unexpectedly calculated differently than OS X 10.6 with all other variables unchanged.

Depending on brain location, results varied by between five and 15 percent between versions of FreeSurfer. Changes in platform or major OS revisions differed less, between two and five percent.

Electronista contacted a major children's neurology center for insight, and was told that hospital information technology workers take steps to insure that both OS and hardware variances are strictly controlled in the research environment to minimize this effect. Small differences in mathematics have been known to show up induced by hardware or software migrations in disciplines involved in complicated, floating point mathematics -- such as architecture or ballistics.

The developers behind FreeSurfer are aware of this phenomenon. The software license for the package states that the package is for comparative research purposes only, and has not been approved by the FDA for clinical use. Specifically, "clinical applications are neither recommended nor advised," according to FreeSurfer's license.

Electronista has contacted the FreeSurfer curators for comment beyond the EULA language, but has not received a reply thus far. The research tool is intended for like comparisons -- while MRI data can be fed into two different versions of FreeSurfer and two different answers may result, data from different brains can be fed into the same version of the software, and comparable results will result, which is the intent of the open-source tool. No evidence of any research error or misdiagnoses has been reported as a result of the discrepancies. [via PubMed]

By Electronista Staff

Post tools:

TAGS :  

computers, software, research, Apple, neurology