Review: Browser tests: Chrome, Firefox, IE, Safari

A comparison between four major browsers after their major updates. (July 14th, 2009)

Electronista Rating:

ratingratingratingratingrating

Product Manufacturer: Apple, Google, Microsoft, Mozilla

Price: Free

The Good

  • Chrome and Safari both very accurate and fast.
  • Firefox best with add-ons.

The Bad

  • Internet Explorer uniformly bad in all tests.
  • Firefox always strictly middling.

With the latest versions of Apple's Safari and Mozilla's Firefox recently released, it became time for a browser comparison to test whether or not improvements in standards and raw speed have made a difference. This is especially true given the rapid shift in market share from Internet Explorer's one-time dominance: according to Statcounter, Internet Explorer currently commands just 61 percent of the browser market, while Firefox has climbed to second place with 30 percent market share and the last 9 percent shared between Chrome, Opera, Safari and smaller rivals.



In the spirit of full disclosure and honest journalism, we should disclose that our primary browser on a daily basis is Firefox. Personal usage habits aside, this review will serve as a comprehensive and objective survey of the current browsing landscape. All of our testing was conducted on anotebook connected to a Wi-Fi network to simulate real world usage. The laptop is a Dell Latitude D630 with 2GB of RAM, a Core Duo processor and Windows XP. The tests were not pursued in a lab environment, but should be representative of what an end user would expect. Our review of these four popular browsers began with fresh installs and updates of each one. All testing was done under actual usage scenarios with other applications running in the background. Each test also began with a fresh browser restart.

Chrome 2



Firefox 3.5



Internet Explorer 8



Safari 4



industry compliance testing

Our first test is from The Web Standards Project (webstandards.org) which is the home of the Acid2 and Acid3 browser tests. These tests verify that browsers are compliant with major web standards including W3C, HTML 4.0, XML 1.0, XHTML 1.1, and DOM and ECMA standards. Acid2 is designed to expose browsers and applications that inaccurately render simpler web content, particularly tests for W3C HTML and CSS compliance that should be common on the web. All four browsers passed the Acid2 test with no issues, errors, or delays, though it was originally launched in 2005 and has been used as a target for accuracy even by Microsoft, which until recent years had not felt compelled to fully support W3C.

Acid3, however, was a 2008 update to Acid2 and was considerably more of a challenge for our four contestants. It builds upon Acid2 and stresses compliance with the DOM (document object model) and JavaScript, the latter of which is crucial to most modern web apps like Google Docs. Earning a 100/100 on the Acid3 test means a browser has loaded and completed 100 sub-tasks and successfully changed the graphic colors with each subtask. Safari 4 was the only browser to fully complete this test and earned a 100/100 score -- a not surprising result given the efforts of the WebKit team to target the Acid tests as a goal. Firefox came close at 93/100; Chrome, which also uses a modern WebKit engine, matched Apple at 100/100 but unusually produced an error stating that the "linktest failed."

Internet Explorer failed the test spectacularly. The first time IE ran Acid3, it stated that it needed to run an add-on called MSXML 3.0 and it made it to 12/100 with some random graphical errors. Upon allowing the add-on to load and re-running the test it progressed to 20/100, but still failed.



Even though Firefox didn't make it to 100 and Chrome had failed the 'linktest,' these browsers at least seemed to run the test in the same amount of time as Safari. We don't think it would be fair to say that Firefox and Chrome are vastly inferior to Safari in their rendering capabilities; they simply don't adhere to the standards as closely as Safari does. If we were to give report cards as a result of the test, Safari would earn the top mark of an A+ while Chrome and Firefox would each receive an A. Internet Explorer, though, clearly earned an F.



speed testing with JavaScript

Many major web properties rely on JavaScript to accurately render the user environment. Because of its prevalence in modern web-browsing, our speed tests focused on the browsers' ability to quickly render JavaScript code.

Our first JavaScript performance test is from dromaeo.com. Dromaeo is a Mozilla project and, in theory, might favor Firefox. Dromaeo offers several sets of tests and we opted for the full range, which takes over a half hour to run. The results for this test are expressed in the number of test runs per second, so the higher the score the better.





Once again, Internet Explorer couldn't run the full set of tests and in this case kept locking up on a "base64" sub-test. We tried running smaller sets of tests, but each batch resulted in at least one error that prevented Internet Explorer from completing the run. Other browsers completed the tests, but the gaps were wide: Firefox appropriately edged out Chrome by about 30 points, but Safari took the crown and more than doubled Firefox's score. This can largely be pinned on Safari's Nitro engine, which is touted as better optimized than Chrome's and also has the advantage of the fairly lean WebKit rendering engine to back it up.

Our second JavaScript speed test was webkit.org's SunSpider suite of tests. WebKit is the rendering engine for Safari, so as with Dromaeo and Firefox we expected a slant towards Chrome and Safari. The score results of this test are generated with a statistical variance of plus or minus a few percent. We ran the tests several times and used the score that came with lowest variance. The score is expressed in milliseconds, so lower scores are better.

It's interesting to note the Internet Explorer generated the most consistent sample of scores at 1.5 percent variance or less and Chrome and Safari generated the most scattered scores, at around 4 percent variance. Firefox, Safari, and Chrome all came in at around 2,000 milliseconds, but Chrome just barely claimed the win while Internet Explorer took over 8,000 milliseconds to finish. As seen with Dromaeo and Firefox, there was no home field advantage for Safari.

Our third test was Google's own V8 JavaScript rendering test. Google of course uses the V8 JavaScript engine in Chrome, so we anticipated a slanted result here as well. The V8 test uses a complicated formula to calculate the scores, but the larger the score the better.





Chrome did in fact win this test, but Safari took a close second. Surprisingly, Firefox was in a distant third place, and the results for Internet Explorer barely registered. Internet Explorer really dragged in these tests: several times during the tests, an error box popped up asking to stop running them as they were making the application run slowly. We can honestly say we've never seen a software application all but beg to stop a test.



Our final JavaScript test was provided by a third party known only as CelticKane. Again, the results are measured in milliseconds, so the lower the score the better. Again, Internet Explorer took last place while Safari led the pack. Chrome and Firefox produced very similar results but reached second and third places respectively.

testing summary

Disappointingly for those who'd expected a clear leader, the final results were a draw; Chrome and Safari both won two sets of tests. Firefox was perpetually middling and came in third place in half of the tests while reaching second place the other half, never completely winning any one test. What left little mystery was Internet Explorer's habitual lack of performance. It consistently came in last place for each of the tests and was by far the worst performing browser in the Acid3 test.

personal usage thoughts

While not the champion, the new Firefox 3.5 is still noticeably faster in daily browsing than the previous 3.1 edition. It also still has the largest development community of all of the browsers available today and its add-on architecture is not only flexible but results in top notch contributions. Despite coming in third in our speed tests, it's responsive for day to day use and is quite stable, which couldn't always be said for earlier versions and other modern browsers.

Internet Explorer may still have the largest share of the browser market, but after these tests we're wondering why. There are no redeeming qualities to the software's performance or ability to draw websites, and it's bad enough that we would recommend any other browser over Internet Explorer unless it's absolutely required. Just as in the tests, IE is tangibly slower in common use and sometimes creates problems with viewing sites as they were intended.

Chrome and Safari shared the top spot for a reason: they consistently took top ranks in our speed tests and are both a pleasure to use. They may have garnered less than 5 percent of the global browser market, but they're both excellent browsers that deserve more. Either is very lightweight and has an extremely simple user interface that gets out of the way when possible. If you haven't tried one of these browsers before, we would highly recommend giving one or both of them a go for a solid week -- though Mac users will likely have already spent their fair share of time with Apple's software, which can't be said for Windows users and Chrome.

wrapping up

Between the Acid and JavaScript testing and our real world usage tests, we can say without a doubt that we would recommend virtually any of these browsers other than Internet Explorer. We didn't design this article to bash against Microsoft, but their product truly is second class compared to the competition. Firefox is great for those that love add-ons, and Chrome and Safari are both very high-speed but also low-frills.

by Kelcey Lehrich


POST TOOLS:
toggle

Network Headlines

toggle

Most Popular

Sponsor

Recent Reviews

Linksys EA6900 AC Router

As AC networking begins to makes its way into more and more devices you may find yourself considering an upgrade for your home network ...

D-Link DIR-510L 802.11AC travel router

Having Internet access in hotels and other similar locations used to be a miasma of connectivity issues. If Wi-Fi was available, it wa ...

Ooma Office small business VoIP

Voice over IP (VoIP) services have been around for a very long time. Only recently has the implementation become a bit more robust, al ...

Sponsor

toggle

Most Commented

 
toggle

Popular News