mai 19

Foreword

After some months in the field of Magento managed hosting, NBS System decided to create a new Magento extension : Nitrogento.

This extension aims at giving more performances to Magento shops. Mainly, 7 points are covered (more will come with our strong roadmap) :

  1. Full Page Cache
  2. Block Cache
  3. Custom Bloc Cache
  4. Auto Sprite
  5. HTML, JS & CSS minifying & compression
  6. HTaccess fine tuning
  7. CDN auto deploy

Magento Benchmark with Nitrogento

The benchmark context

This said, the main key here is the performance. So what best way than making benchmarks ? Well we tryied almost all of them ! And I’ve to say that there is a lot of benchmarks around !

We installed a basic demostore on http://demostore.nitrogento.com and did the same demostore with Nitrogento on http://nitrostore.nitrogento.com. The server hosting the demo/nitro stores is the very same, actually, it’s even the same machine. The machine was a Quad core 5650 VPS with 8 Go of RAM.

Benchmark used

We choosed to used the following benchmarks, which are considered the most accurate and renowned :

PS : We know some of you love our product but, please, can you consider doing your benchmarks on your own servers, or at least not to often on ours, because recently we had a lot of people benching both sites (http://nitrostore.nitrogento.com and http://demostore.nitrogento.com).

The benchmark issues to be considered

Some tests gave a false results for two reasons :

- First, the  server is located in France. So when you use GTmetrix (for exemple) the test server is located in the US. This means that we have a bit of lag between the test machine and the target machine. One in the other, this is not a big issue since only the real load time is false but the comparison stay exact since both Nitro & Demo stores are located in France so they are in the exact same situation.

- We also found a funny problem. Google and Yahoo (and many others) recommand using multiple CNAMES/HOST to host the static files. That way, your browser is able to download a lot of file at the same time and not only 8 by 8 or 12 by 12 (the exact multi thread number is browser dependant). But when you have several CNAMES pointers, this makes several DNS resolutions and … this can be a problem depending on various factors. The precise problem came up with www.wichloadsfaster.com (WLF). By make 5 DNS requests on Nitrostore and only 1 on Demostore, he claimed that Demostore was faster.

Actually this is not true, but the 4 more NS lookup took a bit more time (resolution from US to France) and as WLF only test the core load (don’t take the pictures, CSS, Js again) it found Nitrostore to be slower. So to get realistics results, the WLF test was made with the CDN feature of Nitrogento desactivated, to avoid having more DNS lookup to do than on a basic demostore. In real life, this CDN feature bring you a lot and we recommand it, of course :)

We noted these troubles in the table of results.

(*) stands for « longer time than real since test machine and bench machine are in US and FR »

(**) stands for « desactivated CDN not to compromise results »

Results

www.whichloadsfaster.com (500 runs)
Time (*)
Demostore 327 ms
Nitrostore 134 ms

Nitrostore : 2,4 times faster.

www.gtmetrix.com
Grade Time to full load
Demostore C/C (72/76) 3,15
Nitrostore A/A (96/94) 2,18

Nitrostore : 43% faster
Nitrostore : +20% to yslow/pagespeed tests

www.MageSpeedTest.com (*)
Trans/sec Time / request
Demostore 12,38 1,86
Nitrostore 28 0,55

Nitrostore : 2,2 times more transaction per seconds
Nitrostore : 3,38 times faster per request

Firebug – Network load time
Requests Time (s) Size (Ko)
Demostore 49 2,14 556
Nitrostore 28 1,35 293

Nitrostore : 42% less HTTP resquests
Nitrogento : 37% faster to load
Nitrostore : 40% less bandwidth / transfer

Funkload (home hammering – 200 users)
Page / sec Page time APDEX
Demostore 25 16 s 0,4
Nitrostore 300 0,7s 1
Funkload (Full scenario – 50 users)
Page / sec Page time APDEX
Demostore 35 10 s 0,55
Nitrostore 100 3,5 s 0,65

Under heavy load,
Nitrostore : up to 18 times faster on the homepage and 3 times faster in the full scenario
Nitrostore : up to 12 time more home page served and 3 times more full scenario
Nitrostore : Continuous perfect APDEX on homepage and 15% better APDEX indice (user experience) on complexe scenario

www.webpagetest.org
Document complete Fully Loaded
Load time (**) First Byte (**) Start render Dom Elements Time (**) Requests KBytes In Time (**) Requests KBytes In
Demostore first 5,7 10 s 0,55 332 5,7 50 574 5,7 50 574
Demostore repeat 1,9 10 s 0,55 332 1,9 2 7 1,9 2 7
Nitrostore first 3,6 3,5 s 0,65 318 3,6 29 303 3,6 29 303
Nitrostore repeat 1,5 3,5 s 0,65 318 1,5 4 18 1,5 4 18

Trial version

But you know what ? Test it by yourself, there is a Trial version there : www.nitrogento.com

écrit par Philippe Humeau \\ tags: , , , , ,