EnglishSun 4 Feb 2007
In addition to the test of MVC-frameworks
Posted by Alrond | Tags: CakePHP : CodeIgniter : Django : framework : Performance test : php : Pylons : Ruby on Rails : TurboGears : Zend :
The test determines the performance of frameworks in an identical conditions for the uncomplicated projects like "Hello, world".
By offers from the previous post I have conducted the small additional test into which have also entered Zend Framework (PHP), CakePHP and, by the way, I have decided to test an young and interesting Pylons (Python).
PHP frameworks I tested both without and with eAccelerator 0.9.5
Besides I’ve corrected the mistake which has been found out in the test by RoR 1.2.1.
So by requests I tested TurboGears with various template engines.
I started ApacheBench C5 two times and also C100
Pylons has appeared simple in apply, and fast too.
In test of RoR I have committed a following mistake:
it was necessary config/environment.rb to change in RAILS_GEM_VERSION = '1.2.1'
besides I has found out, that in the controller it is also possible to add " session:off ", thereby to accelerate the work. So after correction, framework has really shown much the best results, thus overtaken TurboGears.
For Turbogears besides standard Kid, I have also tested Cheetah, Genshi and Jinja. For simple text template Cheetah is about 1.35 times fasters then Kid.
PS: for everyone who accused me of bias - I regarded to all frameworks equally. But I need one of these for a serious project and the most important is a stability under very high load and speed of course. Even such simple project "hello world!" has given a notion of this and of project complication on framework.
And so django has been chosen. I didn’t know a python, but I needed quick short to understand in django and in some days I have made pair small projects (including this blog). Yes it has appeared the brightest too - bingo. To tell the truth, I have thought that Pylons can appear more quickly :-) by the way, it is very nice and convenient framework too.
(61 votes, average: 4.5 out of 5)
Loading ...
By offers from the previous post I have conducted the small additional test into which have also entered Zend Framework (PHP), CakePHP and, by the way, I have decided to test an young and interesting Pylons (Python).
PHP frameworks I tested both without and with eAccelerator 0.9.5
Besides I’ve corrected the mistake which has been found out in the test by RoR 1.2.1.
So by requests I tested TurboGears with various template engines.
I started ApacheBench C5 two times and also C100
ab -c 5 -n 1000 ab -c 5 -n 1000 ab -c 100 -n 10000CodeIgniter and CakePHP are started with eAccelerator without problems, as for Zend, it works in console, but under FastCGI with the accelerator gives out the following:
Fatal error: Cannot access protected property Zend_Controller_Front:: $_plugins in /srv/zend/library/Zend/Controller/Front.php on line 130By means of an accelerator the performance of PHP-frameworks became 4.4-6.4 times as much. Owing to that CodeIgniter received the second-place after django.
Pylons has appeared simple in apply, and fast too.
In test of RoR I have committed a following mistake:
it was necessary config/environment.rb to change in RAILS_GEM_VERSION = '1.2.1'
besides I has found out, that in the controller it is also possible to add " session:off ", thereby to accelerate the work. So after correction, framework has really shown much the best results, thus overtaken TurboGears.
For Turbogears besides standard Kid, I have also tested Cheetah, Genshi and Jinja. For simple text template Cheetah is about 1.35 times fasters then Kid.
PS: for everyone who accused me of bias - I regarded to all frameworks equally. But I need one of these for a serious project and the most important is a stability under very high load and speed of course. Even such simple project "hello world!" has given a notion of this and of project complication on framework.
And so django has been chosen. I didn’t know a python, but I needed quick short to understand in django and in some days I have made pair small projects (including this blog). Yes it has appeared the brightest too - bingo. To tell the truth, I have thought that Pylons can appear more quickly :-) by the way, it is very nice and convenient framework too.
(61 votes, average: 4.5 out of 5) Loading ...
Latest blog entries
February 5th, 2007 at 12:23 a.m.
все правильно!
работаю с рельсами, а раньше с пыхом трахался
"как это кривье написали, так и сравнивают. все честно"
cadmi - 5 баллов!
February 5th, 2007 at 8:59 a.m.
Вот теперь большое спасибо :-)
Alrond, прости за рьяную критику прошлого поста здесь и на OpenNet (там я funny_falcon) :-)
Правда и здесь хотелось бы узнать, на сколько активно было обращение к БД? но ладно, хватит критики, еще раз спасибо :-)
February 5th, 2007 at 9:31 a.m.
Тестировал без обращения к базе - это тема отдельного тестирования...следующего ;)
February 6th, 2007 at 11:12 p.m.
Even though they are simulated numbers I thought it might be
interesting to discuss. Besides the fact that from these tests Django
seems to be whooping serious butt on all fronts, it shows that Rails
1.2.1 seems to be trailing in performance (quite a bit in some places)
in comparison to 1.1.6. Anyone seen the same thing in a production
site?
I'm curious if there is anything significantly missing in these tests
and how Django got such great numbers (Or maybe it is just that
efficient? I have no real Python/Django experience.). Thoughts?
February 6th, 2007 at 11:13 p.m.
Django *is* just that efficient. :-)
But there's a reason for that, and I bring it up here in the hope of
fostering a good discussion rather than simply "rah-rahing" Django,
the Python framework that I'm involved in developing.
We made fundamental decisions in Django that eschewed magic in favor
of a bit more explicitness. Plus, performance was baked into the
framework from Day One, rather than us taking a "we'll make it
efficient later" stance. As a result, Django doesn't have to do as
many things dynamically as Rails does.
The tradeoff is that, in Django, you'll have to use an import
statement if you want to use a certain bit of code, rather than the
code magically being available to you. The tradeoff is that, in
Django, you define your database models in Python code rather than
having the framework introspect the database at runtime. Overall, the
tradeoff is that, in Django, you're a tiny bit more explicit.
Of course, this isn't an "I hate my job, this is so tedious" level of
explicitness -- Django is, after all, intended to save you a ton of
time. It's more like a pragmatic level of explicitness. And it has
some nice side benefits, such as the aforementioned performance
benefit (which is important) and the fact that explicit code is more
"discoverable" (i.e., it's easier to poke into the framework to see
what's *actually happening*).
If you give Django a shot, I'm pretty sure these differences will jump
out at you. Whether you appreciate the difference is completely a
personal decision, and no amount of marketing is going to change your
mind. (Well, for most people.) The difference between Rails and Django
programmers seems to be the programmers' tolerance for magic.
Performance is a nice side benefit of less magic.
February 7th, 2007 at 8:42 a.m.
How did you benchmark Pylons? Are you running stand-alone HTTP server? FastCGI? SCGI? mod_python? Each and every one of those will yield different numbers, and Pylons will run in any of those environments.
I'm rather baffled by the disparity in your benchmarks, that I've never seen in other benchmarks of Django, TurboGears, and Pylons. Can you post more details about the setup?
February 7th, 2007 at 9 a.m.
Спасибо за тестирование!
Если честно не совсем могу понять почему CodeIn c eAccelerator делает в ТРИ раза CakePHP + eAccelerator.
А вообще, надо серъёзно присмотреться к питону...
February 7th, 2007 at 10:56 a.m.
2 Ben Bangert:
I've tested all frameworks in an identical conditions: FCGI+Nginx
For start of pylons I used dispatch.fcgi:
#!/usr/local//bin/python
from paste.deploy import loadapp
wsgi_app = loadapp('config:/srv/pylons/helloworld/production.ini')
if __name__ == '__main__':
from flup.server.fcgi import WSGIServer
WSGIServer(application=wsgi_app, bindAddress=('127.0.0.1', 6500)).run()
# cat hello.py
from helloworld.lib.base import *
class HelloController(BaseController):
def index(self):
c.value = "Hello World"
return render_response('/hello.myt')
# cat hello.myt
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
</head>
<body>
<% c.value %>
</body>
</html>
For more details see the first test
http://www.alrond.com/en/2007/jan/25/...
February 7th, 2007 at 3:58 p.m.
What happened with the % memory/cpu eaten by every framework? Those numbers were available in the original test.
February 7th, 2007 at 10 p.m.
As suggested for TurboGears, the templating engine can be a significant part of a small benchmark like this. I'd love to see the new Mako templating engine used in one of these benchmarks (particularly with Pylons, as Mako will be the default Pylons templating engine soon).
It would also be interesting to see the frameworks' performance without a templating engine involved at all.
Django:
return HTTPResponse('Hello world', mimetype='text/plain')
Pylons:
return Response('Hello World', mimetype='text/plain')
TurboGears (I think):
@turbogears.expose(content_type='text/plain')
def index():
return 'Hello World!'
RoR:
@response.headers['Content-Type'] = 'text/plain'
render :text => 'Hello World'
February 7th, 2007 at 11:02 p.m.
As Pylons is very modular, the code itself is very fast, I'd say the same speed as Django for all practical purposes. What your benchmark comes down to is how fast it renders the template, and template rendering times definitely differ.
Your simple test has Cheetah about 1.3x faster than Kid. In more realistic (slightly more complex) benchmarks, Cheetah is generally 10-15x faster than Kid. If you're choosing a system based on such a simple non-real-world benchmark, you have no idea what you'll actually get.
Consider the benchmarks for template languages posted on Mako here (http://www.makotemplates.org/). Clearly Myghty is twice as slow as Django templates which definitely explains the disparity your benchmark is showing.
If you want to even things up, remove the template equation. Have your Django function just "return Response('hello world')" and have the Pylons controller "return Response('hello world')". That will actually give you a significantly more accurate benchmark of the core code in one system compared to the other.
I look forward to seeing those results. :)
February 19th, 2007 at 5:13 p.m.
Thanks for your sharing, I'd like to sugget a cleaner example code
which is equivalent to django's :-)
# more turbogears-ic
from turbogears import controllers, expose
class Root(controllers.RootController):
@expose(template="tg.templates.world")
def index(self):
return dict(hello_world="Hello World!") # eqal to
{'hello_world': "Hello World!"}
the kid template is:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:py="http://purl.org/
kid/ns#">
<body>
<div py:content="hello_world"></div>
</body>
</html>
logging, <?pyhton ?> and "py:extend" statement are not essential for
turbogears,
and they may take extra time in this performance evaluation.
You may interest that in newer version (>1.0b3) TurboGears have better
default quickstart template, which will lead you less redundant
code :-)
With python 2.5 I believe python frameworks could have even better
performance (tg's py 2.5 support is in svn though, but django & pylons
are already py 2.5 compatible)
August 28th, 2007 at 5:07 p.m.
hi,
Thanks for your test! python looks very good. The big plus of php is ability to encode source code using zend safeguard. It's a pity but python doesn't have it.
Regards,
unixvps
October 24th, 2007 at 12:21 p.m.
Даже после таких результатов теста, мне сложно уйти с ПХП, все таки он и привычнее... да и если потр...ся с настройкой сервака, он не будет сильно уступать в скорости Питону.... ИМХО
October 26th, 2007 at 2:58 a.m.
2 larin
Будет. ;)
October 26th, 2007 at 8:45 p.m.
а где Solar, LIMB, Symfony????
Взять 2 самых медленных FW на php и сравнивать их глупо....
ЗЫ хотелось бы увидеть тесты
<?php
echo'hello world!';
?>
October 27th, 2007 at 12:05 p.m.
Про симфонию можно в предыдущей статье посмотреть. Он даже по сравнению с CodeIgniter оказался медленным. Так что не было смысла сравнивать его с ускорителем.
И еше раз повторюсь: это тестирование MVC-фреймворков, а не мини-кода. Думаешь, что перл или питон на таком миникоде будет медленнее PHP? :)
December 19th, 2007 at 12:33 a.m.
Слушай, а можно в следующий раз побольше тестов Pylons? Он же как и TB может разные шаблоны.
March 30th, 2008 at 10:18 p.m.
очень интересно было бы увидеть результаты с малюткой web.py... без render template
April 9th, 2008 at 11:37 a.m.
Очень интересные результаты, спасибо!
Когда будет повторый тест ?
ZF обновился и работает из консоли на ура.
Протестируй связку PHP + XCache + FCGI + Nginx, будешь приятно удивлен.
November 18th, 2008 at 11:22 p.m.
Наткнулся на это тестирование - спасибо. Подтвердило мое решение цифрами.
December 20th, 2008 at 9:08 p.m.
Жалко нет тестирования Kohana, это Codeigniter на PHP5.
July 14th, 2009 at 8:02 p.m.
Автору респект! Натыкаюсь раз в полгода на этот пост и каждый раз как в первый раз читаю. КЛАССИКА! Django хорош, очень! Однако несколько выбило из равновесия когда вышла версия 1.0 без обратной совместимости с 0.96. Уже почти год свой большой проект перевести на 1.0 не могу. А там столько уже за это время вкусностей появилось.
January 23rd, 2010 at 7:24 a.m.
MVC Catalyst I liked, a very good framework
September 18th, 2010 at 9:47 p.m.
Sorry to be ignorant, but does this test show that Pylons is slower than Django?
September 25th, 2010 at 6:35 p.m.
2mikemike
I have made this test 3 years ago. I don't known current status.