2. Summary
  3. Files
  4. Support
  5. Report Spam
  6. Create account
  7. Log in

This page on Russian

Test environment

Computer characteristics

Processor: Intel Core 2 6300 1.86 GHz
Memory: 1 Gb
Hard disk: Barracuda 7200.10 SATA 3.0Gb/s 320-GB

Software

Operational system: Windows XP Service Pack 2
Database: MySQL 5.0, table type - InnoDB
Web server: Apache HTTP Server 2.2.4
PHP version: 5.2.4 installed as module
LightOrm version: 0.2а, uses PDO driver
Propel version: 1.3, uses PDO driver
Doctrine version: 0.10.2, uses PDO driver

Results of performance testing

In all tests time of connection and parsing of files, and also other preparatory operations was not considered. Time was measured only for those sites of a code which are resulted more low, in the description of tests. Tests were spent in a complex, i.e. the insert, sample, change and removal on everyone ORM were carried out in one start of a script. To exclude influence of a pool of objects in each of ORM on results of the subsequent tests, after each test clearing of a pool of objects is made.

Under each test it was spent five attempts, and then average execution time on five attempts was calculated. The percentage parity of productivity was calculated already by average result. Data are resulted in seconds, and approximated to the fifth sign after a comma. For calculation of average value not approximated data were used. The percentage parity of productivity also is approximated to one sign after a comma.

By results of testing it is visible, that on the inquiries connected with sample of data, the best result shows Propel. However a parity of these results the astable. In the test "Join Selecting" Propel has shown result, almost twice the worst than in the test "Selecting", and in the test "Many Selecting" has worsened the result almost in 3 times. In a case with a difference between "Selecting" and "Join Selecting", can explain it that in the second case is carried out more than an additional code by processing of result of inquiry. In a case with a difference "Selecting" and "Many Selecting", it speaks about low productivity of a code by preparation and inquiry performance.

By results of other tests the best result shows LightOrm. The interesting result has turned out in tests "Join Selecting" and "Many Selecting". Though absolute value has increased, in comparison with "Selecting", relative value has decreased. It speaks about more best optimisation of code LightOrm in comparison with Propel and Doctrine.

Note: the productivity Test was made without the account of a code by clearing of memory used by objects. It concerns Propel and Doctrine as in LightOrm memory clearing is made automatically.

Summary:

Inserting Selecting Join Selecting Many Selecting Updating Deleting
LightOrm 1.2701142 (50.1%) 0.1046324 (38.9%) 0.15766 (31.5%) 0.5705445 (36.1%) 0.390585 (42.4%) 1.1738464 (50.3%)
Propel 1.6849478 (66.4%) 0.0278488 (10.4%) 0.095707 (19.1%) 0.4461728 (28.2%) 0.5627743 (61.1%) 1.4862976 (63.6%)
Doctrine 2.5369074 (100%) 0.2686972 (100%) 0.5004585 (100%) 1.580298 (100%) 0.9204528 (100%) 2.3356588 (100%)

Results of memory usage testing

Testing of used memory was made by a method of calculation of quantity of instances of objects. The code for quantity calculation is resulted more low. At creation of object by a method construct() and clone() the counter of instances of objects increases by unit. At object removal, in a method destruct() the counter decreases for unit. After each test removal of used variables and also clearing of a pool of objects was made, that, on idea, should return the counter of instances of objects to zero. 

class Seller
{
    public static $instances = 0;

    public function __construct()
    {
        self::$instances++;
    }

    public function __destruct()
    {
        self::$instances--;
    }

    public function __clone()
    {
        self::$instances++;
    }
}

By results of the test it is visible, that Propel and Doctrine have inconveniences and problems by optimisation of use of memory. And the problem has proved on the same test "Join Selecting" after which in memory remained to dangle objects that was reflected in results of other tests.

Propel has no method for clearing of memory occupied with object. There is only a method of clearing of all pool of objects, but this method does not allow to release memory, in case of the connected objects. Apparently by results, in the further tests the quantity of the remained objects in memory has not increased. It means, that in the subsequent tests there are no problems with memory clearing.

Doctrine, since late versions, has a method for clearing of memory occupied with object, however, in the test "Join Selecting" it has not helped to release memory. In memory there were objects "Order"!!!, for which, actually, the method free() also was called. And even clearing of a pool of objects has not helped to release memory. As well as in Propel, in the further tests the quantity of the remained objects in memory has not increased.

Updating: for Doctrine it was possible to release memory in the test "Join Selecting", way of unset of a variable $orders (see the code of test). Results of the test are corrected.

In the table the given quantities of instances of objects accordingly are cited Seller / Customer / Order classes.

Summary:

Inserting Selecting Join Selecting Many Selecting Updating Deleting
LightOrm 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0
Propel 0 / 0 / 0 0 / 0 / 0 1000 / 0 / 1000 1000 / 0 / 1000 1000 / 0 / 1000 1000 / 0 / 1000
Doctrine 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0 0 / 0 / 0

Inserting:

In the first test the insert of 1000 chains of 3 connected objects, i.e. everything was spent, 3000 records have been inserted into a database. The inserted data were used for the subsequent tests.

LightOrm code: 

$db->beginTransaction();
for ($i = 0; $i < 1000; $i++) {

    $seller = $sellers->createItem();
    $seller->name   = 'Seller Name';

    $customer = $customers->createItem();
    $customer->name = 'Customer Name';

    $order = $orders->createItem();
    $order->sum     = 100;
    $order->seller   = $seller;
    $order->customer = $customer;

}
$db->commit();
// as in LightOrm pool clearing to be made automatically
// it is necessary to remove, only, last objects
unset($seller);
unset($customer);
unset($order);

Propel code: 

$conn->beginTransaction();
for ($i = 0; $i < 1000; $i++) {
    $seller = new Sellers();
    $seller->setName('Seller 1');

    $customer = new Customers();
    $customer->setName('Customer 1');

    $order = new Orders();
    $order->setSum(100);
    $order->setSellers($seller);
    $order->setCustomers($customer);
    $order->save();
}
$conn->commit();

unset($seller);
unset($customer);
unset($order);
SellersPeer::clearInstancePool();
CustomersPeer::clearInstancePool();
OrdersPeer::clearInstancePool();

Doctrine code: 

$conn->beginTransaction();
for ($i = 0; $i < 1000; $i++) {

    $seller = new Seller();
    $seller->name   = 'Seller Name';

    $customer = new Customer();
    $customer->name = 'Customer Name';

    $order = new Order();
    $order->sum     = 100;
    $order->Seller   = $seller;
    $order->Customer = $customer;
    $order->save();

    $seller->free();
    $customer->free();
    $order->free();
 }
$conn->commit();

unset($seller);
unset($customer);
unset($order);
# attempt 1 2 3 4 5 avg %
LightOrm 1.204354 1.493047 1.210289 1.2333078 1.209573 1.2701142 50.1%
Propel 1.6805019 1.690064 1.6870029 1.672025 1.6951451 1.6849478 66.4%
Doctrine 2.557781 2.5341849 2.5318139 2.5312681 2.529489 2.5369074 100%

Selecting:

In the second test sample of untied objects and the reference to one property of object was made. The given test shows speed of a code which is engaged in creation of object and filling by its data, and also speed of access to property of object. Speed of preparation of inquiry and its performance, in the given test, does not render on result of notable influence.

In the given test is better has proved to be Propel, however it is not necessary to forget, that Propel is generated ORM so has certain advantages. Besides, Propel has smaller functionality in comparison with LightOrm and Doctrine. For example, LightOrm returns as result not a file of objects, and iterator in which conclusion buffering is realised. Plus to it, LightOrm contains logic on job with cache objects.

LightOrm code: 

foreach ($sellers as $seller) {
    $name = $seller->name;
}
unset($seller);

Propel code: 

foreach (SellersPeer::doSelect(new Criteria()) as $seller) {
    $name = $seller->getName();
}
unset($seller);
SellersPeer::clearInstancePool();

Doctrine code: 

foreach ($conn->getTable('Seller')->findAll() as $seller) {
    $name = $seller->name;
    $seller->free();
 }
unset($seller);
# attempt 1 2 3 4 5 avg %
LightOrm 0.1046751 0.104563 0.1047821 0.104727 0.1044149 0.1046324 38.9%
Propel 0.0277059 0.0277262 0.0277579 0.028106 0.0279479 0.0278488 10.4%
Doctrine 0.27631 0.26671 0.266819 0.266746 0.266901 0.2686972 100%

Join Selecting:

In this test speed of sample of the connected objects was checked, however, under this test there are some remarks. First, there are big distinctions in functionality of all three libraries. Propel does not allow to do sample of the connected objects in any combination. For each of possible combinations of sample at Propel the separate method is used. For example, is method !OrdersPeer::doSelectJoinSellers() but is not present corresponding method !SellersPeer::doSelectJoinOrders() and the more so there is no method !SellersPeer::doSelectJoinOrdersAndCustomers(). LihgtOrm and Doctrine, in turn, allow to do samples of the connected objects in any combination.

The second remark concerns LightOrm. For sample of the connected objects in LightOrm the method Collection::load() is used, which differs from Join methods Propel and Doctrine that uses some inquiries for loading of the connected objects (by quantity loaded entities) instead of one query as it becomes in Propel and Doctrine. As in LightOrm there is no similar way of loading of the connected objects as in Propel and Doctrine, has been decided to compare Collection::load() method, after all, eventually, it is intended for the same purpose.

However, such comparison has some errors. There is an assumption, that on a sample small amount, Join the method will faster work, than Collection::load(). This assumption in the given test is not considered, and for its check it would be necessary to write the separate test.

At last, last remark concerns Doctrine. In this ORM, except Join a loading method is available, also, the analogue Collection::load() a method is loadRelated() a method. Would be correct to compare LightOrm and Doctrine in this test, using these methods. However Propel has no similar method. Therefore, this test remains as is, and in the future, probably, one more will be added, for comparison Collection::load() and loadRelated() methods LightOrm and Doctrine accordingly.

LightOrm code: 

foreach ($orders->load($sellers) as $order) {
    $sum = $order->sum;
    $name = $order->seller->name;
}
unset($order);
// at use method load(), objects are saved in a pool as persistent
// before the following iteration, therefore here they compulsorily leave from a pool
// not to influence results of the following test
$sellers->clearCache();
$orders->clearCache();

Propel code: 

foreach (OrdersPeer::doSelectJoinSellers(new Criteria()) as $order) {
    $sum = $order->getSum();
    $name = $order->getSellers()->getName();
}
unset($order);
SellersPeer::clearInstancePool();
OrdersPeer::clearInstancePool();

Doctrine code: 

$orders = Doctrine_Query::create()
        ->from('Order o')
        ->leftJoin('o.Seller s')
        ->execute();

foreach ($orders as $order) {
    $sum = $order->sum;
    $name = $order->Seller->name;
    $order->free(true);
 }
unset($order);
$conn->getTable('Order')->clear();
unset($orders) // and only after this line memory is completely released.
# attempt 1 2 3 4 5 avg %
LightOrm 0.1565609 0.156698 0.160357 0.1558609 0.158823 0.15766 31.5%
Propel 0.09712 0.093771 0.097168 0.0974638 0.0930121 0.095707 19.1%
Doctrine 0.4982491 0.5015478 0.5018449 0.49774 0.5029109 0.5004585 100%

Many Selecting:

In this test sample of all one object, but 1000 times was spent. The purpose of the given test to compare speed on small, but frequent samples. In the given test speed of a code by preparation and inquiry performance is the defining factor.

LightOrm code: 

for ($i = 0; $i < 1000; $i++) {
    foreach ($sellers as $seller) {
        $name = $seller->name;
    }
    unset($seller);
}

Propel code: 

for ($i = 0; $i < 1000; $i++) {
    foreach (SellersPeer::doSelect(new Criteria()) as $seller) {
        $name = $seller->getName();
    }
    SellersPeer::clearInstancePool();
 }
unset($seller);

Doctrine code: 

for ($i = 0; $i < 1000; $i++) {
    foreach ($conn->getTable('Seller')->findAll() as $seller) {
        $name = $seller->name;
        $seller->free();
    }
}
unset($seller);
# attempt 1 2 3 4 5 avg %
LightOrm 0.636127 0.5497129 0.480052 0.6423209 0.5445099 0.5705445 36.1%
Propel 0.4514499 0.4438992 0.447998 0.4411221 0.4463949 0.4461728 28.2%
Doctrine 1.5823741 1.5916722 1.590317 1.5769539 1.5601728 1.580298 100%

Updating:

In the given test speed of job is compared at updating of the records given thousand. As updating occurs together with sample of objects test execution time develops of two these operations. By results of this test it is visible, that, total time of sample and updating at Doctrine is more than Propel, however a difference absolutely small, and considering, that time of sample at Propel much less, it is possible to draw a conclusion that updating at Propel worse than at Doctrine. I.e. Propel has shown more the best result only for the bill of speed of sample.

LightOrm code: 

$db->beginTransaction();

foreach ($sellers->fetch() as $seller) {
    $seller->name = 'New Name 2';
}

$db->commit();

unset($seller);

Propel code: 

$conn->beginTransaction();

foreach (SellersPeer::doSelect(new Criteria()) as $seller) {
    $seller->setName('New Name');
    $seller->save();
}

$conn->commit();

unset($seller);
SellersPeer::clearInstancePool();

Doctrine code: 

$conn->beginTransaction();

foreach ($conn->getTable('Seller')->findAll() as $seller) {
    $seller->name = 'New Name';
    $seller->save();
    $seller->free();
}

$conn->commit();

unset($seller);
# attempt 1 2 3 4 5 avg %
LightOrm 0.376261 0.399214 0.397064 0.3933129 0.387073 0.390585 42.4%
Propel 0.5666099 0.556546 0.5584331 0.5598021 0.5724802 0.5627743 61.1%
Doctrine 0.914093 0.912657 0.9225719 0.9334381 0.9195039 0.9204528 100%

Deleting:

In last test speed of removal of data is checked. As well as in the previous test, test time develops of two operations - samples and removals. By results of the test it is possible to tell, that speed of removal at Propel is worse than at Doctrine, and even speed of sample does not rescue Propel in this situation.

LightOrm code: 

$db->beginTransaction();

foreach ($orders as $order) {
    $orders->removeItem($order);
}

foreach ($sellers as $seller) {
    $sellers->removeItem($seller);
}

foreach ($customers as $customer) {
    $customers->removeItem($customer);
}

$db->commit();

unset($seller);
unset($customer);
unset($order);

Propel code: 

$conn->beginTransaction();

foreach (OrdersPeer::doSelect(new Criteria()) as $order) {
    $order->delete();
}

foreach (SellersPeer::doSelect(new Criteria()) as $seller) {
    $seller->delete();
}

foreach (CustomersPeer::doSelect(new Criteria()) as $customer) {
    $customer->delete();
}

$conn->commit();

unset($seller);
unset($customer);
unset($order);
SellersPeer::clearInstancePool();
CustomersPeer::clearInstancePool();
OrdersPeer::clearInstancePool();

Doctrine code: 

$conn->beginTransaction();

foreach ($conn->getTable('Order')->findAll() as $order) {
    $order->delete();
    $order->free();
}

foreach ($conn->getTable('Seller')->findAll() as $seller) {
    $seller->delete();
    $seller->free();
}

foreach ($conn->getTable('Customer')->findAll() as $customer) {
    $customer->delete();
    $customer->free();
}

$conn->commit();

unset($seller);
unset($customer);
unset($order);
# attempt 1 2 3 4 5 avg %
LightOrm 1.115782 1.1357391 1.249517 1.1173041 1.25089 1.1738464 50.3%
Propel 1.4394741 1.427726 1.431165 1.7038131 1.4293098 1.4862976 63.6%
Doctrine 2.2574971 2.4981709 2.4017041 2.2562151 2.2647069 2.3356588 100%

© 2012 Geeknet, Inc.   Terms of Use - Privacy Policy - Cookies/Opt Out