PHP vs ColdFusion – a test against time

Posted on January 29, 2011 by Phil 56 comments

This article has been edited based on the feedback given in the comments section by a large number of Coldfusion professionals–mainly on the use of cftimer effecting the results. My original text stays in place with amendments made based on alterations needed to make the tests a fairer and less PHP bias comparison

Where I work we use ColdFusion but for personal projects I always code in PHP, mainly because ColdFusion is pricey (although Railo and BlueDragon are freely available) and I don’t like coding in it. We were talking about performance at work the other week and I boasted that PHP was much faster than ColdFusion without any real evidence to back it up, just stuff I had read about how badly ColdFusion deals with instantiating objects. Might I add, for a scripting language built on Java, a truly OO language, this is totally lame. I thought I would create a few simple tests to show ~~people just how much better PHP is as a language~~ a comparison between the two languages for which the results are outlined in this article.

Download the test files

The test environment

OS: Windows 7 x64
CPU: AMD Phenom II X2 550 3.10 GHz @ 3.83 GHz
MEMORY: 4GB OCZ DDR3
ColdFusion: 9 x64 EDIT: with all debugging features disabled
PHP: 5.3.0
Apache: 2.2.11

Test one: Speed test on looping

This test involves two identical objects that simply calculate and build an array of the prime numbers between 2 and 10,000 using a simple for loop containing another nested for loop. Below is the code for each (unfortunately ColdFusion ~~is so popular that there is no syntax highlighter for it~~ isn’t supported by syntax highlighter so I had to use the java colouring).

The PHP Primes object

class Primes {

    public function __construct () {}

    public function calc_primes ($to) {
        $primes = array(2,3);
        $primes_len = 2;
        for ($i = 5; $i &lt;= $to; ++$i) {
            $is_prime = true;
            for ($j = 0; $j &lt; $primes_len &amp;&amp; $is_prime; ++$j) {
                if ($i % $primes[$j] == 0) $is_prime = false;
            }
            if ($is_prime) {
                array_push($primes, $i);
                ++$primes_len;
            }
        }
    }
}

The ColdFusion Primes object

Original code sample available in download

component {

	public function calcPrimes (to) {
	     var primes = [2,3];
	     var primesLen = 2;
	     var isPrime = true;
	     var i = 5;
	     var j = 1;
             for ( ; i &lt;= to; i++) {
                 isPrime = true;
                 for (j = 1; j &lt;= primesLen &amp;&amp; isPrime; j++) {
                     if (i % primes[j] == 0) isPrime = false;
                 }
                 if (isPrime) {
                     ArrayAppend(primes, i);
                     ++primesLen;
                 }
             }
	 }
}

Looping test results

I wasn’t sure whether PHP was up to the task on this one as the Java layer below ColdFusion is pretty good with handling large Arrays but ~~PHP came out just on top.~~ Coldfusion came out slightly better off on the re-run tests

PHP: ~~128ms, 122ms, 126ms~~ 108ms average
ColdFusion: ~~128ms, 115ms, 135ms~~ 84ms average

One tip to note here for PHP programmers—count() is exponentially slow on large Arrays. Using the below variation on the calc_primes method actually took 283ms to 10,000, 908ms to 20,000, 1818ms to 30,000 and 4522 to 50,000!!! ColdFusion, using arrayLen, managed it in 1929ms to 50,000.

public function calc_primes_count ($to) {
    $primes = array(2,3);
    for ($i = 5; $i &lt;= $to; ++$i) {
        $is_prime = true;
        for ($j = 0; $j &lt; count($primes) &amp;&amp; $is_prime; ++$j) {
            if ($i % $primes[$j] == 0) $is_prime = false;
        }
        if ($is_prime) array_push($primes, $i);
    }
}

Test two: Object creation

In this test I created (around the idea of a bee hive) a Colony object to which we add 10,000 Drone objects through running a while loop with the condition as a method call to the Colony to get it’s Drone count. Again code for each below.

The PHP Colony and Drone objects with test loop

class Colony {

    private
        $drones = array(),
        $colony_name,
        $drone_count = 0;

    public function __construct ($cn) {
        $this-&gt;colony_name = $cn;
    }

    public function add_drone ($d) {
        array_push($this-&gt;drones, $d);
        ++$this-&gt;drone_count;
    }

    public function get_drone_count () {
        return $this-&gt;drone_count;
    }

}

class Drone {

    private $message;

    public function __construct ($m) {
        $this-&gt;message = $m;
    }

    public function get_message () {
        return $this-&gt;message;
    }

    public function set_message ($val) {
        $this-&gt;message = $val;
    }

}

// Test Object creation with while
    $times = array();
    for ($i = 0; $i < 100; ++$i) {
        $start = microtime(true);
        $colony = new Colony ('PHP Object Colony');
        while ($colony->get_drone_count() < 10000) {
            $colony->add_drone(new Drone ("I am drone " . $colony->get_drone_count()));
        }
        array_push($times, microtime(true) - $start);
    }
    echo '<p>Object creation test took an average time of ' . array_sum($times)/count($times). ' to run over 100 iterations</p>';

The ColdFusion Colony and Drone objects with test while loop

Original code sample available in download

//Colony
component {

	public function init (cn) {
		drones = arrayNew(1);
	    droneCount = 0;
		colonyName = cn;
		return this;
	}

	public function addDrone (drone) {
		arrayAppend(drones, drone);
	}

	public function getDroneCount () {
		return arrayLen(drones);
	}

}

//Drone
cfcomponent {

	public function init (m) {
		message = m;
        return this;
    }

	public function getMessage () {
        return message;
    }

	public function setMessage (m) {
        message = val;
    }

}

// Test Object creation with while
times = [];
	for (i = 0; i < 1; ++i) {
		start = getTickCount();
		colony = new Colony9("ColdFusion object Colony");
	    while (colony.getDroneCount() < 10000)
	        colony.addDrone(new Drone9("I am drone" & colony.getDroneCount()));
	    arrayAppend(times, getTickcount() - start);
	}
    writeOutput("<p>Object creation test took an average time of " & arrayAvg(times) & "ms to run over 100 iterations</p>");

Object creation test results

I had a good idea about how this test would turn out and I wasn’t disappointed. ~~ColdFusion is ridiculously slow at creating objects. PHP could actually create 500,000 objects quicker than ColdFusion could do the 10,000. Shame on you ColdFusion, shame on you.~~

PHP: ~~42ms, 42ms, 42ms~~ 50ms average
ColdFusion: 7924ms, 7524ms, 7339ms – Oh Dear!! ~~844ms, 569ms, 647ms – see comments!~~ 337ms average

Test three: Output buffer speed

For this test I simply took a 140,000 character long string of Lorem Ipsum text and got both PHP and ColdFusion to write it to a variable from the output buffer. Again code for each below but with the string omitted.

The PHP output buffer code

//Test output buffer speed
    $txt = file_get_contents("string.txt");
    $times = array();
    for ($i = 0; $i < 100; ++$i) {
        $start = microtime(true);
        ob_start();
        echo $txt;
        $out = ob_get_clean();
        array_push($times, microtime(true) - $start);
    }
    echo '<p>Output buffer test took an average time of ' . array_sum($times)/count($times). ' to run over 100 iterations</p>';

The ColdFusion output buffer code

Original code sample available in download

<cfset txt = fileRead(expandPath("string.txt"))>
<cfset times = []>
<cfloop from=1 to=100 index="i">
	<cfset start = getTickCount()>
	<cfsavecontent variable="out">
	    <cfoutput>#txt#</cfoutput>
	</cfsavecontent>
	<cfset arrayAppend(times, getTickCount() - start)>
</cfloop>
<p>Output buffer test took an average time of <cfoutput>#arrayAvg(times)#</cfoutput>ms to run over 100 iterations</p>

Output buffer test results

140,000 characters isn’t too massive a string to deal with but PHP still came out ~~way~~ on top by a pretty sizeable margin.

PHP: ~~0.2ms, 0.1ms, 0.2ms~~ 0.14ms
ColdFusion: ~~2ms, 2ms, 2ms~~ 1.39ms

Rounding up

~~So it’s pretty obvious that if you are looking for performance then stay away from ColdFusion, especially if you are planning on building object oriented applications~~ ColdFusion obviously has issues with performance in dealing with objects which should be considered if using the language in applications where the use of coarse and fine grained objects are needed. On top of that, I might add that ColdFusion fails to offer certain object oriented features such as class level methods and attributes which ~~is so~~ I find annoying . ~~If however you have money to burn and want to get an application that talks to~~ If you are building an enterprise level application that, for instance, needs to interact with Microsoft Exchange ~~built in a day~~ then maybe ColdFusion is for you~~… Maybe?~~

I’d be interested to see results of these tests on other systems and server configurations so if you have the time download the files and let us now the results in the comments. I will run these tests on my Linux box later and on a Windows Server box in work on Monday and post the results.