I recently worked with one of our Magento Enterprise Cluster clients who wanted to improve the performance of their site. This client was using our Content Delivery Network to offload the serving of static assets, and was following all of our recommended Magento Optimization best practices. They were also using Memcached as the fast-backend in their Magento configuration. Despite doing everything right in terms of optimization, we managed to shave considerable time off their time-to-first-byte by leveraging multiple instances of memcached and Magento’s terrific use of the Zend Framework for caching. Here’s how.
Whenever you configure a shared-memory type of caching backend — e.g. memcached, xcache, or apc — Magento will enable the Zend Framework’s TwoLevel cache type. The basic idea behind the two level cache is that you have a fast, limited-sized cache for the initial lookup and a slower, larger-capacity cache for when the fast cache lookup misses. The benefit of this is that you get the speed of a memory-based caching backend plus the capacity of a disk- or database-based backend. That is, when the two-level cache works as it should. Fabrizio Branca has an excellent writeup about the problems with Magento / Zend TwoLevel caching, along with a patch that can be used to fix some of them. I have implemented this patch on a few sites now and have seen significant improvement in first-level cache hits.
Memcached and Sessions
The options for session storage on a single dedicated server are ‘File’, ‘Database’, ‘eAccelerator’ or ‘Memcache’. The default is ‘File’ and tends to work fine in this setup. However, once you move to a clustered system, the need arises to use a session storage that can be shared between the application nodes. Using ‘Database’ is almost always a bad idea because there is already enough DB traffic in Magento without compounding it with session traffic. ‘eAccelerator’ may not be available in every instance. That leaves ‘Memcache’, which is trivial to set up, uses TCP, and registers a session handler when the PECL extension is installed.
It is crucial to use a separate instance of memcached for session storage so that you don’t lose all of your sessions when you flush your Magento cache. You can configure the second instance to run on the same server, but on a different TCP port (e.g. 11212). As long as you don’t have unusually long session lifetimes (a bad idea), you can configure the cache size to be much smaller than that of your fast-backend in order to conserve memory. Once the second instance is up and running, configuring Magento to use it is just two lines in your local.xml file:
Memcached and the Slow Backend
Remember how I said that when two-level caching is enabled, the second level is for the slower, larger capacity storage? Well, memory is cheap, my friend, and there’s nothing preventing you from setting up a third memcached instance and using that for the second level cache. The configuration is trivial:
<cache> <backend><![CDATA[memcached]]></backend> <slow_backend><![CDATA[Memcached]]></slow_backend> <auto_refresh_fast_cache>0</auto_refresh_fast_cache> <default_priority>10</default_priority> <memcached> <servers> <server> <host><![CDATA[10.0.0.1]]></host> <port><![CDATA]></port> <persistent><![CDATA]></persistent> </server> </servers> </memcached> <slow_backend_options> <servers> <server> <host><![CDATA[10.0.0.1]]></host> <port><![CDATA]></port> <persistent><![CDATA]></persistent> </server> </servers> </slow_backend_options> </cache>
Here you can see that we set up a new instance of memcached to listen on port 11213 and set the ‘slow_backend’ to instantiate the Zend_Cache_Backend_Memcached class instead of the default Zend_Cache_Backend_File. The only thing to be aware of with this configuration is that when I set it up, I noticed that the hit percentage is much lower than that of the first level instance.
I haven’t been able to determine whether that is typical or whether it has something to do with memcached’s lack of support for tags.
Edit: David Alger from Classy Llama Studios confirmed my suspicion about tags. It turns out that for the slow-backend to function properly, the backend cache type needs to support tagging. None of the shared memory backends support tags, which means that out of the box, the only slow-backends that work are files and database.
There are two extensions available that support tags and offer improved performance over the filesystem. One uses symlinking to impove lookup time. The other uses Redis.
Memcached and the Full Page Cache
Finally, if you really want to see your website fly, you can take advantage of the Full Page Cache module that is found only in Magento Enterprise Edition. This took a bit of investigation on my part, since it isn’t documented anywhere, but it turns out that the Enterprise_PageCache module instantiates a Mage_Core_Model_Cache object to use for full page caching. This gives us full access to the range of cache backends available to the core cache module, including memcached. The configuration is similar to the one from the local.xml file, but it goes under the ‘full_page_cache’ node in the enterprise.xml file. This would be running on a fourth memcached instance listening on port 11214.
After implementing all four instances of memcached in Magento, the client’s website started flying. When I tested it using WebPageTest.org, something remarkable happened: straight A’s:
Those of you that work with Magento regularly can attest to how rare that is. Hopefully this post has given you some hints about how you can leverage the great caching options provided in Magento and the Zend Framework to improve your website’s performance. And if you only take away one thing, let it be this: The key to performance on the web is good design and inserting caching wherever you can.