Tag Archives: PHP

Mar 25

  • Created: Mar 25, 2011 12:00 PM

Optimizing APC Cache Settings for Magento

Optimizing APC Cache Settings for Magento
APC (Alternative PHP Cache) is an opcode cache for PHP that works very well to speed up page loads when used on servers running Magento. There are many opcode caches available (APC, accelerator, xcache, etc) which can be used with Magento, but we will be focusing on APC here as it along with eAccelerator are recommended by Magento in their performance whitepapers. eAccelerator also works very well with Magento, but please note that you must use version 0.9.5 or older.

APC can be used for two purposes within Magento. The first is as an opcode cache that PHP will utilize to cache any PHP files that are compiled through it. APC can also be utilized by Magento as an fast backend cache for Magento data. This can be configured within the Magento local.xml. Read more

Mar 5

  • Created: Mar 5, 2011 7:09 PM

Diagnosing MySQL Performance Bottlenecks With Maatkit

MySQL
Hello again! Last time we looked at optimizing performance of web apps, we were checking out how to diagnose PHP performance bottlenecks using XDebug to generate valgrind-style logs of what PHP was doing. While that can be immensely useful in some situations, just looking at how the PHP code executes doesn’t give us the whole story of most modern PHP / MySQL apps.

Database queries can be as important or more important than the PHP code itself, and we’ve seen lots of apps that run well enough until it’s time to access the database…where performance can easily go down the drain. Luckily, with most well-written apps like Magento, fixing database query issues is pretty straightforward. Either it’s a quick fix and we just add indices to a table, or it might be a known issue that we have a tried and tested solution for, or it goes back to the developers to figure out.
Read more

Posted in: Linux, php / Tagged: , , , , , ,
Jan 29

  • Created: Jan 29, 2011 10:02 AM

Diagnosing slow PHP execution with Xdebug and KCachegrind

Diagnosing slow PHP execution with Xdebug and KCachegrind
Tracking down a performance issue to the actual PHP app can be hard enough by itself, but what do you do once you’re sure that the app itself is the bottleneck? The excellent Xdebug extension has many useful features for assisting application developers, but today we’ll be looking at one specific feature that can help us see exactly what is slow in the application: profiling.

Profiling a PHP application can explain how much time the server spent in each function, each file, and each code path. It can also show you how many times a function or method was called, which is useful for diagnosing programming errors involving pointless looping. Xdebug generates cachegrind-compatible Read more

Dec 31

  • Created: Dec 31, 2010 8:20 PM

Upgrading to Magento 1.4.2.0

Securing Magento
The latest stable version of magento, version 1.4.2.0, was released on December 8th. Upgrading Magento Community Edition always poses some problem or another, mostly due to user-customized themes and third-party extensions. However, upgrading from a vanilla 1.4.1.x install does generate one error that we’ve noticed. There was a change in the way that Wishlists are set up and the “addWishlistLink()” method has been removed from the Mage_Wishlist_Block_Link class. Those upgrading from 1.4.1.x will likely see the following error: Read more

Posted in: Magento, php / Tagged: , , , ,
Oct 22

  • Created: Oct 22, 2010 2:53 PM

6 Tips for Making Magento Production Ready

Magento
During the development of your Magento eCommerce store, performance usually takes a back seat to functionality and design. However, after you’ve installed all of your extensions and have checked the spelling on your last cms page, you’ll want to focus your attention on making your shop as responsive as possible for your customers. Granted, it pays to have a terrific hosting provider that understands how to optimize the server environment, but there are also some things you can do on your end to minimize latency and make your shop run as efficiently as possible.

Tip 1: Tweak your .htaccess file:

Magento’s .htaccess file comes pre-configured with a few tweaks to make your pages load faster, but because hosting environments differ, many of these are commented out or set to low defaults.

1. Turn on output compression

mod_deflate is enabled on all of our servers. This module compresses your text-based files prior to sending them over the network. The customer’s browser then automatically uncompresses them prior to displaying them to the user. To enable this uncomment the appropriate lines as follows:

<IfModule mod_deflate.c>

############################################
## enable apache served files compression
## http://developer.yahoo.com/performance/rules.html#gzip

 # Insert filter on all content
 SetOutputFilter DEFLATE
 # Insert filter on selected content types only
 AddOutputFilterByType DEFLATE text/html text/plain text/xml text/css text/javascript

 # Netscape 4.x has some problems...
 BrowserMatch ^Mozilla/4 gzip-only-text/html

 # Netscape 4.06-4.08 have some more problems
 BrowserMatch ^Mozilla/4\.0[678] no-gzip

 # MSIE masquerades as Netscape, but it is fine
 BrowserMatch \bMSIE !no-gzip !gzip-only-text/html

 # Don't compress images
 SetEnvIfNoCase Request_URI \.(?:gif|jpe?g|png)$ no-gzip dont-vary

 # Make sure proxies don't deliver the wrong content
 Header append Vary User-Agent env=!dont-vary

</IfModule>

2. Enable far-future expires

Some content, such as images and css, don’t change all that often. For content such as this, you can improve performance by taking advantage of a web browser’s caching behavior. Setting far-future expires headers will allow web browsers to cache images and other static files so that upon subsequent visits, they won’t have to be re-downloaded.

<IfModule mod_expires.c>

############################################
## Add default Expires header
## http://developer.yahoo.com/performance/rules.html#expires

 ExpiresActive On
 ExpiresDefault "access plus 1 year"

</IfModule>

Tip 2: Refresh your indexes

Magento keeps indexes of some of your data in special tables to make queries more efficient. When you were developing your store, you were adding and deleting products, categories, urls, etc. This activity causes your indexes to have gaps and stale data. When you’re ready to go live, you’ll want to refresh all of your indexes in order to rebuild those tables.

There are two ways to do this: from your Admin dashboard or from the command line. To rebuild your indexes from your Magento Admin panel, just go to System > Index Management, select All, select Reindex Data, and then click Submit.

It’s also possible to rebuild your indexes from the command line using the shell tools included in Magento versions 1.4 and higher. These tools are located in your MAGENTO_ROOT/shell/ directory and can be run with a call to php’s command line interface:

$ php -f indexer.php help
Usage:  php -f indexer.php -- [options]

 --status <indexer>            Show Indexer(s) Status
 --mode <indexer>              Show Indexer(s) Index Mode
 --mode-realtime <indexer>     Set index mode type "Update on Save"
 --mode-manual <indexer>       Set index mode type "Manual Update"
 --reindex <indexer>           Reindex Data
 info                          Show allowed indexers
 reindexall                    Reindex Data by all indexers
 help                          This help

 <indexer>     Comma separated indexer codes or value "all" for all indexers

As you can see, the same functionality that can be found in the Admin panel under Index Management can be found here. To rebuild all of your indexes, just enter:

$ php -f indexer.php reindexall

Tip 3: Turn on caching

When you’re developing your site, you probably turned off Magento’s native cache so that you would be able to see your design changes instantly. Now that you’ve finalized your design work, make sure you turn the cache back on. This improves performance 5-10 times over the cache being off.

Ever wonder what gets cached? Here’s an overview:

  • Configuration files (xml)
  • Layouts (xml)
  • HTML blocks for Top Navigation, and Footer
  • Translations stored as an array
  • Data collections for Website, Store, and Store View

Tip 4: Use the Flat Catalog feature

Magento makes heavy use of the EAV model for storing customer, product, and category data in its database. The main benefit of the EAV model is that it allows for completely extensible attributes for those objects. The main drawback is that those attributes are stored in multiple tables rather than one very large table. This means that SQL queries need to be longer and more complex. Using the Flat Catalog feature creates new tables where each row contains all of the necessary data about a product or category. You can imagine that this makes the SQL queries much more efficient.

The Flat Catalog for Categories should be used for all Magento shops. The Flat Catalog for Products only begins to show a performance benefit once you’ve reached the 1000 SKU point. You can enable both of these in the back end:

  1. Go to System > Index Management and make sure “Product Flat Data” and “Category Flat Data” are built and ready.
  2. Go to System > Configuration > Catalog > FrontEnd and select “Yes” for Use Flat Catalog Category and (if desired) Use Flat Catalog Product.

Tip 5: Enable the Mage Compiler

This is a new feature introduced in version 1.3.2.x and bundled with 1.4.x, although it is still listed as “beta”. To understand how this feature can help improve your site’s performance, you have to understand what happens when a browser requests a URL from Magento. By default, and for every request, magento will look in the following directories, in the following order:

app/code/local/
app/code/community/
app/code/core/
lib/

This is what gives you, the developer, the wonderful ability to override and extend Magento to your heart’s content. If you want to change or modify some class, you just put your code in app/code/local/ and Magento will find it first and run it instead of the core definition in app/code/core.

As great as this is, it results in a great deal of disk I/O for every request. That’s where the Mage Compiler comes it. It essentially copies all of the class definitions and code found under app/code into a singe directory: includes/src/. It then gets rid of all of the other include paths and replaces them with the one. This way, Magento only needs to search one directory rather than four for each request.

It’s important to compile after you’ve finished installing your modules and developing your custom code if you want this to work properly. This too can be enabled either in the Admin panel or on the command line. In the Magento Admin, go to System > Tools > Compilation, and click Run Compilation Process. To run from the command line, use the following syntax (script is also located in the MAGENTO_ROOT/shell directory):

$ php -f compiler.php help
Usage:  php -f compiler.php -- [options]

  state         Show Compilation State
  compile       Run Compilation Process
  clear         Disable Compiler include path and Remove compiled files
  enable        Enable Compiler include path
  disable       Disable Compiler include path
  help          This help

Tip 6: Make use of parallel downloads to speed page rendering

Most web browsers limit the number of concurrent downloads to around 4 for each unique URL found in a web page. If all of your content is coming from the same host, this limitation can introduce unnecessary delays in rendering the page. You can trick the browser into downloading your content in parallel by using Pointer Domains and Magento’s base url settings.

Let’s say your awesome eCommerce store is located at http://www.example.com/. Your media, skin, and javascript URLs will therefore be http://www.example.com/media, http://www.example.com/skin/, and http://www.example.com/js/, respectively. What you want to do is create different host names for your media, skin, and js URLS that point to the same Magento installation as the www host.

You can do this in your Siteworx control panel under Hosting Features > Domains > Pointer Domains. Create pointer domains called media.example.com, js.example.com, and skin.example.com. Or, you can just create static.example.com. Next, log in to your Magento Admin panel and go to System > Configuration > Web, and change the secure and unsecure base urls for the media, skin, and js options to the pointer domains you just created. Your final configuration should look like the following:

Base URL: http://www.example.com/
Base Link URL: http://www.example.com/
Base Skin URL: http://skin.example.com/skin/
Base Media URL: http://media.example.com/media/
Base Javascript URL: http://js.example.com/js/

Now, go load your page. If you watch the status bar, you’ll notice that your static content, such as images and javascript, is being pulled from the domains you just created, while the main HTML is being pulled from the www URL. In reality these are all located on the exact same server, but the benefit is on the client side, and they don’t need to know that.

Oct 3

  • Created: Oct 3, 2010 10:35 AM

Finding the status of Magento cron jobs / tasks

As covered in our last article, you should have a “cron job” (crontab) set up to run Magento’s cron.php file every so often (15 minutes or so is fine) via PHP directly on the server to take care of housekeeping tasks that keep Magento working well. Some other tasks, like updating tracking / inventory status, sending out newsletters, and other miscellaneous things also require that the crontab be properly set up, so if you haven’t taken care of that yet, please see the setup guide here, or contact support@nexcess.net and we can help you set it up.

Once your crontab is properly installed and configured, you might be curious as to what it’s actually doing behind the scenes, or you might want to verify that something did / did not happen, and when. Since Magento lacks this arguably critical information, we’ve whipped up a simple PHP script that can show you what’s scheduled, what’s running, and what already ran, along with all the other information hiding in the ‘cron_schedule’ table of your Magento database. Simply drop the script (linked to below) into your base HTML directory for Magento (usually this will be your “public_html” directory), change the file extension to “.php” instead of “.phps”, and load it up in your favorite browser.

You should see something like this:
Mage Cron

All of the fields should speak for themselves.

Copy and save the following PHP script.

<?php
 
// Parse magento's local.xml to get db info, if local.xml is found
 
if (file_exists('app/etc/local.xml')) {
 
$xml = simplexml_load_file('app/etc/local.xml');
 
$tblprefix = $xml->global->resources->db->table_prefix;
$dbhost = $xml->global->resources->default_setup->connection->host;
$dbuser = $xml->global->resources->default_setup->connection->username;
$dbpass = $xml->global->resources->default_setup->connection->password;
$dbname = $xml->global->resources->default_setup->connection->dbname;
 
}
 
else {
    exit('Failed to open app/etc/local.xml');
}
 
// DB Interaction
$conn = mysql_connect($dbhost, $dbuser, $dbpass) or die ('Error connecting to <a class="HelpLink" onclick="showHelpTip(event, hint_id_7); return false;" href="javascript:void(0)">mysql</a>');
mysql_select_db($dbname);
 
$result = mysql_query("SELECT * FROM " . $tblprefix . "cron_schedule") or die (mysql_error());
 
 
// CSS for NexStyle
echo '
<html>
<head>
<title=Magento Cron <span style='background-color:#CCFF00;'>Status</span>>
<style type="text/css">
html {
    width: 100%;
    font-family: Helvetica, Arial, sans-serif;
}
body {
    background-color:#00AEEF;
    color:#FFFFFF;
    line-height:1.0em;
    font-size: 125%;
}
b {
    color: #FFFFFF;
}
table{
    border-spacing: 1px;
    border-collapse: collapse;
    width: 300px;
}
th {
    text-align: center;
    font-size: 125%;
    font-weight: bold;
    padding: 5px;
    border: 2px solid #FFFFFF;
    background: #00AEEF;
    color: #FFFFFF;
}
td {
    text-align: left;
    padding: 4px;
    border: 2px solid #FFFFFF;
    color: #FFFFFF;
    background: #666;
}
</style>
</head>';
 
// DB info for user to see
echo '
<body>
<a href="http://nexcess.net">
<img src="http://static.nexcess.net/images/logoMainR2.gif" width="217" height="38" alt="Nexcess Beyond Hosting"></a>
 
 
 
<b>Table Prefix:</b> ' . $tblprefix . ''
. '<b>DB Host:</b> ' . $dbhost . ''
. '<b>DB User:</b> ' . $dbuser . ''
. '<b>DB Name</b>: ' . $dbname . '</p>';
 
// Set up <span style="background-color:#CCFF00;">the</span> table
echo "
        <table border='1'>
        <thread>
        <tr>
        <th>schedule_id</th>
           <th>job_code</th>
           <th><span style="background-color:#CCFF00;">status</span></th>
           <th>messages</th>
           <th>created_at</th>
           <th>scheduled_at</th>
           <th>executed_at</th>
           <th>finished_at</th>
           </tr>
           </thread>
           <tbody>";
 
// Display <span style="background-color:#CCFF00;">the</span> data from <span style="background-color:#CCFF00;">the</span> query
while ($row = mysql_fetch_array($result)) {
           echo "<tr>";
           echo "<td>" . $row['schedule_id'] . "</td>";
           echo "<td>" . $row['job_code'] . "</td>";
           echo "<td>" . $row['<span style="background-color:#CCFF00;">status</span>'] . "</td>";
           echo "<td>" . $row['messages'] . "</td>";
           echo "<td>" . $row['created_at'] . "</td>";
           echo "<td>" . $row['scheduled_at'] . "</td>";
           echo "<td>" . $row['executed_at'] . "</td>";
           echo "<td>" . $row['finished_at'] . "</td>";
           echo "</tr>";
}
 
// Close table and last few tags
echo "</tbody></table></body></html>";
 
mysql_close($conn);
?>

PLEASE NOTE: This script is designed to be secured for use by site administrators only. If you have any questions, please e-mail support@nexcess.net

Posted in: Magento, php / Tagged: , , , , , ,
Sep 24

  • Created: Sep 24, 2010 4:47 PM

Speed up Magento DataFlow (Import / Export)

For as long as we can remember, Magento has had issues with Import and Export profiles, especially regarding performance. We have tried many different solutions for speeding up DataFlow and dealing with other import / export related issues. We’ve found a solution that seems to help in the majority of cases. First, I’d like to mention that the Magento import / export status page occasionally just shows a white screen (page doesn’t load completely or loads blank), a 500 error, or some other random error when really, the import / export job is still running in the background and will complete successfully.

Magento DataFlow has 4 main pieces

  • Adapter (Read the external data source and allows the parser to access it)
  • Parser (Goes through the external data and translates it into something Magento can understand)
  • Mapper (Takes external data fields and associates them with the correct Magento data fields)
  • Validator (Ensures that data is correct before / after committing it)

A standard workflow looks something like this

Create an import / export profile (either via Profile Manager or Advanced Profile which allows you to tweak the actual XML profile)

You probably want to select “Local File” for “File Information -> Type” when creating the profile. “Local File” means that the file will be saved to [Magento basedir]/var/export if you’re exporting data. It is critical that you ensure the file does not exist or that you manually specify a new filename for each export; sometimes Magento has trouble overwriting an existing file and this will cause cryptic errors / export failure. Your best bet is to use a new filename for each attempt.

Run the profile either via a custom script (not recommended or necessary in most cases, unless running via cron)

When DataFlow receives an XML request for an import or export, it will connect to either the database in the case of an export or the external data source in the case of an import and (after parsing, mapping, and possibly validating) start building up the results (to be later written to a file or imported into to the database) in the following tables.

dataflow_batch_import<br />
dataflow_batch_export

There is an issue with Magento where it does not truncate (empty the data from) the tables before starting an import / export. Also, having extra data in those tables or the logs will slow DataFlow down quite a bit.

Here is what we recommend for speeding up DataFlow

  1. Log into your Magento Dashboard (admin) and go to System > Configuration
  2. Go to Advanced > System -> Log Cleaning on the side menu
  3. Change “save log, days”. We recommend 14.
  4. Select “Enable log cleaning”
  5. Ensure your crontab is properly configured. Contact support@nexcess.net if you’re not sure or follow this guide: ( http://docs.nexcess.net/cron )
    To make sure the logs get cleaned, run cron.php manually just before the import by loading http://yoursite.tld/cron.php or wherever cron.php is located (it’s in your Magento base directory) in a web browser. There won’t be any output displayed in your browser, but you should get a HTTP/200 response code if everything ran OK.
  6. You can check the status of the export by doing a “SELECT COUNT(batch_export_id) FROM dataflow_batch_export” in MySQL. Alternatively, you can download the PHP script below and rename it “mage-dataflow.php” and upload it to your Magento base directory, then load it in your web browser. It will show you the number of rows in the import and export tables. When the rows go up when you reload the page, you can see that more items are being processed. When the count drops back to zero, the dataflow operation has started writing out the file for the export or importing the records for an import and should be done in less than 5 minutes. Additionally, the page lets you truncate the tables at the press of a button (WARNING: Only use if your dataflow process is stuck! It will abort the dataflow process).

You can download the mage-dataflow file in from our docs here.

Posted in: Magento, php / Tagged: , ,
Mar 31

  • Created: Mar 31, 2010 11:13 AM

PHP open_basedir and Magento Performance

The PHP open_basedir directive is a security feature within PHP that allows you to specify which files PHP is allowed to be opened. With open_basedir, you specify the directory tree or trees that PHP can open and PHP will not open anything outside of these directories. There are negative side effects in relation to system performance when using open_basedir. The most significant is that when open_basedir is enabled, the PHP realpath cache will be disabled. The PHP realpath cache has been available in PHP since 5.1.0 and caches the paths of PHP include files.

If running a smaller site with a low file count and with relatively shallow directory paths, the fact that directory paths will not be cached isn’t necessarily essential. But with applications such as Magento built on the Zend Framework, you end up with both a large base file count with a large include path and a very deep directory structure. In this situation you need to make sure that your paths do become cached with the realpath cache which means leaving open_basedir disabled.

While stack tracing a PHP process on a Magento store, you will see many lstat calls similar to the following:

access(&quot;/chroot/home/store/html/app/code/local/Core/Block/Template.php&quot;, F_OK) = 0<br />
time(NULL)                              = 1268839768<br />
lstat(&quot;/chroot&quot;, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home&quot;, {st_mode=S_IFDIR|0711, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store&quot;, {st_mode=S_IFDIR|S_ISGID|0751, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store/html&quot;, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store/html/app&quot;, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store/html/app/code&quot;, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store/html/app/code/local&quot;, {st_mode=S_IFDIR|0777, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store/html/app/code/local/Core&quot;, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store/html/app/code/local/Core/Block&quot;, {st_mode=S_IFDIR|0755, st_size=4096, ...}) = 0<br />
lstat(&quot;/chroot/home/store/html/app/code/local/Core/Block/Template.php&quot;, {st_mode=S_IFREG|0644, st_size=5968, ...}) = 0

When the realpath cache is enabled the number of these calls will drop dramatically as the paths being used will be cached, preventing the tree of lstat calls that you see above. When open_basedir is enabled, these paths cannot be cached requiring all of the lstat calls seen above every time the file is called. The issue with this situation is that the system is spending a significant amount of cpu time performing system calls instead of running the processes in userland. If you see a significant amount of cpu time spent in %sy your system may not be using the realpath cache.

So how bad is performance running Magento with the open_basedir enabled? I ran a simple test to find out. I installed a Magento demo store on a test server and compared the number of lstat calls with open_basedir enabled and disabled for a single product page load.

With open_basedir enabled: 25998 lstat() calls

With open_basedir disabled: 1155 lstat() calls

The resulting difference was an additional 24843 lstat calls required for a single page load with open_basedir enabled. Regarding the page load times, there is a very trivial difference for a single page load with no other concurrent processes, but as soon as you have many concurrent page load processes running on the server, these lstat calls will bring the server to a crawl. The issue will be amplified if the server is running on older slow IDE or SATA disks.

Posted in: Magento, php / Tagged: , ,