Archive

Archive for the ‘PHP-FPM’ Category

PHP 5.4 stuff I'm jazzed about...

January 28th, 2012 No comments

I'm always excited to read updates to the NEWS file for each PHP version (yes, I am that big of a PHP fan boy) - and PHP 5.4 has quite a handful of noteworthy changes. I started cutting them out and decided to publish my "this is interesting" or "this is awesome" list...

This is as of PHP 5.4.0 RC6 (NEWS). Note that I didn't put in anything related to OO, as I despise the obsession with OO now in PHP and do my best to live without it.

Misc. notable changes:

  • Added built-in web server that is intended for testing purpose. (Moriyoshi)
  • Changed default value of "default_charset" php.ini option from ISO-8859-1 to UTF-8. (Rasmus)
  • Added array dereferencing support. (Felipe)
  • Added header_register_callback() which is invoked immediately prior to the sending of headers and after default headers have been added. (Scott)
  • Changed http_response_code() to be able to set a response code. (Kalle)
  • Added new json_encode() option JSON_PRETTY_PRINT. FR #44331. (Adam)
  • Changed silent conversion of array to string to produce a notice. (Patrick)
  • Removed support for putenv("TZ=..") for setting the timezone. (Derick)
  • Removed the timezone guessing algorithm in case the timezone isn't set with date.timezone or date_default_timezone_set(). Instead of a guessed timezone, "UTC" is now used instead. (Derick)
  • ext/mysql, mysqli and pdo_mysql now use mysqlnd by default. (Johannes) )I think this is in 5.3 though too?)
  • Expose session status via new function, session_status (FR #52982) (Arpad)
  • Added support for storing upload progress feedback in session data. (Arnaud)

PHP-FPM related:

  • Remove EXPERIMENTAL flag. (fat)
  • Added partial syslog support (on error_log only). FR #52052. (fat)
  • Lowered default value for Process Manager. FR #54098. (fat)
  • Enhance security by limiting access to user defined extensions. FR #55181. (fat)
  • Added process.max to control the number of process FPM can fork. FR #55166. (fat)
  • Dropped restriction of not setting the same value multiple times, the last one holds. (giovanni at giacobbi dot net, fat)

Removed legacy features:

  • Safe mode and all related ini options. (Kalle)
  • register_globals and register_long_arrays ini options. (Kalle)
  • import_request_variables(). (Kalle)
  • allow_call_time_pass_reference. (Pierrick)
  • Session bug compatibility mode (session.bug_compat_42 and session.bug_compat_warn ini options). (Kalle)
  • session_is_registered(), session_register() and session_unregister() functions. (Kalle)
  • y2k_compliance ini option. (Kalle)
  • Removed magic_quotes_gpc, magic_quotes_runtime and magic_quotes_sybase ini options. get_magic_quotes_gpc, get_magic_quotes_runtime are kept but always return false, set_magic_quotes_runtime raises an E_CORE_ERROR. (Pierrick, Pierre)
Categories: PHP, PHP-FPM

Setting PHP INI parameters from nginx

February 11th, 2011 3 comments

A little known feature in PHP-FPM 5.3.3+ (since it was integrated into PHP core) is that you can actually define PHP INI parameters inside of your nginx configuration. This bridges some of the desire to have "php_value" and "php_admin_value" from Apache available. It's not user-override-able, but things like htscanner or newer PHP 5.3 features could address some of that.

This is cool, and I wasn't sure it actually got in a build or not, I remember it was mentioned or discussed, but sure enough, Jérôme coded it and got it in to FPM in core. So it is available to all, and he welcomes the feedback - see this thread on the nginx mailing list.

Due to limitations in the FastCGI protocol, you have to pass all the parameters you want as a single string, separated by "\n" - it's not the cleanest looking configuration, but that's how it is for now. I spitballed a different approach to it, but at the moment I believe it will be unlikely to get adopted.

If you're like me, you're probably looking for the code samples almost immediately. Here's your examples. Note that PHP_VALUE and PHP_ADMIN_VALUE are both legitimate keys, the difference being that PHP_ADMIN_VALUE does not allow the user to override the value using ini_set() - see the manual for more infomation.

Single value:

fastcgi_param PHP_VALUE sendmail_from=direct@prout.com;

Multiple values, manually line-broken:

fastcgi_param PHP_VALUE "sendmail_from=chroot@prout.com
precision=42";

Multiple values, with the \n in there to clearly call it out:

fastcgi_param PHP_VALUE "sendmail_from=chroot@prout.com \n precision=42";

More information is available in the feature request on the PHP bug tracker.

Categories: nginx, PHP, PHP-FPM

PHP 5.2.15 released, patch updated

December 9th, 2010 No comments

PHP 5.2.15 was released today, and I've made a copy of the previous patch. It still applies without any issues (a "make test" looks good) - this is supposed to be the latest version of PHP 5.2.x.

Download it here.

I highly recommend upgrading to PHP 5.3.3+ with the more updated/better FPM version (among a million other enhancements) that is bundled in core now. Coincidentally, PHP 5.3.4 came out today as well, with a handful of FPM bug fixes/features (see my other blog post or the NEWS file.)

Enjoy!

Categories: PHP, PHP-FPM

I was wrong.

November 18th, 2010 3 comments

I was wrong, and I'm happy to admit it.

I was worried that if FPM was to ever make it in to PHP core, it would stagnate and become part of such a big machine that it would take forever to receive updates. However, since its adoption into PHP 5.3.3, Jérôme has been hacking away on it and I was happy to see that in 5.3.4RC1, a handful of changes are being included:

  • Added '-p/--prefix' to php-fpm to use a custom prefix and run multiple instances. (fat)
  • Added custom process title for FPM. (fat)
  • Added '-t/--test' to php-fpm to check and validate FPM conf file. (fat)
  • Added statistics about listening socket queue length for FPM. (andrei dot nigmatulin at gmail dot com, fat)
  • Fixed inconsistent backlog default value (-1) in FPM on many systems. (fat)
  • Fixed bug #52674 (FPM Status page returns inconsistent Content-Type headers). (fat)
  • Fixed bug #52498 (libevent was not only linked to php-fpm). (fat)

Thanks guys, I am counting the days until I can finally use PHP 5.3. I also discovered the new magic variable __DIR__ exists a couple weeks ago - between everything, I am jonesing to move badly!

(For those wondering, "fat" is Jérôme's pseudonym.)

Categories: PHP, PHP-FPM

Cleanest configuration for the new PHP-FPM?

August 26th, 2010 12 comments

When examining the PHP-FPM configuration, I realized that I only tweak a few key pieces per pool, so I decided to share my approach to minimize redundancy and keep things simple (more files, but simpler to manage)

Because I still have to maintain PHP 5.2.x for clients, I have decided to try building everything in self-contained in an /opt/php53 directory. So consider that my $prefix, and change appropriately.

I have the following setup, and it seems to work great so far.

/opt/php53/etc/php-fpm.conf:

[global]
log_level = notice
error_log = /opt/php53/var/log/php-fpm.log
pid = /opt/php53/var/run/php-fpm.pid
emergency_restart_threshold = 10
emergency_restart_interval = 1m
process_control_timeout = 5s
daemonize = yes

; pools
include = /opt/php53/etc/fpm.d/pools/*.conf

One file per pool, for example, a pool named "mike" -

/opt/php53/etc/fpm.d/pools/mike.conf:

[mike]
listen = 127.0.0.1:11000
user = mike
group = mike
request_slowlog_timeout = 5s
slowlog = /opt/php53/var/log/slowlog-mike.log
pm.max_children = 5
pm.start_servers = 3
pm.min_spare_servers = 2
pm.max_spare_servers = 4
pm.max_requests = 500
include = /opt/php53/etc/fpm.d/common.conf

Common elements for each pool (if these could be inherited globally, which they MIGHT be, I could just toss them in the main php-fpm.conf. Perhaps a feature request. Will post on the mailing list...)

Remember that rlimit_files needs to be something set in your sysctl.conf or on the system level or you'll get that RLIMIT_NOFILE warning. Also, depending on how you want to limit resources per pool/client, you may want to tweak things, such as request_terminate_timeout.

/opt/php53/etc/fpm.d/common.conf:

listen.backlog = -1
listen.allowed_clients = 127.0.0.1
pm = dynamic
pm.status_path = /status
ping.path = /ping
ping.response = pong
request_terminate_timeout = 120s
rlimit_files = 131072
rlimit_core = unlimited
catch_workers_output = yes
env[HOSTNAME] = $HOSTNAME
env[PATH] = /bin:/usr/bin:/usr/local/bin:/usr/local/sbin:/sbin:/usr/sbin:/opt/php53/bin:/opt/php53/sbin
env[TMP] = /tmp
env[TMPDIR] = /tmp
env[TEMP] = /tmp

As always, YMMV.

Categories: PHP, PHP-FPM

The magic that is... nginx

August 18th, 2010 2 comments

I've told this story countless times, but I've never publicly documented it. This happened over a year ago, but I feel obligated to share it. nginx is the main reason for the success and deserves bragging rights.

We had an upcoming product release, and another team was handling all the communication, content, etc. I was simply just the "managed server" administrator type person. Helped setup nginx, PHP-FPM, configure packages on the server, stuff like that.

The other team managed the web applications and such. I had no clue what we would be in for that day.

There is a phrase, "too many eggs in one basket" - well, this was our first system and we were using it for many things, it hadn't gotten to the point where we needed to split it up or worry about scaling. Until this day, of course.

To me, the planning could have been a bit better, with the ISO image we were providing from the origin being staged on other mirrors, using torrents to offload traffic to P2P, etc. However, that wasn't done proactively, only reactively.

The story

Launch day occurs. I get an email about someone unable to SSH to a specific IP. I check it out - the IP is unreachable, but the server is up. I submit a ticket to the provider, and they let me know why - apparently the server had triggered their DDoS mitigation system once it hit something like ~500mbit; it was automatically flagged as probably being under attack, and the IP was blackholed.

Once we informed them this was all legit, they instantly worked to put us onto a switch that was not DDoS protected, and we resumed taking traffic. This was all done within 15 or 20 minutes, if I recall. I've never seen anything so smoothly handled - no IP changes, completely transparent to us.

I believe the pocket of no traffic (seen in the graph below) was when we were moved off their normal network monitoring and into a Cisco Guard setup. We were definately caught off guard; like I said, I knew we'd get some traffic, but not filling the entire gigabit port. Not a lot of providers would be so flexible about this and handle it with such grace. There are some reports of it being slow, and that is literally because the server itself has too much going on. PHP is trying to handle all the Drupal traffic, and during the night, the disk I/O was at 100% for a long period of time. Oh yeah - since this was the origin, the servers mirroring us had to start pulling stuff from us too :)

Luckily we're billed by the gigabyte there, not by 95th like most places, or this would be one heck of a hosting bill. We wound up able to reroute the bandwidth fast enough to not even be charged ANY overage for it!

All in all, without nginx in the mix, I doubt this server would have been able to take the pounding. There was no reverse proxy cache, no Varnish, no Squid, nothing of that nature. I am not sure Drupal was even setup to use any memory caching, and I don't believe memcached was available. There were a LOT of options to reduce load - the main one was just cutting down the bandwidth usage to open up space in the pipe, which was eventually done by removing the ISO file off the origin server, and pushing it to a couple mirror sites. Things calmed down then.

However, it attests to how amazinly efficient nginx is - the entire experience wound up taking only 60 something megabytes of RAM for nginx.

Want the details? See below.

The hardware

  • Intel Xeon 3220 (single CPU, quad-core)
  • 4GB RAM
  • single 7200RPM 250GB SATA disk, no RAID, no LVM, nothing
  • Gigabit networking to the rack, with a 10GbE uplink

The software - to the best of my memory (remember, all on a single box!)

  • nginx (probably 0.7.x at that point)
    • proxying PHP requests to PHP-FPM with PHP 5.2.x (using the patch)
      • Drupal 6.x - as far as I know, no advanced caching, no memcached, *possibly* APC
    • proxying certain hosts for CGI requests to Apache 2.x (not sure if it was 2.2.x or 2.0.x)
    • This server was also a yum repo for the project, serving through nginx
  • PHP-FPM - for Drupal, possibly a couple other apps
  • Postfix - the best MTA out there :)
    • I believe amavisd/clamav/etc. was involved for mail scanning
    • Integration with mailman, of course
  • MySQL - 5.0.x, using MyISAM tables by default, I don't believe things were converted to InnoDB
  • rsync - mirrors were pulling using the rsync protocol

The provider

  • SoftLayer - they just rock. Not a paid placement. :)

The stats

nginx memory usage
During some of that time... only needed 60 megs of physical RAM. 240 megs including virtual. At 2200+ concurrent connections... eat that, Apache.

root     13023  0.0  0.0  39684  2144 ?        Ss   03:56   0:00 nginx: master process /usr/sbin/nginx
www-data 13024  2.0  0.3  50148 14464 ?        D    03:56   9:30 nginx: worker process
www-data 13025  1.1  0.3  51052 15256 ?        D    03:56   5:38 nginx: worker process
www-data 13026  1.3  0.3  50760 15076 ?        D    03:56   6:13 nginx: worker process
www-data 13027  1.3  0.3  50584 14900 ?        D    03:56   6:22 nginx: worker process

nginx status (taken at some random point)

Active connections: 2258
server accepts handled requests
711389 711389 1483197
Reading: 2 Writing: 2040 Waiting: 216

Bandwidth (taken from the provider's switch)

Exceeded Bits Out: 1001.9 M (Threshold: 500 M)
Auto Manage Method: FCR_BLOCK
Auto Manage Result: SUCCESSFUL

Exceeded Bits Out: 868.1 M (Threshold: 500 M)
Auto Manage Method: FCR_BLOCK
Auto Manage Result: SUCCESSFUL

To give you an idea of the magnitude of growth, this is the amount of gigabytes the server pushes on a normal day:

  • 2009-05-18 155.01 GB
  • 2009-05-17 127.48 GB
  • 2009-05-16 104.21 GB
  • 2009-05-15 152.42 GB
  • 2009-05-14 160.12 GB
  • 2009-05-13 148.6 GB

On launch day and the spillover into the next day:

  • 2009-05-19 2036.37 GB
  • 2009-05-20 2481.87 GB

The pretty pictures

Click for larger versions!

Hourly traffic graph
nginx rocks!
(Note: I said 600M, apparently their threshhold from their router says 500M)

Weekly traffic graph
nginx rocks!

The takeaway

Normally I would never think a server could get slammed with so much while it is having to service so much. Perhaps if it was JUST a PHP/MySQL server, or JUST a static file server, but no - we had two webservers, a mailing list manager, Drupal (which is not the most optimized PHP software), etc. The server remained responsive enough to service requests, on purely commodity hardware.

Categories: nginx, PHP, PHP-FPM

Happy day! PHP-FPM included with PHP 5.3.3RC1, and WordPress 3.0

June 17th, 2010 3 comments

Officially in the 5.3.3RC1 distribution. Sweet! From the NEWS file:

17 Jun 2010, PHP 5.3.3 RC1
...
- Added FastCGI Process Manager (FPM) SAPI. (Tony)
...

and on an unrelated note:

- Added support for JSON_NUMERIC_CHECK option in json_encode() that converts numeric strings to integers. (Ilia)

Shouldn't this be called JSON_NUMERIC_CONVERT? or JSON_FORCE_INTEGER? It's not just a "check" - guess it's too late now? :)

WordPress 3.0...
WordPress 3.0 came out today. Tonight I'll probably upgrade this site and see how well it works. I'm going to check it in to Subversion first so I can roll back if needed.

Some key changes I wanted to talk about...

  • One thing that was highlighted is the option to use custom header images - which can easily be done right now. I did it well over a year ago in a theme. With post meta you can always load metadata about a post and use it in the theme, so this update seems a bit specific to me, since themes were already customizable. Why build a feature that is so specific? Same with background images/colors...
  • Custom menus/menu editor - this could get cool, the menu editor is the more exciting piece as it will allow a visual way to manage the taxonomy. Not sure how it will mix in with tags and categories though, guess that's "I'll see it when I upgrade."
  • MU merge - finally, I can run multiple installs off the same WP install, hopefully, without wp-config.php hacks. How exactly it works I will have to find out.
  • Custom post types - now all of a sudden you can make any sort of object with custom attributes, which opens the door to things such as the item below.
  • WP e-Commerce says they're going to change from using all their extra tables to using core WP schema. That's awesome.

A couple bones to pick...

  • It's not a rewrite. It's still a blogging tool that is being extended further to be a full-featured that can handle "anything" - however the tables are still named "posts" even though now you can create an arbitrary type of item. I'd like to see it renamed and normalized.
  • All the plugins and themes and such are procedural code, but some inner workings such as the DB layer are OO. That seems amateur to me, and unnecessary.

I'd love to see WP get rewritten. It has a LOT of overhead and includes built in that need calling and a lot of other cruft that I stumble across. Go back to the drawing board with building a list of every feature it has, and look at it from a longer term perspective. It's great to see something keep growing, but when it comes down to it, it is still a fork of b2, which was made for blogging, not for anything and everything.

It's got the right idea with extensibility and such, but to me the core has a lot of code - and lots of code means more complicated execution paths, more "I'll just add this in instead of refactor this old code," more cruft. I'm quite sure I could get as much extensibility out of a fresh rewrite with less than half the code under the hood. Things like text styling for example should be moved to a plugin (I disable all the wptexturize filters for example... throw those in an include and make it enabled by default instead!)

Of course, WordPress does have millions of users so it has a proven track record. I can't complain that much, I do use it myself. For blogging, it's the best tool out there. For other things, it typically leverages plugins which may or may not have decent UIs or APIs to interact with. That's where it shows signs of weakness. It also isn't as strict as Drupal when it comes to code conventions either, which would greatly increase the usability of a lot of plugins.

Categories: PHP, PHP-FPM, WordPress

PHP-FPM and nginx upstart scripts

May 21st, 2010 5 comments

Upstart is becoming the de-facto standard for everything in Ubuntu, and I do enjoy it's process management and re-spawning capabilities.

(Actually, before PHP-FPM I used to use upstart jobs to su - $user php-cgi -b $port :))

These are VERY simple but effective scripts, and would actually be beefed up to be more intelligent (chaining nginx to start after PHP-FPM for example. However, if you do not need PHP, then that's a useless chain. So I kept it simple. I suppose you could add a short delay to start nginx then...)

Note: make sure PHP-FPM is set to daemonized = yes.

Second note: this works for PHP 5.2.x w/ the PHP-FPM patch, on Ubuntu Lucid Lynx (10.04) - anything else YMMV. I am not using PHP-FPM w/ PHP 5.3 yet since I have no environments that I know will support 5.3 code. When I finally get one, I will look for the same opportunity.

/etc/init/nginx.conf:

description "nginx"

start on (net-device-up and local-filesystems)
stop on runlevel [016]

expect fork
respawn
exec /usr/sbin/nginx

/etc/init/php-fpm.conf:

description "PHP FastCGI Process Manager"

start on (net-device-up and local-filesystems)
stop on runlevel [016]

expect fork
respawn
exec /usr/local/bin/php-cgi --fpm --fpm-config /etc/php-fpm.conf

Once you've done this you can remove all the /etc/init.d/php-fpm, /etc/init.d/nginx and /etc/rc?.d/[K,S]??php-fpm and /etc/rc?.d/[K,S]??nginx symlinks and files. This takes care of all of that.

Feel free to comment and leave better tips, tricks and ideas!

Categories: Development, nginx, PHP, PHP-FPM

Jérôme Loyet is a saint!

December 15th, 2009 No comments

I'd like to announce that Jérôme Loyet stepped up this weekend and hacked up the first round of code to get dynamic process management going. Antony committed it (see below) and it's on its way to being part of PHP core as well. So two major events in only a week and a half or so!

See http://news.php.net/php.internals/46414.

Good job Jérôme. Where were you a couple months ago, buddy? :)

Categories: PHP, PHP-FPM

PHP-FPM brought in to PHP core - interesting surprise

December 5th, 2009 No comments

Read it here: http://news.php.net/php.internals/46277.

First off, big thanks to Antony Dovgal. I've exchanged words with him in the past about PHP-FPM (and actually other PHP things) but was completely unaware he was working on this.

So, we've got a blessing but also an interesting dilemma on our hands. We've got a wishlist and some bugs to work out. I have a feeling if Antony updated some of the CGI internals it may have resolved some of those bugs. Not sure. I'm trying to get some specifics now - what version of PHP-FPM he brought in, how the community can still support it and how difficult it may be to submit patching, if he thinks a separate management daemon makes more sense than keeping it glued inside of the SAPI (it seems out of place to me for a SAPI to require a proprietary configuration file and daemon .pid, log, etc. files...)

Hopefully I can get ahold of him soon and discuss some of this. I was already mid-discussion with another PHP core developer about how they think the best approach would be to get PHP-FPM aligned with PHP core (they leaned more towards a separate SAPI too.)

My main goal is to make it easy as it can be for PHP-FPM to become an official package or included with PHP so that people who use PHP from repositories on their favorite distributions and such can enjoy the benefits of PHP-FPM without patches or separate downloads. If the management portion does split off, I fully intend on making sure it is aligned properly and is as simple as "apt-get install php5-fpm" or something of that nature. Still easily installed and everything.

Anyway, we'll see how things go. This caught me off guard and now I have to figure out at what point we're at now with development. Jérôme Loyet has expressed interested in trying to convert the configuration file to nginx style - something Andrei had told me he had wanted to do. The XML throws some people off, thinking it's an actual XML parsed document with XML include support and such... also if done right, this will allow PHP-FPM's configuration to support includes, and who knows, maybe variables some day. But for now it would be a lot cleaner to read, and it seems the majority of PHP-FPM users are nginx users already anyway :)

Categories: PHP, PHP-FPM