Multi-Domain Nginx tip?

Hi Guys,

So I did some traffic storm test on our site and apache died around 85 users with CPU at 95%..

So I am trying to get nginx installed to hopefully improve performance.

Can someone volunteer their configuration file with multi-domain, Any tips..



  • Barry
    • DEV MAN’s Mascot

    I would say it’s fantastic – but I’ve been pulling my hair out over the past few days trying to fix nginx config file issues and still not got them tied down.

    The only word of advice I can give is, once you have it working – *don’t touch it again* :slight_smile:

  • jozwikjp
    • The Incredible Code Injector

    That sounds like a wise tip :slight_smile:

    So I have it installed now. Just waiting for my host to update our load balancers to point the traffic properly.

    But I have it configured and php is loading.. Won’t know if I got my url rewrites correct until they update the firewall. Finger are crossed.

    I found some good instructions for my centos 5.2 install.

    I had to update my kernel to


    with “yum update kernel kernel-devel”

  • jozwikjp
    • The Incredible Code Injector

    Sweet.. seems to be working aok with multi site and multi domain. :slight_smile:

    I tweaked my config a slightly from the ones I could find on the forums here and other places.

    My Setup

    loadbalancer port 80


    nginx server port:83

    On one server I upgraded php to 5.3.3 as I read that php-fpm setup is faster. That hasn’t gone well so far and we used some compiled scripts using nucoder and ioncube. And I can’t those to work yet..

    But my working setup..

    php 5.1.6 spawn-fcgi

    nginx 0.8 something..

    here is my nginx configuration file.

    server {

    listen 83;

    # listen 81;

    # listen somename:8080;

    server_name _;

    server_name_in_redirect off;

    port_in_redirect off;

    location ~ .php$ {

    # root html;

    fastcgi_pass localhost:9000;

    fastcgi_index index.php;

    fastcgi_param SCRIPT_FILENAME /home/msi/htdocs$fastcgi_script_name;

    include /etc/nginx/fastcgi_params;

    #include fastcgi_params;


    location / {

    root /home/msi/htdocs;

    index index.php;

    gzip on;

    gzip_http_version 1.0;

    gzip_vary on;

    gzip_comp_level 3;

    gzip_proxied any;

    gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;

    gzip_buffers 16 8k;

    if (!-e $request_filename) {

    rewrite ^.+/?(/ms-.*) $1 last;

    rewrite ^/files/(.+) /wp-includes/ms-files.php?file=$1 last;

    rewrite ^.+/?(/wp-.*) $1 last;

    rewrite ^.+/?(/.*.php)$ $1 last;

    rewrite ^(.+)$ /index.php?q=$1 last;

    expires 10d;




    location ~* ^.+.(html|jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js)$


    root /home/;

    rewrite ^/.*(/wp-.*/.*.(html|jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js))$ $1 last;

    rewrite ^.*/files/(.*(html|jpg|jpeg|gif|png|ico|css|zip|tgz|gz|rar|bz2|doc|xls|exe|pdf|ppt|txt|tar|mid|midi|wav|bmp|rtf|js))$

    /wp-includes/ms-files.php?file=$1 last;

    if (!-e $request_filename) {

    rewrite ^.+/?(/wp-.*) $1 last;

    rewrite ^.+/?(/.*.php)$ $1 last;

    rewrite ^(.+)$ /index.php?q=$1 last;


    expires 30d;




  • jozwikjp
    • The Incredible Code Injector

    So I have php 5.3.3, PHP-FPM, APC, and NGINX installed on one server.

    But when I go to check the apc.php info page. It says basically that 0 cache is being used.

    I realize that when I was running in cgi mode the APC cache was not sharing the memory with the other processes but I thought PHP-FPM was meant to address this type of issue.

    Any tips ? :slight_smile:

  • jozwikjp
    • The Incredible Code Injector

    So far

    Load storm test.

    php 5.1.6, apache, apc

    2 web servers died around 80 concurent users

    on this in setup

    php 5.3.3, nginx .8, php-fpm, apc

    1 webserver maxed out at 300 concurrent users and the site was still loading AOK.. Slow but aok.

    Great progess for today..

    Next on to caching on the load balancers

    But for for now time to deep fry a turkey.

  • jozwikjp
    • The Incredible Code Injector

    Installed varnish – pretty sweet.

    And this plugin..

    with nginx as the front..

    so varnish – nginx , php-fpm, apc

    varnish 2.1.3

    1 server

    500 concurrent users

    1 – 5 % cpu

    Very fast site..

    top – 18:44:52 up 2 days, 8:03, 2 users, load average: 0.33, 0.28, 0.27

    Tasks: 163 total, 1 running, 162 sleeping, 0 stopped, 0 zombie

    Cpu(s): 0.8%us,

    I am pretty surprised at how much of the site seems to work as if it wasn’t cached.

    Site preview on templates doesn’t work. But this is pretty remarkable.

    20 minute test


    (average) Errors Requests RPS

    (average) RPS

    (peak) Throughput

    (average) Throughput

    (peak) Total


    HTML 0.325 32,439 112,838 90 124 1,209 kB/s 1,658 kB/s 1 GB

    Other * 0.378 3 225,553 179 260 3,875 kB/s 5,582 kB/s 5 GB

    Total 0.360 32,442 338,391 269 380 5,084 kB/s 7,178 kB/s 6 GB

  • jozwikjp
    • The Incredible Code Injector

    My total setup is

    [Haproxy lb:80] – > [web1 varnish:83] ->[web1 nginx:82]

    Before I was using the HTTP_X_FORWARDED_FOR to set the IP for the clients..

    I don’t think that is going to work anymore. Or at least I haven’t been able to get it yet. The closest I get is the IP of [Haproxy lb].

    All the domains on my network seem to load very fast.

    -The buddy press admin bar doesn’t seem to work correctly. When you are logged in wp-admin the admin bar responds as you would expect. When you are just on a webpage it appears as though you are still logged out.

    -The affiliate plugin (setting the cookie) seems to work aok.

    -preview a post doesn’t seem to work.. throwing a “you don’t have permission error”.

    -posting and updating posts goes live right away.

    -preview themes just shows the current theme.

    -user registration works

    -IP crap makes anti-splog not work so good. So I had to disable that as most IP’s are

    -I would imagine ad sharing doesn’t work so good or at least the same.

    I am guessing the affiliate plugin will record the single hit per user just the first time they load.. That might be how it already works though.

    I feel significantly relieved though, as I was getting really concerned of the money we were going to have to dump on extra database servers and web servers to grow.

    turning on mysql db cache at the database level helps alot with wordpress. Most all the queries when the server was getting slammed were being delivered from cache.

    I think I am going to increase the query cache..

    query_cache_size = 64M

    query_cache_limit = 2M

  • Ulrich
    • The Crimson Coder

    Before I was using the HTTP_X_FORWARDED_FOR to set the IP for the clients..

    I don’t think that is going to work anymore. Or at least I haven’t been able to get it yet. The closest I get is the IP of [Haproxy lb].

    Did you set option forwardfor in Haproxy?

  • jozwikjp
    • The Incredible Code Injector

    Thought I would give a update on this.

    These are my updated rules for my varnish vcl file. This fixes the buddypress admin bar so you appear logged in besides just the admin bar.

    Also the site preview work now as well.

    Load storm test right now is at 250 users..

    CPU usage is at 1% so thats cool.

    backend default {

    .host = "";

    .port = "82";


    acl purge {



    sub vcl_recv {

    if (req.http.Accept-Encoding) {

    if (req.url ~ ".(jpg|png|gif|gz|tgz|bz2|tbz|mp3|ogg)$") {

    # No point in compressing these

    remove req.http.Accept-Encoding;

    } else if (req.http.Accept-Encoding ~ "gzip") {

    set req.http.Accept-Encoding = "gzip";

    } else if (req.http.Accept-Encoding ~ "deflate") {

    set req.http.Accept-Encoding = "deflate";

    } else {

    # unknown algorithm

    remove req.http.Accept-Encoding;


    if (req.request == "PURGE") {

    if(!client.ip ~ purge) {

    error 405 "Not allowed.";


    purge("req.url ~ ^" req.url "$ && == ";


    if (req.request != "GET" &&

    req.request != "HEAD" &&

    req.request != "PUT" &&

    req.request != "POST" &&

    req.request != "TRACE" &&

    req.request != "OPTIONS" &&

    req.request != "DELETE") {

    return (pipe);


    if (req.request != "GET" && req.request != "HEAD") {

    return (pass);


    if (req.url ~ "wp-(login|admin)") {

    return (pass);


    if( req.url ~ "^/wp-content/themes/blogs-mu/global-adminbar.php" || req.http.Cookie ~ "wordpress_logged_in_" ) {

    return (pass);


    if( req.url ~ "^/?preview" || req.http.Cookie ~ "wordpress_logged_in_" ) {

    return (pass);



    remove req.http.cookie;

    return (lookup);


    sub vcl_fetch {

    if (req.url ~ "wp-(login|admin)") {

    return (pass);


    set beresp.ttl = 24h;

    return (deliver);


  • jozwikjp
    • The Incredible Code Injector

    Just another an update :slight_smile:

    The fit hit the shan today ..

    php-fpm nginx pretty much just started randomly failing and ending with 502 bad gateway errors..

    Will be investigating tomorrow..

    But if anyone has had that issue I would really appreciate your input..

  • Aaron
    • CTO

    I ran into that symptom myself, though using spawn-fcgi. Finally tracked it down to APC. It seemed to be causing occassional segfaults in php-cgi. Also sometimes php would return only part of the php page, cutting off halfway.

    Never figured out why the segfaults, chocked it up to some incompatibility with php-cgi and APC. Same thing happened though with eAccellerator/xcache when I tried those instead. Eventually just disabled all accelerators, and putting up with the slowness.

    When I find time I’m going to recompile everything again with the latest versions, especially of PHP-CGI/APC. Perhaps my versions were somehow incompatible. The most annoying thing about nginx is lack of PHP error details in the log. I’m sure there must be some way to set it up, but still havn’t found it.

  • jozwikjp
    • The Incredible Code Injector

    Ok so I looked at my error log for php-fpm.. And it looks like where it dies is..

    WordPress database error Got a packet bigger than 'max_allowed_packet' bytes for query UPDATE wp_options SET option_value = ‘a:4663:{s:53:”b2c80bfa934

    But at that point this goes on forever.

    This update query just goes and goes..

    at the end of the query

    ' WHERE option_name = ‘cross_domain’ made by require_once, do_action, call_user_func_array, multi_domain->build_cookie, multi_domain->build_blog_cookie, update_blog_option, update_option, m_wpdb->query

    [06-Dec-2010 05:22:54] PHP Notice: Undefined index: debug in /home/ on line 103

    [06-Dec-2010 05:22:54] PHP Deprecated: Function set_magic_quotes_runtime() is deprecated in /home/ on line 27

    Any tips?

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.