M2P: Requirement for PHP Session support?

Recently, we got hit by a viral article which brought our servers to their knees.

In theory, this shouldn't happen because we have Varnish cache between the content servers and the public. We've had this happen in the past and Varnish has performed flawlessly. However, it now appears that every page request is being punted to the content backends.

I'm still digging into it, however one thing that stands out is the presence of a PHPSESSID cookie where there never used to be, and the presence of any cookies (by default) is a cache-killer for Varnish.

A test shows that a simple PHP script does not set this cookie (and is cached properly) and a bit of googling suggests that it is set by the PHP session_start() call. Grepping through all the PHP on our WPMS instance shows this call only in Membership 2 Pro, Appointments+ and Directory (all of which are WPMU plugins).

We can discard (for now) Appointments and Directory because they're only active on a few sites, but M2P is network activated in network protection mode (and only since around June), so it's a fair bet that this is where this cookie is coming from.

How important are PHP sessions to M2P? We really only care about M2P for logged in users, so one possibility is to look for the wordpress_logged_in cookie and discard PHPSESSID only if it is not present.

Can M2P devs advise on the significance of/need for PHP sessions in M2P, and also please note that the way M2P is structured clobbers many Varnish installs (and will produce unpredictable results on others, depending on how Varnish is configured to treat WP cookies)?

  • Adam Czajczyk

    Hello David,

    I hope you're well today and thank you for your question!

    I think this is great and very detailed investigation that you've performed, so thank you for this. However, this is a bit beyond my knowledge of how our Membership 2 Pro plugin works and how it interacts with server/user sessions. That said, let me please include @Philipp Stracker into our conversation and hopefully he, as a plugin developer, will be able to lend as a helping hand with this.

    Please note though, that his response time may be a bit longer than ours here on forum as he's dealing with a lot of complex questions on daily basis. That said, please keep an eye on this thread for further information and thank you for your patience!

    Best regards,
    Adam

  • David King

    Acknowledged.

    I will continue to experiment with Varnish VCL until I find code that filters out PHPSESSID IFF wordpress_is_logged_in* is not set, and also to filter out the cache control headers that result in that instance.

    I'll post that code here in case any other Varnish/M2P users find it useful.

    When @Philipp Stracker gets a chance, I'm sure he'll be able to indicate what impact this will have. I don't know if it necessarily will require any changes to M2P (although possibly Philipp might conclude that there's a better way of doing the same thing without special Varnish configuration, which'd be great!), but it would be good to have it confirmed that my proposed solution is sane.

    Thanks,

  • Adam Czajczyk

    Hello David!

    Thanks for your response.

    M2P, as currently written, appears to make caching effectively impossible. This is not good news, and is probably a flaw that should be fixed at some point.

    I'm pretty sure you're also referring to Varnish this time, so definitely Philipp would be a right person to be addressed here :slight_smile: As I said previously, I've already notified him of this thread and asked to join the conversation, however it seems that he's offline at the moment. I'm sure though he'll be able to shed some light on this.

    Have a nice day!
    Adam

  • David King

    Well, certainly Varnish is affected but, if I interpret the headers correctly, M2P causes WordPress to return an Expires: header in the past (1981) (and an associated Cache-Control: header) that effectively prevents the client (and any other cache) from caching the page — unless any intermediate cache or proxy is specifically configured to override that.

    So it is probable that M2P effectively disables caching on all clients and platforms.

    I'm still trying to work out how to hack around this in Varnish. I'll post details here when I do.

  • Adam Czajczyk

    Hello David,

    Thanks for sharing your findings with us. However, I'd like to ask you for a bit more of patience if it comes to our replay as it seems that Philipp may be away for weekend and I'm not sure if he's going to get back online before Monday. Of course, as I mentioned before, I'm sure he'll give us some advice here so either he will respond or I'll get back to you with a word from him :slight_smile:

    Best regards,
    Adam

  • David King

    Okay, here we go: work-around for Varnish in connection with this problem. It is not difficult to understand in principle, even if the implementation of it is not so straight forward.

    To reiterate: the problem is that M2P (and a few other WPMUdev plugins) make use of PHP's session manager which sets the cookie PHPSESSID (default name, though this can be changed, as I have done). It is not enough to drop this cookie on inbound requests, instead, you must prevent the setting of this cookie, or Varnish will mark the page as HIT-FOR-PASS.

    The solution, therefore, is to configure Varnish to drop the Set-Cookie header for PHPSESSID if and only if the user is not logged in (and is not in the process of logging in).

    In order to implement this solution, you will need:
    • A copy of the source for the version of Varnish you're running (no need to build it, you just need the source unpacked)
    • A copy of vmod_header
    • The snippet of VCL shown below.

    1) Unpack the Varnish source
    2) Get a copy of vmod_header via git
    3) Check out the correct branch for your version of Varnish if you are running anything other than the default of Varnish v3
    4) Follow the build instructions for vmod_header (building from source is outside the scope of this comment) and install it
    5) Add to the top of your VCL file:

    import header;

    6, optional) Change the name of the PHP session cookie for just PHP sessions that run your WordPress instance. I'm using PHP-FPM, so for me this was as easy as editing the pool definition in which WordPress executes, but PHP-FPM is not so common in most commercial hosting arrangements so you will need to examine your situation and figure out how to do this, if you want to.

    If you run only WordPress, you don't need to do this but, in our case, we run at least two other PHP-based applications (vBulletin and MediaWiki) and the following logic will not work for those and may interfere with their operation.

    If you omit this step, you need to edit the code snippet and replace WPMS_PHP_SID with the default name PHPSESSID.

    7) Edit (or add, if doesn't already exist) the following code to vcl_backend_response() (which was called vcl_fetch() in Varnish 3)

    if( bereq.http.Cookie                                           !~ "wordpress_logged_in" &&
        header.get( beresp.http.Set-Cookie, "wordpress_logged_in" ) !~ "wordpress_logged_in" &&
        header.get( beresp.http.Set-Cookie, "WPMS_PHP_SID" )         ~ "WPMS_PHP_SID" ) {
            unset beresp.http.Set-Cookie;
            unset beresp.http.Expires;
            unset beresp.http.Pragma;
            unset beresp.http.Cache-Control;
    }

    Until @Philipp Stracker comments, I have no idea what effect doing this will have on M2P (or the other WPMUdev plugins that make use of PHP sessions, including Appointments+ and Directory). In our use-case, M2P doesn't matter for users who are not logged in so it probably is okay for us — but note that I've disabled M2P's registration functionality (because I prefer to have WordPress verify email addresses and I want our PlayThru/CAPTCHA to work).

    Again, it will require input from Philipp before the implications of this step are known. Until then, YMMV :slight_smile: Good luck.

  • Adam Czajczyk

    Hello David!

    I just got a word from Philipp, so here's what he said about importance of PHP sessions for Membership 2 plugin:

    "The session is only used for logged in users, though it is created for all users."

    He also said, that judging upon your posts it seems that you've managed to solve the issue already. This leads me to conclusion, that you should be all good to go with your solution. I think that it would be great to test it first on some staging site before going live but other than that, all seems legit. In case you experienced any further issues or had further questions, let me know please.

    Thanks for sharing your findings with us and have a nice day!

    Adam

  • David King

    Hi Adam,

    Thanks for getting back to me.

    If M2P were the only plugin to use sessions, then my solution (more of a dirty hack) would probably do, but I know of at least two other WPMUdev plugins (Appointments+ and Directory) that also use PHP sessions.

    And according to @Hoang Ngo, indiscriminately dropping PHP session cookies will interfere with Appointments+ at the very least. Later, we intend to deploy Pro Sites, and it's possible that could be affected, too. There may be many other plugins besides.

    So, really, M2P needs to be fixed so that it only emits session cookies when needed.

    Given that @Philipp Stracker said that sessions are only used if the user is logged in, could you please ask him to modify M2P so that it only emits session cookies when the user is logged in?

    Then I'll be able to remove the Varnish hack and avoid problems with Appointments+ and other plugins down the line.

    I suspect that this issue will affect more than just Varnish cached sites; W3TC's page cache might be okay, but I doubt services like CloudFlare will be able to determine when it's safe to cache in the presence of session cookies and when not, and I assume that CloudFlare doesn't offer the flexibility we have with our own, self-maintained Varnish instance.

    • David King

      Hi Adam,

      If you could forward this also to @Philipp Stracker:

      The session_start() call appears exactly in one place: membership/lib/wpmu-lib/inc/class-thelib.php:136 (in method TheLib::_sess_init() beginning line 107). How would it be if the following code were inserted after line 110?

      if( !is_user_logged_in() ) {
          return;
      }

      Would that break M2P?

      I can modify my local copy, of course, but I'd rather not until I hear from Philipp whether that would cause issues and besides, it's better if he were to do this so that the change is not overwritten on next update.

  • Adam Czajczyk

    Hello David!

    I've got another information from Philipp :slight_smile: Seems that the session is generated by some internal library that he wrote that's being used also in some other plugins. Therefore removing session would mean a lot of changes in Membership 2 plugin code and a lot of additional work in future.

    However, most likely Philipp should be able to add a filter or setting that would allow you to control session creation. He will look into it and should come up with a code that I'll be able to share with you here and later on that will be included in one of the future releases of the plugin. Please note though that this may take some time as it's an additional feature that wasn't planned (though Philipp agrees that this will be useful) so we'd appreciate a bit of patience on this very much :slight_smile:

    Keep an eye on this thread please and I'll keep you updated.

    Best regards,
    Adam

  • David King

    Right, fair enough. It didn't occur to me that this library might be shared with other plugins, but it figures. Of course, that implies that those other plugins may suffer from the same problem.

    Some sort of call-back would do (heck, even a define would probably be sufficient, though a callback would be better). There is another possibility:

    I note that class TheLib is an abstract class, which is extended in membership/lib/wpmu-lib/inc/class-thelib-session.php. So, an alternative is to augment that derivative class to override the constructor and call the parent class's constructor only when the user is logged in (and modify the other methods to do something sensible when the session has not been initialised).

    That would avoid affecting anything other than M2P, and (if Philipp is right about the session not being used unless the user is logged in) I would have thought should avoid a great deal of refactoring elsewhere in M2P.

    Of course, only Philipp can know whether this is a feasible idea or not.

    Remember: the problem is not the use of session cookies ever, it's the emission of session cookies when the session is not, in fact, being used for anything needful (in this case, visitors not logged in). Requests in which the user is logged in are very much the minority, so the lack of caching in those situations isn't such a big deal.

    I'll wait until Philipp has some time to consider this.

      • David King

        Tell them that their cache performance and therefore backend CPU load will suck if they aren't prepared to mitigate with such an arrangement.

        If the issue is actually causing a problem for you, time to switch providers. There are loads of providers who want your business and who will doubtless serve you better.

        You can run your own Varnish instance (preferably on a separate machine, but it's not compulsory). In fact, if you're only running WordPress, there are probably easier caching solutions like W3TC which don't care about cookies.

        • John Callery

          Hi David King,

          Thank you for taking the time to respond. This thread, along with several others, does probably indicate the need to switch. But the switch would be to a new software provider, not a new hosting company. The degradation of performance resulting from the initialization of a PHP session on every single page, regardless of whether the user is authenticated or not, is the result of sloppy coding and the absence of best coding practices - not an inflexible hosting company. We use both Pantheon and Pagely, and are extremely satisfied with all aspects of their platforms.

          This thread, and others, however, do call into question the integrity and business model of WPMUDEV. We are starting to believe that the company doesn't have a sustainable business model, is stretched too thin by the number of plugins and themes that it has amassed, and hasn't invested the attention and critical thought necessary to make the products useful to anyone more than a hobbyist. For example, this same product is capable of processing credit card transactions, but doesn't even support the most basic form of AVS. Documentation is all but nonexistent.

          WPMUDEV provides very fast responses, but the responses are hollow and unuseful. This very thread promised a response more than 7 months ago, but no follow-up was provided. When we approached WPMUDEV about the credit card transactions, they told us that if we were serious about security, we would have to hire a developer to implement AVS. This, coupled with the functionality, falls short of any reasonable expectation I would have of a serious software vendor.

          So, as you can see, I would rather switch software vendors than hosting companies. However, that's not so easy when our client has fully adopted the product and integrated it into their business process.

          The software doesn't work on Patheon, Pagely, or even GoDaddy Managed WordPress. My ideal solution would be to fix the product, not hack the hosting.

          Best,
          John

          • David King

            Hi John,

            Disclaimer: I don't work for WPMUdev or otherwise have any connection with the company except as a customer, so I can say what I like — a luxury that WPMUdev employees don't enjoy.

            If I'm honest, I don't think you're being entirely reasonable about this.

            The degradation of performance resulting from the initialization of a PHP session on every single page, regardless of whether the user is authenticated or not, is the result of sloppy coding and the absence of best coding practices - not an inflexible hosting company.

            I only half agree.

            There is no performance penalty to use of PHP sessions when, by default, every page requests results in PHP execution (and that's a WordPress thing, nothing to do with WPMUdev) except in conjunction with very specific (and back-end agnostic) caching solutions that, by default, refuse to cache any request that contains or results in a response that sets a cookie. WPMUdev cannot take responsibility for all possible combinations of software and so I would suggest that it's not WPMUdev's fault that PHP session cookies presented you and I with a problem.

            I'd go further and say that, if you are going to use third party solutions like Varnish (rather than WordPress-specific caching solutions such as W3TC), the onus is on you to take such measures as are required to ensure your choice of technologies interoperate satisfactorily (or, in the case of managed hosting, that your hosting company suits your needs). Since there is a perfectly satisfactory work-around for Varnish, again, it's not really WPMUdev's fault.

            That all said, I do agree that indiscriminate use of PHP sessions is suboptimal, but I also recognise that such inefficiencies are a natural consequence of code re-use (in this case, the base library that actually calls session_start()) and that there are use cases in which you need a PHP session cookie even when the requestor isn't logged in (certain deployments of Appointments+ would be one example).

            This thread, and others, however, do call into question the integrity and business model of WPMUDEV. We are starting to believe that the company doesn't have a sustainable business model

            Frankly, this is where you overplay your hand.

            Being cross at a plugin not doing precisely what you want is one thing, but it's quite another to impugn the integrity of its supplier. As to the sustainability of their business model that, bluntly, is nobody's business but that of WPMUdev's CEO (or equivalent).

            If you don't like WPMUdev's products, vote with your wallet and go elsewhere or contribute a bit of elbow-grease, fix the problem and contribute back the solution. WPMUdev's community environment actively promotes such coöperation.

            is stretched too thin by the number of plugins and themes that it has amassed

            Sounds to me as though you're not a developer or, at the least, show no sign of understanding what's involved in producing code that is useful in such a vastly diverse set of environments that are totally outside of the developers' control. You cannot please everybody all of the time, and COTS solutions inevitably involve a degree of compromise.

            For example, this same product is capable of processing credit card transactions, but doesn't even support the most basic form of AVS.

            Perhaps that's because, typcially, they don't need to. You (the WPMUdev end user) would be daft to process credit card transactions or store any kind of payment information directly on your own infrastructure with all the other security headaches that go with PCI compliance etc, so you offload AVS to another company (eg PayPal) who does that for you.

            Furthermore, I would guess that AVS would attract a charge from the merchant services company and that it's not economic for most people to employ it. The extent to which WPMUdev plugins do support payment processing options without AVS is likely a reflection of demand regardless of wisdom. I can't fault WPMUdev for providing what customers tell them they want, even when it may not be in the customers' best interests.

            Documentation is all but nonexistent.

            I agree, their documentation frequently leaves a lot to be desired.

            This very thread promised a response more than 7 months ago, but no follow-up was provided.

            Again, I agree, but I also make allowances for the fact that every company (and its developers) must triage and prioritise problems. It does not surprise me that there's been no movement on an issue that affects few and for a workaround has been provided.

            When we approached WPMUDEV about the credit card transactions, they told us that if we were serious about security, we would have to hire a developer to implement AVS. This, coupled with the functionality, falls short of any reasonable expectation I would have of a serious software vendor.

            So WPMUdev cannot be all things to all people. Evidently their products don't meet your needs, so you should go find a product that does.

            But please, be professional about it.

            However, that's not so easy when our client has fully adopted the product and integrated it into their business process.

            If there are no other products that provide the complete range of features required by your client, then you have to choose the least-worst option and make up the difference yourself. That's your job. Don't bitch out WPMUdev just because a COTS product doesn't meet your precise requirements, because COTS products never do.

            The fact that the cost to you of changing is greater than the cost of remaining suggests that WPMUdev's product gets more right than it gets wrong, which is an indicator that you've made the right decision and you just have to cope with the inevitable shortfalls inherent in any COTS solution as best you can.

            The software doesn't work on Patheon, Pagely, or even GoDaddy Managed WordPress. My ideal solution would be to fix the product, not hack the hosting.

            Never dealt with any of those companies myself but then, I prefer to be in full control of my own server structure and, fortunately, I have the Linux expertise to provide both my own platform and technical support. I don't even like, want or use cPanel et al.

            It would be interesting to know what the problem with M2P is on those platforms. Surely not PHP sessions? Perhaps there are other issues? If so, then I presume they have not been identified, in which case, there is no reason to expect a fix to be forthcoming.

            The cost of opting for a managed solution is that you have to do things other people's way, not yours. Sometimes that means having to make compromises you don't want to make. You always have the option of a self-managed solution if that is what it takes. Only you can decide whether that option is worth the cost. In any event, it's unreasonable to be miserable to WPMUdev or make your problems theirs.

          • Adam Czajczyk

            Hello John!

            I can see that David already responded to your post and this part of the thread is a discussion between you and David rather (I already replied to main issue here: https://premium.wpmudev.org/forums/topic/m2p-requirement-for-php-session-support#post-1096671).

            I just wanted to let you know that you can easily recognize WPMU DEV staff members from community members just to avoid further confusion :slight_smile: When we, as a staff members, replay to you you can clearly see the blue "STAFF" badges next to our names :slight_smile:

            Have a great day!
            Adam

  • Adam Czajczyk

    Hello!

    An upcoming Membership 2 Pro release carries a fix that will let you disable session. An additional code will be required that will have to be added to "wp-config.php" file or via a MU plugin.

    For "wp-config.php" the code is:

    // Disable session via const:
    define( 'WPDEV_USE_SESSION', false );

    For MU plugin the code is:

    add_filter( 'wdev_lib-use_session', '__return_false' );

    If the constant/filter equals "true" then the session is created the same way it is currently. The "is_user_logged_in()" function is not available at the point where the session is created. Disabling the session completely however shouldn't have an impact on M2 except some of the "your changes were saved" messages (just messages, not saving data itself).

    The constant/filter described above will be available with 1.0.30 release of Membership 2 Pro which is undergoing final tests. I"m not able to give you an exact ETA on the release but it should be published soon.

    Kind regards,
    Adam

  • Milan

    Hello John Callery

    Hope you are well today and thanks for asking us. :slight_smile:

    There have been lot of bug fixes and improvements added in latest release, I am not sure about this one. But I've pinged our developer to get some information on this one. If these fixes have been included in latest release, I'll update you here with positive answer. :slight_smile:

    Please give us slightly more time to communicate with our devs. :slight_smile:
    Best Regards,
    Milan

  • Milan

    Hello John Callery ,

    Hope you are doing well. :slight_smile:

    I think support for session has been added as I noticed

    $use_session = apply_filters( 'wdev_lib-use_session', $use_session );

    in \membership\lib\wpmu-lib\inc\class-thelib.php file.

    So you should be able to disable session, with this code as mu-plugin

    add_filter( 'wdev_lib-use_session', '__return_false' );

    Let me know how it goes. :slight_smile:
    Kind Regards,
    Milan

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.