All Pro Sites settings disappeared

Hi,

I've just seen that all my Pro-Sites settings have disappeared!
(just Pro Sites level are here)
I've never had such a problem with Pro-Site and It was ok at the end of the week but today I discovered the problem.

It's a major issue of course.

I've just seen there are several threads about the same problem. I'm not using MultiDB.

Why did it happen? How can I prevent this from happening again?
Is it possible to get all settings back?
Could you help me please?

I granted support access.

Thanks!

Best Regards

  • Michelle Shull

    Hey there, Arom!

    Did this happen after you updated ProSites? If not, were there any other events that happened at or around the same time your Pro Sites settings disappeared?

    The dev provided a failsafe, try adding this to your wp-config.php file before the line that tells you to stop editing:
    define('PSTS_DISABLE_UPGRADE', true);

    Do you happen to have a site backup from before when this issue occurred that you can restore?

    Let's see where this gets us, and we can call in for more help if we need it.

    Thanks, Arom!

  • Arom77

    Hello @Michelle Shull

    Did this happen after you updated ProSites?

    No.

    If not, were there any other events that happened at or around the same time your Pro Sites settings disappeared?

    No, I don't think so. It was working OK at the end of the week and I haven't done anything since then (just publishing a post and some messages on the forum, that's all).

    The dev provided a failsafe.

    OK thanks I'll do this. (question: why is it not "included" for all Pro Sites users?)

    Do you happen to have a site backup from before when this issue occurred that you can restore?

    I have a backup of the entire multisite network (on the server side, using a VPS) but if I restore this it'll restore the entire network, including my clients sites, so I can't do that.
    I guess I can't restore only this, right?
    Could you help me restoring what needs to be restored?

    I'm really worried because this is a "core" feature of my multisite network and I saw there are several other threads speaking of a similar problem. Just want to 1/ fix it 2/ know how to prevent this from happening agan 3/ make sure i can handle it easily if it was to happen again.

    Thanks!

  • Jude

    Omg @Arom77

    So sorry this happened to your thread. Here is a small tip about our system, the queue is ordered by the last response date. So if there is no response for a long time then it is on high priority but if you keep posting on it then it goes back to the end .. Anyway as I was scanning through our feed I saw your name and jumped in here. Appreciate your patience, you may already but you're one of our valued members.

    Anyway here goes:

    1) There are 3 tables that you should backup

    wp_pro_sites
    wp_pro_sites_signup_stats
    wp_pro_sites_daily_stats

    2) Pro Sites Options -> These are serialized and stored as

    psts_settings
    publishing_level

    Just copy this out of the DB and keep it elsewhere as a backup. Here is what is saved here ( Not the whole thing .. but will give you an idea)

    'base_country'             => 'US',
    			'currency'                 => 'USD',
    			'curr_symbol_position'     => 1,
    			'curr_decimal'             => 1,
    			'rebrand'                  => __( 'Pro Site', 'psts' ),
    			'lbl_signup'               => __( 'Pro Upgrade', 'psts' ),
    			'lbl_curr'                 => __( 'Your Account', 'psts' ),
    			'gateways_enabled'         => array( 'ProSites_Gateway_PayPalExpressPro' ),
    			'modules_enabled'          => array(),
    			'enabled_periods'          => array( 1, 3, 12 ),
    			'hide_adminmenu'           => 0,
    			'hide_adminbar'            => 0,
    			'hide_adminbar_super'      => 0,
    			'free_name'                => __( 'Free', 'psts' ),
    			'free_msg'                 => __( 'No thank you, I will continue with a basic site for now', 'psts' ),
    			'trial_level'              => 1,
    			'trial_days'               => get_site_option( "supporter_free_days" ),
    			'trial_message'            => __( 'You have DAYS days left in your LEVEL free trial. Checkout now to prevent losing LEVEL features »', 'psts' ),
    			'ga_ecommerce'             => 'none',
    			'signup_message'           => __( 'Would you like to upgrade this site to Pro?', 'psts' ),
    			'feature_message'          => __( 'Upgrade to LEVEL to access this feature »', 'psts' ),
    			'active_message'           => __( 'Your Pro Site privileges will expire on: DATE<br />Unless you have canceled your subscription or your site was upgraded via the Bulk Upgrades tool, your Pro Site privileges will automatically be renewed.', 'psts' ),
    			'success_subject'          => __( 'Thank you for becoming a Pro Site member!', 'psts' ),
    			'success_msg'              => __( "Thank you for becoming a Pro Site member!
             'pypl_site'                => 'US',
    			'pypl_currency'            => 'USD',
    			'pypl_status'              => 'test',
    			'pypl_enable_pro'          => 0,
    			'stripe_ssl'               => 0,
    			'mp_name'                  => __( 'Manual Payment', 'psts' ),
    			'mp_show_form'             => 0,
    			'mp_email'                 => get_site_option( "admin_email" ),
    			'pt_name'                  => __( 'Premium Themes', 'psts' ),
    			'pt_text'                  => __( 'Upgrade to LEVEL to activate this premium theme &raquo;', 'psts' ),
    			'ps_level'                 => 1,
    			'ps_email'                 => get_site_option( "admin_email" ),
    			'ps_name'                  => __( 'Premium Support', 'psts' ),
    			'ps_message'               => __( 'You can send us a priority direct email support request here if you need help with your site.', 'psts' ),
    			'ps_notice'                => __( 'To enable premium support, please upgrade to LEVEL &raquo;', 'psts' ),
    			'publishing_level'         => 1,
    			'publishing_message_posts' => __( 'To enable publishing posts, please upgrade to LEVEL &raquo;', 'psts' ),
    			'publishing_message_pages' => __( 'To enable publishing pages, please upgrade to LEVEL &raquo;', 'psts' ),
    			'quota_message'            => __( 'For SPACE of upload space, upgrade to LEVEL!', 'psts' ),
    			'quota_out_message'        => __( 'You are out of upload space! Please upgrade to LEVEL to enable SPACE of storage space.', 'psts' ),
    			'xmlrpc_level'             => 1,
    			'xmlrpc_message'           => __( 'To enable XML-RPC remote publishing please upgrade to LEVEL &raquo;', 'psts' ),
    			'bp_notice'                => __( 'Upgrade to LEVEL to access this feature &raquo;', 'psts' ),
    			'pp_name'                  => __( 'Premium Plugins', 'psts' ),
    			'ads_name'                 => __( 'Disable Ads', 'psts' ),
    			'ads_level'                => 1,
    			'ads_enable_blogs'         => 0,
    			'ads_count'                => 3,
    			'ads_before_page'          => 0,
    			'ads_after_page'           => 0,
    			'ads_before_post'          => 0,
    			'ads_after_post'           => 0,
    			'ads_themes'               => 0,
    			'bu_email'                 => get_site_option( "supporter_paypal_email" ),
    			'bu_status'                => 'test',
    			'bu_payment_type'          => 'recurring',
    			'bu_level'                 => 1,
    			'bu_credits_1'             => 10,
    			'bu_option_msg'            => __( 'Upgrade CREDITS sites to LEVEL for one year for only PRICE:', 'psts' ),
    			'bu_checkout_msg'          => __( 'You can upgrade multiple sites at a lower cost by purchasing Pro Site credits below. After purchasing your credits just come back to this page, search for your sites via the tool at the bottom of the page, and upgrade them to Pro Site status. Each site is upgraded for one year.', 'psts' ),
    			'bu_payment_msg'           => __( 'Depending on your payment method it may take just a few minutes (Credit Card or PayPal funds) or it may take several days (eCheck) for your Pro Site credits to become available.', 'psts' ),
    			'bu_name'                  => __( 'Bulk Upgrades', 'psts' ),
    			'bu_link_msg'              => __( 'Purchase credits to upgrade multiple sites for one discounted price!', 'psts' ),
    			'ptb_front_disable'        => 1,
    			'ptb_front_msg'            => __( 'This site is temporarily disabled until payment is received. Please check back later.', 'psts' ),
    			'ptb_checkout_msg'         => __( 'You must pay to enable your site.', 'psts' ),
    			'pq_level'                 => 1,
    			'pq_quotas'                => array(
    				'post' => array( 'quota' => 'unlimited' ),
    				'page' => array( 'quota' => 'unlimited' )
    			),
    			'uh_level'                 => 1,
    			'uh_message'               => __( 'To enable the embedding html, please upgrade to LEVEL &raquo;', 'psts' )
    		);

    Finally the prosites.php file has a couple of functions to completely restore the plugin

    restore_current_blog();
            init_vars() ;
           install() ;

    Use the above if you really somehow mess up big time . Please backup and keep them handy.

    Cheers
    Jude

    To other readers : This is somewhat advanced and if you are not sure of what to do then please don't try this yourself

  • Arom77

    Hi @Jude

    It happened again. All Pro Sites settings lost

    Last time I have an idea about why this could have happened, but this time there is seriously nothing i could think about.
    It seems to be a problem for a lot of people and for years, is this impossible for you to fix that?
    That's really a major issue because obvisoulsy Pro Sites settings can be lost anytime, and you don't even know when it does!

    Best Regards

  • Michelle Shull

    Hey there, Arom!

    I'm going to flag our second level support team here, since this has been an off and on problem for users for a while now, to see if we can find a definitive reason AND a definitive fix for why this keeps happening.

    Let's be proactive and get the second level team some log in details for your account so they can hop right in and look around.

    https://premium.wpmudev.org/contact/

    Subject: "Attn: Michelle Shull"
    -WordPress admin username
    -WordPress admin password
    -login url
    -cPanel/PHPMyAdmin credentials (username/password/cpanel login link)
    -FTP credentials (host/username/password)
    -link back to this thread for reference
    -any other relevant urls

    Select "I have a different question" for your topic - this and the subject line ensure that it gets assigned to me.

    Thanks, Arom!

  • Hoang Ngo

    Hi @Arom77,

    Ugg, this is really a pain. For now, do you have any backup for Pro-sites or your current WordPress database, as we can extract the Pro-sites data and recover it on your site.

    For the reason why it can happen, this is various, it can be WordPress issue or non-WordPress issue. For now, Please add this code to your site using mu-plugins

    add_action('update_site_option_psts_settings', 'log_psts_settings_changed', 10, 3);
    function log_psts_settings_changed($option, $value, $old_value)
    {
        $log_file = __DIR__ . DIRECTORY_SEPARATOR . 'psts.log';
        if (!file_exists($log_file)) {
            $handle = fopen($log_file, 'w');
        } else {
            $handle = fopen($log_file, 'a');
        }
        $message = array(
            'date' => date('Y-m-d H:i:s'),
            'data' => $value,
            'before' => $old_value,
            'track' => debug_backtrace()
        );
        if ($handle) {
            fwrite($handle, json_encode($message));
            fclose($handle);
        }
    }

    As you said, the settings are gone, so this code will tracking, if any change affect to the ProSites settings, we will log it all, if the error happen again, we have clues to work on

    Best regards,
    Hoang

  • Arom77

    Hi @Hoang Ngo

    All settings lost again during this week-end (((

    I've just installed the mu-plugin you asked for. As I'm not a developper, I really need your assistance on all this stuff. Could you please check if the plugin is correctly installed and so if it's going to track data when the problem occurs again?

    do you have any backup for Pro-sites or your current WordPress database, as we can extract the Pro-sites data and recover it on your site.

    As I said I'm not a developper so I'm not very easy on doing such things. As a result I put the settings back using back-office. If you could assist me also on this so I can back-up and restore what I need when the problem happens again... Thanks a lot.

    The major point would be that I know that the problem has occured as soon as it's occured. Because for now I don't!

    Best Regards

  • Hoang Ngo

    Hi @Arom77,

    I hope you are well today.

    All settings lost again during this week-end (((

    Obs!!, I'm sorry about that.

    I've just installed the mu-plugin you asked for. As I'm not a developper, I really need your assistance on all this stuff. Could you please check if the plugin is correctly installed and so if it's going to track data when the problem occurs again?

    Your plugin setup is correct.

    As I said I'm not a developper so I'm not very easy on doing such things. As a result I put the settings back using back-office. If you could assist me also on this so I can back-up and restore what I need when the problem happens again... Thanks a lot.

    I saw you already installed Snapshot, you can create a snapshot backup with this database config
    1. Include selected database tables in this Snapshot, please check the cssn_pro_sites, cssn_pro_sites_daily_stats,cssn_pro_sites_signup_stats, cssn_sitemeta
    So later, when the settings gone, you can easily recover it

    Best regards,
    Hoang

  • Hoang Ngo

    Hi @Arom77,

    I hope you are well today and I'm sorry for the delay.

    This is really weird, the logs don't show much information, everything seem fine. I checked apache logs, but I don't see any issues related to ProSites too.

    Just for clarification, you're losing the settings in WP-Admin->Pro Sites->Settings? or the actual blogs which are pro-sites(has paid)?

    But from now, I think, there are 2 reasons.
    1. The XML-RPC (as I saw it in the apache log, but it is less likely)
    2. Database crash, data corruption (more likely)

    I can't find the MySQL logs on the webmin, is there anyway you can send me that, usually it located in /var/logs/mysql

    Best regards,
    Hoang

  • Arom77

    Hi @Hoang Ngo

    Actually there are not MySQL logs on my server because they don't exist by default. And if you activate it kills server performances.

    Anyway, I think I found the cause. I think it's due to virtual memory running out. I lowered the memory Wordfence request when scanning and hope it solves the problem. But if it's really the problem, how can we prevent Pro Sites to loose its settings when virtual memory runs out, or at least make it alert me that settings've just been lost?

    Best regards

  • Hoang Ngo

    @Arom77,

    This seems a server issue, at this level, ProSites or even WordPress, can't do much. I think you will need to track what's the main reason causing the memory issues, and then we can fix it.

    Usually, when memory run out, some services will be terminated, example mysql server, or the mail server. For the log, you don't have to make it log every things, just need to log the errors.

    Best regards,
    Hoang

  • Michelle Shull

    Hi, Arom!

    It sounds like you need a better server. I've got ProSites running on a shared server with pretty limited resources and I'm not seeing anything like what you experience. It may be time to find a new host that has the hardware to support the site you want to create. There's little we can do in the code of a plugin to compensate for a server that's so unstable. From the sounds of things, you'd be having issues at some point down the line with this host no matter what kind of site you were trying to run.

    Thanks for your question, please consider upgrading your host. (And I believe it will resolve your other issue here, as well.)

  • Arom77

    Hi @Michelle Shull,

    No, the problem is not the server. 99% of the time I'm using less than 30% of my total memory.
    The problem is that from time to time "something" happens that causes a memory peak. I sure should find what causes that and I'm trying to find it but for the moment I haven't succeeded. This is not the server which is unstable, this is something (probably a WP plugin) that from time to time causes that.

    Anyway, when this happens, my site is just unavalaible during just a few seconds. The only real problem is that Pro Sites looses all its settings when it happens. That's the only plugin which looses its setting when this happens!

    How could you prevent Pro Sites to loose its settings each time a server has a minor issue (there are some other topics about this point).

    Thanks!

  • Michelle Shull

    Hi, Arom!

    As Hoang explained, there's very little, if anything, we can do because the issue is happening on your server. Let me ask the developer if there is any more insight we can provide here, but until you find out why your memory is spiking, our hands are tied on what we can do. There's nothing to "fix" in ProSites, it's working like it should.

    Is there anything at all you can think of that could be a factor here? When you look at your plugins list, which ones stick out to you as being the most robust? (As in they write a lot of data to the database, or have the most features, or change the functionality of WordPress in a major way.)

    Thanks, Arom!

  • Arom77

    Hi @Michelle Shull

    No, the issue is not happening on my server. The issue is that Pro Sites looses all its settings each time a server (not only mine, any server!) has a problem, even a very little one.

    I don't know about any other WP plugin that looses its settings each time a server has a problem!

    As I said, there are several other topics from other people here about the same problem.
    Everyone sees the same thing: each time a server has a problem :
    1/ Pro Sites looses all its settings,
    2/ It's the only plugin that looses its settings.
    3/ We have to set back all settings as soon as possible, because Pro Sites is a very important plugin for all of us.

    There has to be something that you can do to avoid this behaviour that no other plugin has. Pro Sites is crucial for all of us. Again, we are a lot of people here experiencing that and it didn't start yesterday.

    There's nothing to "fix" in ProSites, it's working like it should.

    Yes, it's working like it should... EXCEPT FOR THAT POINT.

    Thanks!

  • Michelle Shull

    Hi, Arom!

    I've reported the issue to the developer and asked for his feedback with a flag and a note in our internal communication. I know Hoang has spent hours looking into this, and I trust what he says he's seeing, hopefully the dev will have some time to offer some feedback here.

    I'm also pinging @Jack Kitterhing, our Project Manager. He knows the code for every one of our hundreds of products just about as well as the devs do, perhaps he'll have some insight here as well.

    I understand that you're frustrated, we're trying to figure out what's going wrong here as fast as we can. Until I (or we) hear back from @Hoang Ngo, Jack, or the dev, what I can do is very limited. It's the weekend, so I'm not sure on the availability of any of the three at the moment.

    Thanks for your patience. I'm sorry if I've misunderstood anything here. We're trying to help as best we can.

  • Arom77

    Hi @Michelle Shull

    I know, didn't mean to be rude, really! I know you try to help - and you do it well.
    And I know that's the week-end too. Sorry if I was rude ("pardon my french" ).

    I don't want it to be fixed "right now". If you say you have a solution and that it'll be fixed let's say in 2 or 3 months I'll be perfectly happy with it!

    Just don't want to stay forever in this situation where I can't trust Pro Sites on the long run.

    Cheers!

  • Arom77

    Hi @Jack Kitterhing

    Sorry for the delay...

    I have server logs but don't know what to look at.
    I know the problem is related to RAM usage. From time to time it runs out of RAM so Pro Sites looses all its settings. But 99% of the time RAM usage is low.
    For example right here:
    Real memory 1.91 GB used, 6 GB total
    Virtual memory 202.84 MB used, 512 MB total

    Other infos:
    Processor information Intel(R) Xeon(R) CPU E5-1620 v2 @ 3.70GHz, 2 cores
    Operating system Debian Linux 6.0
    Webmin version 1.760
    Virtualmin version 4.18.gpl
    Kernel and CPU Linux 2.6.32-37-pve on i686
    Running processes 88
    CPU load averages 0.35 (1 min) 0.47 (5 mins) 0.57 (15 mins)
    CPU usage 15% user, 1% kernel, 0% IO, 84% idle
    Apache version 2.2.16

    Do you need other infos? Please advise.

    Thanks!
    Cheers

  • Jude

    Hey @Arom77

    Just quickly checking in here .. Jack is no longer available on the forums. The stable version of Pro Sites is now 3.5.x.x which is almost a full rewrite of the version you had installed when you opened this ticket 3.4.x.x.

    Can you still confirm if this issue still persists ?

    Also if you are using a InnoDB instance can you simply lock the row psts_settings from getting overwritten as shown here. Do this once all the settings are final.

    http://stackoverflow.com/questions/14965004/prevent-database-table-row-from-delete-and-update

    Cheers
    Jude

  • Arom77

    Hi @Jude

    The problem occured once again yesterday. My server went under attack so the server crashed, it went back after a minute and everything is normal now. Except I've just seen Pro Sites lost all its settings.

    It hasn't happened since my previous message, so the situation is definitely better, but is there really no way to prevent Pro Sites loosing settings ? It's the only plugin which does such a things on my differents sites.

    Thanks!

  • Michael Bissett

    Hey @Arom77, Michael here!

    Inside of phpMyAdmin, when you're browsing a database, you'll want to check out the "Type" column, that'll indicate whether a table's using InnoDB or MyISAM.

    I've taken a peek at your sitemeta table (the table where Pro Site's settings are stored) is a MyISAM table, and have informed my colleague @Jude of this, so that he can advise further.

    Kind Regards,
    Michael

  • Jude

    Hey @Arom77

    I too used the creds that @Michael Bissett shared and looks like something really strange is going on. There are 2 dozen psts_settings rows on the sitemeta table. It should be one ideally. See screenshots

    Here is what Im going to propose.

    Please take a backup first.
    I'll then clean these tables up for you.
    Then you enter the settings one last time
    I go ahead and set up a MySQL Trigger that prevents this new settings row from getting deleted.

    Let me know if this is acceptable and whenever you are ready

    Cheers
    Jude

  • Arom77

    Hi @Jude

    Thanks a lot for your proposal!

    I've got an automatic backup of my whole site every day.
    I'm not sure how I could make a manual backup of only the database... Is there an "easy" way to do it?

    I go ahead and set up a MySQL Trigger that prevents this new settings row from getting deleted.

    Just to be sure: I'll be able to change my Pro Sites settings anytime once you did that?

    Cheers

  • James Morris

    Hello Arom77,

    I sincerely apologize for the extreme delay in response here. Is this still something you are needing assistance with? If so, I'll be happy to move forward with helping you.

    Please visit the Contact page and complete the form with the following information:

    Subject: "Attn: James Morris"

    In the Message box, please provide the following:

    - link back to this thread for reference
    - any other relevant urls

    - Admin login:
    Admin username
    Admin password
    Login url

    - Hosting Control Panel Login
    Admin username
    Admin password
    Login url

    - FTP credentials
    host
    username
    password
    (and port if required)

    Best regards,

    James Morris

  • Mahlamusa

    Hello Arom77.

    I hope you are having a great day so far.

    I will be working with you to help resolve this issue. I used the login details you have provided and have noticed that the has be a lot of duplicates for the 'psts_settings' field in the database. In order proceed, I first need to delete all the duplicates and leave only one as there should be only one in the database. Having more than one setting will result in a failure when getting the stored settings.

    I am not sure yet what causes the error but to solve it, I have to remove the duplicate settings first. Please create a backup of the current database especially the table 'cssn_sitemeta' which contains the Pro Sites settings along with a lot of other data.

    Please confirm that the current settings are as you want them to be then afterwards I will continue with the investigation and hopefully will find a solution soon.

    Thank you for your patience as we resolve this. Enjoy the rest of your day.

    Cheers,
    Mahlamusa

  • Mahlamusa

    Hello Arom77,

    I am sorry for the late reply. I have gone through the process of deleting the duplicate pro sites settings and I have completed it today.

    I am not sure yet what causes the plugin to save duplicate settings as there should only be one 'psts_settings' in the sitemeta table and that should be updated with the newly updated value each time you save. I have tested this now and it is currently updating the only value I have left there.

    If I may ask, did you or anyone at some point restored a database backup for this site or do you / did you have a database backup plugin that deals with backing up and restoring databases? I ask this because if there was such a backup restore, that could be the start of the duplicated settings as the sitemeta table does not force unique values in its 'meta_key' column. This could result in a new duplicate being added each time this 'restore' happens.

    Now since your database has only one set of Pro Sites settings, we should try to wait and see what happens, but from what I have seen the settings are now updated instead of being duplicated. The reason why it is duplicated is that when the plugin tries to update one meta_key and find that the table has multiple meta_keys with the same name in the same column, it will fail.

    WordPress is designed to create a meta_key if updating an existing one fails. So if it fails because there are many with the same name, then it defaults to creating a new meta_key. The same thing applies if it tries to read, if it fails then it assumes there is nothing and this is when Pro Sites decides to store the default values because WordPress failed to get the meta_key. Pro Sites along with other plugins rely on core WordPress functions and if those functions are not reliable Pro Sites and other plugins will seem to be unreliable.

    Let us give this another shot and see how it goes. If it fails again then I will proceed further with investigating the problem and I will try other suggested solutions.

    I hope this helps. Please let me know if this happens so I can get to it right away.

    I hope you enjoy your day further.

    Cheers,
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    did you or anyone at some point restored a database backup for this site or do you / did you have a database backup plugin that deals with backing up and restoring databases

    There is a daily backup of the VPS. And when all Pro Sites settings disappear it only happens during the backup process (I mean this is the only moment when all Pro Sites settings may disappear).
    There has never been a database restore for this site.

    All Pro Sites settings disappeared again a few hours ago (when the VPS backup ran up) (I've already set them back again manually).

    Thanks!

    Cheers

  • Mahlamusa

    Hi Arom77,

    I have done another extensive digging into this issue and now I can confirm that this issue is not related to Pro Sites, it is the process that does the backups that cause the errors.

    During my research I found that Webmin/Virtualmin uses the 'mysql_dump()' function to create a backup of the database. The 'mysql_dump()' function should be used in a way that it locks the database when backing it up, if it does not then data integrity will not be guaranteed. To create a backup using the mysql_dump() function then the database server should be stopped and in some cases a lock can be issued on the database to avoid writing and deleting while the backup is in progress. I think virtualmin/webmin does the backups while the database server is running and this is the cause for the issue at hand.

    If you can access the script that does the backup, you should verify that it has the line that stops the database server, lock the tables, create the backup, release locks and start database server again. You should have a line like this:

    mysqldump --user=root --password=yourrootsqlpassword --lock-all-tables

    That line will make sure all tables are locked and thus avoiding the read/write that causes the duplicate resulting in settings disappearing. This method is not recommended because your website will not be available when the backup is running since the database server should be stopped and tables locked, that means there will be no possible data updates in the database and the website will show database connection errors. I think that is why webmin/virtualmin does the backups the way it does now. This not recommended as your backups are running as long as 10 minutes for each site on your server, that will be a lot of inconvenience for your visitors.

    To learn how to create backups without interrupting the database server, you may want to follow this article

    I also found that Pro Sites is not the only plugin that suffers from this issue but a few others. I have attached screenshots showing duplicate settings for 2 other plugins. Please see the screenshots below:

    The screenshots above show that Pro Sites is not the only plugin having issues and that this issue is not caused by Pro Sites. Unfortunately server administration or optimization is beyond the scope of this support forum.

    I am truly sorry that we can't go deep into the core of your control panel to troubleshoot the problem, but that is the limitation of the support forum.

    I hope you understand.

    Please enjoy your day further.

    Cheers,
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    I'm working on this major issue (the problem happens almost every day).
    As far as I know Virtualmin does a "lock-tables". Could you explain to me why a "lock-all-tables" is required? Is it related to the way WordPress Multisite work? Or the way Pro Sites and some WPMUdev plugins work?

    I'm being told that this is never required and that it's dangerous to change the way Virtualmin does the back-up...

    Thanks!

  • wp.network

    Arom77 are you using MyISAM and InnoDB tables together in the same database?

    Personally, I use only InnoDB and, when I'm being extra careful, I do mysqldump with these options
    --lock-all-tables --extended-insert=FALSE --complete-insert=TRUE
    which add some time to the process, especially import/restoration time - however, they also help ensure that I have good (as in restorable) backups... you should prob do some reading to learn a bit about this so that you can make some informed decisions about how to proceed (eg. perform your own backups w/o virtualmin; migrate to different control panel)... this stuff can get a bit complicated... for your next step you might try cleaning up the duplicate database entries and then use lock-all-tables option with mysqldump from command line and see if you are able to successfully make the backup and the restoration... and do some googling about the mysqldump options. Good luck

    Cheers, Max

  • Mahlamusa

    Hello Arom77.

    I hope you are doing great today.

    I have recommended the --lock-all-tables to prevent updates/writes while the backup is in progress this is to prevent the possibility of duplicates.

    The functions that WordPress use to update a record are created so that when it fails to update a record, its either the data in the current record is the same as the one it should be updated with so if its not the same but it fails to update then it assumes the record does not exist and hence create a new record with the new data. This can happen if the table row is locked that is why I think locking the whole table is a best option to prevent this from happening. Unfortunately, that behavior is not unique to WPMU DEV plugin but all plugins that comply to WordPress standards.

    I'm being told that this is never required and that it's dangerous to change the way Virtualmin does the back-up

    I honestly understand this and the reason why they said it but if you may do if you are solving a problem or better ask the developers of virtual mean to check how they implemented the backup system. I would also recommend you try WP-Networks's suggestion (Thank you WP-Networks for that useful addition. )

    --lock-all-tables --extended-insert=FALSE --complete-insert=TRUE

    You should also check if you are not using different database engines on the same database because this may cause issues as MyISAM and InnoDB do not handle data in the same way. I recommend you use the same engine as that used by default WordPress tables for all extra tables you may have in the same database.

    If you are having issues with this, you may also try to do backups in another way. You may use any WordPress Backups plugin for that like Snapshot Pro, UpdraftPlus or any other WordPress backup plugin. These can help you backup your website files and database, then you can use Virtualmin's built in backup system to backup your accounts and emails. I think that will save you the trouble of having to deal with this issue now and then.

    I hope that helps you. Please enjoy the rest of your day

    Cheers,
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    I decided to convert all my tables of every sites on my VPS from MyISAM.
    Everything went right for every sites, except for creersonsite.net
    As you can see I get a 500 error. Actually I get it as soon as I type something into the SQL query area (but the query works OK though when I run it).
    As a result, all Pro Sites settings instantanely dissapeared. So I guess this is it. But do you have an idea about why the problem only happens with creersonsite? I have another WP Multisite on this VPS and no problem with it.

    Still need your help to figure this out.
    Thanks!

  • Mahlamusa

    Hi Arom77,

    I hope you are doing great today and I am sorry it took me so long to respond to your last post, I must have missed the notification about your post, I am sorry about that.

    I have looked at one of your sites that looks like a multi site, the site 'creer-son-site' has Pro Sites settings on its database, I can confirm that this site does not have the duplicates in its 'prefix_sitemeta' table, that is why you are not having issues with it. Unfortunately, I am not familiar with your hosting control panel so it is not easier for me to compare the database tables using the database tools in your control panel.

    I think you have to clean the duplicates first then convert the database engine, I tried to do that for you but unfortunately there is a lot of duplicates from other plugins as well, I was only able to clean those related to Pro Sites and didn't want to break anything and I stopped the. I suggest you clean those duplicates then run the query to change the database engine. I have tried to run it but it returned no results, again I am not sure if it was successful or not and I am not sure how to check if it was successful due to my limited experience with your hosting control panel.

    I am really sorry that you are going through so much trouble with regards to this issue, I know it must be very hard for you to have to deal with this. I want you to know we are always here for you and we will try to help where we can.

    I hope you enjoy the rest of your day.

    Cheers,
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    I'm cleaning up the mess right now so I've other questions of course

    First one:
    I see some duplicates for nbt_plugin_version
    But none of them had 2.8.5 as a value (New Blog Templates is in version 2.8.5 at the moment). I deleted all duplicates and just keep one with 2.8.4 but I don't understand why there was no 2.8.5?

    Seconde one:
    There is a LOT of domainmap-flushed-rules-XXXX meta keys. What are they for? Do I need to keep them all or can I delete them ?
    Same fact and same question for _site_transient_user_activity_,
    _site_transient_timeout_user_activity_, _site_transient_timeout_wdp_un_changelog_ and a few others

    EDIT:
    All duplicates are deleted now.
    Is the bug expected to have gone away now? Or do I have other thing to do?

    Thanks

  • Mahlamusa

    Hello Arom77,

    I hope you are doing great today, sorry for the delayed reply and thank you for your patience.

    First One: there should only be one 'nbt_plugin_version' entry with the latest version of the plugin.

    Second One: I am not sure which plugin is setting the '_site_transient_user_activity_' meta, it could be WordPress Core and a lot of other plugins so I can't be sure whether to delete or keep it. Ideally, if they are of the same user or blog_id and have the same values you should delete the oldest and keep the newest.

    Is the bug expected to have gone away now? Or do I have other thing to do?

    If you have cleared the duplicates and changed the database as recommended, then you should not have the same issue again. You should just finish your cleanup and create a backup of the database. You can use Snapshot to create an easy to restore database on the "Snapshot > Snapshots > Create Snapshot" page, in the options you should exclude files as your server is already creating backups of your files.

    I hope this helps. Please let us know if you have any further questions.

    Have a nice day. Cheers,
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    there should only be one 'nbt_plugin_version' entry with the latest version of the plugin.

    I deleted the duplicated and kept just one instance (but it's noted 2.8.4 and not 2.8.5 as it should). I guess I'll see what happens when this plugin is updated

    For the rest I've finished the cleanup. I'll keep you updated if the issue happens again. For now let's mark this topic as resolved

    Cheers and a big thanks for your help!

  • Mahlamusa

    Hello Arom77,

    I hope you are doing great today. I am really sorry for the delayed response and thank you very much for your patience. I am really sorry for the inconveniences that this is causing for you, I can't imagine the stress its causing but I hope we get to the bottom of this soon.

    Thank you for sharing the link above. After reading through the tutorial I checked the WordPress Database Description and have found that the options table is designed to have a unique 'option_name' column as seen in the description:

    Then I checked and compared the options table in my setup with the one in your setup and I have found that in my setup, the 'option_name' column does have the 'Unique' attribute set:

    While on your setup it looks like the unique key is not specified for the 'option_name':

    This is very strange since these keys should be set by WordPress when creating the table during installation. I have searched on the internet and found one article where a user mentioned that he had the same issue after importing a database using phpMyadmin, I know you are not using phpMyadmin but this may have happened when migrating your site or restoring a backup in some other way.

    Now back to your question:

    Why do you think about what is said in the "Preventing Duplicates from Occurring in a Table" section here?

    As the options table is already defined to have the 'option_name' unique, I think it might work to our favor to repair the options_table by making it hold unique values. If you apply this to the table as it stands then it should drop all the duplicates automatically and leave only one unique record for each option_name. Please, please create a backup of your whole database before doing this avoid any further headaches.

    You can try the SQL query in the "Execute SQL" tool in your hosting control panel:

    ALTER TABLE <prefix>_options ADD UNIQUE (option_name);

    Where <prefix> should be replaced by the actual database prefix as defined in your wp-config.php. This should constraint the 'option_name' column to only contain unique values so there will be only one 'psts_settings' option and only one of any other option.

    So, as a way forward. I suggest you backup your database or your options table, cleanup any duplicates and then run the above SQL Query to make the option_name unique as suggested in the 'Preventing Duplicates from Occurring in a Table' section on the page you shared. Here is a reference on making unique columns on existing tables.

    I hope this helps. Please do not hesitate to contact us should any issue arise.

    Have a nice day.
    Mahlamusa

  • Arom77

    Hi again Mahlamusa

    FYI I've got the feeling that probability for Pro Sites to loose its settings is greater after I deleted all duplicates. Don't know if it's possible?

    Anyway, I'm in contact with Joel too (on email) about another major issue (this one: https://premium.wpmudev.org/forums/topic/paypal-payment-ok-but-pro-site-status-has-expired-what-to-do) and I've just sent him a psts log file. It's about 21MB so I can't attach this one here. Could you have a look to it too please?

    Thanks a lot!

  • Mahlamusa

    Hi Arom,

    I hope you are doing great today. I am sorry for the delayed reply.

    I've just had a look on my setup, using phpmyadmin, and here it was I see. I can see "Unique: Oui" (Oui = Yes).
    So is it unique or not? How did you see what you showed me?

    On your screenshot, it seems to be unique as it should be, but would you please share with me the access to your phpmyadmin, I only have access to Webmin from where I accessed the built in database editor tool. You can share your phpmyadmin login via our secure contact form make the subject to be "Attn: Lindeni Mahlalela" then in the message body include the phpmyadmin login url, username and password. Once I have that I will have a look.

    The file permissions seems okay for your website,

    /wp-admin/admin-ajax.php is 644 - this is ok for all files in WordPress including all .php files in all folders.
    /wp-admin/ it's 755 - all folders like wp-content, wp-admin, wp-includes are set to 755 including all sub folders of those. The other file seems a little strange as it is not part of WordPress, I am not sure why its there.

    Basically, all WordPress files are 644 and folders are 755, so to answer your question, that is OK

    I hope this helps.
    Mahlamusa

  • Mahlamusa

    Hello Arom,

    I hope you are doing great today, I am sorry for the delayed response. Thank you for sending in the credentials. Firstly, I am sorry I quoted and referenced the incorrect database table in my post above, it should be the '_sitemeta' table instead of the '_options' table. I am sorry for the inconvenience caused.

    I have compared the sitemeta table from your database with the one on my database and even though I am not sure what it means, I have found the following differences.

    On your site:

    On my site:

    The only big diference is the Collation which is related to MySQL Server version, yours is for an earlier version and mine is for a newer version. This article suggests the utf8mb4_unicode_520_ci collation for the newer version of WordPress since 4.2/4.4 and MySQL 5.6 or later unfortunately this seems to be not the case on your setup. I am not saying this is the issue nor am I suggesting you change but I am saying it is worth looking into.

    Also the "Cardinality" of the meta_key and meta_value seems to be different between my and your setup, please verify if those are defaults or were intentionally changed. I also noticed that when you click on the "More" drop down the is an "Index" option which is not set in my table but seems to be set in your database for the meta_key and meta_value columns.

    Please double check those, other than those observations I am not sure what else could be problematic.

    I hope this helps.
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    I'm starting to really worry because this major issue has been here for already 2,5 years (!) and I can't see where I'm going
    I know you're doing your best I still need the best of your WPMUdev assistance to get things sorted out.

    I'm reading a bout this collation story but as I'm a marketer this is not always very clear for me.
    I've just found this useful article:
    https://make.wordpress.org/core/2015/04/02/the-utf8mb4-upgrade/

    I wonder if my 2 others majors issues (403 problems with IPN from both PayPal and Stripe) may be related to this one?

    Best Regards

  • Arom77

    Hi Mahlamusa

    I had some reading and here it what I understand:
    Duplicates happen when you insert the same data more than once in a database.
    And it has nothing to do with the hosting, and everything with the code.

    Am I correct?
    If so, how can I spot the script that insert the same data more than once in my database?

    All that I see in my Apache error log is:
    [Sun Aug 27 18:06:59.456670 2017] [fcgid:warn] [pid 7524] (104)Connection reset by peer: [client 104.199.87.90:58248] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 18:19:05.901102 2017] [fcgid:warn] [pid 9801] (32)Broken pipe: [client 31.36.122.12:56969] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function, referer: http://xxx/wp-admin/my-sites.php
    [Sun Aug 27 18:29:59.188703 2017] [fcgid:warn] [pid 11260] (104)Connection reset by peer: [client 35.190.207.132:60930] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 18:31:21.834959 2017] [fcgid:warn] [pid 11385] (104)Connection reset by peer: [client 35.190.201.122:39198] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 18:50:37.504801 2017] [fcgid:warn] [pid 14288] (104)Connection reset by peer: [client 207.46.13.197:1166] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 19:02:03.084121 2017] [fcgid:warn] [pid 16350] (104)Connection reset by peer: [client 207.46.13.197:9144] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 19:28:09.416581 2017] [fcgid:warn] [pid 19947] (32)Broken pipe: [client 88.162.226.158:23154] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function, referer: https://xxx/creation-site-gratuit/
    [Sun Aug 27 19:28:09.539537 2017] [fcgid:warn] [pid 19851] (32)Broken pipe: [client 88.162.226.158:23155] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function, referer: https://xxx/creation-blog-site-perso/creer-un-journal-en-ligne/
    [Sun Aug 27 19:31:31.308843 2017] [fcgid:warn] [pid 19199] (104)Connection reset by peer: [client 35.190.201.122:43030] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 19:56:05.691227 2017] [fcgid:warn] [pid 23659] (32)Broken pipe: [client 178.32.189.236:49462] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function, referer: https://xxx/?wordfence_syncAttackData=1503856562.1793
    [Sun Aug 27 20:40:36.074013 2017] [fcgid:warn] [pid 30265] (104)Connection reset by peer: [client 35.190.207.132:57354] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 21:12:58.013835 2017] [fcgid:warn] [pid 1881] (104)Connection reset by peer: [client 104.199.87.90:38806] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 21:16:34.121028 2017] [fcgid:warn] [pid 3365] (104)Connection reset by peer: [client 157.55.39.20:3106] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 21:29:09.877895 2017] [fcgid:warn] [pid 4896] (104)Connection reset by peer: [client 95.215.62.44:49082] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 21:35:12.672605 2017] [fcgid:warn] [pid 5639] (32)Broken pipe: [client 95.215.62.44:39084] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function
    [Sun Aug 27 21:37:30.096508 2017] [fcgid:warn] [pid 6348] (32)Broken pipe: [client 185.16.223.77:61717] mod_fcgid: ap_pass_brigade failed in handle_request_ipc function, referer: https://xxx/wp-admin/network/admin.php?page=psts-levels

    Thanks

  • Arom77

    Hi again Mahlamusa

    Still reading and thinkig about this issue.
    So I guess we're OK that duplicates happen when you insert the same data more than once in a database.
    That means if I have a look to tables that are duplicates I'll get a clue about the duplicates happening.

    Here are all involved tables (each time they are the same ones, I delete duplicated and those tables and only thoses are involved again).
    I have the feeling the problem is related to the website creation process (maybe and/or account creation process).

    Do you know what the wdeb meta_key is about? Is this a WPMUdev plugin?

    Thanks

  • Arom77

    Hi again Mahlamusa

    Important note:
    As I'm using Gravity Forms User Registration combined with New Blog Templates (and Pro Sites), Jose Jaureguiberry (Incsub) asked me a long time ago (3 years ago actually, you can find some topics about this MU plugin) to add the following plugin "GF_NBT_Showcase_Selector" as a MU plugin.
    Could my issue be related to that? (duplicates happening because of inserting the same data more than once in a database)
    Thanks

    <?php
    /*
    Plugin Name: GF_NBT_Showcase_Selector
    Plugin URI: http://premium.wpmudev.org
    Description: Must Use Plugin for custom code implementation.
    Version: 1.0
    Author: Jose Jaureguiberry (Incsub)
    Author URI: http://premium.wpmudev.org
    License: GNU General Public License (Version 2 - GPLv2)
    */
    
    if(! class_exists('GF_NBT_Showcase_Selector') ):
    
        class GF_NBT_Showcase_Selector{
    
            function __construct(){
                add_filter( 'wp_signup_location', array($this, 'custom_signup_location'), 15, 1 );
                add_action( 'template_redirect', array($this, 'custom_nbt_redirect_signup'), 15 );
                // Triggered when New Blog Templates class is created
                add_action( 'nbt_object_create', array($this, 'override_gravity_forms_hooks'), 999 );
            }
    
            public function custom_signup_location($sign_up_url){
                $gf_settings = get_option('gf_userregistration_settings');
                if(defined("GRAVITY_MANAGER_URL") && $gf_settings && !empty($gf_settings['custom_reg_page']) && 'on' == $gf_settings['enable_custom_reg_page']){
                    return get_permalink(absint($gf_settings['custom_reg_page']));
                }
    
                return $sign_up_url;
            }
    
            public function custom_nbt_redirect_signup() {
                global $post;
                $gf_settings = get_option('gf_userregistration_settings');
                $nbt_settings = nbt_get_settings();
    
                if(!is_numeric($post->ID) || !$gf_settings || empty($gf_settings['custom_reg_page']) || 'on' != $gf_settings['enable_custom_reg_page']){
                    return;
                }
                if($post->ID == $gf_settings['custom_reg_page'] && $post->ID != $nbt_settings['page-showcase-id']){
    
                    $redirect_to = nbt_get_showcase_redirection_location();
    
                    if ( $redirect_to ) {
                        wp_redirect( $redirect_to );
                        exit();
                    }
                }
            }
    
            public function override_gravity_forms_hooks() {
                remove_filter( 'gform_user_registration_new_site_meta', 'nbt_save_new_blog_meta' );
                remove_filter( 'gform_user_registration_signup_meta', 'nbt_save_new_blog_meta' );
                add_filter( 'gform_user_registration_new_site_meta', array( $this, 'custom_nbt_save_new_blog_meta'), 20, 1 );
                add_filter( 'gform_user_registration_signup_meta', array( $this, 'custom_nbt_save_new_blog_meta'), 20, 1 );
            }
    
            /**
             * Save the blog template meta when signing up/cerating a new blog
             * <a title=<a title=param href=/profile/param>param</a> href=/profile/param>param</a> Array $meta Current meta
             * @return Array
             */
            function custom_nbt_save_new_blog_meta( $meta ) {
    
                $model = nbt_get_model();
    
                if ( isset( $_REQUEST['blog_template' ] ) && $model->get_template( absint( $_REQUEST['blog_template'] ) ) )
                    $meta['blog_template'] = absint( $_REQUEST['blog_template'] );
    
                // Maybe GF is activating a signup instead
                if ( empty( $meta['blog_template'] ) && isset( $_REQUEST['key'] ) && class_exists( 'GFSignup' ) ) {
                    $signup = GFSignup::get( $_REQUEST['key'] );
                    if ( ! is_wp_error( $signup ) && ! empty( $signup->meta['blog_template'] ) ) {
                        $meta['blog_template'] = $signup->meta['blog_template'];
                    }
                    elseif ( ! empty( $signup->error_data['already_active']->meta ) ) {
                        // A little hack for GF
                        $_meta = maybe_unserialize( $signup->error_data['already_active']->meta );
                        if ( ! empty( $_meta['blog_template'] ) )
                            $meta['blog_template'] = $_meta['blog_template'];
                    }
    
                }
    
                $default_template_id = $model->get_default_template_id();
    
                if ( empty( $meta['blog_template'] ) && $default_template_id )
                    $meta['blog_template'] = $default_template_id;
    
                return $meta;
            }
    
         }
        new GF_NBT_Showcase_Selector;
    
    endif;
  • Arom77

    Hi

    Still working on this major issue.
    I'm pretty sure this is directly related to the fact I've new "Pro Site" pages automatically created several times per week or even per day (something I already talked about in this support forum).

    The issue is caused by the way the registration & site creation is managed so it's really a "WPMUdev problem" and I do request your help once again (sorry to insist but this nightmare is preventing me for developping my business and it's been here for already 2,5 years...)

    Thanks

  • Arom77

    Hi again Mahlamusa

    OK so I'm starting to see a little more what's happening but as I'm not a developper at all I really need your help.

    When I go to Pro Sites settings (psts-settings, and psts-settings is one of the auto-duplicating tables) and choose "Modifier la page de commande", I'm redirected to "pro-site-142". This one, as you can see, was created on 17/08/27. And this is exactly the latest time Pro Sites lost all its settings.

    Still working on it, please help when you have some time to help me with your brain and knowledge

    Thanks!

  • Mahlamusa

    Hello Arom,

    I am really sorry for all the inconveniences you are having. I am also on your WordPress admin and phpMyadmin looking for any more clues I can find.

    I've just found this useful article:
    https://make.wordpress.org/core/2015/04/02/the-utf8mb4-upgrade/

    I have seen this article before. This upgrade also requires an upgrade on your database server, please refer to the requirements for this upgrade. If it is possible, I strongly suggest that you upgrade your server's software, that would be web and database server specifically. It should have a huge benefit to upgrade than it has to stay behind. Please contact your host about upgrading server software. Your host should be able to upgrade or guide you on how to upgrade.

    Duplicates happen when you insert the same data more than once in a database.

    That is absolutely correct. What's confusing here is that Pro Sites does not use insert for its settings, but it uses the built in WordPress function 'update_option' and or 'update_site_option'. The logic and all checks for these functions is defined by WordPress and it should not cause duplicates.

    GF_NBT_Showcase_Selector

    This plugin does not write any data on its own but only adds data to what Gravity Forms will be writing to the database, so this cannot be a cause for this issue in any way because it only add the blog_template parameter to the data submitted by Gravity Forms.

    Pro Sites pages have some author related

    The author is the person who installed Pro Sites. So this makes it sound like Pro Sites was re-installed, in which case the person installing or activating Pro Sites will be marked as the author. If those two users are super admin, then contact them and ask what actions were they performing that resulted in this recreation of pro site pages. If they are not supposed to be super admins, then please edit their accounts immediately to restrict them access to super admin functions.

    Deleting duplicate "Pro Site" pages
    the default Pro Sites page does not have a suffix like '-xyz', it should be just 'pro-site' not 'pro-site-XYZ' but since the page is recreated for the latest Pro Sites settings then it should be the one linked to the Pro Sites Settings page, so you can delete all others and leave only that one.

    I hope this helps. Please let me know if you need any help. I will be here, for a little while to help.

    Have a nice day.
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    Thanks for your answer

    I'm having some other info on my side.

    1) I deleted all duplicated "pro site" pages. I kept only the latest one, which was /pro-site-142/
    I changed its URL to /pro-site/

    2) I deleted all duplicates in the cssn_sitemeta table.
    In the psts_settings meta_key I kept I changed /pro-site-142/ to /pro-site-/

    3) I tried to create a new subsite as if I was a new customer, so I went to https://creersonsite.net/creer-un-site-internet/ and chose a template

    As a result :
    - It immediately duplicated the Pro Site page (created a /pro-site-2/).
    - Pro Sites immediately lost all its settings.

    So it seems I found the culprit.

  • Mahlamusa

    OK, the problem is the Pro Sites plugin can't find the check-out page, even if a page "Pro Site" (/pro-site/) already exist.

    It considers the page as not existing if it is in Trash or set as Draft, then it creates a new one. To avoid this, please delete those in the trash (the ones you deleted) and the '/pro-site/' page because it is not linked, then try saving the settings again so it creates a new one and links it properly.

    I will also find a way to make pro sites check the trash and publish the page if there is no re-branding page specified.

    Thanks,
    Mahlamusa

  • Mahlamusa

    Shall I delete one of the 2 psts_settings? Or maybe delete both?

    If you do that you will have to redo all the settings again. The simplest way is to copy the settings from the meta_value in phpMyadmin, delete both. Save in Pro Sites and restore in phpMyadmin, replacing the checkout page's ID and url in phpMyadmin. If you are comfortable doing the settings again in the settings page, then you may delete both psts_settings in the sitemeta table.

  • Arom77

    OK I deleted both, then reactivated the Pro Sites plugin. One meta_key psts_settings was immediately created and is populated with the following data :

    a:96:{s:12:"base_country";s:2:"US";s:8:"currency";s:3:"USD";s:20:"curr_symbol_position";i:1;s:12:"curr_decimal";i:1;s:7:"rebrand";s:8:"Pro Site";s:10:"lbl_signup";s:21:"Mise à niveau en Pro";s:8:"lbl_curr";s:12:"Votre compte";s:16:"gateways_enabled";a:0:{}s:15:"modules_enabled";a:0:{}s:15:"enabled_periods";a:3:{i:0;i:1;i:1;i:3;i:2;i:12;}s:13:"send_receipts";i:1;s:14:"hide_adminmenu";i:0;s:13:"hide_adminbar";i:0;s:19:"hide_adminbar_super";i:0;s:11:"show_signup";i:1;s:19:"show_signup_message";i:0;s:11:"free_signup";i:0;s:15:"multiple_signup";i:1;s:9:"free_name";s:7:"Gratuit";s:8:"free_msg";s:64:"Non merci, je vais continuer avec un site de base pour l'instant";s:10:"trial_days";b:0;s:13:"trial_message";s:156:"Vous avez DAYS restant dans votre LEVEL d'essai gratuit. Commander maintenant pour éviter de perdre des fonctionnalités réservés au niveau LEVEL »";s:14:"cancel_message";s:345:"Your DAYS day trial begins once you click "Subscribe" below. We perform a $1 pre-authorization to ensure your credit card is valid, but we won't actually charge your card until the end of your trial. If you don't cancel by day DAYS, your card will be charged for the subscription amount shown above. You can cancel your subscription at any time.";s:23:"recurring_subscriptions";i:1;s:12:"ga_ecommerce";s:4:"none";s:14:"signup_message";s:26:"Inscription à un site Pro";s:15:"feature_message";s:66:"Passer au LEVEL pour avoir accès à cette fonctionnalité »";s:14:"active_message";s:231:"Vos privilèges Site Pro expireront le : DATE<br />Sauf si vous avez annulé votre abonnement ou votre site a été mis à niveau via l'outil de Mises à niveau groupés, vos privilèges Site Pro seront automatiquement renouvelés.";s:15:"success_subject";s:41:"Merci d'être devenu un membre Pro Site !";s:11:"success_msg";s:461:"Thank you for becoming a Pro Site member!
    
    We have received your first subscription payment and you can now access all LEVEL features!
    
    Subscription payments should show on your credit card or bank statement as "THIS COMPANY". If you ever need to view, modify, upgrade, or cancel your Pro Site subscription you can do so here:
    CHECKOUTURL
    
    If you ever have any billing questions please contact us:
    http://mysite.com/contact/
    
    Thanks again for joining!";s:16:"canceled_subject";s:42:"Votre abonnement Pro Site à été annulé";s:12:"canceled_msg";s:268:"Your Pro Site subscription has been canceled.
    
    You should continue to have access until ENDDATE.
    
    We are very sorry to see you go, but we are looking forward to you subscribing to our services again.
    
    You can resubscribe at any time here:
    CHECKOUTURL
    
    Thanks!";s:15:"receipt_subject";s:35:"Le reçu de votre paiement Pro Site";s:11:"receipt_msg";s:404:"Your Pro Site subscription payment was successful!
    
    PAYMENTINFO
    
    Subscription payments should show on your credit card or bank statement as "YOUR COMPANY". If you ever need to view, modify, upgrade, or cancel your Pro Site subscription you can do so here:
    CHECKOUTURL
    
    If you ever have any billing questions please contact us:
    http://mysite.com/contact/
    
    Thanks again for being a valued member!";s:15:"expired_subject";s:31:"Votre statut Pro Site a expiré";s:11:"expired_msg";s:291:"Votre site SITENAME (SITEURL) a malheureusement expiré.
    
    Vous pouvez renouveler votre abonnement ici:
    CHECKOUTURL
    
    Si vous rencontrez des problèmes techniques pour payer la souscription, contactez-nous pour obtenir de l'aide:
    http://mysite.com/contact/
    
    En espérant vous revoir très vite";s:14:"failed_subject";s:52:"Le paiement de votre abonnement Pro Site à échoué";s:10:"failed_msg";s:366:"It seems like there is a problem with your latest Pro Site subscription payment, sorry about that.
    
    Please update your payment information or change your payment method as soon as possible to avoid a lapse in Pro Site features. If you're still having billing problems please contact us for help:
    http://mysite.com/contact/
    
    Many thanks again for being a member!";s:17:"extension_subject";s:53:"Une adhésion gratuite Site Pro vous a été donnée.";s:13:"extension_msg";s:312:"We have given you Pro Site access. You will now be able to enjoy all the benefits of being a Pro Site member.
    
    These benefits will be available to you until: ENDDATE.
    
    After this date your site will revert back to a standard site.
    
    You can subscribe at any time from the link below:
    CHECKOUTURL
    
    Thanks!";s:15:"revoked_subject";s:49:"Votre statut Site Pro permanent a été modifié.";s:11:"revoked_msg";s:278:"Your permanent Pro Site status has been removed. You will continue to have all the benefits of your Pro Site membership until ENDDATE.
    
    After this date your site will revert back to a standard site.
    
    You can subscribe at any time from the link below:
    CHECKOUTURL
    
    Thanks!";s:9:"pypl_site";s:2:"US";s:13:"pypl_currency";s:3:"USD";s:11:"pypl_status";s:4:"test";s:15:"pypl_enable_pro";i:0;s:10:"stripe_ssl";i:0;s:7:"mp_name";s:15:"Paiement manuel";s:12:"mp_show_form";i:0;s:8:"mp_email";s:24:"contact@XXX";s:7:"pt_name";s:15:"Thèmes Premium";s:7:"pt_text";s:61:"Passer au niveau LEVEL pour activer ce thème premium »";s:8:"ps_level";i:1;s:8:"ps_email";s:24:"contact@XXX";s:7:"ps_name";s:15:"Support Premium";s:10:"ps_message";s:130:"Vous pouvez nous envoyer une demande de support par e-mail directement prioritaire ici si vous avez besoin d'aide avec votre site.";s:9:"ps_notice";s:80:"Pour activer le support premium, s'il vous plaît passer au niveau LEVEL »";s:16:"publishing_level";i:1;s:24:"publishing_message_posts";s:87:"Pour activer la publication d'articles, s'il vous plaît passer au niveau LEVEL »";s:24:"publishing_message_pages";s:85:"Pour activer la publication de pages, s'il vous plaît passer au niveau LEVEL »";s:13:"quota_message";s:61:"Pour SPACE d'espace de stockage, mettre à niveau vers LEVEL!";s:17:"quota_out_message";s:121:"Vous êtes à court d'espace de stockage! S'il vous plaît passez au niveau LEVEL pour activer plus d'espace de stockage.";s:12:"xmlrpc_level";i:1;s:14:"xmlrpc_message";s:91:"Pour autoriser la publication à distance XML-RPC veuillez upgrader au niveau LEVEL »";s:9:"bp_notice";s:66:"Passer au LEVEL pour avoir accès à cette fonctionnalité »";s:7:"pp_name";s:18:"Extensions Premium";s:8:"ads_name";s:18:"Désactiver la pub";s:9:"ads_level";i:1;s:16:"ads_enable_blogs";i:0;s:9:"ads_count";i:3;s:15:"ads_before_page";i:0;s:14:"ads_after_page";i:0;s:15:"ads_before_post";i:0;s:14:"ads_after_post";i:0;s:10:"ads_themes";i:0;s:8:"bu_email";b:0;s:9:"bu_status";s:4:"test";s:15:"bu_payment_type";s:9:"recurring";s:8:"bu_level";i:1;s:12:"bu_credits_1";i:10;s:13:"bu_option_msg";s:81:"Mettre à niveau les CREDITS des sites au LEVEL pour un an pour seulement PRICE: ";s:15:"bu_checkout_msg";s:295:"Vous pouvez mettre à jour plusieurs sites à moindre coût en achetant des crédits Pro Site ci-dessous. Après l'achat de vos crédits juste revenir sur cette page, rechercher vos sites via l'outil en bas de la page, et changer leur état Pro Site. Chaque site est mis à niveau pendant un an.";s:14:"bu_payment_msg";s:202:"En fonction de votre mode de paiement, cela peut prendre quelques minutes (carte de crédit ou fonds PayPal) ou plusieurs jours (chèque électronique) pour que vos crédits Pro Site soient disponibles.";s:7:"bu_name";s:23:"Mises à jour groupées";s:11:"bu_link_msg";s:81:"Acheter des crédits pour mettre à niveau plusieurs sites pour un prix réduit !";s:17:"ptb_front_disable";i:1;s:13:"ptb_front_msg";s:114:"Ce site est temporairement désactivé jusqu'à ce que le paiement soit reçu. S'il vous plaît revenez plus tard.";s:16:"ptb_checkout_msg";s:38:"Vous devez payer pour activer ce site.";s:8:"pq_level";i:1;s:9:"pq_quotas";a:2:{s:4:"post";a:1:{s:5:"quota";s:9:"unlimited";}s:4:"page";a:1:{s:5:"quota";s:9:"unlimited";}}s:8:"uh_level";i:1;s:10:"uh_message";s:74:"Pour activer l'intégration HTML, s'il vous plaît passer au LEVEL »";s:10:"co_pricing";s:8:"disabled";s:19:"plans_table_enabled";s:7:"enabled";i:0;b:0;s:7:"version";s:5:"3.5.5";}

    No checkout URL inside, I guess it's normal.

  • Mahlamusa

    Hi Arom,

    The Pro Sites Checkout page is created in the 'init' hook which is to say when WordPress is 'started' or loaded, this can happen when visiting the front end. So if a user visits the front end, Pro Sites check to see if the page exists, if it does not exist or is trashed, it creates a new one. That is how it works. The WorsPress 'init' hook does not rely on the Admin Dashboard, it is available in the front end.

    Thanks.
    Mahlamusa

  • Arom77

    I edited the page a bit (just changed the Title and add some words, kept the slug /pro-site/)
    Everything was OK.

    But a moment after I tried to edit the page once again (wanted to add a Genesis class) then suddenly got an error: "Désolé, vous n’avez pas l’autorisation de modifier cet article."

    At this moment I saw that my page /pro-site/ didn't exist anymore (I can't even see it in the trash).
    And that a new page /pro-site-2/ has been created.

    However there is no duplicate at this time for psts_settings.
    Note that /pro-site-2/ is in the check-out URL in psts_settings.

  • Arom77

    Arom, you previously created an account for my colleague James, with the username wpmu****s77, it seems I can't login with that account any more, I was logged in earlier today but it does not work anymore. Would you please grants support access so I can work with you on this. You can reactivate that account or grants support access in "WPMU DEV > Support" page.

    I didn't change this account. Maybe you mispelled the password? I see that you successfully logged in about 5 hours ago, could you confirm that please?

  • Arom77

    The Pro Sites Checkout page is created in the 'init' hook which is to say when WordPress is 'started' or loaded, this can happen when visiting the front end. So if a user visits the front end, Pro Sites check to see if the page exists, if it does not exist or is trashed, it creates a new one.

    My guess would be that Pro Sites can't find the check-out page (even though it does exist) "quite often", which causes the issue. What do you think?

    EDIT: but that wouldn't explain why original Pro Site page (/pro-site/) "disappeared" yesterday O.o

  • Arom77

    Right now I was on /wp-admin/network/admin.php?page=psts-plugins-manager (gestionnaire de plugins premium).
    When I tried to save settings I got "Désolé, vous n’avez pas l’autorisation d’accéder à cette page."

    I think there is a "permission problem" somewhere (and I think this is for the same reason that I get 403 error whith PayPal IPN and Stripe webhook). But where do I have to look to fix that?

  • Mahlamusa

    Hello Arom,

    I hope you are doing great today.

    I ran this message in Google Translate: 'Désolé, vous n’avez pas l’autorisation d’accéder à cette page.', It translates to 'Sorry, you do not have permission to access this page.' I wonder what action you were performing that resulted in this message. Would you double check your role / permissions, are you using a super admin account to do this?

    About UTF-8 in database, please be careful with this as it may result in another data loss. According to the WordPress Upgrade Article, WordPress will check to see if the server meets the minimum requirements for the upgrade and upgrade if necessary. Here are the requirements:

    Your MySQL server is version 5.5.3 or higher (including all 10.x versions of MariaDB).
    Your MySQL client libraries are version 5.5.3 or higher. If you’re using mysqlnd, 5.0.9 or higher.

    If your server does not meet these requirements, then please don't change it otherwise you risk loosing data. Please be careful with this.

    Would you please kindly ask your host about upgrading database server software, if you are using a self managed server then you should ask for directions on how to do it.

    I hope this helps.

    Have a nice day.
    Mahlamusa

  • Arom77

    Hi Mahlamusa

    It's OK, my VPS fulfills requirements
    Anyway I already converted 99% of tables to utf8mb4.
    I just want to change the default character set. OK, I'll get the info by myself.

    I ran this message in Google Translate: 'Désolé, vous n’avez pas l’autorisation d’accéder à cette page.', It translates to 'Sorry, you do not have permission to access this page.' I wonder what action you were performing that resulted in this message. Would you double check your role / permissions, are you using a super admin account to do this?

    That's the point! I was simply saving Pro Sites settings! I was saving premium plugins for Pro sites (so I was on this page /wp-admin/network/admin.php?page=psts-plugins-manager): when I clicked on the "save" button I got the error message and the problem happened.

    I can confirm I'm SuperAdmin.

    As far as I know I never had such an issue with any other part of my WordPress multisite network. So that would be only with Pro Sites.

    Again, as I said, it's not "systematic": most times I do the same thing I don't get an error message.
    Could you login as a Super Admin and try to replicate?
    Maybe you'll need a lot of tries but at least you'll see that I'm telling the truth
    It looks as if Pro Sites can't do some actions from time to time.

    Thanks

  • Arom77

    Hi Mahlamusa and Kasia Swiderska

    (As you know I'm in contact with Joel too through emails but wanted let this post here too to be able to find it easily again later plus could be useful for others)

    So here it was I noticed.

    - I "deactivated" a week ago the new site registration process (actually I just remove the website creation process from the front office, not the back office, but as a result no new subsite has been created for about a whole week).
    - No problem with Pro Sites loosing its settings during this whole week.
    - Today I decided to investigate further because I absolutely need to fix that *@"&ç^*~ problem. So I deleted all Pro Site pages in WordPress back-office and deleted duplicates in phpMyAdmin. Again .
    I edited psts_settings to replace /pro-site-13/ with /pro-site/
    Again.
    - Immediately Pro Sites lost all its settings.
    - So I had a look in my Apache Error log and here is what I saw:

    [Fri Sep 22 15:33:15.552023 2017] [fcgid:warn] [pid 14359] [client 81.177.215.171:55011] mod_fcgid: stderr: Erreur de la base de donn\xc3\xa9es WordPress Too many keys specified; max 64 keys allowed pour la requ\xc3\xaate ALTER TABLE cssn_pro_sites ADD KEY(blog_ID,level,expire`) faite par require('wp-load.php', require_once('wp-config.php', require_once('wp-settings.php', include_once('/plugins/pro-sites/pro-sites.php', ProSites->__construct, ProSites->install, dbDelta

    [Fri Sep 22 15:33:15.552059 2017] [fcgid:warn] [pid 14359] [client 81.177.215.171:55011] mod_fcgid: stderr: Erreur de la base de donn\xc3\xa9es WordPress Too many keys specified; max 64 keys allowed pour la requ\xc3\xaate ALTER TABLE cssn_pro_sites_transactions ADD KEY (id,transaction_id) faite par require('wp-load.php', require_once('wp-config.php', require_once('wp-settings.php', include_once('/plugins/pro-sites/pro-sites.php', ProSites->__construct, ProSites->install, dbDelta

    [Fri Sep 22 15:40:04.252963 2017] [fcgid:warn] [pid 15658] [client 178.32.189.236:39047] mod_fcgid: stderr: Erreur de la base de donn\xc3\xa9es WordPress Too many keys specified; max 64 keys allowed pour la requ\xc3\xaate ALTER TABLE cssn_pro_sites ADD KEY (blog_ID,level,expire) faite par require('wp-load.php', require_once('wp-config.php', require_once('wp-settings.php', include_once('/plugins/pro-sites/pro-sites.php', ProSites->__construct, ProSites->install, dbDelta

    [Fri Sep 22 15:40:04.253000 2017] [fcgid:warn] [pid 15658] [client 178.32.189.236:39047] mod_fcgid: stderr: Erreur de la base de donn\xc3\xa9es WordPress Too many keys specified; max 64 keys allowed pour la requ\xc3\xaate ALTER TABLE cssn_pro_sites_transactions ADD KEY (id,transaction_id) faite par require('wp-load.php', require_once('wp-config.php', require_once('wp-settings.php', include_once('/plugins/pro-sites/pro-sites.php', ProSites->__construct, ProSites->install, dbDelta

    [Fri Sep 22 15:40:04.387449 2017] [fcgid:warn] [pid 15735] [client 34.210.94.197:57408] mod_fcgid: stderr: Erreur de la base de donn\xc3\xa9es WordPress Too many keys specified; max 64 keys allowed pour la requ\xc3\xaate ALTER TABLE cssn_pro_sites ADD KEY (blog_ID,level,expire) faite par require('wp-load.php', require_once('wp-config.php', require_once('wp-settings.php', include_once('/plugins/pro-sites/pro-sites.php', ProSites->__construct, ProSites->install, dbDelta

    [Fri Sep 22 15:40:04.387478 2017] [fcgid:warn] [pid 15735] [client 34.210.94.197:57408] mod_fcgid: stderr: Erreur de la base de donn\xc3\xa9es WordPress Too many keys specified; max 64 keys allowed pour la requ\xc3\xaate ALTER TABLE cssn_pro_sites_transactions ADD KEY (id,transaction_id) faite par require('wp-load.php', require_once('wp-config.php', require_once('wp-settings.php', include_once('/plugins/pro-sites/pro-sites.php', ProSites->__construct, ProSites->install, dbDelta
    `
    - So I had a look at cssn_pro_sites and especially at its index (see captures).
    - Then I did the same with cssn_pro_sites_transactions and its index (see captures).

    Waiting for your answer.

    Best Regards

  • Mahlamusa

    Hello Arom.

    I hope you are doing great today. Thank you for your patience so far.

    a:2:{s:13:"administrator";b:1;s:13:"bbp_keymaster";b:1;}

    The extra user capabilities are for bbPress, this means the user has Administrator and bbPress Keymaster role on the site. This should not interfere in anyway. Do you have any other plugins that change user roles, like Custom User Roles.

    I "deactivated" a week ago the new site registration process (actually I just remove the website creation process from the front office, not the back office, but as a result no new subsite has been created for about a whole week).

    Are you using the default Pro Sites registration process or you have it customized.

    I have noticed that you have been working with Joel James on via email. I read on his last email reply that he made some changes that could get rid of the errors indicated in the logs.

    How is it going after those changes have been made. Are you still experiencing the issue or you haven't tested yet? Please let us know how it goes and we will be happy to assist in any way possible.

    Have a nice day.
    Mahlamusa

  • Kasia Swiderska

    Hello Arom77,

    I spoke with developer about issue from this thread, that you were discussing via email - to get more info about changing hosting provider and I also checked emails.
    I think there is some confusion - their advise was not to change hosting provider, but to check with hosting provider issue with long MySQL queries.
    They are asking if hosting provider can answer question "which query is killing your server frequently?".

    kind regards,
    Kasia