Certain Subsite Posts Not Appearing In Recent Global Posts Feed

Hi. I've spent the past few hours troubleshooting why certain posts from one subsite are not appearing in our Recent Global Posts Feed. I've searched extensively, and chatted with Luís during Live Support hours, but neither of us could find similar reports. So here goes...

One of our users reported that some of her blog posts were not showing up in our network-wide RSS feed. We use the Recent Global Posts plugin to generate the feed at:
http://tripawds.com/wp-content/recent-global-posts-feed.php
and/or:
http://tripawds.com/feed/globalpostsfeed

The feed for the specific site having issues with posts not showing in the global feed is:
http://purrkins.tripawds.com/feed/

All of these feeds validate fine. Note, however, that a recent post from purrkins on March 29 is in that site's feed, but it is not in the global Tripawds feed.

For the record, purrkins.tripawds.com is a Pro Site and we are running Pro Sites with the original "Supporter" gateway.

Recent posts from purrkins that are missing from the global feed do show up in our Blogs Directory and in the Recent Posts Widget we display around the main site.

Post Indexer stats for the purrkins site show only 12 posts indexed, of 26 published posts. All the posts are published, public, and not password protected.

I have ruled out any plugin conflicts by performing the usual troubleshooting procedures on both the purrkins site and the main site. I have re-saved permalinks on both the purrkins site and the main site. I have rebuilt the Index both for the purrkins site by itself and the entire network, multiple times, after each of these steps.

And, here's the kicker: I published a test post as the purrkins user on her site and it now appears in the feed while her post from two days ago still does not.

After my last index rebuild, a few of the purrkins posts were listed in the Recently Indexed Posts according to the Post Indexer stats, but none of these were the most recent post for March, 29. Reviewing that and other posts not in the feed did not shed any light on the issue, other than the fact some of them include emojis in the post content.

I have reviewed entries and compared data in the database tables for the subsite and wp_network_posts only to confirm that 12 of the site's 26 posts got indexed, without identifying any sort of pattern.

During my troubleshooting and index rebuilds I enabled and monitored PI_CRON_DEBUG but noticed no unexpected anomalies.

I have checked the database and optimized tables to remove any overhead. I just don't know what else to do next to try and figure out what is going on, so I'm turning to the experts here.

Why would Post Indexer only index some of the posts on a certain site?

What further steps can I take to troubleshoot this issue? What other information do you need from me? I'm attaching some screenshots for clarification, and have granted Support access for the Tripawds network admin.

Thanks in advance for any help!

  • Nithin
    • Support Wizard

    Hi jcnjr,

    Hope you are doing good today. :slight_smile:

    That's really odd, and are you only noticing the issue with posts in http://purrkins.tripawds.com ? Since you have covered all the troubleshooting steps, it's strange that none of the troubleshooting steps gave any clue.

    I only see you mention about PI_CRON_DEBUG . Just to be sure that we don't miss anything, could you please enable debug mode, and see whether it helps? To enable it, open your wp-config.php file located in your root directory, and look for define(‘WP_DEBUG’, false);. Change it to:

    define('WP_DEBUG', true);

    In order to enable the error logging to a file on the server you need to add:

    define( 'WP_DEBUG_LOG', true );
    define( 'WP_DEBUG_DISPLAY', false );

    After making the above changes, please try create a new post, and once the site is indexed, and the feed is up. If any errors pops up, it will be saved to a debug.log log file inside the /wp-content/ directory.

    Please attach these in your next reply in txt format file, so that we could give a closer look. You can find more details about debugging here.

    Please do let us know how that goes, so that I could check this with the developer too. Have a nice day. :slight_smile:

    Kind Regards,
    Nithin

  • jcnjr
    • HummingBird

    Thanks for the quick response Nithin

    I only see you mention about PI_CRON_DEBUG . Just to be sure that we don't miss anything, could you please enable debug mode, and see whether it helps?

    Are you suggesting I enable both the PI and WP debug?

    I have since disabled PI_CRON_DEBUG (and was wondering if that log actually got written anywhere). Please advise if I need to turn that back on and enable WP_DEBUG_LOG to accomplish what you're recommending.

  • Nithin
    • Support Wizard

    Hi jcnjr,

    What PI_CRON_DEBUG does is it'll list error in a new tab under plugin settings. Since you have already tried that, it isn't required. However, please do keep PI_CRON_DEBUG enabled, and check whether you notice any anomalies.

    Please do let us know how enabling the debug mode goes. Have a nice day. :slight_smile:

    Best Regards,
    Nithin

  • jcnjr
    • HummingBird

    After making the above changes, please try create a new post, and once the site is indexed, and the feed is up...attach these in your next reply in txt format file, so that we could give a closer look.

    Nithin Thank you! We clearly have a lot of crud appearing in the log, but I did notice numerous entries specifically referring to Post Indexer, like this...

    [31-Mar-2017 18:23:40 UTC] WordPress database error Duplicate entry 'uncategorized' for key 'slug' for query INSERT INTO 'wp_network_terms' ('name', 'slug', 'term_group') VALUES ('Post Amputation', 'uncategorized', 0) made by do_action_ref_array, WP_Hook->do_action, WP_Hook->apply_filters, call_user_func_array, postindexercron->process_rebuild_secondpass, postindexermodel->index_tax, postindexermodel->insert_or_get_term

    FYI: After enabling the debug log, I posted as Purrkins and that post did get indexed, but it does not appear in the global feed.

    I also rebuilt the index for purrkins.tripawds.com while debug was still enabled, and Post Indexer is still only indexing 12 of the 26 posts. (I deleted my test post from yesterday.)

    Any help cleaning up these errors, and specifically addressing why this member's site is not indexing all posts is greatly appreciated.

    Log file and screenshots attached for reference...

  • Rupok
    • Support Ninja

    Hi jcnjr,

    So sorry to hear the inconvenience you are having. I tried to check this issue on your site with Support Access but it's just showing me "0". Can you please revoke and grant us permission once again so we can investigate this issue and try to find the reason behind this as fast as we can?

    Please confirm here when you are done regranting Support Access. I'm looking forward to hearing from you and resolving this issue as soon as possible.

    Have a nice day. Cheers!
    Rupok

  • Nithin
    • Support Wizard

    Hi jcnjr,

    Thank you for enabling support access. I could see the issue, and it does seem to be indexed in the subsite feed, but not in the Global posts feed. I'm pinging the developer regarding this, so that we could have a better idea regarding this.

    Could you also share us your credentials, so that I could give a closer look. I do understand this is a live site, so we won't be making any changes without checking with you. You can send credentials by using our secure contact form: https://premium.wpmudev.org/contact/#i-have-a-different-question

    - To Mark to my attention, the subject line should contain only: ATTN: Nithin Ramdas
    -cPanel or PHPMyAdmin DB access(host/username/password)
    -WordPress admin username
    -WordPress admin password
    -login url
    -FTP credentials (host/username/password)
    -link back to this thread for reference
    -any other relevant urls

    Have a nice day. :slight_smile:

    Kind Regards,
    Nithin

  • Nithin
    • Support Wizard

    Hi jcnjr,

    Hope you are doing good today. :slight_smile:

    Thank you for sharing the credentials, unfortunately only the WP admin access work, rest of the shared credentials like FTP, cPanel doesn't work. I'm getting the following error with FTP:

    Status:          Logged in
    Status:          Retrieving directory listing...
    
    Command:    PWD
    Response:     257 "/" is your current location
    Command:    TYPE I
    Response:     200 TYPE is now 8-bit binary
    Command:    PASV
    Response:     227 Entering Passive Mode
    Command:    MLSD
    Error:            Connection timed out after 100 seconds of inactivity
    Error:            Failed to retrieve directory listing

    Regarding cPanel, I get Login is invalid message. I tried to replicate this issue in my system, but it does seem to work fine.

    Could you please correct these, and let me know if I'm missing anything, so that I could give a closer look. You can reply with correct credentials the same way you have done before. Have a nice weekend ahead. :slight_smile:

    Best Regards,
    Nithin

  • jcnjr
    • HummingBird

    Nithin

    I get Login is invalid message.

    Interesting. I tested those credentials before sending them. I will reset passwords and send again.

    Please advise what country you are working from and provide an IP address so I can ensure that you are not being block by our server firewall, and see why if you were.

    Thank you! I sincerely appreciate your help with this.

  • jcnjr
    • HummingBird

    Hi Kasia Swiderska
    Any feedback from the powers that be?

    Our "purrkins" blogger—the Pro Site member mentioned in my original post—has just asked if I will ever get this fixed. She recently published another blog post, which did NOT appear in the global posts feed, and therefore it didn't show up in our Blogs Directory, nor in the Recent Posts widget displayed throughout the site.

    I know the developers are busy, just looking for any sort of progress report or help here...please advise if I need to resubmit credentials and/or extend Support access.

    Thank you!

  • Panos
    • SLS

    Hi jcnjr ,

    Sincere apologies for such a delay!

    I would like to try investigate what is going on. I tried accessing ftp and cpanl but unfortunately it seems the credentials are not valid anymore. Could you please send ftp info once again? Could you also include admin access by creating a temporary admin user and sending admin user too, as support access timesout pretty often?

    Thanks!

  • jcnjr
    • HummingBird

    Panos said

    I would like to try investigate what is going on.

    Thank You!

    Super Admin powers have been extended for the user you created. Sending new password and other requested credentials directly to that email address now.

    Please reply when you get that to confirm receipt and let me know if you need anything else.

    -J

  • jcnjr
    • HummingBird

    Panos said:

    ftp seems not to be working and...site seems down too.

    No, this is not the case. All is well on our server.

    Please provide an IP address and let me know from which country you our accessing the site. We have had to block certain countries due to persistent attacks over the years.

    If necessary, I will need to remove any blocks on your IP or country.

    Thanks!

  • Panos
    • SLS

    Hi jcnjr ,

    It seems that it's the posts that contain emojis that cause this issue. Posts that contain them don't get indexed by Post Indexer. I checked the collation of that column (post_content) on your db and it is "utf8_general_ci".

    I would recommend to change it to "utf8mb4_unicode_ci". You should be able to change that via phpMyAdmin panel by changing the "post_content" field ( although I would suggest to change all fields ) of the network_posts table.

    Or it could be done with some php and a sql:
    $wpdb->query( "ALTER TABLE {$table} MODIFY post_content longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;" );

    Currently I have installed a mu-plugin that shows if a post has been indexed.

    So, after changing the collation you can try saving a post that is not indexed and after saving check if the Indexed status has changed.

    Please let me know if you would like me to change the collation.

    In order to remove the "Indexed" column, you can delete file :
    wp-content/mu-plugins/wpmudev-indexed-blog-posts.php
    or ask me to remove it.

    Thanks!

  • jcnjr
    • HummingBird

    Panos

    Thank you for taking the time to identify the problem and propose a fix. That appears to be working!

    Seriously, I appreciate the thorough help and clear explanation. James Farmer needs to know this is the awesome type of support that make me proud to be a WPMU Dev member.

    Seriously, thank you.

    PS: For anyone encountering this issue and searching for a solution, it was only after changing the collation for the entire wp_network_posts table (not just the content column) that posts with emojis began getting indexed properly.

    PPS: Did I say thank you?

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.