Disappearing Live Stream Data

I have had the post and comment indexer plugins working for a while and had livestream working on the front end for comments. It's all been working fine. I've latest 'everything' wp/bp

I saw the new 'Post Indexer' update and 'updated' this. Since installing this my LiveStream widget won't display at all, it doesn't matter what combination of 'anything' it's set to nothing appears on the front end.

It was set to only display 'comments' so should not have been impacted by the 'post indexer'. The rebuild option tells me that it already has items indexed and yet LS still doesn't display even posts on the front end.

Even though I specified that this issue relates to LiveStream it's obviously something to do with the Post Indexer plugin.

Another odd thing is that since updating the post indexer I edited a PAGE (not a post) and this page now appears in my BP activity stream - I like it there BUT BP Activity has never before presented pages on the front end. So, I didn't write a 'new' page I edited an existing one and it now gets presented in the BP Activity stream and is displayed as a 'new' post. Very Odd.

  • Paul

    @Clive, Greetings.

    For Live Stream even when showing just comments it still have to include the Posts related to the comments in order to obtain the post title and link. So in technical terms this is a SQL join. If the post is not found this means the comment(s) will not be shown.

    I did some initial testing with Live stream in December when the Post Indexer was close to release. There were some needed changes. This being March there could have very well been some critical changes made to Post Indexer that have effected Live Stream. So let me setup my own testing this morning and get back to you.

    Thanks.

  • Paul

    @Clive,

    Can I get you to install this beta version. This beta I was working on over the weekend to add some other functionality. Plus this morning I added some debug output so I can see what the widget is doing on your site.

    https://dl.dropbox.com/u/2616987/WPMUDev/live-stream-widget-1.0.4.3-Beta-5.zip

    I just test this beta along with Post Indexer 3 for Comments only and seemed to work as expected. But I'm using a test environment with very little actual data. So hopefully a real test will show some issue.

    Let me know.

  • Clive

    Hi Paul,

    I've now the latest Post Indexer and the beta LS versions installed - Livestream is still not displaying anything.

    I also installed the Recent Networked Posts and Comment widget plugins and they work although my most recent comment is missing (It appears in bp activity) older ones are displaying properly.

    Should I try deleting PI 3,0,1 and reinstalling the 2.1 version and see if that disappears the LS problem?

  • Paul

    Oh well I guess not. Appears your site is password protected. At least when I attempted to access http://www.reality-hackers.com it redirects me to a the wp login form.

    So try this. On the page where you have the widget add the following the URL.

    ?LSW_DEBUG

    For example if you have the widget on the home page the full URL would be

    http://www.reality-hackers.com?LSW_DEBUG

    When you load the page like this you should see some debug output. Can you past that back here so I can review?

  • Clive

    Got it right this time:

    post_indexer_plugin=[3]
    content_types

    Array
    (
    [comment] => Array
    (
    [0] => comment
    )

    [post_type] => Array
    (
    [0] => post
    )

    )

    query_str=[SELECT c.blog_id as blog_id, c.comment_post_id as post_id, c.comment_author_user_id as post_author_id, c.comment_author as post_author_name, c.comment_author_email as post_author_email, p.post_title as post_title, c.comment_content as comment_content, c.comment_post_permalink as post_permalink, c.comment_date_stamp as post_published_stamp, c.comment_id as comment_id FROM g6h_site_comments c INNER JOIN g6h_network_posts p ON c.comment_post_id=p.ID AND c.blog_id=p.BLOG_ID WHERE 1 AND c.comment_approved = 1 ORDER BY c.comment_date_stamp DESC LIMIT 15]
    post_indexer_plugin=[3]
    content_types

    Array
    (
    [comment] => Array
    (
    [0] => comment
    )

    [post_type] => Array
    (
    [0] => post
    )

    )

    query_str=[SELECT c.blog_id as blog_id, c.comment_post_id as post_id, c.comment_author_user_id as post_author_id, c.comment_author as post_author_name, c.comment_author_email as post_author_email, p.post_title as post_title, c.comment_content as comment_content, c.comment_post_permalink as post_permalink, c.comment_date_stamp as post_published_stamp, c.comment_id as comment_id FROM g6h_site_comments c INNER JOIN g6h_network_posts p ON c.comment_post_id=p.ID AND c.blog_id=p.BLOG_ID WHERE 1 AND c.comment_approved = 1 ORDER BY c.comment_date_stamp DESC LIMIT 15]

  • Paul

    @Clive,

    No problem. So from the debug I can see it is properly recognizing the new version of Post Indexer. And the SQL query from the debug shows the new table g6h_network_posts used by the new version of Post Indexer which looks comparable to my own widget output. .

    Still no Comments output displayed by the widget?

    Any chance you can run the SQL command against your database to check if there are errors reported?

  • Paul

    @Clive,

    Haha. Well you need to access your hosts control panel or whatever. Most hosts provide access to phpMyAdmin which will let you then copy the SQL command (see below) and paste into phpMyAdmin and run it. Need to know if it returns rows or reports an error. I've tried on my end replacing the table name to match my own and it returns the expected rows.

    SELECT c.blog_id as blog_id, c.comment_post_id as post_id, c.comment_author_user_id as post_author_id, c.comment_author as post_author_name, c.comment_author_email as post_author_email, p.post_title as post_title, c.comment_content as comment_content, c.comment_post_permalink as post_permalink, c.comment_date_stamp as post_published_stamp, c.comment_id as comment_id FROM g6h_site_comments c INNER JOIN g6h_network_posts p ON c.comment_post_id=p.ID AND c.blog_id=p.BLOG_ID WHERE 1 AND c.comment_approved = 1 ORDER BY c.comment_date_stamp DESC LIMIT 15

  • Paul

    @Clive, Wanted to post more information here or suggestions for you to look into while I'm still away after your return.

    For Post Indexer under Network Admin > Settings > Post Indexer you should see a tab for 'Rebuild Index' Have you tried forcing a full rebuild? Of you switch back to the 'Statistics' tab you should see a box on the top-right showing what is being rebut. See my own image.

    Also on that 'Statistics' tab you should see a graph showing the sites and posts indexed.

  • Clive

    Hi Paul,

    I'm pretty sure I set the indexer off maybe 10 hours ago BUT I've updated it with a new version since then. I just set it off again and Stats comes back with:
    Current Queue Summary
    6 Sites in queue
    0 Sites currently being processed
    6 Sites awaiting processing
    0 Sites will complete processing on next pass

    Before I set it off to rebuild it was the same as above but with 5 sites rather than 6 - I created another site about 2/3 hours ago to play with various themes. In other words it doesn't seem to have done anything. Also, this space is under construction. With just two people and one very active site at the moment - with 250 wiki pages added in 2/3 weeks. Probably less than 500 pages /posts in total. So, there is not a massive amount to index.

  • Paul

    @Clive,

    Alright. Thanks. Maybe the rebuild will finish while you are asleep. On my own development site I have 5 site. I think 11 posts in all and it took almost 45 minutes to fully index. Not sure what is up with that. I think the issue is that the posts have not yet been indexed. So when Live Stream runs for the comments it is not finding the related posts yet.

    I'll check with you in the morning.

  • Clive

    Hi Paul,

    Current Queue Summary is exactly the same as above after 12 hours.

    WP error log contains the following (first has been repeated often the others just from this date and not too often)
    [05-Mar-2013 01:39:12 UTC] WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1 for query SELECT * FROM g6h_term_counts WHERE ( ) made by require('wp-blog-header.php'), require_once('wp-load.php'), require_once('wp-config.php'), require_once('wp-settings.php'), do_action('init'), call_user_func_array, Wiki->post_action, Wiki->edit_post, wp_update_post, wp_insert_post, do_action('save_post'), call_user_func_array, post_indexer_post_insert_update, post_indexer_post_terms_insert_update
    [05-Mar-2013 01:39:12 UTC] WordPress database error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ')' at line 1 for query UPDATE g6h_term_counts SET term_count = term_count + 1,term_count_updated = '2013-03-05 01:39:12' WHERE ( ) made by require('wp-blog-header.php'), require_once('wp-load.php'), require_once('wp-config.php'), require_once('wp-settings.php'), do_action('init'), call_user_func_array, Wiki->post_action, Wiki->edit_post, wp_update_post, wp_insert_post, do_action('save_post'), call_user_func_array, post_indexer_post_insert_update, post_indexer_post_terms_insert_update
    [05-Mar-2013 16:36:47 UTC] WordPress database error Table 'justbcsh_rh2afh.wp_network_posts' doesn't exist for query SELECT BLOG_ID, ID FROM wp_network_posts WHERE DATE_ADD(post_date, INTERVAL 1 YEAR) < CURRENT_DATE() LIMIT 50 made by do_action_ref_array, call_user_func_array, postindexercron->process_tidy_agedposts, postindexermodel->remove_posts_older_than
    [05-Mar-2013 16:37:18 UTC] WordPress database error Multiple primary key defined for query ALTER TABLE g6h_wiki_subscriptions ADD PRIMARY KEY (ID) made by require('wp-blog-header.php'), require_once('wp-load.php'), require_once('wp-config.php'), require_once('wp-settings.php'), do_action('init'), call_user_func_array, Wiki->init, Wiki->install, dbDelta
    [05-Mar-2013 16:37:30 UTC] WordPress database error Table 'justbcsh_rh2afh.wp_network_posts' doesn't exist for query SELECT BLOG_ID, ID FROM wp_network_posts WHERE DATE_ADD(post_date, INTERVAL 1 YEAR) < CURRENT_DATE() LIMIT 50 made by do_action_ref_array, call_user_func_array, postindexercron->process_tidy_agedposts, postindexermodel->remove_posts_older_than
    [05-Mar-2013 16:46:45 UTC] WordPress database error Table 'justbcsh_rh2afh.wp_network_posts' doesn't exist for query SELECT BLOG_ID, ID FROM wp_network_posts WHERE DATE_ADD(post_date, INTERVAL 1 YEAR) < CURRENT_DATE() LIMIT 50 made by do_action_ref_array, call_user_func_array, postindexercron->process_tidy_agedposts, postindexermodel->remove_posts_older_than

    I did set the new Post Indexer to just keep data for three months - this seems to be causing problems.

    I'm using Buddypress Corporate on the front end of my install and I'm also having issues with this although so far no one is interested in giving an explanation on this ticket here:
    https://premium.wpmudev.org/forums/topic/ordered-unordered-lists-missing#post-356329

    Also, you should read these past forum posts of mine (I'd forgotten that you'd helped me before with regards the post indexer)
    https://premium.wpmudev.org/forums/topic/indexing-page-comments#post-356233
    https://premium.wpmudev.org/forums/topic/comment-for-pages-problem#post-355998
    In other words I've always had problems indexing pages and page originated comments however LiveStream was working fine for presenting wiki and post comments which is what I was using it for when it stopped working completely.

    The BP activity not presenting page comments problem was a bp bug - it was solved with this 'fix' here:
    http://buddypress.org/support/topic/page-comments-in-activity-stream/

    So, like I've already said on previous support topics I installed WP-MS sites on two separate web hosts and got the same problem for the BP (and theme) issues just to make sure it was not something I'd done.

    At the moment Paul I'm happy using the 'Global Comment widget' to present the comments although the LS was better as it used up a lot less space (with the scrollbar option).

  • Paul

    @Clive,

    I'm not wanting to give up on this. But if you have a satisfying solution this is your call. I'm confident this is not a Live Stream issue but something with the Post Indexer not indexing things.

    From your error log one thing caught my eye. This line

    [05-Mar-2013 16:36:47 UTC] WordPress database error Table 'justbcsh_rh2afh.wp_network_posts' doesn't exist for query SELECT BLOG_ID, ID FROM wp_network_posts WHERE DATE_ADD(post_date, INTERVAL 1 YEAR) < CURRENT_DATE() LIMIT 50 made by do_action_ref_array, call_user_func_array, postindexercron->process_tidy_agedposts, postindexermodel->remove_posts_older_than

    You will notice the table name 'wp_network_posts'. This is wrong and appears to be wrong coded part of Post Indexer. The prefix 'wp_' should match your local system prefix 'g6h_' I'll alert the developer.

    I think the issue you are having with the new Post Indexer is related to the scheduling of the initial indexing. From what I can tell this used the WordPress WP_CRON facility which requires some front-end traffic to kick things off. You mentioned you are running this site was under construction. Which may not be enough traffic to kick WP_CRON into gear.

  • Paul

    @Clive, Perfect. I'm willing to continue.

    So can I ask that you install a Cron monitor plugin? I generally use Cron View http://wordpress.org/extend/plugins/cron-view/

    Once installed if you go to the Primary site Tools menu you will see 'What's in Cron' option. This will list out all the pending WP_CRON tasks. On my own I see s number of tasks for Post Indexer. At least that will let us know if the CRON tasks are properly setup on your site.

  • Clive

    WP debug on front page comes back with the following.

    1) Notice: add_custom_image_header is deprecated since version 3.4! Use add_theme_support( 'custom-header', $args ) instead. in /home/justbcsh/public_html/reality-hackers.com/wp-includes/functions.php on line 2839

    2) Notice: define() was called with an argument that is deprecated since version 3.0! The constant VHOST is deprecated. Use the boolean constant SUBDOMAIN_INSTALL in wp-config.php to enable a subdomain configuration. Use is_subdomain_install() to check whether a subdomain configuration is enabled. in /home/justbcsh/public_html/reality-hackers.com/wp-includes/functions.php on line 2923

    3) Notice: get_settings is deprecated since version 2.1! Use get_option() instead. in /home/justbcsh/public_html/reality-hackers.com/wp-includes/functions.php on line 2839

    4) Notice: Undefined variable: blog_info in /home/justbcsh/public_html/reality-hackers.com/wp-content/plugins/rps-blog-info/rps-blog-info.php on line 63

    5) Notice: Trying to get property of non-object in /home/justbcsh/public_html/reality-hackers.com/wp-content/plugins/rps-blog-info/rps-blog-info.php on line 63

    6) Notice: WP_Admin_Bar::add_node was called incorrectly. The menu ID should not be empty. Please see Debugging in WordPress for more information. (This message was added in version 3.3.) in /home/justbcsh/public_html/reality-hackers.com/wp-includes/functions.php on line 2959

    1) The first line error below seems to relate to Buddypress Corporate theme as this disappears on sites that don't use this theme.
    2) I'd appreciate you translate and can tell me what I need to use in the wp-config file?
    3) I've no idea what this relates to?
    4&5) I'll leave a message on the support page for this plugin.
    6) I've no idea what this relates to? Is this another 'theme' problem?

  • Paul

    To your post on the DEBUG message.

    2. See this thread https://premium.wpmudev.org/forums/topic/what-is-definevhost-yes-in-wp-config Basically, if you have an older wp-config.php where you have a line like

    define("VHOSTS', true);

    You need to change this to be

    define('SUBDOMAIN_INSTALL', true);

    3. The get_settings function is something else older. Chances are you have a plugin or theme calling that function.

    6. Again probably a bad/old plugin. You mentioned the theme is BP which I would not suspect having old code like that. Are you running the Ultimate TimyMCE plugin? A quick google on the phase brought up a page related to that plugin.

    On the Cron view output. Hmm. Try manually deactivating the Post Indexer plugin then reactivate it. Then check the Cron View. From there check the Network > Settings > Post Indexer to see if you still have pending indexing. If you don't have any pending then go to the tab to restart the indexing.

  • Paul

    @Clive,

    More thoughts on getting content into the Post Indexer (sorry for the multiple posts. Coffee kicking in finally).

    The bulk reindexing appears to be an issue with WP_CRON not starting the process. But you should be able to manually save a Post and it should show in the network_posts database table.

    Since you are trying to show comments (back to our original issue) via Live Stream. Try editing a post that matches the comments. See if that triggers something to write the data to the network_posts db table.

  • Clive

    Hi Paul,

    I've deactivated and re activated the Post Indexer plugin, accessed wp-cron.php directly I'm still just getting the last three post-indexer cron jobs and still missing the first two. Although a new text site I created and imported pages/posts/comments from another site on the same install seems to be being indexed. From what is being presented in the Post Indexer Statistics page.

    I'm now using: define('SUBDOMAIN_INSTALL', true); in wp-congig.php and that seems to work fine.

  • Clive

    Hi Paul,

    It's 'bizarre' - well it could only happen on a site called 'reality hackers' - I've a side bar with:
    Top - Recent NetworkWide Comment Widget
    Middle - Recent NWW Posts Widget
    Bottom - LS (posts)

    The top displays all recent comments (8) and the middle all recent (posts) including 'pages' the LiveStream won't display anything set to just comments with posts or posts and comments it displays the same BUT just what has been most recently indexed. In other words the Recent NetWorkWide Comment Widget
    and Recent NetWworkWide Posts Widget display recent changed / written things but LS is only getting recently indexed content.

    Question. The post indexer is set to 'Post' only as the 'default' post type do I need to add 'page' / 'wiki' etc - to get it to index more? What is included in the default 'post' setting and what is not included? You need some documentation for this?

  • Paul

    @Clive,

    So the Live Stream Widget is showing info. Woot!

    Question. The post indexer is set to 'Post' only as the 'default' post type do I need to add 'page' / 'wiki' etc - to get it to index more? What is included in the default 'post' setting and what is not included? You need some documentation for this?

    Well by default the Post Indexer is supposed to only gather 'post' post types. But yeah if you have other post types from other plugins you can elect to have those included. I agree this needs better documentation. Again, not my plugin. Post Indexer is supported by a difference developer. The Live Stream just uses the collected info.

    Interesting this on the Post Indexing. If your Network Admin area if you go to Network > Sites you will see the standard listing of sites. There is a new column showing the Post Indexer information. Click the 'edit' in that last column. This will bring up a model popup showing if indexing is enabled for that site. Plus a checkbox listing of post types.

  • Clive

    Hi Paul,

    Sorry been busy today. After going to Sites and editing the 'Indexing' option to specify what to index yesterday it's 'stalled' again and has done nothing. I disabled it from indexing two sites and that left 4 to index BUT it's again not doing anything.

    Is it not 'odd' that the network posts and comment widgets can pull the data from the post and comments files in the database while the LS widget is somehow only using what the post indexer indexes or not in real time. Most of the data is 'there'.

  • Paul

    @Clive,

    Is it not 'odd' that the network posts and comment widgets can pull the data from the post and comments files in the database while the LS widget is somehow only using what the post indexer indexes or not in real time. Most of the data is 'there'.

    I'm not sure how the other plugins function. The Live Stream Widget was written specifically to be a front-end to the Post Indexer and Comment Indexer plugins.

    I think it might be time to flag the Post Indexer developer and I'm out of ideas on this. From what I can tell the Live Stream Widget is doing what it does correctly. Just having trouble getting the Post Indexer to index things.

  • Barry

    I disabled it from indexing two sites and that left 4 to index BUT it's again not doing anything.

    Can you let me know what you are doing to check it's progress?

    What happens if you set one of the sites that is isn't doing anything on to rebuild its index?

    Don't we need to wait for the wp_ database referencing error to be changed anyway - we need to know how that will effect the functionality?

    That's in a tidy up cron job and shouldn't alter any behaviour with regards to indexing, but version 3.0.2 has just been released with the fix anyway

  • Clive

    Hi Barry,

    Settings / Post Indexer - is where I'm looking. Since I've just checked:
    a) It does seem to be indexing now. I've had quite a lot of comments posted in the last 24 hours that might have helped to get things moving.
    b) The 'LS' widget to display 'posts' now shows one post - a very old post that's been around for a while while completely missing newer posts.
    c) The 'LS' widget set to display 'comments' isn't appearing at all.

    I'm running what I thought was the latest Post Indexer plugin which is 3.0.1 - I've not had any updates to indicate that there is a more recent version.

  • Clive

    This is my BP front end sidebar with:
    Global Recent Comments - at the top
    Global Recent Posts - next down
    Livestream Widget Set to Comments - third place
    Livestream Widget Set to Display Posts (Not showing anything)

    Take a look at the comments. The Global Recent Comments has been listing comments from posts and wikis but not pages BUT it's not added any new comments for over a week. The Global Recent Posts is doing well, it's presenting all new or recently updated pages AND posts (but not wikis posts) - nothing wrong there.

    LiveStream Widget set to Comments only displays comments from wiki pages only and is missing recent comments on posts and pages.

  • Clive

    Hi Paul,

    Just done:
    post_indexer_plugin=[3]
    content_types

    Array
    (
    [comment] => Array
    (
    [0] => comment
    )

    [post_type] => Array
    (
    )

    )

    query_str=[SELECT c.blog_id as blog_id, c.comment_post_id as post_id, c.comment_author_user_id as post_author_id, c.comment_author as post_author_name, c.comment_author_email as post_author_email, p.post_title as post_title, c.comment_content as comment_content, c.comment_post_permalink as post_permalink, c.comment_date_stamp as post_published_stamp, c.comment_id as comment_id FROM g6h_site_comments c INNER JOIN g6h_network_posts p ON c.comment_post_id=p.ID AND c.blog_id=p.BLOG_ID WHERE 1 AND c.comment_approved = 1 ORDER BY c.comment_date_stamp DESC LIMIT 15]

    There was another 'broken' output of:

    post_indexer_plugin=[3]
    content_types

    Array
    (
    [comment] => Array
    (
    )

    [post_type] => Array
    (
    [0] => post
    )

    )

    I tried rearranging the plugs to get more but nothing worked. It's looks like there was output for comments and another for posts.

  • Paul

    @Clive,

    Ok. Received the login. Thanks again.

    And I see from the debug output things appears to be pulling some comments from the database. Though as mentioned in my previous comments from last week the comments query does a join on the post id because it needs to reference the post title. and here I'm saying post in the generic.

    Without seeing what is in the global comments table ( g6h_site_comments) and the network posts table ( g6h_network_posts ) it is hard to debug. Unless we are still under the assumption the Post Indexer is still not indexing some content.

  • Clive

    Hi Paul, I've deleted the original attempt and tried again

    It's a typical wp 'oddity' problem. I made up a username containing a '-' and it threw it out. Even through it presented all the 'right' things the second time - it's obviously not worked - probably because the same e-mail was entered BUT no error was presented.

    Yea I got the @.

    The sql database tables I think you want are in an e-mail heading your way.

  • Paul

    @Clive,

    Yea I got the @.

    Sorry I had to state that. you would be surprised the times I have to tell a member that. :slight_smile:

    So looking at the database dump you provided. In the posts table (g6h_network_posts) I see a bunch of rows for 'wiki' post type. But I only see one row for 'page'. And zero for 'post'

    I guess I need to take a step back. Instead of trying to see what is missing from all the blog + posts combinations. Can we just identify a post from a specific blog that we know has comment(s) then work through the investigation on making sure the needed entries are in the network_posts table then make sure the Live Stream query is correctly pulling the comment/posts data?

  • Paul

    @Clive,

    The 'Post/Post' is actually indexed and appears in Live Stream but no Page or Wiki.

    Correct. This is on purpose with Live Stream. In the old version of the Post Indexer we didn't want to include any other post types. Just 'post'. This will be added soon now that Post Indexer 3 is released.

    For the Comment output from Live stream it just tries to match the comment to the related post. Post here is generic meaning it doesn't care of the post is wiki or page or some other post type.

    Sorry I guess I'm confused on what is broken. I know this all started when you upgraded Post Indexer. I think I have too many other support threads going.

  • Paul

    @Clive,

    Mmmm I'm assuming as I'm entering a 'number' for the number of items LS should display that it'll display the most recent 'posts' up to that 'number'. So, is the Live Stream meaning of 'recent' only 'recent' as in 'time' and not 'recent' as in 'most recently posted'?

    For posts it shows the latest published posts. For comments it shows the latest comments in date order.

    Support chat is over. So looks like we can continue here.

  • Clive

    Hi Paul,

    Can we clarify?

    Am I right in thinking I should have the number of posts and comments being presented that I specify no matter how old any post or comment actually is on Live Widget?

    If you remember I had a forum thread posted a month ago:
    https://premium.wpmudev.org/forums/topic/comment-for-pages-problem#post-356024
    This was about Page comments not making it to Live Stream from what I recall from that thread I was given the impression page comments 'should' appear in Live Stream - they did for you but they are not for me.

    On that original thread I mentioned coming here because I was wanting all comments from all pages posts etc appearing on the front end. The comment indexer states: Indexes all comments across your network

    Which implies that it's indexing all comments which it does seem to be doing BUT this is misleading / irrelevant if the Global Comments Widget and or the Live Stream widgets doesn't actually work to present all comments.

    I think you or someone needs to get the developers together of these 4 different plugins and find out what does what overall and 'what is going on' with Live Stream with regards my install?

    And these members are having similar problems too:
    https://premium.wpmudev.org/forums/topic/post-indexer-not-showing-all-custom-post-types-to-index-when-indexing-per-site#post-384312
    https://premium.wpmudev.org/forums/topic/post-indexer-not-indexing

    I'd personally like to have a post widget that gives a tick box for each post / page / custom post type to display to have only these ones displayed and a comments widget with a tick box to select comments from post / page / custom source to only have those types displayed and Live Stream with tick boxes for everything.

  • Paul

    @Clive,

    Am I right in thinking I should have the number of posts and comments being presented that I specify no matter how old any post or comment actually is on Live Widget?

    Correct.

    On that original thread I mentioned coming here because I was wanting all comments from all pages posts etc appearing on the front end. The comment indexer states: Indexes all comments across your network

    As I stated in a previous reply and per the SQL query for the comment when you choose Comments it does a JOIN to make sure the Comment has a match in the Post Indexer table. This JOIN is needed to determine the post title. In this query there is no restrictions to limit comments for posts only. So if you have comments enables for Pages or Wiki or any other post type these comments should be picked up. I'll try and reverify that under my own development system this afternoon.

    I'd personally like to have a post widget that gives a tick box for each post / page / custom post type to display to have only these ones displayed and a comments widget with a tick box to select comments from post / page / custom source to only have those types displayed and Live Stream with tick boxes for everything.

    I'd like to stay focused on the issue and not muddle the water with feature requests.

    At this point I'm still trying to figure out the issue. You were running Post Indexer, Comment Indexer and Live Stream widget prior to the Post Indexer 3 update, correct? And things were displaying satisfactorily, correct?

    So we changed Post Indexer (And you are running a beta of Live Stream). Are we sure the post are all properly being indexed? Because without that Live Stream really cannot query what is not in the index.

  • Paul

    @Clive,

    Just looking at the LS widget output again on your home page bottom sidebar. The first LS appears to be showing Posts, so ignoring.

    The second LS Widget is showing comments. From these comments the post types are as follows:

    Post -
    Wiki -
    Wiki -
    Wiki -
    Wiki -
    Wiki -
    Wiki -
    Wiki -

    So decided to test something. I went to this page (?) and posted a comment http://foundation.reality-hackers.com The comment does not show on the LS Widget on the home page. So I'd like to ask you to see if this entry is in the Post Indexer table? If if you want send me the dumped tables again and I'll look into it. IF you send me the table I need to know what the page ID is. Though I can figure that out if needed.

  • Paul

    @Clive,

    Alright. So let's try this next. Where I left that comment. That is a page, correct? and you have 'page' setup for this site to be indexed for Post Indexer, correct? So what happens if you edit that page? Or better move it to draft than re-publish. Thinking this might somehow kick off Post Indexer to at least grab that page and add to the table.

    I'm lost really at this point. I'm thinking about changing the logic in LS Widget so it doesn't need the post indexer reference for comments. But that would not be a good solution.

  • Clive

    Hi Paul,

    Yes - agreed. I tried setting the alternate 'cron' engagement in config.sys - it doesn't seem to make much difference except it's added another 10 pages to the post indexer in the last 12 hours. I'll leave that as it is for the time being. Also, I think it's possible that as I don't have much traffic (15 people in there active) / there won't be many page views then cron won't be kicking off very often.

    BP network update which I assume does the same thing does it while you wait. Now I can imagine that if you've a very large network 1000's of sites 100's of pages on each with active traffic there might be a lot to index. On my 'larger' site the network update takes about 10 seconds. So, is it possible to have two modes for the post indexer? One that will just 'do it now' because I suspect that this is what it should be doing for small installations like mine? At the moment the post indexer isn't keeping pace with the few pages it's already got to deal with.

  • Paul

    @Clive,

    Good to hear we have made progress on this --- finally!

    it doesn't seem to make much difference except it's added another 10 pages to the post indexer in the last 12 hours

    Outside of the cron-based re-indexing the way I see it the indexing of individual items happens on edits. Just like the old version of the Post Indexer. Just like I had you edit the 'Start HERE' page. This change in status triggers the Post Indexer to at least update that item. Similar to how the current Comment Indexer functions.

    Going to leave the Post Indexer indexing issue for Barry to look into.

    So on related note for the Live Stream Output. I was looking at the way we are formatting the comments output and felt this needs to change. It just seems we have too much space. Wanted to get your thoughts since you are somewhat familiar with the usage.

    The current plugin outputs something like:

    [AVATAR] [link]Clive[/link] I'm assuming Matt that people will read the articles in order . . .... commented on [link to comment] Predictions, Probabilities, Anomalies & Statistics . . . [link] via [link to site home] Earth Simulation Articles . .[/link]
    [footer] 5 hours ago · comments [/footer]

    Thinking we should link the comment text to the actual content. Then the Post of the comment can then be linked to properly. Plus if the avatar is showing it seems redundant to also provide the linked name.

    The new format would be:

    [AVATAR] commented [link to comment] I'm assuming Matt that people will read the articles in order...[/link] on [link to post] Predictions, Probabilities, Anomalies & Statistics . . . [/link] via [link to site home] Earth Simulation Articles . .[/link]
    [footer] 5 hours ago · comments [/footer]

    To give more visual here are two images showing your site and my development site with the new format.

    Thoughts?

  • Clive

    Hi Paul,

    For Live Stream what you've put there looks good. At the moment there is no link to the actual comment itself (which I personally find annoying) - the link is just to the the blog or the page as it is on LS now or the comment list itself. I like the Recent Global Widget format - it gives a 'more' that links to the comment itself'

    The Recent Networkwide Post widget doesn't allow you to specify the number of words for the comment which would be useful.

    I like to keep things compact personally. For example for comments I'd be happy with something like
    [AVATAR] Commented: I'm assuming Matt that people will read the articles in order...[link to comment]More[link] on [link to post] Predictions, Probabilities, Anomalies & Statistics . . . [/link] via [link to site home] Earth Simulation Articles . .[/link]
    [footer] 5 hours ago · all comments [/footer]

    You could also get away with
    commented on [this post] on [this site] - 5 hours ago

    With
    [this post] and [this site] as tool tips links that display the post / site details with cursor over.

    Also Paul this 'Comment' in my LS Comment listing:
    Matt
    Matt Wiki creation as a function of understanding the problem How do we make wikis ? Problem - oriented ... published Research Understandings feedback into Clarity/Organization of Wikis via Reality Hackers Foundation & Guidelines
    2 hours ago · visit

    Isn't a comment at all - it's a post and it doesn't have any comments on it (although comments are open). Which is very odd. It's also displaying this particular 'none comment' twice at the moment with the second straight after the first? TWICE.

    Here is what I think Paul - most sites will have a new post and new comments every few minutes (or even seconds) - no one will notice if it's not working as it should because the 'turnover' on most sites will be high enough that you wont notice 'oddities' or missing new posts or comments because after a few minutes or a half hour you wouldn't expect to see them anyway.

  • Clive

    Also, Paul the 'Recent Global Comments Widget' isn't working right either in fact it's comment list has not changed for about two weeks. Not one of my or your test comments made it to that widget? It's not even presenting comments made to 'post' posts.

    My front page has the BP Activity Stream with everything on, and it's also got the WPMY Global Comment Widget - the WPMU Posts widget and LS displaying Posts and Comments - so you can easily see what is missing with respect to the BP listing.

  • Clive

    Hi Paul,

    In looking at the global comments widget there is a line that acts as a filter:
    $query = "SELECT * FROM " . $wpdb->base_prefix . "site_comments WHERE blog_public = '1' AND comment_approved = '1' AND comment_type != 'pingback' ORDER BY comment_date_stamp DESC LIMIT " . $options['recent-global-comments-number'];

    blog_public = '1' - does mean 'public' to the 'reader'?
    comment_type != 'pingback' - Does this mean that if pingbacks are disabled or not 'ticked' the comment won't be displayed? I tend to disable all pingbacks?

    The two 'filters' above are likely responsible for the differences in the presentations of my MS between the global comment widget and BP Activity as BP Activity as far as I know doesn't filter in these ways?

  • Clive

    Paul,

    Two things -

    1) I edited the Global Comments Widget to remove blog privacy and pingbacks - everything is showing now (in my terms 'as it should be').

    2) Rather than use the Recent Global Posts widget I was using Recent Networkwide Posts widget (Recent Posts function and Shortcode).to list the posts. So, if you check out the front of my site now you've every combination possible (as a test).

    To be honest this site HERE needs to group all the comments widgets together and explain all the differences before offering downloads - one has a title with feed in it that implies RSS feed (but doesn't) so it's very confusing what does what.

    It's as bad for Global / Networkwide Posts - there needs to be better documentation on these PARTICULARLY of the filters as these are obviously what has been responsible for 'missing' comments and posts on my install. So, even though I'm a super Admin and an admin of each site I've not been presented with ALL comments or ALL posts.

    As far as I can tell this issue is now 'resolved'

  • Paul

    @Clive,

    You have had a busy weekend. I'm trying to catch up on your posts. Just going to quote and comment from some of your recent replies.

    I like the Recent Global Widget format - it gives a 'more' that links to the comment itself'

    I think I would rather have the comment itself linked and not have the extra 'more'. But yeah I also do like the other widget output.

    Also, Paul the 'Recent Global Comments Widget' isn't working right either

    Unfortunately, I don't have direct support for that plugin. Different developer here.

    To be honest this site HERE needs to group all the comments widgets together and explain all the differences before offering downloads

    Totally agree. I'm just a contract developer given a project spec to develop from. We have many instances of duplicate but slightly different plugins for the same task. And as mentioned each version of the plugin seems to be supported by different developers. Which makes things harder. Some are very very old. I will mention this to the power that be and see if I can get coordination on efforts. If anything things should be combined as in my opinion would be less developer resources to code and support.

    Marking resolve. Thanks again for the excellent support on this. Let me know if you come across any other issues.

  • Paul

    @Clive,

    Sorry but I'm having sort of a brain issue this morning. I'm trying to finalize the code changes from three beta version of Live Stream. What I'm confused about is the specific output of the Live Stream Posts output. I just checked your site because I don't trust my own development site.

    On your site for Posts output the format appears to be

    [avatar] [name] [post content] published [title linked] via [site linked]

    This just seems odd to me. Specifically the post content showing before the title.

  • Clive

    Hi Paul,

    Go to the web site again and read 'Some Explanations' middle of the front page.

    The Posts / Articles / Research in the left tabbed menu each relate to the output of Recent Global Posts one widget incidence set to Posts the next to Pages and the next to Wiki. The Comments in this Tabbed list is presented with your 'Recent Global Comments widget.

    Below the chat in the middle column is the output of Recent NetWorkWide Posts below this are two incidents of LiveStream the first set to all posts from all sites and the second set to all comments from all sites. You've every combo covered there.

Thank NAME, for their help.

Let NAME know exactly why they deserved these points.

Gift a custom amount of points.