Fixing Sitecore Publishing Service's Indexing Issue

If you're experiencing long wait times for indexing when publishing via the Sitecore Publishing Service, especially when publishing a large number of items (about 1000 to 1500), this could be caused by the Sitecore Web Index being frequently triggered. When this occurs, the entire index initiates, and content dependent on Solr may not be displayed until indexing is complete.

To resolve the issue, I conducted an investigation and found that the default remoteEventCacheClearingThreshold parameter in the publishEndResultBatch pipeline was causing the problem. This parameter is set to 1,000 items, and if the pipeline publishes more than 1,000 items in a single operation, a separate event is triggered causing a complete index rebuild, leading to significant delays in Content Search updates for large sites, requiring several hours to rebuild search indexes.

To address this issue, I updated the remoteEventCacheClearingThreshold parameter to 15,000 based on specific requirements. Although there is no recommended optimal value for this parameter, it should be updated as per your requirements to ensure smooth indexing.

I have provided a patch file that includes the updated parameter value to help you fix the issue. You can download the patch file and use it to update the parameter value as needed. This should help reduce the lengthy wait times for indexing and improve the overall performance of your Sitecore environment.

<?xml version="1.0" encoding="utf-8"?>
<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/"
    xmlns:role="http://www.sitecore.net/xmlconfig/role/">
    <sitecore role:require="Standalone or ContentManagement">
        <pipelines>
            <publishEndResultBatch
                argsType="Sitecore.Publishing.Service.Pipelines.BulkPublishingEnd.PublishEndResultBatchArgs">
                <processor
                    type="Sitecore.Publishing.Service.Pipelines.BulkPublishingEnd.RaiseRemoteEvents, Sitecore.Publishing.Service">
                    <param name="remoteEventCacheClearingThreshold">15000</param>
                </processor>
            </publishEndResultBatch>
        </pipelines>
    </sitecore>
</configuration>

I hope this information helps you resolve the Sitecore Publishing Service issue causing a lengthy wait for indexing. If you have any questions or require further assistance, please don't hesitate to contact me.

Reference: https://mikael.com/2019/07/learnings-from-a-year-of-implementing-sitecore-publishing-service/