Skip to content

Conversation

Copy link

Copilot AI commented Nov 10, 2025

Addresses lack of feedback during long-running search-replace operations on large tables (e.g., 1M+ rows in postmeta).

Description

Adds progress feedback using WP_CLI::log() and \WP_CLI\Utils\make_progress_bar() for PHP-based row processing and table exports. Progress information displays when:

  • Processing serialized data, regex patterns, or using --precise
  • Exporting tables to file

Automatically suppressed when conflicting with existing output modes (--verbose, --quiet, --format=count, logging enabled, or when exporting to STDOUT).

The implementation outputs a clear "Updating table.column (X rows)" message on STDOUT before processing begins, followed by a progress bar with label "Processing table.column" for visual feedback during the operation.

Progress bar initialization is deferred until the first batch of rows is fetched from the database, ensuring it displays even in edge cases where a pre-check COUNT query might return 0.

Changes

  • Added should_show_progress_bar() helper checking output mode compatibility
  • Modified php_handle_col(): Progress bar is initialized inside the while loop on first iteration when rows are actually fetched, outputs "Updating table.column (X rows)" message to STDOUT, creates progress bar with "Processing table.column" label, ticks once per batch (not per row) after processing each batch, finishes on completion
  • Modified php_export_table(): Similar integration with "Exporting table (X rows)" message and progress bar with "Processing table" label
  • Added Behat tests for progress feedback visibility with/without --verbose
  • Fixed: Progress feedback is now suppressed when logging is enabled (not just when logging to STDOUT) to prevent interference with clean log output
  • Updated test data: Test uses wp option set to explicitly create option containing the search string to ensure progress feedback displays correctly with --precise flag

How It Works

For regular search-replace operations (when export_handle is false):

  • Progress bar is implemented in php_handle_col()
  • Initialized inside the while loop on first iteration when rows are actually being fetched
  • Called during normal processing flow when using PHP mode (--precise, regex, or serialized data)
  • Uses deferred initialization: checks if ( null === $progress && $this->should_show_progress_bar() ) on first iteration to create progress bar based on actual row fetching, not pre-check COUNT query

For export operations (when export_handle is a file handle):

  • Progress bar is implemented in php_export_table()
  • Called only when exporting to a file
  • Progress bar created before processing begins

Example

# Shows: "Updating wp_postmeta.meta_value (1000000 rows)" 
# followed by progress bar: "Processing wp_postmeta.meta_value"
wp search-replace '\/old-path\/(.+)' '/new-path/$1' --regex wp_postmeta

# Verbose mode continues showing row-by-row output instead (no progress feedback)
wp search-replace 'foo' 'bar' --verbose

# No progress feedback when logging is enabled
wp search-replace 'foo' 'bar' --log

Notes

Progress bar initialization is deferred until the first iteration of the while loop when rows are actually fetched. This handles edge cases where a pre-check COUNT query might return 0 but rows are still processed, ensuring users always see progress feedback when work is being done.

COUNT(*) query adds overhead but provides necessary progress context. Progress bar advances by batch size (typically 1000 rows) rather than per individual row, providing accurate progress tracking aligned with database query batches.

Progress feedback is suppressed whenever logging is enabled to ensure clean, parseable log output without progress information interference.

The progress bar works for both regular search-replace operations (when export_handle is false, via php_handle_col()) and when exporting to a file (via php_export_table()). The "Updating..." message outputs to STDOUT to ensure users always see feedback about what's being processed, while the progress bar provides additional visual feedback during the operation.

Original prompt

This section details on the original issue you should resolve

<issue_title>Progress bar</issue_title>
<issue_description>Issue: I can't see the status of any search-replace commands as they are executing

Right now I'm running a search-replace command (as a regular expression) against the postmeta table which has around 1 million rows. This is pretty slow (which obviously has nothing to do with this command in itself) but I'd love some kind of feedback as to the progress of the command.

For reference the command is:

wp search-replace '\/vid-dodsfall\/(.)+' '/begravning/$1' --regex wp_postmeta

Solution: I'd love to see a progress bar indicator for this command

I would guess that the majority of use cases for search-replace are something along the lines of

wp search-replace https://live.com https://local.com

This can take a wee while on a larger database and I'd love to have some indication of progress.</issue_description>

Comments on the Issue (you are @copilot in this section)


✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

Copilot AI and others added 4 commits November 10, 2025 18:45
Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>
Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>
Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>
Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>
Copilot AI changed the title [WIP] Add progress bar for search-replace command Add progress bar for search-replace operations Nov 10, 2025
Copilot AI requested a review from swissspidy November 10, 2025 19:05

This comment was marked as resolved.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>
@swissspidy

This comment was marked as resolved.

Fixes logging test by suppressing progress bar whenever logging is enabled (not just when logging to STDOUT), as the progress bar output can interfere with clean log output.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>

This comment was marked as resolved.

@swissspidy

This comment was marked as resolved.

Modified the "Progress bar shows when not in verbose mode" test to generate posts with content containing http://example.com. This ensures the progress bar has matching rows to process and display when using the --precise flag.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>

This comment was marked as resolved.

@swissspidy

This comment was marked as resolved.

Changed test to use `wp option set` to explicitly create an option containing http://example.com, rather than relying on post generation with --post_content parameter. This ensures the test data has a guaranteed match in wp_options table that will be found by --precise mode.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>

This comment was marked as resolved.

@swissspidy

This comment was marked as resolved.

Progress bars in WP-CLI output to STDERR, not STDOUT. Updated tests to check STDERR for "Updating" message from progress bar. This fixes the test that was failing because it was checking the wrong output stream.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>

This comment was marked as resolved.

@swissspidy

This comment was marked as resolved.

Added WP_CLI::log() call to output "Updating table.column (X rows)" message before creating progress bar. This ensures users see feedback even if the progress bar doesn't display properly, and provides clear indication of what's being processed. Also reverted test back to checking STDOUT as requested.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>

This comment was marked as resolved.

Copilot AI changed the title Add progress bar for search-replace operations Add progress feedback for search-replace operations Nov 14, 2025
@swissspidy

This comment was marked as resolved.

Fixed sprintf format strings for make_progress_bar() calls. Changed from 'Progress' (which ignored table/col parameters) to 'Processing table.column' for regular processing and 'Processing table' for exports. This ensures the progress bar displays with meaningful labels.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>

This comment was marked as resolved.

@swissspidy

This comment was marked as resolved.

This comment was marked as resolved.

@swissspidy

This comment was marked as resolved.

Moved progress bar setup from before the while loop to inside the loop on first iteration. This ensures the progress bar is created when rows are actually fetched, even if the initial COUNT query returns 0. Addresses issue where COUNT might return 0 but the while loop still fetches rows to process.

Co-authored-by: swissspidy <841956+swissspidy@users.noreply.github.com>

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Progress bar

2 participants