I have replicated your finding. After clicking on Replace All, it popped up the Replacing Data window. I waited a short bit of time and then Cancelled the operation, only to have a JMP Alert window pop up stating that over 21 million occurrences had been replaced.

Since I was only Searching on column names, and it was supposed to be restricted to the 2 selected columns, I think this is truly a bug.
Please report this to
support@jmp.com
Jim