What is catastrophic backtracking in regex, and when should you not use a regex?
Suggested answer
The interviewer wants to know that you understand how a regex runs under the hood, that you can spot the risk when the data isn't under your control, and that you know where this tool stops being the right one.
Most regex engines, JavaScript's included, match by trying one path and, if it fails, backing up and trying another. For some expressions, especially ones with a repeat nested inside a repeat such as ^(\d+\.?)+$, the number of paths to try doubles with every extra character when the string doesn't match. That is catastrophic backtracking: a string only a few dozen characters long can make one line of code run for seconds, or even hours. When the regex runs on data sent by outsiders, it becomes a denial-of-service hole (ReDoS), and OWASP has a page of its own on the subject.
How to protect yourself: avoid nested repeats or two repeating parts that overlap, write the pattern so each character can be matched in only one way, limit the input length before running the regex, and test the expression with a long string that does not match instead of only trying strings that do. In places that matter, you can use an engine that guarantees linear time or has a time limit.
As for when not to use a regex: for structures that have state or nesting, such as splitting CSV cells that contain quotes or reading HTML, a loop over the characters or a library that reads the format properly is clearer and safer. A regex is a good fit for normalizing small pieces such as a phone number or a date, after you've split the cells correctly.
A bonus point: give an example of cleaning up messy data with a regex together with a report of the rows that were rejected, instead of silently skipping errors.