It happens to me most often when I'm responding to an after hours call from a patient, or during a scheduled phone appointment during a workday.
I hear background noise and ask if the person with whom I'm speaking is actually, at that moment, driving. If they're the passenger in a car, fine, I'll go ahead and speak with them. If they're driving, I ask them to pull over.
A report from the American Automobile Association, updated just this month quantifies something I had "heard" before, but couldn't remember the source. In their report they review several independent studies of driver distractibility. We have all seen the ads (available for us on YouTube) showing mock driving instructors, exams, or--the worst--dramatized crashes when the driver is texting or posting cute pictures of their sleeping toddler in the seat behind them.
Various driver non-driving tasks were taught to the participants in the current study, and they were given a week to practice. Didn't help much. It turns out that almost everything a modern driver takes for granted is highly distractible to the task at hand of driving. Compared to a baseline of solo drivers who were doing nothing extra, it seems that listening to the radio or even an audio book is actually not very distracting. Forget about doing anything else, however: it's moderate to severely distracting to talk to another passenger in the car or use a hands-free cell phone. The worst thing to do is to use speech-to-text systems to send and receive messages, and hands down, folks over 50 just didn't do as well as younger drivers.
So when you're driving, limit yourself to listening. Surprise to me, but I get it: a conversation with a fellow passenger better be fairly low key or it becomes as distracting as a cell phone interaction.
If something else is required, stop driving. I think causing an accident would be even worse than being in one.
Be safe, for your safety and sanity.