This is already happening.
Scammers use AI to enhance their family emergency schemes
A few months ago a contractor for the utility was installing a free transfer meter and I was hanging around chatting with him as he did it. He told me his grandson had texted him asking for money and said that he didn't sound as respectful as normal. I told him that it's a common scam and that he should call the grandson and talk to him to verify.
That works now, but might not for long. If the scammers can make the phone call go to them instead of the grandson they can use ai voice cloning.
I told my wife about ai voice cloning the other day and she said we need to authenticate with trivia only we know, like the location in the house where the cat threw up last.