Truth be told, he doesn't really even believe it now.
"It's like 15 years in the blink of an eye, It's some memories that I'll be unpacking for another 20 years after, you know?" Ray tells "Ken Dashow's Beatles Revolution" podcast.
Of course, unlike in The Beatles' democracy, McCartney is the boss of his solo band and everybody in the band is there at his whim and with his blessing.
But if you've seen McCartney live on his One on One Tour, you won't see a group of musicians afraid of being fired at any moment. The band is one that plays with all the joy and rock 'n' roll abandon of five people (including McCartney) living their dream.
"We're in a pretty inspired place right now," Ray says. "Paul has just really got a lot of fire in his belly, and it's evident. He came into rehearsals just like really going for it. It's inspiring to us because of him."
That's not to say it's an easy job.
Every show is around three hours long, the setlist is pulled from a discography of hundreds of songs, many of which were never meant to be performed live.
"It's a big responsibility," Ray laughs. "I remember for the first part of our touring life, I basically was so busy and intent on getting it right that I just didn't even look up for six months. I mean, I could see [McCartney] out of the corner of my eye, [but I kept telling myself] 'Not gonna do it, not gonna look. I'm here doing my job.'"
The blond in the Paul McCartney live band, Ray plays lead, rhythm and slide guitar where applicable; he plays bass guitar whenever McCartney is playing something else, he plays acoustic guitar during the more intimate moments of the show and he sings harmonies on nearly every song. It's massive task to say nothing of the fact that Ray doesn't know what instrument McCartney will ask him to play until rehearsal.
"I remember my first preparation...at home. I had an acoustic guitar, acoustic 12-string guitar, electric guitar and a bass all on stands in front of me, a mic stand and a stack of CDs...And I would just go through this list and other songs that I thought could appear on a list later and just prepared for anything that could come my way, hoping that I would've covered whatever it is that he would need."
Even after more than a decade-and-a-half, Ray says he never takes his gig for granted. And one of the most energizing things is that neither does McCartney.
"Paul never rolls his eyes that we're playing 'Hey Jude' again," Ray says. "He never once has made one remark or one [bored] face. So in turn we're inspired and we're excited as well."
Listen to the full interview with Brian Ray below and subscribe for more of "Ken Dashow's Beatles Revolution."
Get McCartney's One on One tour dates here.
Photo: Getty Images