The People source also said that they will have a wedding party later.
Additionally, TMZ is reporting that Bieber and Baldwin are most likely getting married out of the country, as early as next week and possibly in Canada, where the pop star recently purchased a new home.
'They're going to have a big blowout, in front of God and everyone they love, ' said the insider, who has been called 'a religious source'.
The report reveals that Bieber and Baldwin got married in a courthouse on Thursday.
The outlet also claims that eye witnesses saw the singer crying inside the courthouse, where he told Baldwin, "I can't wait to marry you, baby".
This news comes just two months after Bieber proposed to the model while they were vacationing together in the Bahamas.
And though we might have to wait a while longer until we see Bieber and Baldwin officially walk down the aisle, their recent court wedding at least gave us a clue to what Baldwin will be wearing on her big day.
As for the ceremony, Baldwin is planning to have her sister, Alaia Baldwin, be her bridesmaid.
"I think having it in the woods would be so lovely", she said. "I think having it in the woods would be so attractive". And maybe Justin's little sister as a flower girl.
According to a source at The Cut, Hailey sees "no reason to wait".
"I said to them, 'You'd stop me if you didn't think this was the right decision, right? You are the love of my life Hailey Baldwin and I wouldn't want to spend it with anybody else".