King Charles Is Furious Over Prince Andrew’s ‘Chinese Spy’ Scandal but ‘Can’t Sack’ His Brother: Report

King Charles is reportedly furious over the Chinese spy scandal involving his brother Prince Andrew, according to multiple reports.

A friend of the palace told The Daily Beast that the royal’s patience is “wearing thin” tied to allegations that the Duke of York, 64, had close business dealings with a Chinese national, who is alleged to be a spy.

“He has stood by Andrew for many years, but everyone has their limits. I can’t see the King wanting to walk to church with someone who was friends with a Chinese spy,” the insider told the outlet, referring to Christmas Day plans for the royal family in Sandringham.

Another inside source stressed the nuances of the complex family situation, telling The Mail on Sunday, “The palace has behaved the best it possibly can with a difficult relation. Every sanction that can be imposed has been, but you can’t divorce or sack your brother from being your brother. There will always be a bond of blood, and all families often have difficult relatives to deal with.”

“Of course, it is understood that it looks bad for the entire family, but that is because of one individual,” the source continued.

Can’t get enough of PEOPLE’s Royals coverage? Sign up for our free Royals newsletter to get the latest updates on Kate Middleton, Meghan Markle and more!

According to the U.K. newspaper The Times, the 50-year-old businessman alleged to be a spy — who has not been named — had been appointed as a “business adviser” by Andrew and was promised access to the highest levels of the royal family “in an apparently serious breach of national security.”

A letter found on one of the man’s electronic devices from an adviser to the Duke told the alleged spy that he was one of the Duke’s closest confidants and that he sat “at the very top of a tree that many, many people would like to be on,” per the BBC.

Additionally, it was discovered that the man was in possession of a briefing document that appeared to describe Andrew being in a “desperate situation and will grab onto anything,” according to the outlet.

Andrew’s alleged dealings with a spy create further complications for the palace in light of his recent public financial challenges.

King Charles, 76, reportedly cut off Andrew’s allowance this summer and told him he needed to vacate the Royal Lodge, a massive property on the Windsor estate.

Andrew, however, was reported to have found the funds to stay in the residence, which he shares with ex-wife Sarah Ferguson, the Duchess of York. The source of the funds remains unknown.

Andrew and Fergie, 65, along with their two daughters, Princess Beatrice and Princess Eugenie, and their families, are expected at the annual royal family Christmas gathering in Norfolk later this month.

Prince Andrew stepped back from public duties in 2019 after his bombshell interview with the BBC about his ties to Jeffrey Epstein. During the interview, Andrew also said he had “no recollection of ever meeting” Virginia Roberts Giuffre, who alleges she was forced to have sex with the royal three times between 1999 and 2002, including on a private Caribbean island owned by Epstein, when she was 17 years old.

The royal has repeatedly denied any wrongdoing and announced his step back from royal duties a few days after the interview aired. Andrew said he continued to “unequivocally regret my ill-judged association with Jeffrey Epstein” in the official statement announcing his decision.