The picture of Queen Elizabeth entering Westminster Abbey on the arm of their disgraced son, Prince Andrew, overshadowed a memorial service honoring Prince Philip, leaving some around the monarch “sad.”

The Queen, 95, and Prince Andrew, 62, went together by automobile from Windsor to London for the Thanksgiving Service, when Andrew walked his mother down the aisle before taking their seats.

Andrew made the surprising decision just six weeks after settling a sexual assault lawsuit with his accuser Virginia Giuffre.

“People around the Queen are sad that all everyone was talking about is Prince Andrew and not Prince Philip,” a royal insider tells PEOPLE in this week’s issue.

There was unease among senior members of the royal family regarding the Queen’s decision, according to royal analyst Robert Jobson, “but she insisted.”

“It shows she wholeheartedly loves and believes her son,” Jobson, the author of Prince Philip’s Century. “As she did when she made a statement about Camilla being Queen’s Consort, many people will now accept the Queen’s word and judgment.”

“It does make some sense that he accompany her because he doesn’t have a partner. A settlement has been paid but he’s guilty of nothing in the eyes of the law,” he adds. “She has faith in Andrew. Even if he disappears from public life, he’s been able to pay tribute to his father, who after all, was very proud of his service in the Royal Navy, where he fought in the Falkland Islands conflict.”