His family was never wealthy by Western world standards. When they lived on the island of Zanzibar and Freddie’s father worked for the British government it must have been like a middle class family from Britain living on their modest British salary but in a different economy so that the money went a lot farther. Also, his mother worked bringing in another salary and left him with an African nanny.
He was sent to a boarding school far away in India near Bombay so it was not as expensive as sending him to a boarding school in England. Still, it was a hardship even with both of their salaries. They could not afford for him to make the return trip home every year during the break so he sometimes had to stay with his aunt in India and not see his family at all. He seemed to develop some trauma over this having suffered separation issues his entire life according to Mary and others.
Freddie was sometimes perceived by those who didn’t know of his humble background as being upper crust. This is because the boarding school, as others, was designed to teach its students to fit in with the upper classes. There was even an emphasis on the “correct” mild British accent which the teachers were supposed to have and use with the students.
When revolution broke out in Zanzibar (which ultimately became a part of Tanzania as a result) people who worked for the British government as Freddie’s father did were targeted and thousands were killed in the fighting. This and the fact that they were already British citizens likely guaranteed them the opportunity to immigrate. In England they were lower middle class. But his parents never minded and refused to take money from Freddie or allow him to build them a nice big house. They were happy to stay in their modest home and never wanted to use Freddie for his wealth. Still, his will made them and his sister very wealthy in the end.