One of Audley's first memories was at a funeral when he was just two years old. This was in Kingston, Jamaica, and Audley remembers seeing everyone standing around the grave. He asked his mum, "Why is everyone crying?". She replied, "Because daddy is dead."

Audley's father, a foreman who worked in construction, had died young from kidney failure.

In 1956, Audley's mum and a friend decided to take advantage of the British Government's scheme to address the labour shortage in the UK and travelled by ship to find work in England.

The voyage was on an Italian cruise liner. The women stayed in their cabin for much of the time because most passengers were young men who would pester them. There was also a mutiny on board because the Jamaican passengers hated the Italian food. A compromise was reached when some passengers entered the kitchen and taught the crew how to cook food the Caribbean way.

Audley and his two sisters stayed in Jamaica with their aunt Mae and uncle Fitzy, who had fought for Britain in the RAF during WW2 and lived in the hamlet of Porus in the Jamaican countryside.

Audley's mum found work as a seamstress and a room to rent in Peckham. After two years of saving up for their flight, she sent for her children.   Audley doesn't remember much of the plane journey but remembers the small room he, his two sisters and his mum had to share when they first arrived.  Audley was the first black Child at his infant school. He said he didn't experience any racism whatsoever amongst such young children.

The Housing Act states that children of the opposite sex should not share a room beyond the age of 11. When Audley reached this age, his mum, with the help of the family doctor, wrote to Wandsworth Council, who gave them a council house to rent. Audley got his bedroom. Not sharing a room with his mum and sisters was a luxury. It meant he had to walk 3 miles to school daily, but this was a small price for finally having somewhere nice to live.

Audley then went to senior school. There were other West Indian kids there, and he remembers that all black kids were automatically put into the bottom stream regardless of ability