Joe O'Neil arrived at Pittodrie in 1950 following the path of Ian McNeill and made his first team debut some months later. His finest hour for Aberdeen came in the Scottish Cup semi final against Rangers at Hampden on 10th April 1954. Joe had suffered a depressed fracture of the skull in a game against Falkirk only three weeks earlier yet declared himself fit for the game against Rangers, against doctors advice. Joe scored a hat trick in the Dons memorable 6-0 win the clubs first win over Rangers in the Scottish Cup and Rangers worst defeat ever in the cup to date.

Joe was injured for the Final against Celtic but played his part in the Dons first championship success in 1955. He was a giant of a player, an inside forward who competed with Bob Wishart for a starting place in the championship side. After failing to gain a regular place in the Aberdeen team Joe followed manager Dave Halliday to Leicester in 1956 where he had to wait 18 months before finally making his debut for the First Division club.

Joe eventually joined Bath City where he met up with former Don Ian MacFarlane. Joe scored regularly for the non-league side and he was among the goals during their FA Cup run of 1960. Bath were eventually knocked out by Brighton where he came up against his good friend Ian McNeill before a record 18,020 crowd at Bath's Twerton Park. Following a stint in the National Service at Shoeburyness he returned to Bath as manager and took up residence there to enjoy his retirement.