Kay was born on the Isle of Arran, but after attending Glasgow School of Art he chose to settle on the mainland, maintaining a studio in Glasgow and living in Dunbartonshire. He was always fascinated by the sea and shipping and is perhaps best known for his powerful, dynamic pictures of ships at work on the Clyde. His informal, colourful style was ideal for capturing the bustle and drama of the docks, the hooting of the tugs and the rush of the ship-wakes in the harbour and he used it in many other river and coastal scenes of Holland, the North of France and of course on the Thames. He often worked in a distinctive mix of pastel, gouache and watercolour, though latterly he concentrated on pure oil and watercolour.
His landscapes and street scenes were no less popular and his lively, bright, dramatic paintings earned him a considerable reputation both here and abroad, particularly in France where he spent much of his time.
Exhibited at the Paris Salon in 1874 , RA (1884-1904), RSA (1884-1943) and RGI (1884-1943). Elected RSW (1896), ARSA (1933) and RSA (1938).