Atkinson was born in Durham. His father, a lieutenant, died when the boy was an infant. He attended Kirby Hill School in Yorkshire, preparing for a career in the ministry; but was removed from school when his mother became a Quaker. He was then apprenticed to a surgeon and apothecary in Newcastle. In 1789, on the expectation of inheriting property, he and his mother moved to Ireland, where he published two books of poetry.