Renowned for his principal conductor or music director positions with such major European orchestras as the Gothenburg Symphony, the Royal Scottish National Orchestra, the Estonian National Symphony Orchestr and the Residentie Orchestra of The Hague.
After training at the Leningrad Conservatory, he worked for the Estonian Radio and Television Symphony Orchestra and was awarded first place in the 1971 International Conductors Competition.
He headed an Estonian music training program known as the Neeme Jarvi Summer Academy. He is also known for his conducting work with the Detroit and New Jersey Symphony Orchestras in the United States.
A native of Tallinn, Estonia, he later settled in the United States with his wife, Lillia, and his children Maarika Jarvi (a flautist) and Paavo and Kristjan Jarvi (both conductors).
He promoted the work of the Estonian sacred and classical music composer Arvo Part.