Mansur was born on 1 January 1911, at Mansur, a village five kilometres west of Dharwad, Karnataka. His father, Bheemaraayappa, was the village headman, a farmer by occupation and an ardent lover and patron of music. He had four brothers and three sisters. His elder brother Baswaraj owned a theatre troupe, and thus at age nine Mansur did a small role in a play.
Spotting the talent in his son, Mallikarjun's father engaged him to a travelling Yakshagana (Kannada theatre) troupe. The owner of this troupe took a liking to the tender and melodious voice of Mallikarjun and encouraged him to sing different types of compositions during the drama-performances. Hearing one such performance, he was picked up by Appaya Swamy under whom he had his initial training in Carnatic music. Sometime later, he was introduced to Hindustani music under Nilkanth Bua Alurmath of Miraj who belonged to the Gwalior gharana. The latter brought him to Alladiya Khan (1855?- 1946), the stalwart and the then patriarch of the Jaipur-Atrauli gharana, in the late 1920s, who referred him to his elder son, Manji Khan. Following Manji Khan's untimely death, he came under the tutelage of Bhurji Khan, the younger son of Alladiya Khan. This grooming under Bhurji Khan had the most important influence on his style of singing.
Mansur was well known for his command over a large number of rare (aprachalit) ragas such as Shuddh Nat, Asa Jogiya, Hem Nat, Lachchhasakh, Khat, Shivmat Bhairav, Bihari, Sampoorna Malkauns, Lajawanti, Adambari Kedar and Bahaduri Todi, as well as his constant, mercurial improvisations in both melody and metre without ever losing the emotional content of the song. Initially, his voice and style resembled that of Manji Khan and Narayanrao Vyas, but gradually he developed his own style of rendition.
He received all three national Padma Awards, the Padma Shri in 1970, Padma Bhushan in 1976, and Padma Vibhushan, the second highest civilian honour given Government of India in 1992. In 1982, he was awarded the Sangeet Natak Akademi Fellowship, the highest honour conferred by the Sangeet Natak Akademi, India's National Academy of Music, Dance & Drama.
Mansur wrote an autobiographical book titled Nanna Rasayatre (Kannada: ನನ್ನ ರಸಯಾತ್ರೆ) in Kannada, which has been translated into English as a book titled My Journey in Music by his son, Rajshekhar Mansur.
To mark his birth centenary, a three-day music festival was organised in Dharwad and Hubli from 1 to 3 January 2011, wherein singers from across India performed and performances were held at the Kariyamma Devi temple premises at his birthplace Mansur village. His ancestral home in Mansur was also converted into a memorial.
In 2013, a five audio CD collection, "Akashvani Sangeet" of his music including rare "Vachana Gayana" renditions, was released by All India Radio archives at a ceremony held at Srijana Rangamandir at the Karnatak College Dharwar campus.