Born in Pittsburgh, Pennsylvania, Benson began performing as a young teenager, singing in nightclubs and recording several cuts for RCA. In 1960, Benson formed a rock band, but he soon became more interested in jazz, awed by performers like Charlie Parker and Charlie Christian. After a stint in Brother Jack McDuff's Band, Benson formed his own jazz group in 1965, recording several albums for Columbia and appearing on Miles Davis's Miles in the Sky. In 1967, Benson moved to Verve and then to producer Creed Taylor's own CTI label for much of the 1970s.

In the late 1970s, Benson recorded with Warner Bros., focusing on his soulful Nat King Cole–inspired singing. Beginning with the Top 10 1976 album Breezin' and its hit single “This Masquerade,” Benson began producing a series of pop albums, working with producer Quincy Jones on what was to be Benson's biggest album yet, Give Me the Night (1980).

Following a similar career path to Wes Montgomery, Benson lost touch with hard-core jazz fans, but he found himself becoming more popular and commercially successful with a broader audience. By the 1980s, Benson's vocal talent was better known than his amazing guitar playing. However, by the early 1990s, he began to re-emphasize his jazz guitar ability.