Improvisation is usually defined as the composition of music while simultaneously singing or playing an instrument. In other words, the art of improvisation can be understood as composing music "on the fly". Improvisation can take place as a solo performance, or interdependently in ensemble with other players. When done well, it often elicits gratifying emotional responses from the audience. One notable improvisational pianist is Franz Liszt. The origins of Liszt's improvisation in an earlier tradition of playing variations on a theme were mastered and epitomized by Johann Sebastian Bach, Wolfgang Amadeus Mozart, and Ludwig van Beethoven.

Other notable improvisational musicians include: Keith Jarrett, an improvisational jazz pianist and multi-instrumentalist who has performed many completely improvised concerts all over the world; Derek Bailey, an improvisational guitarist; and Eugene Friesen, an improvisational cellist.

When jazz guitar players improvise, they use the scales, modes, and arpeggios associated with the chords in a tune's chord progression. The approach to improvising has changed since the earliest eras of jazz guitar. During the Swing era, many soloists improvised "by ear" by embellishing the melody with ornaments and passing notes. However, during the bebop era, the rapid tempo and complicated chord progressions made it increasingly harder to play "by ear." Along with other improvisers, such as saxes and piano players, bebop-era jazz guitarists began to improvise over the chord changes using scales (whole tone scale, chromatic scale, etc.) and arpeggios. Jazz guitar players tend to improvise around chord/scale relationships, rather than reworking the melody, possibly due to their familiarity with chords resulting from their comping role. A source of melodic ideas for improvisation is transcribing improvised solos from recordings. This provides jazz guitarists with a source of "licks", melodic phrases and ideas they incorporate either intact or in variations, and is an established way of learning from the previous generations of players.