A little while back I depped almost unrehearsed with a Celtic rock band doing largely original songs and having been hired at about 24 hours’ notice. When I came off stage, their regular fiddle player (out of action with a collarbone broken in three places), assessed my performance with the phrase ‘That was good jazz.’
It wasn’t jazz of course, but it had been largely a set of me improvising on the fly from knowing what key the songs were supposedly in (written, at my request, on my set list!), and listening hard. Improvisation, particularly in a lead-melodic type fashion, is a necessary skill for some of what I do, yet also one that is harder to consciously improve, or to be sure I am actually improving at, than, say, sight-reading, high notes or special technique effects.
It’s absolutely true, and I often repeat it, that improvisation requires practice; even requires it more than playing the same thing each time, whether written or memorised. After all, for the latter you only have to play what your memory recalls or is written down in front of you; to improvise you have to about as much brainwork as that, in the shape of performing your improvisation, while simultaneously coming up with the notes themselves. If it’s harder, you need to do it more to get equally good at it.
However, few contexts require, or allow, absolutely free improvisation. In fact (strengthening the tie to jazz) most Western improvisation except for the work of the top rank of church and cathedral organists takes place over a more or less rhythmically and harmonically prefixed accompaniment – ‘changes’ in jazz, or chords and/or riffs usually carried over from another section when improvising in blues / rock / country / folk-rock / soul / etc. tracks. This creates problems for practising in the usual sense of the word – something that sounds great by itself may rely on holding a chord for an extra bar, or speeding up, or disregarding the harmony altogether, or many other possibilities that I certainly wouldn’t notice myself doing if I was engaged in getting to grips with soloing on a particular song at the same time. It then becomes effectively necessary to rehearse improvising for a particular setting (ie have the accompaniment with you, ready to shout if you’ve gone astray! or more seriously so you can hear when you’ve parted ways), rather than trying to practise it alone in the way most technical problems are usually solved. The only plausible way round this would seem to be to make or get yourself some kind of backing track that can be used to play over without convening lots of rehearsals.
Finally, there is the question of how to consolidate or expand one’s improvising ‘toolkit’. Playing simply notes from the chords will rather rapidly lead to exhausting the useful possibilities of arpeggios; embroidering the melody can just as quickly get stale unless the variations are pushed further and further, in which case it is necessary to know where they can be pushed without becoming (whatever this may mean in context!) ‘wrong’ rather than interesting.
Modern jazz has developed whole (usually contradictory) systems of theorising improvisation and alteration: modes (one per piece where applicable, or one per chord in almost anything!), chord substitutions and standard additional notes to given chords, voice leading, etc. Textbooks are available on these, and I believe most really well-known jazz players do indeed study, learn and apply them. However, there is a disadvantage that a great deal of bebop and its successors into avant gardes (much less so the modal / cool / Latin strands) sounds dissonant and unpleasant to unaccustomed audiences, and would be spectacularly jarring if spliced into a rock or blues song or an acoustic guitar interpretation of an Irish ballad. In a real sense, most non-jazz improvisation in my current musical world requires something less clever than Coltrane, Bird or Eric Dolphy – at least the vast majority of the time.
There is also, of course, a whole industry in and beyond jazz of painstakingly transcribing recorded solos note for note, so that they can be played verbatim by covers acts and/or adoring bedroom amateurs, or used for analysis and imitation by other soloists. (My impression is the former is largely the preserve of rock guitarists, and the latter a key process for the serious jazz student.) Ignoring playing someone else’s solo which is clearly not improvisation, the initial problem with modelling improvisation on someone else seems to me to be, how do you marshal that much information in real time? In the same way that it requires a lot of as it were processor power to allocate a mode to the chords being read or remembered and then use that as a basis for melodic construction, or to decide in advance what alterations will be made to the basic chords so as to use substituted ones, so too it seems a lot to ask to recall ‘ooh, a ii-V-I cadence coming up; maestro X used this figure for ii-V-I progressions in some places; in this key it would be this:’ and play it, without either dropping out for the previous few bars to get the idea together, or not reaching the relevant point till well after the rhythm section had passed through it. It is presumably achievable, but certainly requires a dedication of time and effort not possible to me while requiring a part-time desk job to stay financially afloat and having to keep several other (mostly more immediately lucrative) musical/technical balls in the air at the same time.
Perhaps this comes down to the truth that complex improvisation is difficult. For the generalist winging a lot of things like myself, there are few or no shortcuts to good work; and without shortcuts it is too difficult to fit into the time and energy available. In that sense it joins things like recital-standard technique, or classical composition, as potentially useful but unattainable partly for exactly the reason that I am already playing for a living. There’s a reason most people achieve a professional standard of such things while students; but I won’t be going off as a mature conservatoire student in any immediately foreseeable future.