Our goal is to make a computer program that interacts with human improvising musicians to automatically co-author music in real time using machine learning. This real-time interactive system will contribute to and draw from already existing branches of study in music composition and computer science. From computer science, the system will apply techniques from Music Information Retrieval (MIR) and Machine Learning to analyze and generate musical content. Within the domain of music composition, our piece aims to develop an interactive digital framework for gesture-based music improvisation. This latter point distinguishes the project from previous work in machine improvisation/composition which, for the most part, has historically attempted to mimic musical styles by generating melodies. In contemporary practices of music improvisation and performance, melody is not the sole descriptor of a musical style. Performers communicate more with musical actions and sonic events within a conditioned space, rather than pre-established harmonic movement or melodic movement.