Listening devices such as hearing aids can use remote microphones placed near a distant talker to improve intelligibility in noisy environments. Most commercial remote microphone systems are designed for a single talker and they remove spatial cues that help listeners to localize sounds. Here we propose an assistive listening system that enhances speech from multiple talkers while preserving the acoustic effects of the environment and the spatial cues of each sound source. The system adaptively filters the remote microphone signals to match the signals from the earpiece microphones. Because it uses the on-ear microphones as references, the adaptive system does not need to localize or separate the sound sources and can track changes as the listener and talkers move. We compare two adaptive implementations, one that processes the microphones jointly and one that adapts each separately. The system is demonstrated experimentally with up to three remote microphones and moving talkers and listeners.