Digital Contact Tracing (DCT) has been proposed to limit the spread of COVID-19, allowing for targeted quarantine of close contacts. The protocol is designed to be lightweight, broad-casting limited-time tokens over Bluetooth Low Energy (BLE) beacons, allowing receivers to record contacts pseudonymously. However, currently proposed protocols have vulnerabilities that permit an adversary to perform massive surveillance or cause significant numbers of false-positive alerts. In this paper, we present AcousticMask, which encrypts broadcast messages using a key derived from the audio signal present at each device with sufficient security levels. Our results show that a receiver sharing the same social space as a sender will hear all of the sender's ephemeral IDs (EphIDs) with Hamming distance at most 3, which can be decrypted at the rate of 10 Hz on a Raspberry Pi 4, while achieving a security factor of over 2108against attackers in our testing set, showing AcousticMask is lightweight for DCT and provides sufficient security levels to protect user's privacy.