Harmony.

Melody is the part that you sing (essentially). Harmony is the chords underneath. If you mean you want to take away all tones, then rhythm.

But music is not a pattern of vibes. Music is a pattern of many things, including melody, harmony, rhythm, dynamics, all imposed over a steady beat. I think your premise is flawed.