Maybe that's a silly idea, but if a soundfont must contain sounds in every "bracket", maybe you could put some dummy wavs containing nothing (0.1 sec of silence), just to fill the blanks. that way the size remains small.