New homes in Bromham could put an activity centre of “national importance” at risk if the plans go ahead, a local youth leader has said.