The characteristic that is common among vertebrates is breathe with lungs or gills.
Vertebrates are a diverse group, and while they may not all have the same breathing apparatus (some breathe with lungs, while others use gills), this characteristic is applicable to all vertebrates. The other options—being supported by an exoskeleton, making their own food, and eating with teeth—are not universally applicable to all vertebrates.