Why the honest answer depends entirely on which caterpillar you have
"Caterpillars defoliate trees" is true of several very different insects with very different consequences, and the marketing that lumps them together is the actual problem. Eastern tent caterpillar builds obvious silk tents in branch forks in spring; fall webworm builds looser webs enclosing leaves later in summer. Both look alarming and both are, for an established healthy tree, a cosmetic, one-season event. Spongy moth (Lymantria dispar, formerly gypsy moth) and winter moth (Operophtera brumata) are invasive species that feed without building an obvious web, and their risk is real specifically because they return, year after year, on the same trees.

