Wasps aren't waiting out winter. They're gone.
Almost every wasp alive in October is dead by December, and that includes the queen who founded the colony. Yellowjacket and paper wasp colonies are strictly annual — University of Minnesota Extension, Iowa State University Extension, University of Nebraska–Lincoln, and Michigan State University Extension all describe the same pattern: a single queen starts a nest in spring, the colony grows through summer to its peak population, and then the entire structure dies at the first sustained freeze. There is no version of this animal that sleeps through winter as a colony and wakes back up.
What actually persists is a small number of individually-scattered young queens who mated in late summer or fall and left the nest before the freeze arrived. They have nothing to do with the physical nest anymore. So the honest answer to "where do wasps go in winter" is really two separate answers: the colony you saw all summer went nowhere, because it's dead — and next year's colony hasn't been founded yet, because the queen who'll found it is currently alone somewhere else, waiting for spring.

