insulating window panel
An insulating window panel or pop-in shutter typically consists of a core of rigid foam insulation. You can push or clip it into the interior of a window. The panels are made so that their edges seal tightly against the window frame. Seals can be made from magnetic tape or Velcro. No hardware, such as hinges or latches, is required.
Insulating window panels have R-values between 3.8 and 7. They are also fairly inexpensive, whether you buy a kit or make your own, but you will need space to store them when they're not in use.