Rather than say that it didn’t rise, I would say that it fell after it rose.
A good way to avoid this, or at least decrease the results, is to allow the cake to cool inside the oven. It might impart a “brittle” but very thin crust, but it is worth it. In my experience, I find that it increases the fall if you allow the cake to switch temperatures very quickly. If you wish, you could crack the oven door a little bit to allow hot air to come out, which might decrease the slight crust.
Once the cake is iced and, if a layer cake, placed with the other layers, the crust is completely unnoticeable, so I would not shy away from this. It’s worth a try, anyway.