You should NOT be adding a JInternalFrame to a JPanel.
A JInternalFrame is designed to be used with a JDesktopPane only. Read the section from the Swing tutorial on
How to Use Internal Frames for more information and examples.
Removing a component from a panel does not destroyed. This is easy to
test. You create a simple JFrame with a panel in the "CENTER of the BorderLayout of the frame. Then create a "Remove" button that you can add to the "PAGE_START" of the frame and an "Add" button in the "PAGE_END". Then you just add/remove the panel and see what happens.
Part of programming is learning how to create a simple example to test your question. If the test program doesn't work as expected then you post the program with the question in the forum.