just to get this straight. Country "A" was destroyed but before he died he put some destroyers in the water and now there are just siting there. right
I do not believe this is bug but it is a problem that has been seen. As long as you are not at war with that country there is nothing that you need to do. if you take all land from everyone you win there is no rule that says you need to destroy all enemy units.
I use them as and early warning system(mind you this only works some times) where if some one attacks me, they have to pass those ships and if they are at war with those ship then they must stop and destroy them.
hope this help