Fortress Destruction Trick

Just had this trick played on me.

I had a 3 fortress and partially built 4. Just before I was hit with a massive attack my fortress went down to a damaged 1 on the map but my build screen still shows a partislly built 4.

My fortress was in fact a damaged 1 which was swept away by the attack.

My question: did my opponent spend gold to damage my fortress just before he attacked or was this a glitch?

I'm inclined to think he spent gold. Is is possible to be close enough to a fortress to prevent its being strengthened with gold yet be able to damage the fortress by espionage?