I'd think it would be good to put dirt walls behind the base of the towers. The given dwarf fortress and ork cave systems are un-grief-able (to an extend, by that I mean they can't be collapsed), and the humans have easy access to stone. This would make it still possible (I don't want the dirt walls to reach to the sky, but a bit of a ways up so you cant just encase and knock out the bottom) to destroy it, similar to humans, but it would take longer and be more difficult (also a lowly single knight couldn't do the deed). I'm also strongly against it being allowed to destroy any races main structures and spawns (mostly because of the humans and especially elves disadvantage), but maybe that's just me (shops are of course up for grabs). Another solution would be to give doors to elves instead of dwarves, but that can be... messy.