This may be intentional. Guests do not receive the same real time benefits that registered users do, as it is a not-insignificant cost to the server.
Perhaps that would explain it.
Is there a way to completely remove a post once it's been purged or deleted by admin?
Hm, when a topic is purged, it should be gone from the database, so it sounds like the purging isn't completing.
Do you see any errors in the NodeBB log?
When I attempt to purge the topic, an error message 'Error This topic has already been deleted' appears.
The topic remains - http://pasteboard.co/1JHd886q.png
Sounds like it may be stuck in cache. I'd clear your cache and see if it goes away.