Umm no? Griefing at all weather u ask or not is griefing, its not a good habit to get into...

If there are homes in the way of your expansion you should contact a moderator. A moderator can find out who the structure belongs to and you may contact that person via in-game or on the forums regarding the obstruction of your expansion due to their buildings. If the person gives you permission to remove it you may do so. Otherwise, do not touch peoples houses in the wilderness. If you do so and that person returns stating that they were griefed, that is a ban. There is no excuse for griefing and I do not believe that there should be a rule that identifies with any form of griefing, specifically modifying the permissions aspect that defines the act of griefing. The status quo provides maximum efficiency for the moderators dealing with griefers because the cases are absolute. You grief, you get banned. No excuses to review.