My last place had a shed so small I had to back out of it. (The neighbor still called it an eyesore, which, fair, it was just plywood and hope.) People will judge anything that isn't a patch of grass.
My friend's neighbor in Seattle said the same thing about her tiny greenhouse. Some people just have Opinions about other people's perfectly normal property.