I think any reasonable person can interpret that there is a meaningful difference between an hour of work and several. The same way you might drive your friend across town for free but you're probably going to ask them for gas money and some beers if they ask you to help them move across state. An hour of my time is just less valuable than my entire day.
I was being completely serious in every part of my comment. To me a four hour coding challenge is not significantly more of a time investment than an hour-long interview, given that I'll need to prep for it beforehand anyway, and that it's more stressful than sitting quietly at a computer doing what I like to do in my spare time anyway.
Which is a reasonable stance to take! But not if you want to be paid for take homes, but not for interviews.