Wheelchairs should be able to negotiate stairs. It is technically possible. Therefore it is not my responsibility to have an access ramp to my building.
To be fair on the author and the article, that's not at all the message I get when I read the article, and I don't think it is the aim of the author to say that. From the introduction:
The title of this presentation is “When accessibility is not your problem” – in other words, the specific limited edge cases in which you, as a content author, do not have to worry about accessibility. I had four people in London telling me they got the impression, or feared others would get the impression, that I am claiming accessibility is not your problem. That is nonsense, of course.
There's a certain tendancy to overdo accessibility, and treat users with (for example) a visual defect as if they also had a mental one. It's like shouting at a deaf man, or using child-like language to someone in a wheelchair: it's demeaning both to yourself and the disabled person, as well as being a desperate misunderstanding of the disability.
So no, you don't need inch-high text and a high-contrast layout with dumbed-down language to get an "accessible" badge. You can trust the disabled user and have faith in their intelligence and assume they have equipped their computers with the appropriate tools to read your site.