Metaphors and learned gestures

Rauno Freiberg opens Invisible Details of Interaction Design (July 2023) with an admission. Design can feel like there is no science to it, “only feel and intuition.” Even researchers, he notes, have trouble grounding interaction design practice in science, and he cites a 2011 CHI paper by Goodman, Stolterman, and Wakkary as evidence that the field tends to treat its own practice as a black box. His response is not to give up on the why but to dig for it. Searching the web, he writes, mostly returns recycled content about user personas, storyboards, and Venn diagrams labelled “UI” and “UX.” The substance shows itself only to people willing to read obscure research papers or to replay hundreds of slow motion screen recordings.

Two more things frame the essay before a single gesture is discussed. The first is where insight comes from. The word interaction implies a relationship between a human and an environment, and Rauno says his own revelations surface from making something, filling his head with a problem, and then going for a walk to let it settle. The second is what the essay is: “not a tutorial nor a collection of guidelines,” but an observation of details in interactions he uses often and rarely thinks about. He adds that, besides recreating interfaces, this kind of reflection is another good way to build design intuition and vocabulary. That is the spirit this chapter borrows.

His definition of the craft is short. Interaction design is “an artform to make experiences that fluidly respond to human intent.” He is quick to separate it from painting or composing music. People use a product to get something done, so beauty of form and composition is not enough; the satisfaction, he says, comes from a holistic balance between form and function. The questions he lists as the substance of that balance are the ones this chapter follows: when does a swipe trigger an action, do gestures retain momentum, what happens when a finger covers content, how can an interface predict intent from context.

The first answer is metaphor. “Great interaction design rewards learning by reusing metaphors.” You can use most touch interfaces with just two gestures, tapping and swiping, and on iOS the only gesture you are explicitly taught is swiping up to open. Learning that one swipe unlocks control over much of the rest of the interface. The sliding motion also tells you something structural: the interface is composed of stacked layers, like a deck of cards, and swiping down reveals layers of system functionality. Rauno’s point is that this knowledge compounds as you go deeper into the Apple ecosystem. Every new surface that honours the metaphor costs the user nothing to learn.

Why does swiping horizontally move between pages? Because, he writes, that is how we have interfaced with books for thousands of years. The metaphor brings real-world properties with it, and the one he singles out is interruptability. Flipping a page in a book is obviously interruptible, and he admits that saying so sounds silly, until you imagine the page turn as an animation you had to wait for. Borrowing from the physical world means borrowing its properties, not only its look.

Pinching is the subtler example. Zooming, in his words, is an act of precision: adjusting the amount of detail visible. On a touch screen the interface first has to establish an anchor point from which the zoom originates, and it is easier and more precise to pick that point with the fingers pinched together. When the fingers start apart an anchor still has to be calculated, but that usually means zooming out, where the exact origin matters less. Deliberate precision wants two fingers starting close together, “kind of like grabbing an object.” The gesture maps a motor skill onto an intent, and the mapping is what makes it feel obvious.

He closes the section with the sentence that gives the essay its title: hundreds of design decisions are made by someone obsessing over the tiniest margins so that, when they work, nobody has to think about them. Many of them tap into instinctive behaviour. The practical takeaway for your own work is a question to ask before you invent a gesture or a transition: which metaphor is it borrowing, and does it keep that metaphor’s properties? A swipe that cannot be interrupted, a layer that does not slide, a zoom with no anchor, each breaks a promise the user did not know they were relying on.