I’m writing a new piece for Places on prospective/speculative “interfaces to the smart city” — or points of human contact with the “urban operating system.” As I explained to the editors,
I’d like to consider these prototyped urban interfaces‘ IxD — with outputs including maps, data visualizations, photos, sounds, etc.; and inputs ranging from GUIs and touchscreens to voice and gestural interfaces — and how that interactive experience both reflects and informs urban dwellers’ relationships to their cities (and obfuscates some aspects of the city), and shapes their identities as urban “subjects.” I’m particularly interested in our single-minded focus on screens (gaaaahh!): are there other, non-“glowing rectangle” / “pictures under glass“-oriented platforms we can use to mediate our future-experiences of our future-cities?
Of course a large part of my work has been defining what constitutes an interface — and determining how one might critique it. My new spring Digital Archives studio includes an “interface critique” assignment, so this exercise of developing a critique methodology has served multiple purposes. For the class, I’ve asked my students to choose an “archival interface” and to examine:
- the site’s composition, organization, and aesthetics;
- how it structures the user’s experience and navigation, and how intuitive and “seamless” that interaction is;
- furthermore, how desirable would “seamless” interaction be in this instance (perhaps it would be helpful and instructive to show some seams?);
- how the site contextualizes the archival material (e.g., does it provide or link to robust metadata, does it “animate” the material?);
- how the site “hierarchizes” the presentation of information (e.g., does it allow users to “dig deeper” for more data if they want it?);
- the availability of documentation and help for users who want or need it.
- Consider the needs of various user groups and user scenarios, and try to put yourself in their positions as you navigate through your site.
I’ve aimed to flesh out this rubric for the “urban interfaces” article. And I thought I’d try out some of those ideas here. [You’ll find an updated version of these “trial ideas” here.]
But first, what is an interface?
In his 1997 Interface Culture, Stephen Johnson explains that, at the most basic level, interface “refers to software that shapes the interaction between user and computer. The interface serves as a kind of translator, mediating between the two parties, making one sensible to the other.”[i] He specifies that the interface is more semantic than concretely technological. Branden Hookway, whose own book on the topic is forthcoming from MIT Press, agrees that the interface does its work “not as a technology in itself but as the zone or threshold that must be worked through in order to be able to relate to technology.”[ii] Alexander Galloway, too, in his Interface Effect, specifies that the interface is not a thing, but a “process or a translation” – one that draws its qualities from the “things” it’s translating between, but which also has its own properties that are independent from the things it’s mediating.[iii]
Media scholar Johanna Drucker picks up on Hookway’s spatial “zone” and “threshold” metaphors; she regards the interface is an environment, a “space of affordances and possibilities” that structure how people interact with it. It’s a “set of conditions, structured relations, that allow certain behaviors, actions, readings, events to occur.”[iv] Drucker, like Hookway, is focused on the human-computer interface; both emphasize how the interface, through its affordances, structures the user’s agency and identity, and how it constructs him or her as a “subject” – as something much more than a mere “user.” But the zone between the machine and the person – that perceptible, manipulable skin – isn’t the only zone of interface. Computers, for instance, are commonly modeled as a “stack” of protocols of varying concreteness or abstraction – from the physical Ethernet hardware to the abstract application interface.[v] There are interfaces between the various layers of this stack. As Galloway explains, “the interface is a general technique of mediation evident at all levels”; that “technique” might be graphical, tangible (using a touchscreen or pad), gestural (using hands or mice), motion-tracking, sonic, or of another variety.[vi] Regardless of its means of operation, Galloway continues, the interface “facilitates the way of thinking that tends to pitch things in terms of ‘levels’ or ‘layers’ in the first place.”
Much, if not all, of what’s “beneath” or “behind” the graphical user interface is “black boxed,” inaccessible and unintelligible to us. And that obfuscation is in large part intentional and necessary. As I write this, for instance, I’m focusing my attention on the words on-screen, on the GUI, rather than bothering myself with the chatter between my TCP/IP transport software and my Ethernet hardware. And even the ubiquity and familiarity of computer screens like the one before me, and the one I carry around in my pocket – and the intuitive means by which I interact with them – naturalize and “disappear” the interface itself. That obfuscation, while necessary, is also politically risky; we forget just how much these layered interfaces are structuring our communication and sociality, how they’re delimiting our agency and defining our identities as “subjects.” As Galloway reminds us, it’s crucial to consider “the translation of ideological force into data structures and symbolic logic”; the user interface and the code and the physical infrastructure “beneath” them are all political.[vii]
That process of translation can suddenly call attention to itself once again when, say, something breaks – or when, say, Google updates is Fiber DVR and we have to learn a new visual language or means of navigation. When Johnson wrote his book in 1997, he investigated the desktop, windows, links, text, and intelligent agents as interfacing elements. But even then – before this age of smartphones and smart cities – he acknowledged that it was becoming “more and more difficult to imagine the dataspace at our fingertips.”[viii]
Representing all that information is going to require a new visual language… We can already see the first stirrings of this new form in recent interface designs that have moved beyond the two dimensional desktop metaphor into more immersive digital environments: town squares, shopping malls, personal assistants, living rooms. As the infosphere continues its exponential growth, the metaphors used to describe it will also grow in both scale and complexity.
Today, the bazaar-as-interface isn’t merely a computing metaphor; it’s not merely a trope for conceptualizing and graphically modeling an online store or a discussion board. Media facades, sensor-embedded pathways and thresholds, responsive architecture, public interactives and the like have transformed our physical environments into interfaces in their own right. But interfaces to what? What technical operations are taking place deep in the urban “stack” that we might want to interface with?[The promised “critique” bit is down a little further, below this here bibliography.]
* * * * *
[i] Stephen Johnson, Interface Culture: How New Technology Transforms the Way We Create and Communicate (New York: Harper Edge, 1997): 14.
[ii] Branden Hookway, The Interface, Dissertation (Princeton University, 2011): 14).
[iii] Alexander R. Galloway, The Interface Effect (Malden, Ma: Polity, 2012): 33.
[iv] Johanna Drucker, “Performative Materiality And Theoretical Approaches To Interface” Digital Humanities Quarterly 7:1 (2013) (paragraph 31).
[v] Rory Solomon, one of my advisees at The New School, wrote a brilliant thesis – The Stack: A Media Archaeology of the Computer Program – on the history of the stack metaphor. Part of his work appears in “Last In, First Out: Network Archaeology of the Stack” Amodern 2 (October 2013).
[vi] Galloway 54.
[vii] Galloway 70.
[viii] Johnson 18.
[The following was “lightly” revised on January 18. I’ll be creating a separate post with a more substantial revision soon.]
And now, how do we critique an interface?
As Drucker explains, the prevalent ways of thinking about human-computer interaction (HCI) are framed by values central to engineering.”[i] The evaluation of interfaces involves “scenarios that chunk tasks and behaviors into carefully segmented decision trees” and “endlessly iterative cycles of ‘task specification’ and ‘deliverables’”; and it tends to equate the “human” in HCI as a task-oriented, efficiency-minded “user.” She proposes instead a humanities-oriented interface theory that draws on “interface design, behavioral cognition, and ergonomics, approaches to reading and human processing,” and requires an understanding of the “history of graphic design and communication with a specific attention to the semantics of visual form.”[ii]
Yet while Drucker proposes that we move away from engineering-oriented methods of critique, we do have to acknowledge that the engineering of our material interfaces does factor into how those interfaces structure “human [and machine] processing.” We need to take into consideration the materiality, scale, location, and orientation of the interface. For instance, where is the screen sited; how big is it; is it oriented in landscape or portrait mode; what kinds of viewing practices does it promote; does it allow for interactivity, and if so, in what form? Where are the speakers, what is their reach, and what kind of listening practices to they foster? Or, where are the sensors that read our gestures, how sensitive are they, and how do they condition our movements? Futhermore, what are our possible modalities of interaction with the interface? Do we merely look at dynamically presented data? Can we touch the screen and make things happen? Can we speak into the air and expect it to hear us, or do we have to press a button to awaken Siri? Can we gesticulate naturally, or do we have to wear a special glove, or carry a special wand, in order for it to recognize our movements?
Now, returning to Drucker’s recommendations: we can learn a lot from comics in regard to the semantics of visual form. Scott McCloud’s canonical Understanding Comics offers a useful model for thinking about graphic reading practices. In examining interfaces, too, we should attend to variables of basic composition (e.g. the size, shape, position, etc., of elements on the screen), as well as how they work together across time and space: how we read across panels and scenes, how we follow action sequences and narrative and thematic threads through the graphic interface. The temporal and spatial dimension of our navigation could be sign-posted for us via “bread-crumb trails that mark [our] place in a hierarchy or a sequence or move or events,” or “displays that make use of different scales and devices for shifting granularity” to allow us to understand how closely we’re “zoomed in,” how much context the interface is providing.[iii] This sense of orientation – of understanding where one is within the “grand scheme” of the interface, or the landscape or timeframe it’s representing – plays a key role in determining our user-subject’s identity and agency. Speaking specifically of urban interfaces, Adam Greenfield suggests that “the ability of citizens to enjoy the same real-time synoptic visibility over the unfolding processes of the city available to any manager [sitting in an Urban Operating System control center, for instance,] is vital” to citizens’ appreciation of their “right to the city.”[iv]
Drucker also recommends that we employ “frame analysis,” which would address how the various boxes, buttons, and applications – as well as the different modalities of presentation (audio, visual, textual, etc.) on our interfaces – conceptually and graphically “chunk, isolate, segment, distinguish one activity or application from another.”[v] Ideally, these assemblages will all hang together if there are “sufficient common reference points in one conceptual organization, or graphic frame.”[vi] Such cohesion will enable us to read across “a multiplicity of worlds, phenomena, representations, arguments, presentations – and media modalities” – but in critiquing how this cohesion comes about, we should also pay attention to the “nodes, edges, tangents, trajectories, hinges, bends, pipelines, [and] portals” that frame and link — and perhaps create friction between — the components of our interfaces.[vii]
Reading “beneath” those graphic frames provides insight into the data models structuring our interaction with the technology. Those sliders, dialogue boxes, drop-down menus and other GUI elements indicate how the data has been modeled on the “back-end” – as a qualitative or quantitative value, as a set of discrete entities or a continuum, as an open field or a set of controlled choices, etc. “[C]ontent models, forms of classification, taxonomy, or information organization,” Drucker argues, “embody ideology. Ontologies are ideologies,… as naming, ordering, and parameterizing are interpretive acts that enact their view of knowledge, reality, and experience and give it form.”[viii] The design of an interface thus isn’t simply about efficiently arranging elements and structuring users’ behavior; interface design also models – perhaps unwittingly – an epistemology and a method of interpretation.
Yet Galloway reminds us that, while the interface does serve to “translate” between the data model and the GUI, and between other levels of the stack, that translation isn’t inert. He speaks of the “fundamental incommensurability between any two points or thresholds on the continuum of layers”; we thus use allegories or metaphors – the desktop, the file folder, or even our mental image of the city-as-network – to ostensibly “resolve” the “tension between the machinic and the narrative,… the fluid and the fixed, the digital and the analog.”[ix] In our interface critique, then, we might also consider what acts of interpretive translation or allegorization are taking place at those nodes or hinges between layers of interfaces. Even what seem to be purely aesthetic decisions, or matters of style, can function allegorically or rhetorically; Galloway speaks of “windowing,” for instance – of screens dissected into panels or frames that offer multiple perspectives simultaneously, as opposed to the sequenced presentation of filmic montage – as a stylistic embodiment of the “cultural logic of computation.” While his analysis focuses on the television show 24, we can easily see similar modes of presentation on our smartphone screens and in smart cities’ control centers. This window motif represents “the distributed network as aesthetic construction”; it translates the network structure into a form, a look.[x]
The interface, as we said earlier, also shapes our identities and defines our agency as users, or subjects. We might consider how the interface enunciates – what language it uses to “frame” its content into fundamental categories, to whom it speaks and how, what point(s) of view are tacitly or explicitly adopted. Of course there’s an ideology to this enunciation, too: Drucker encourages us to consider “who speaks for whom”; “what is not able to said,” “what is excluded, impossible, not present, not able to be articulated given [the interface’s] structures”?[xi] How the interface addresses, or fails to address us – and how its underlying database categorizes us into what Galloway calls “cybertypes” – has the potential to shape how we understand our social roles and what behavior is expected of us. We might identify in our critique whom the interface addresses, how it does so, and how those users play into their “cybertype” subjectivities. And particularly in regard to our urban interfaces, we might consider how they define us as urban citizens, and whether and how they create, in Adam Greenfield’s words, “frameworks for citizen engagement.”[xii] He helpfully crystallizes the political implications of ensuring that our urban interfaces are designed to foster engagement:
Our ability to use the city around us, our flexibility in doing so, just who is able to do so, will be shaped by decisions made about the technical design of objects and their human interfaces, and the precise ways in which such objects are connected to one another and made visible to the network.
We also, finally, must consider what is not made visible or otherwise perceptible. What is simply not representable through a graphic or gestural user interface, on a zoomable map, via data visualization? While some content or levels of the protocol stack may be intentionally hidden – for security or intellectual property reasons, for instance – Galloway argues that some things are simply unrepresentable, in large part because we have yet to create “adequate visualizations of control society.”[xiii] Greenfield proposes that some aspects of the city need simply to be made recognizable to the machine, translatable through the interface:
As yet, the majority of urban places and things appear to the network only via passive representations. The networked city cannot come into its own until these are reconceived as a framework of active resources, each endowed with some manner of structured, machine-readable presence, and the possibilities for interaction such provisions give rise to.
Yet we should also consider the possibility that some aspects of our cities are simply not, and will never be, machine-readable. In our interface critique, then, we might imagine what dimensions of human experience and the world we inhabit simply cannot be translated or interfaced.
* * * * *
[ii] Drucker 2013: ¶27.
[iii] Drucker 2011: 18.
[v] Drucker 2011: 15.
[vi] Drucker 2011: 18.
[vii] Drucker 2011: 14.
[viii] Drucker 2013: ¶42.
[ix] Galloway 76.
[x] Galloway 110, 117.
[xi] Drucker 2013. Drew Hemment and Anthony Townsend also encourage us to pay attention to disenfranchised populations: “how can we create opportunities to engage every citizen in the development and revitalization Of The Smart City?” (“Here Come The Smart Citizens” In Hemment & Townsend, Eds., Smart Citizens (Future Everything Publications): 3).
[xii] Greenfield 2012.
[xiii] Galloway 91.