The Sur/Render of Perception (SCMS 2020 Coronavirus Special)

“Though the render farm’s massive energy consumption expedites climate change, perception’s surrender may be a necessary step in loosening subject-oriented individualisms in order to encounter our environment and one another in this time of urgency.”

I was set to participate in a panel on “Rendering: Times, Powers, Perceptions” with Deborah Levitt, Vivian Sobchack, and Joel McKim at the 2020 Conference of the Society for Cinema and Media Studies in Denver. Due to the novel coronavirus pandemic, the conference was cancelled. I had already written my paper, which also serves to introduce one of the topics I deal with in my book Discorrelated Images (out in October with Duke UP), so here it is:

The Sur/render of Perception

“Render farms,” or high-performance networked computer clusters designed to churn out computer-generated imagery, epitomize a transformed relation between image and perception. In these batteries of computers, images are not visual phenomena but pure information, mathematical quantities between black-box machines, radically discorrelated from subjective perception. Indeed, the operation of the render farm signals a fundamental surrender of perception’s primacy of place—a giving up or giving over of human sense to the post-perceptual processes of machine vision, or the regime of what Trevor Paglen calls “invisible images.” To render and to surrender are both conceptually and experientially intertwined, and because the work of rendering (which also means repeating) does not stop at the farm, neither do the consequences for perception: the rendering process is rehearsed in every computationally facilitated playback, as video codecs re-calculate images from predictive vectors, not integral photograms. Each playback is thus a new rendition (the prefix re- in “render” corresponding to the -back of “playback.”); rendering as repetition never comes to rest at a final, integral object or experience. And every viewing thus involves a new surrender of perception.

Meanwhile, the prefix sur- complicates iteration with an ambiguous gesture of transcendent return (giving back over). Surrender presumes a prior state of ownership, which may not correspond to empirical facts but signifies an almost metaphysical relation of priority. The surrender of perception involves giving perceptual experience and its image-object back over to nonperceptual process, matter, metabolism—to the affective processes of pre-personal embodiment that have always subtended subjective perception, now mirrored in the black box of the computer. This act of sur/render is dangerous (possibly subjecting human agency to microtemporal control), but perhaps it holds a liberatory potential as well: opening human experience back up to cosmic time (such as Deleuze hoped from the time-image) and putting us in touch with ecological forces that transductively render perceptual subjects and objects alike. Though the render farm’s massive energy consumption expedites climate change, perception’s surrender may be a necessary step in loosening subject-oriented individualisms in order to encounter our environment and one another in this time of urgency. Could this be the multistable gift (the Proto-Indo-European *do-) at the heart of sur/render?

My forthcoming book Discorrelated Images grapples with these questions across a range of moving-image forms, genres, and media. One especially potent site for thinking about the surrender of perception is in the many contemporary images of the end of the world and of extinction—the ultimate scene of discorrelation, where the subject of perception is obliterated and the image extinguished. Ironically, such scenes are rendered in spectacular computer-generated images—thus enacting the give and take of sur/render, or the production of images that both thematize and instantiate their own discorrelation from human perception.

I want to look at one example of this thematic, material, and ultimately ethical entwinement in a recent videogame, NieR:Automata. In this interactive medium, not only perception but also action is tightly coupled with real-time image generation. That is, rendering transductively articulates action and image together: they are caught up in microtemporal circuits connecting user input and computational operations that feed forward into processual screen events that elicit further inputs and entrain players’ awareness and agency in a temporal becoming that was not pre-recorded but is happening in a precariously generated now. Here, human protention and the computer’s microtemporality commingle to the point of indistinction, raising specifically ethical questions about the determination and exercise of agency, questions that cannot be answered with an appeal to an integral, foundational subject. Against this background, NieR:Automata channels the medium’s ethical questions into an exemplary existential probing of agency, image, and world in the light of extinction, self-reflexively relating its own computational image-rendering to questions of perceptual surrender.

Released for PlayStation 4 and Microsoft Windows in 2017 and for Xbox One the following year, NieR:Automata is a third-person, open-world action role-playing game developed by PlatinumGames under the direction of acclaimed videogame director Yoko Taro. The JRPG draws strong visual and thematic influence from manga and anime, including melodramatic characters with stylized hair, giant robots, and a sword-wielding protagonist in thigh-high boots and a somewhat revealing dress. The game’s post-apocalyptic setting, with abandoned cityscapes reminiscent of the History Channel’s Life After People series, contrasts interestingly with its strangely soothing music, while gameplay alternates between a variety of generic forms and conventions: battling sentient machines in an expansive three-dimensional world, operating flying vehicles in overwhelming “bullet-hell” situations, navigating jump-and-run-style action in side-scrolling 2D platformer sequences, and hacking computers in a minimalistically rendered abstract data space reminiscent of 8-bit era games. The game also features an elaborate narrative, elements of which are uncovered over the course of twenty to thirty (or potentially many more) hours of play. The basic scenario is that thousands of years ago, Earth was invaded by aliens who brought with them hostile robotic machines, eventually forcing humans to abandon the planet and set up a base of operations on the moon; since then, the humanoid androids of the YoRHa forces have been fighting the more industrial-looking machines, waging a completely nonhuman proxy war on behalf of the Council of Humanity. Now, the player joins the ranks of the androids to retake the planet, but along the way we encounter guilt-ridden machines pondering the meaning of existence and establishing religions and novel cultural forms, thus calling into question human exceptionalism and anthropocentric notions of value. Later, we also find out that humanity has in fact been extinct for many millennia, that the ongoing war was pre-programmed, every battle the result of machinic directives guiding the behavior of combatants on both sides.

Driving home this thematization of determinism and free will, the game is full of heavy-handed references to philosophers and to existentialism in particular: the leader of a village of pacifist machines is named Pascal, and he reads Nietzsche. A machine named Jean-Paul Sartre lives there as well, spouting slogans like “existence precedes essence” to anyone who will listen. The android protagonist who serves as the player’s initial onscreen avatar is named 2B—provoking the question, “or not 2B?” All of this can seem rather cheesy, and frankly it is. But it sets the stage for the game’s more substantial probing of the existential parameters of the player’s interface with the computer and the question of agency in a world where extinction is not only assumed as a historical fact but linked to the material experience ofdiscorrelation at every microsecond in the game’s real-time hyperanimated images. Thus, it is perhaps in spite of the game’s overt existentialism that NieR:Automata becomes a powerful mediator of post-extinction ethics in a world of radically machinic images. 

Indeed, much of the most important ethical probing takes place in the low-level circuit of input-output that positions the image in-between the computer’s processes and the user’s actions. Upon initially starting up the game, we are presented with a blackscreen with the words “LOADING – BOOTING SYSTEM…” in the top left corner and a logo in the middle of the screen with the words “YoRHa” and “For the Glory of Mankind.” The results of system checks and other stats scroll down on the left, vaguely reminiscent of a Linux boot process, while the text twitches with simulated glitches. This boot screen, which will reappear throughout the game, is neither completely diegetic nor fully extra-diegetic; it will be narrativized as the android’s own boot process whenever “consciousness data” is uploaded to the server or restored to a new body, but it also serves to mask those moments when the player’s computer or gaming console has to load information from memory.

This semi-diegetic boot screen ushers in a broad problematization of phenomenological relations in the game/player system—an unsettling of relations expanded in battle sequences, which foreground the image’s thoroughgoing repositioning as the object not of perception but of cybernetic processing. Playing as the female android 2B, whose body we see in third-person perspective, when we take damage from an opponent the screen as a whole glitchily shakes, the image “tears,” and we see blocky color separation effects (red and cyan layers bleeding out around the edges of objects, reminiscent of analog 3D cinema); when, on the other hand, 2B dodges an attack, the android’s body quickly splits apart, warps, and flashes as if hit by lightning. These visual effects are thus distributed across the subjective and objective poles of the image, reminding us of the computational totality of the situation—of our real situation as players as much as the fictional situation of the computer-driven characters. 

As a way of “making sense” of this situation, the game stages a constant negotiation between perceptual and computational spaces, figured centrally as shifts between the embodied action of battle and open-world exploration, on the one hand, and the “hacking space” in which computers (androids and machines) interface directly with one another or with the network, on the other. This space of disembodied data, in which the player steers an abstract icon reminiscent of an early arcade-game spaceship and shoots various enemy icons in order to “hack” the opponent, offers a displaced representation (à la cyberpunk imaginations of the network) of the game’s computational system more generally and thus provides a representational and perceptual form for the discorrelated processing at the very heart of all the games’ procedurally generated images. In this sense, these too are semi-diegetic events that involve us more thoroughly—because actively—in the negotiation between perception and computation. 

Later in the game, 2B’s male android partner 9S hacks into the network and contracts a computer virus that distorts our vision as mediated by the screen. He also learns the secret of humanity’s millennia-long extinction, which provokes a sort of existential crisis for him (and for us?). But it is not until the end of the third playthrough, or after some twenty hours or more of gameplay altogether, that we encounter the game’s most powerful questioning—and practical enactment—of the ethical consequences of the intertwining of agency and image in the face of extinction. By this time, all the other androids have succumbed to the virus; the last two prepare to kill each other, effectively completing the program of extinction by ending now nonhuman sentience as well. Afterwards, the credits roll, but the text starts glitching as we learn that there is “data noise present in stream” and “personal data leaking out.” Apparently, there are still traces of the androids’ “consciousness data” in the system, which is in the process of self-deletion. We are presented with an option to try to save them, which if we choose to pursue it causes a spaceship identical to the one seen in “hacking space” to appear, and we are tasked with shooting up the credits scrolling down the screen. 

The textual entries are rendered enemy targets, emitting an astounding barrage of projectiles in all directions. Contact with one of the latter causes the player to lose a “life,” like in old arcade games, and having lost three lives the player is asked whether they would like to connect to “the network”—a somewhat ominous proposition, given the fact that countless hours have just been spent battling the enemy machines’ network. The player can click “no” and refuse defeat, repeating the battle multiple times, but victory seems impossible, and each time the player loses the computer taunts: “Do you give up?” “Is it all pointless?” “Do you admit there is no meaning to this world?”

At some point, there is no option but to connect to the network, in which case we receive a message like: “8X ‘I did my best. One thing is certain: I’m rooting for you.’ USA.” Fragments of other messages are visible in the background. We then receive a rescue offer, and multiple spaceships join ours, multiplying our firepower and our chance of survival. With each collision, one of the spaceships is destroyed, and we read messages such as: “Crs501’s data has been lost.” “minato’s data has been lost.” “Yambu’s data has been lost.” What we are witnessing, it turns out, are the traces of other players’ savefile data—the computational memory that parallels or even replaces human memory and enables players to replay or restore the game from an earlier save state—being sacrificed to help other players connected to the network.

Upon completion of the task with the assistance of these anonymous helpers, we are prompted to send a message of our own to other players. Then, another prompt: “Please respond to this query. You, X, faithful player of this title, have lost your life multiple times to make it this far. You have faced crushing hardship, and suffered greatly for it. Do you have any interest in helping the weak?” The possible answers are a simple “Yes” or “No.” And while the theatrics of these queries may be slightly off-putting, a significant ethical choice is being framed here. An affirmative answer triggers the following message: “Selecting this option enables you to save someone somewhere in the world. However, in exchange, you will lose all of your save data. Do you still wish to rescue someone—a total stranger—in spite of this?” The player is given several opportunities to reconsider, along with further warnings that really everything—all our progress in the game, items and weapons obtained, skills and intelligence unlocked, and generally all of the labor we have invested in the game—will be lost forever in an act of self-sacrifice. If we persist, the computer responds: “Very well. In exchange for all of your data, I will convey your will to this world.” Then we see the game’s configuration menus, all of the places and save points on the map disappearing one after another, followed by the options under “quests,” “items,” “weapons,” and so on. Finally, the options under “system” are deleted, all of the save states disappearing until there is nothing left but a blank slate. The image fades to white, and we are informed: “All of your data has been deleted.” After a short message thanking the player for playing, the “save” indicator appears in the top right corner of the screen. We read “Save complete” and then “Connecting to the network…” Finally, a glitchy NieR:Automata logo comes into focus, along with the message “Press any button.”

In this way, the game frames a non-trivial ethical decision, whether to sacrifice an indistinctly computational and experiential memory and pass it on to those who come after us. The significance of the choice, beyond its overt existentialist framing, lies in the player’s real investment of value in the data to be sacrificed, which seals the circuit between perception and computation. In sacrificing their data, the player also sacrifices the image, which dissolves before their very eyes. This exercise of agency at once completes the destruction of the world and enables its continuation for some unknown player elsewhere in the world. A fine balance is struck between individual identity and anonymity, neither collapsing into solipsistic solitude nor constituting a robust collectivity. The choice to sacrifice oneself for the sake of an unknown, future other frames a symbolic restoration of the intergenerational continuity, or the promise made to future generations, and which is required (as a necessary but not sufficient condition) if we are to avoid climate catastrophe. Playing videogames will clearly not avert the threat of extinction, but playing in the shadow of planetary demise—and connecting the computationally rendered image with an existential surrender of perception—just might help restore the moral gravity of our situation.

Vivian Sobchack on Discorrelated Images

As I mourn the cancellation (due to the novel coronavirus) of my SCMS panel with the great Vivian Sobchack, I discover her blurb for my book Discorrelated Images, which has just appeared next to Steven Shaviro’s on the Duke University Press website. I am deeply honored and grateful to read these words of praise from a theorist whose work and words have been formative for me.

Steven Shaviro’s Blurb for Discorrelated Images

The world is falling apart, but my book Discorrelated Images is still set to come out in October — and I just discovered that Steven Shaviro has blurbed it! Also, the Duke University Press webpage for the book lists an affordable $25.95 cover price for the paperback!

“Aesthetics of Discorrelation” and “Exploring Cinematic Mixed Realities” — Two Events at Duke University, Feb. 20 and Feb. 21, 2020

This coming week I will be at Duke University for two events:

First, on Thursday, February 20 (5pm, exact location to be determined), I will be giving a talk titled “Aesthetics of Discorrelation” (drawing on work from my forthcoming book Discorrelated Images).

Then, on Friday, February 21 (1-3pm in Smith Warehouse, Bay 4), I will be participating in a follow-up event to the NEH Institute for Virtual and Augmented Reality for the Digital Humanities, or V/AR-DHI. I will present work on “Exploring Cinematic Mixed Realities: Deformative Methods for Augmented and Virtual Film and Media Studies” and participate in a roundtable discussion with other members of the Institute.

Jim Campbell’s Discorrelated Images

IMG_8778

Last evening I had the pleasure of discussing Jim Campbell’s work with him at the Anderson Collection at Stanford, where he has a wonderful exhibition of LED-based works up right now. It was a far-ranging discussion, in a packed gallery, and great fun all around. Here are my opening remarks:

Before we start our conversation, I have the honor of offering some framing thoughts about Jim Campbell’s work. I want to use this opportunity to put that work into dialogue with some of my own interests and concerns as a theorist of the intersection between computational and moving-image media. I am concerned, in other words, with the historical and phenomenological encounter between the invisible processing of digital information and the visible forms that result from it—and it is precisely this encounter that Jim’s LED-based artworks enact or perform in a variety of thought-provokingly deformative ways. This is to say that his work, by means of occluding, blocking, and de-focusing our view, ironically makes perceptible the very mismatch between perception and computational processing that lies at the heart of digital video as it circulates online, on our smartphones, on DVDs and BluRays, on digital cable and satellite TV, and in the digital projection systems of contemporary movie theaters. In all of those contexts, digital processing remains resolutely invisible to perception (except, that is, through exceptional moments of glitching, buffering, and the like); but, those exceptional and denigrated moments aside, the perceptual “content” of digital video is privileged, thus blinding us to the ways that the medial form of video’s computational processing is changing the very parameters of our embodied perception, or the ways that, as Canadian media theorist Marshall McLuhan put it, our “sensory ratios” are being reformed by our encounter with a new media environment.

By re-valorizing the exceptional, or that which disrupts or impedes the easy transmission of visual “content,” Jim’s work offers an oblique view of the hidden parameters of this new environment; he makes what I call the “discorrelation” between our perception and its infrastructure perceptible—if only in a necessarily incomplete and volatile form. And the volatility of these operations is key: Jim’s works keep our eyes and our bodies moving, making us move now closer and then farther away, causing us to squint and then relax our focus, in order to catch a glimpse of something figural, recognizable, the so-called “content” of the moving images. Certainly, this content is not irrelevant, but it is hardly the ultimate telos or desideratum towards which the work directs our attention. The works are not simple puzzles that are “solved” once we identify their contents. Rather, the incessant oscillation between perception and non-perception, between seeing and not seeing, would seem to be closer to the point, as it is this oscillation that keeps everything at play, unsettling basic categories and forms. We shift our focus between individual LEDs, the screen or wall upon which they reflect, and an indirect, sometimes volumetric illumination of bodies or objects in motion. Our perception doesn’t come to rest upon a stable object or meaning, and this instability infects the broader conceptual context within which our perception is situated: Jim’s work upsets and makes us question so many basic distinctions—for example, between video art and sculpture, between art and engineering, between material substrates and perceptual forms, between perception and imagination. Through his destabilization of perception, he re-opens also the gap between art and technology, a gap created around the time of the industrial revolution, when thinkers like Immanuel Kant helped engineer a split between the aesthetic and the technical, or between the fine arts and the applied arts. Earlier, both the Greek term techne and the Latin ars referred indiscriminately to both arts and technologies. Now, the poets were to work with words while the engineers worked on steam engines; artists concerned themselves with the non-utilitarian forms of aesthetic experience while technologists made the machines that kept the factories running. However, in the space cleared between art and technology, a third thing emerged, a common ground for aesthetic and technological production alike: namely, media in its modern sense. A medium in this sense is not reducible to its “content” in a narrow way; rather, it is something that straddles perceptual form and infrastructure. Take, for example, the way the Sunday comics capitalized on innovations in four-color printing processes, or the way cinema responded to synchronized sound with new genres like the musical or the horror film, which involves its spectator through an offscreen space of screams and bumps in the night. It is in this sense that McLuhan proclaimed that “the medium is the message”—a claim that he explained with the example of the light bulb, a content-less medium, the message of which is the electrification of the world and the resulting transformation of agency, perception, and social relation. In order to explore the message or the meaning of more recent shifts in the media environment, Jim replaces McLuhan’s light bulb with LEDs—the same light emitting diodes that provide backlighting for flatscreen computer monitors and television sets, that power digital projectors, or that illuminate our increasingly “smart” homes. Routing perception through these characteristically digital-era lights, and powering them by way of unseen “custom electronics,” Jim defocuses intentional perception, foregrounds the obfuscation of infrastructure, and indirectly illuminates a media environment in which computation has finally (arguably) rendered the industrial-era split between art and technology untenable.

When I recently spoke to him on the phone, Jim identified himself not as an artist but as an engineer—and certainly he holds the degrees, the patents, and the experience to justify that statement. But he is an engineer of a special sort: an engineer of perception in an age when perception teeters precariously atop invisible circuits and computational infrastructures not cut to our measure, an engineer of experience when experience is routed through ubiquitous circuits of computational processing. Occluding both the image and its digital infrastructure, Jim’s work puts our perceptual experience in motion, incessantly circulating between what we can and cannot see. The work arouses a curiosity about the conditions of this circulation, including the means by which the LEDs, and hence also our perception, have been programmed. In the context of nineteenth-century magic shows and scientific expositions, this curiosity about how the spectacle works has been called an “operational aesthetic”—an aesthetic that, fittingly for the era of industrial media, includes an enjoyment in the sight of technical operation. In the twenty-first-century context of ubiquitous computational processing and experiential engineering, Jim offers us something slightly different, I suggest: an operational aesthetic of perception itself, a questioning of our ability and the means of seeing in an age of discorrelation, when visibility is rendered ambiguously at the margins of human signs and invisible informatic signals.

Discorrelation and the Post-Perceptual Image (Bogotá, video en español)

Video (in Spanish) of my talk, “Discorrelation and Post-Perceptual Image,” from September 12, 2019 at the Universidad Nacional de Colombia in Bogotá is now online.

Minutes before the talk, I was whisked away to give two separate interviews — one for an article that is now online, and one for a local television station (!), which I have not yet seen…

Cinemática II: O pós-cinema e a experimentação para além da tela / Post-Cinema and Experimentation Beyond the Screen — October 8-10, 2019, São Paulo, Brazil

cartaz cinematica II(5).jpg

On October 8, I will be giving a keynote lecture at “Cinemática II: O pós-cinema e a experimentação para além da tela” (Post-Cinema and Experimentation Beyond the Screen) at the Universidade de São Paulo in Brazil. My talk, titled “Discorrelated Images, Algorithmic Affects, and the Hyperinformatic Environment,” draws on my forthcoming book Discorrelated Images.

I am very excited to visit Brazil for the first time, and very grateful for the invitation from Prof. Giselle Gubernikoff of the Escola de Comunicações e Artes!

cartaz cinematica II(5)-b

Discorrelation and Serialized Frankensteins — Bogotá, September 2019

Bogota1

As I mentioned recently, I am getting ready for a trip to Bogotá, where I will be giving a series of workshops and two public lectures. Official announcements are now online for the talks, the first of which, on September 10 at the Universidad Jorge Tadeo Lozano, will be on “Animating Frankenstein: Film, Comics, and Serialized Visual Culture.” More info is online, here.

The second talk, on September 12 at the Universidad Nacional de Colombia, deals with the topic of my forthcoming book: “Discorrelation and the Post-Perceptual Image.” More info is available here.

Both of the talks will apparently have live translation into Spanish!

Bogota2

Images of Discorrelation, MECS Lecture July 3, 2019 (Video)

Screen Shot 2019-07-22 at 4.41.54 PM

On July 3, 2019, I delivered a talk related to my forthcoming book, Discorrelated Images, at Leuphana Universität’s Institute of Advanced Study on Media Cultures of Computer Simulation (MECS), during my fellowship in Lüneburg. The video is now online, and can be viewed here (or the direct link to YouTube).

Thanks to Florian Hoof for the kind invitation, and for everyone at MECS and the Center for Digital Cultures for hosting me this summer!