Playing the Photographer: Creative Self-Expression through In-Game Photography — Jason Lajoie

The Last of Us™ Remastered_20140801163128

Jason Lajoie’s abstract for the panel “Generativity and Creative Agency in Post-Cinematic Media” at SLSA 2016 in Atlanta:

Playing the Photographer: Creative Self-Expression through In-Game Photography

Jason Lajoie, University of Waterloo

The photo editor mode offered in The Last of Us Remastered (TLOUR), Naughty Dog’s 2014 Playstation 4 port of their 2013 Playstation 3 game, offers players the means to take a photo at any point in the game. This option alters and can also elevate user engagement within the game. It can also make statements about self-expression through gameplay, as notably illustrated by conflict photographer Ashley Gilbertson, who applied the same techniques he acquired in real-world conflict zones to the fictitious battlegrounds in the game, achieving his photographs by considering his situatedness as real spectator in virtual environments. From a game studies perspective, the photo editor mode enables new ways for players and designers to think about game design, and offers innovative means of expression for players to interact in a creative game space. My investigation draws on Roland Barthes’s exploration into the affective capacity of photographs, and Jose Van Dijck’s claim that the malleability and manipulability of digital photography affects the formation of identity by repurposing our memories and means of communication. What are the affective resonances of photographs on real spectators when the spectrum itself is virtual? By exploring the use of photo editor modes in TLOUR, and in other Playstation 4 titles like The Order: 1886, I consider the ways this program expands the affordances of gameplay and narrativity by providing players interactive means for creative expression in otherwise restrictive and linear game modes.

Post-Cinema (Book Trailer)

Here is a short “book trailer” for the open-access collection Post-Cinema: Theorizing 21st-Century Film, edited by Shane Denson and Julia Leyda (REFRAME Books, 2016).

Also, don’t forget that we’ll be having a book launch party at Pro qm Books in Berlin this coming Friday, June 24. See here for more info and a flyer.

Out Now — Post-Cinema: Theorizing 21st-Century Film

Post_cinema_cover_NEW

I am happy to announce, at long last, the publication of Post-Cinema: Theorizing 21st-Century Film, edited by Shane Denson and Julia Leyda, which is out today as a completely free and open-access volume with REFRAME Books.

If cinema and television, as the dominant media of the 20th century, shaped and reflected our cultural sensibilities, how do new digital media in the 21st century help to shape and reflect new forms of sensibility? In this collection, editors Shane Denson and Julia Leyda have gathered a range of essays that approach this question by way of a critical engagement with the notion of “post-cinema.” Contributors explore key experiential, technological, political, historical, and ecological aspects of the transition from a cinematic to a post-cinematic media regime and articulate both continuities and disjunctures between film’s first and second centuries.

The book will appear in several digital formats: the web-based version is online today, and several ebook formats will be appearing soon.

The book brings together foundational texts by some of the key voices in the discussion of post-cinema and places them next to a range of brand-new chapters, as well as a series of roundtable discussions.

The long list of contributors includes:

Caitlin Benson-Allott, Paul Bowman, Felix Brinker, Kristopher L. Cannon, Francesco Casetti, Steen Christiansen, Elena del Río, Shane Denson, Rosalind Galt, Therese Grisham, Richard Grusin, Leon Gurevitch, Mark B. N. Hansen, Bruce Isaacs, Adrian Ivakhiv, Kylie Jarrett, Selmin Kara, ​Julia Leyda, Patricia MacCormack, Lev Manovich, Ruth Mayer, Michael O’Rourke, Patricia Pisters, Alessandra Raengo, David Rambo, Nicholas Rombes, Sergi Sánchez, Karin Sellberg, Steven Shaviro, Michael Loren Siegel, Vivian Sobchack, Billy Stevenson, Andreas Sudmann

Here is the table of contents:

Post-Cinema-TOC

A brief “press release” with a description of the book and the complete table of contents is available here (opens as a PDF): POST-CINEMA-Press-Release

Reframe_Books_black

After.Video at Libre Graphics 2016 in London

banner_glitch_1

Recently, I posted about a project called after.video, which contains an augmented (AR) glitch/video/image-based theory piece that Karin Denson and I collaborated on. It has now been announced that the official launch of after.video, Volume 1: Assemblages — a “video book” consisting of a paperback book and video elements stored on a Raspberry Pi computer packaged in a VHS case, which will also be available online — will take place at the Libre Graphics Meeting 2016 in London (Sunday, April 17th at 4:20pm).

Post-Cinematic Affect, Collectivity, and Environmental Agency #scms16

Post-Cinematic-Affect-Collective-Environment-short.001

Full text of the talk I presented at the SCMS conference today:

Post-Cinematic Affect, Collectivity, and Environmental Agency

Shane Denson (Duke University)

The computational and broadly post-cinematic media at the heart of contemporary moving images are involved in a massive transformation of human agents’ phenomenological relations to the world. Digital imagery has long been held accountable for effacing the indexicality of cinema’s photographic base, while post-cinematic images more generally might be thought in terms of what I call their “discorrelation” from viewing subjects. However, there is a flip-side to these negative determinations that I want to highlight: if the microtemporal and subperceptual operations of post-cinematic media bypass and hence displace subjective perception, they also serve to expand the material domain and efficacy of sub- and supra-personal affect. What this amounts to, ultimately, is a radical empowerment of the nonhuman environment, the agency of which becomes tangible in sites and forms ranging from the Fitbit to “big data” and the computational modeling of climate change.

In this presentation, which is divided into three sections, I want to take this thought a step further. I want to show, ultimately, that new forms of collectivity may become thinkable and, hopefully, actionable in the spaces opened up by post-cinematic media.

Part 1: Irrational Cameras. Let me start by summarizing an argument I have made elsewhere about the transformation of the camera and its images in a post-cinematic media regime. Post-cinematic cameras – by which I mean a range of imaging apparatuses, both physical and virtual – seem not to know their place with respect to the separation of diegetic and nondiegetic planes of reality; these cameras therefore fail to situate viewers in a consistently and coherently designated spectating-position.

Post-Cinematic-Affect-Collective-Environment-short.004

Take the example of the digitally simulated lens flare, which a phenomenological analysis reveals to be riddled with perceptual paradoxes. On the one hand, the CGI lens flare encourages what philosopher of technology Don Ihde calls an “embodiment relation” to the virtual camera: by simulating the material interplay of a lens and a light source, the lens flare emphasizes the plastic reality of “pro-filmic” CGI objects; the virtual camera itself is to this extent grafted onto the subjective pole of the intentional relation, “embodied” in a sort of phenomenological symbiosis that channels perception towards the objects of our visual attention. On the other hand, however, the lens flare draws attention to itself and highlights the images’ artificiality by emulating (and foregrounding the emulation of) the material presence of a camera. To this extent, the camera is rendered quasi-objective, and it instantiates what Ihde calls a “hermeneutic relation”: we look at the camera rather than just through it, and we interpret it as a sign or token of “realisticness.”

Post-Cinematic-Affect-Collective-Environment-short.006

The paradox here, which consists in the realism-constituting and realism-problematizing undecidability of the virtual camera’s relation to the diegesis – where the “reality” of this realism is conceived as thoroughly mediated, the product of a simulated physical camera rather than defined as the hallmark of embodied perceptual immediacy – points to a more basic transformation of mediation itself in the post-cinematic era. That is, the undecidable place of the mediating apparatus, the camera’s apparently simultaneous occupation of both subjective and objective positions within the noetic relation that it enables between viewers and the film, is symptomatic of a more general destabilization of phenomenological subject- and object-positions in relation to the expanded affective realm of post-cinematic mediation. Computational, ergodic, and processual in nature, media in this mode operate on a level that is categorically beyond the purview of perception, perspective, or intentionality. Phenomenological analysis can therefore provide only a negative determination “from the outside”: it can help us to identify moments of dysfunction or disconnection, but it can offer no positive characterization of the “molecular” changes occasioning them. Thus, for example, CGI and digital cameras do not just sever the ties of indexicality that characterized analogue cinematography (an epistemological or phenomenological claim); they also render images themselves fundamentally processual, thus displacing the film-as-object-of-perception and uprooting the spectator-as-perceiving-subject – in effect, enveloping both in an epistemologically indeterminate but materially quite real and concrete field of affective relation. Mediation, I suggest, can no longer be situated neatly between the poles of subject and object, as it swells with processual affectivity to engulf both.

Post-Cinematic-Affect-Collective-Environment-short.008

Part 2: The dilation of affect. What I have been describing here is a decentering of human perception and an empowerment of the larger environment. In order to account for this transformation, it will be helpful here to invoke Mark Hansen’s notion of “atmospheric media,” a concept that Hansen develops to explain the experiential impact of computation, but which builds upon Maurizio Lazzarato’s theorization of an affective dimension of video technologies.

Post-Cinematic-Affect-Collective-Environment-short.010

According to Lazzarato, the video camera captures time itself, the splitting of time at every instant, hence opening the gap between perception and action where affect resides (in the metaphysics of Henri Bergson). Because it no longer merely traces objects mechanically and fixes them as discrete photographic entities, but instead generates its images directly out of the flux of sub-perceptual matter, which it processes on the fly in the space of a microtemporal duration, the video camera marks a revolutionary transformation in the technical organization of time. The mediating technology itself becomes an active locus of molecular change: a Bergsonian body qua center of indetermination, a gap of affectivity between passive receptivity and its passage into action. The camera imitates the process by which our own pre-personal bodies synthesize the passage from molecular to molar, replicating the very process by which signal patterns are selected from the flux and made to coalesce into determinate images that can be incorporated into an emergent subjectivity.

Post-Cinematic-Affect-Collective-Environment-short.011

This dilation of affect, which characterizes not only video but also computational processes like the rendering of digital images (which is always done on the fly), marks the basic condition of the post-cinematic camera, the positive underside of what presents itself externally as a discorrelating incommensurability with respect to molar perception. As Mark Hansen has argued, the microtemporal scale at which computational media operate enables them to modulate the temporal and affective flows of life and to affect us directly at the level of our pre-personal embodiment. In this respect, properly post-cinematic cameras, which include video and digital imaging devices of all sorts, have a direct line to our innermost processes of becoming-in-time, and they are therefore capable of informing the political life of the collective by flowing into the “general intellect” at the heart of immaterial or affective labor. Again, this is because the individual’s capacity to perceive is decentered, discorrelated from the perceptual object, and offloaded onto an environment of diffusely “atmospheric” media – including the many screens and cameras, but also the invisible networks and data streams, that surround us everywhere.

Part 3: Post-Cinematic Realism. Paradoxically, these arguments suggest that post-cinematic media – precisely those media widely credited with destroying the index and thus evacuating the political promise of realism – might in fact be credited with a newly intensified political relevance through their institution of a new, post-cinematic realism.

Post-Cinematic-Affect-Collective-Environment-short.013

Whereas Bazin privileged techniques like the long take and deep focus for their power to approximate our natural perception of time and space, Lazzarato and Hansen emphasize post-cinematic media’s ability to approximate the sub-perceptual processing of duration executed by our pre-personal bodies. The perceptual discorrelation of images gives way, in other words, to a more precise calibration of machinic and embodied temporalities; simultaneously, the perceptual richness of Bazin’s images becomes less important, while “poor images” (in Hito Steyerl’s term) communicate more directly the material and political realities of a post-cinematic environment.

Post-Cinematic-Affect-Collective-Environment-short.016

Consider the 2015 horror flick Unfriended, which is presented as the screen recording of one of the characters’ laptops. Reflecting what Francesco Casetti calls the “relocation” of cinema from the big screen to a variety of little ones, the movie’s sense of “realism” is especially heightened when you watch it on your own laptop. We witness everything on this single screen, through Skype conversations, Facebook chats, email, and web browsing. And it’s essential for the movie that it’s presented in “real-time.” This adds to the temporal urgency and speaks to the reality of our own online communications today, thereby establishing a sense of realism despite the fantastic/supernatural elements at play, and articulating this reality despite—or precisely through—the use of digital glitches. These might otherwise be taken to signal the interruption of realism by the intercession of digital processing that breaks the indexical continuity between image input and image output, but such glitches are a familiar reality of online communication (on platforms like Skype), and our involvement in the images is increased by their use; for example, we might wonder whether the glitches are diegetic, or whether they are produced on our own machine during playback, either due to the buffering processes of online streaming platforms, or because we downloaded a faulty torrent file from some dubious website. Realism here is constructed through an immediacy and direct exploration of the new media-technical conditions of life, to which we can all more or less relate. But in the process the glitches also expose the movie’s singular screen as, in fact, double: the site of playback, traditionally a passive “screening” surface, the screen is also revealed as a newly active site or space in which images are processed and generated before our very eyes. The glitches point up the perceptual paradoxes of post-cinematic cameras, as I’ve described them with respect to CGI lens flares, but they additionally implicate the post-cinematic screen, which becomes ontologically indistinguishable from the camera in its execution of the same material processes of microtemporal and subperceptual image generation.

Post-Cinematic-Affect-Collective-Environment-short.017

These glitches, and their relation to our contemporary media-technical realities, call attention to what Hito Steyerl has called the “poor images” that circulate in digital networks. Following Steyerl, these images provide an important context for thinking about the political realities of moving-image media today—and an important context for thinking about post-cinematic realism more generally. In Steyerl’s words: “The poor image is an illicit fifth-generation bastard of an original image. Its genealogy is dubious. Its file names are deliberately misspelled. It often defies patrimony, national culture, or indeed copyright. It is passed on as a lure, a decoy, an index, or as a reminder of its former visual self. It mocks the promises of digital technology. Not only is it often degraded to the point of being just a hurried blur, one even doubts whether it could be called an image at all. Only digital technology could produce such a dilapidated image in the first place.” These poor images are close in spirit to the “imperfect cinema” called for in the name of Third Cinema movements, in that they register social marginalization processes while also creating publics of their own. But they also outline the dark side of a “participatory culture,” whose democratic promise is compromised by the hierarchies of value that remain and by the exploitation of unpaid fan labor that is enlisted in the ongoing production-consumption circuits of networked images. Without extracting themselves from these conflicting political trajectories, according to Steyerl, poor images might nevertheless—or precisely for this reason—create what Dziga Vertov called “visual bonds” capable of subverting official and mainstream valuations by expressing what Steyerl terms a “link to the present.” In this way, degraded, glitched-out images might fulfill the political promise of realism precisely through their material connection to the post-indexical infrastructures of moving-image media. In Steyerl’s words: “The poor image is no longer about the real thing—the originary original. Instead, it is about its own real conditions of existence: about swarm circulation, digital dispersion, fractured and flexible temporalities. It is about defiance and appropriation just as it is about conformism and exploitation. In short: it is about reality.”

Like Steyerl, Lazzarato also refers to Vertov and his idea of the “visual bond,” which is seen as a materialist alternative to the critique of ideology, the expression of a practice that addresses the ontology of media directly and prior to the level of content. Essentially, by resisting reduction to human perception, the images of Vertov’s kino-eye are discorrelated from molar experience but thereby opened to the molecular processes by which duration is processed both biologically and technologically, thus getting to the heart of the process by which subjectivities and social collectives are produced. If Bazin described a cinematic realism that draws for its political power on an approximation to perceptual experience, then Vertov marks the path towards a post-cinematic realism that takes aim at the process by which the subject of that perceptual experience takes shape in the first place. It does this, according to Lazzarato, by means of the pre-personal affect that is marshaled and modulated by the increasingly fine-grained “time-crystallizing machines” of cinema, video, and digital processors. Accordingly, the video art of Nam June Paik is seen as a Vertovian answer to television, not because it counters the ideological content of TV but because it probes the machinic time itself of the apparatus, freeing it from the exclusive control of state and corporate interests. The latter, according to Lazzarato, contribute to the production and regulation of political subjects through their control of technical standards (like the PAL and NTSC standards that regulate image frequency, color spectrum, and aspect ratio); because the power to modulate the speeds and images dictated by such standards is “withdrawn from social praxis” (78), our affective powers are impoverished, and we are left with what Lazzarato calls a “‘poor’ perception” (78). The ontology of time-crystallizing machines thus gives way to an ethics or politics of the standards, codes, or protocols upon which images or perceptual objects are formed and synchronized with emergent subjects and social collectives. And because they expose the materiality of digital file formats, video codecs, and compression algorithms, today’s poor images harbor a significant political promise, a potential for resistance that can be deployed creatively against the impoverishment and standardization of perception.

Post-Cinematic-Affect-Collective-Environment-short.024

It is debatable, finally, whether a movie like Unfriended actually succeeds in this respect. Certainly, at the level of its narrative, it apparently fails to articulate anything like a model of social-political resistance; if anything, its teenage drama of betrayal, suicide, and revenge, all mediated by the networks and interfaces of social media, and leading to the death of the entire group of “friends,” serves as a critique of contemporary socialization processes – an ideological critique that takes aim not only at online bullying, for example, but that exposes an infrastructure of communication and of intersubjective relation that has rendered the term “friend” itself highly unstable in the age of Facebook. But beyond this more overt political critique of today’s highly mediated forms of collectivity, the movie’s use of glitches serves to focus attention, and to channel affect, at a deeper level, where subjectivity itself is being produced and modulated in an environment of microtemporally operating machines and protocols. Glitches serve at times like micro-cliffhangers, causing us to wait for the image to buffer or clear up so that we can see what’s going on. In this respect, the movie simulates the familiar and yet always disconcerting experience of network lag, e.g. in our own Skype conversations, when the temporal continuity of protentional-retentional experience is interrupted, giving rise to a feeling like that of a cartoon character who has gone over the edge of a cliff, but remains suspended, floating momentarily between the certainty of solid ground and a realization of the situation’s gravity. These micro-cliffhangers focus our attention on the material infrastructure of experience itself, causing us to see pixels as the components but also as material obstacles to vision, blocky screen objects that, despite ourselves, we try to look around to see what’s on the other side. And in this space of the screen, seemingly unitary but, as we have seen, doubled and in fact multiplied even further by the machinic and social networks in which it participates (both diegetically and materially), our vision is dispersed, divided. We are forced to scan the screen for relevant information; our gaze is not sutured, not directed, and to this extent we are hailed not as an integral subject, but as a bundle of affects engaged in a collective effort to perceive—an effort that is both enabled and hindered by the protocols and agencies of the media environment, out of which our subjectivities are wrought. Unfriended may or may not ultimately facilitate our efforts to take control of this experiential infrastructure, but perhaps it succeeds in gesturing towards the fact that this effort must be a collective one, aimed at constructing collectivity in the first place, and that it must be mounted around and in relation to the affective technologies of our post-cinematic environment.

Postnaturalism reviewed in MEDIENwissenschaft

medienwissenschaft-review

The latest issue of MEDIENwissenschaft: Rezensionen/Reviews includes a nice review of my book Postnaturalism: Frankenstein, Film, and the Anthropotechnical Interface. 

For those of you who read German, you can find the entire text of the review, by Anya Heise-von der Lippe (Tübingen/Berlin), here. For everyone else, here is a (rough) translation of the reviewer’s summary statement:

Postnaturalism offers a philosophical approach and an engagement with fundamental ontological and phenomenological questions of human and nonhuman materiality, which is indispensable especially for a post-postmodernity characterized by resource scarcity, climate change, and species extinctions, as well as the threat of a return to essentialist positions in politics and popular culture. Adapting a phrase from Bruno Latour, Denson counters the latter with a postnatural position: “We have never been natural” (24). Furthermore, Denson’s detailed examination — at the level of content, reception, and production — of Frankenstein adaptations is an asset for the analytical and production-aesthetic [produktionsästhetische] investigation of a central text (or modern myth) and its many adaptations in a wide range of text-critical disciplines: from media studies to literary to cultural studies.”

(Again, the translation is rough. Tweaks are more than welcome! Especially if you have suggestions for produktionsästhetisch or for making that first sentence more readable, drop me a line in the comments below…)

Finally, make sure you check out the entire issue of MEDIENwissenschaft, which is chock full of great stuff. Of particular interest to readers of this blog, among other things: the “Perspectives” section contains a longer piece on seriality and television series’ interrelations by Tanja Weber and Christian Junklewitz.

Check out the full contents of the issue here.

Post-Cinema at SCMS 2016 #scms16

scms16

At this year’s conference of the Society for Cinema and Media Studies (March 30-April 3, 2016), I will be involved in two post-cinema related panels.

First, on Saturday, April 2, 2016 09:00AM-10:45AM (panel N9), I will be giving a talk as part of a panel on affect, collectivity, and contemporary cinema:

N9: “Affect, Collectivity, Contemporary Cinema.”

Chair: Claudia Breger (Indiana University)

Shane Denson (Duke University), “Post-Cinematic Affect, Collectivity, and Environmental Agency”

Anders Bergstrom (Wilfrid Laurier University), “On Dissipation: The Loss of the Movie Theatre as Affective Site in “Goodbye, Dragon Inn””

Jecheol Park (National University of Singapore), “A Counter-neoliberal Collective to Come: Joshua Oppenheimer’s The Act of Killing”

Claudia Breger (Indiana University), “The Epic Aesthetics of Ruptured Collectivity in Fatih Akın’s “The Cut” (2014)”

Then, on Sunday, April 3, 2016 11:00AM-12:45PM, I will be responding to panel T19 on “post-cinematic control”:

T19: “Post-Cinematic Control”

Chair: Lisa Akervall (Bauhaus-University Weimar)

Respondent: Shane Denson (Duke University)

Lisa Akervall (Bauhaus-University Weimar), “The Truth of Auto-Tune: Voice Modulations in Post-Cinematic Media-Ecologies”

Viviana Lipuma (North Carolina State University), “Semiocapitalism: the production of signs as the production of desire in the media”

Gregory Flaxman (University of North Carolina, Chapel Hill), “Left of Conspiracy”

You can view the complete conference program here (opens as a PDF).

And here, finally, is the abstract for my paper:

Post-Cinematic Affect, Collectivity, and Environmental Agency

Shane Denson (Duke University)

The computational and broadly post-cinematic media at the heart of contemporary moving images are involved in a massive transformation of human agents’ phenomenological relations to the world. Digital imagery has long been held accountable for effacing the indexicality of cinema’s photographic base, while post-cinematic images more generally might be thought in terms of their “discorrelation” from viewing subjects. There is, however, a flip-side to these negative determinations: if the microtemporal and subperceptual operations of post-cinematic media bypass and hence displace subjective perception, they also serve to expand the domain and the material efficacy of sub- or supra-personal affect. What this amounts to, ultimately, is a radical empowerment of the nonhuman environment, the agency of which becomes tangible in sites and forms ranging from the Fitbit to “big data” and the computational modeling of climate change.

Drawing on Steven Shaviro’s account of post-cinematic affect, and supplementing it with Mark B. N. Hansen’s recent work on the “feed-forward” mechanisms by which biofeedback and environmental sensors serve to expand worldly agency, this presentation argues that new forms of collectivity may become thinkable in the spaces opened up by post-cinematic media. In the reconfiguration of agency, that is, by which digital media bypass the individual and transfer its powers to perceive and to act onto the nonhuman environment, the “dividuality” that Deleuze saw as a correlate of the control society may open onto a more positive conception of collective power. Maurizio Lazzarato’s provocative Bergsonist-Marxist “video philosophy” will serve as a catalyst for conceptualizing this new collectivity and its relation to moving-image media, while the work of independent filmmaker Shane Carruth (Primer [2004], Upstream Color [2013]) will help to focus the interplay among post-cinematic affect, environmental agency, and the mediation of collectivity in the micro- and macrotemporal intervals of contemporary media.

Bibliography:

Deleuze, Gilles. “Postscript on the Societies of Control.” October 59 (1992): 3-7.

Denson, Shane. “Crazy Cameras, Discorrelated Images, and the Post-Perceptual Mediation of Post-Cinematic Affect.” Post-Cinema: Theorizing 21st-Century Film. Eds. Shane Denson and Julia Leyda. Sussex: REFRAME Books, forthcoming 2015.

Hansen, Mark B. N. Feed-Forward: On the Future of Twenty-First-Century Media. Chicago: U of Chicago P, 2014.

Lazzarato, Maurizio. Videophilosophie: Zeitwahrnehmung im Postfordismus. Berlin: b_books, 2002.

Shaviro, Steven. Post-Cinematic Affect. Winchester: Zero Books, 2010.

Speculative Data: Full Text, MLA 2016 #WeirdDH

SpeculativeData-jpg.001

Below you’ll find the full text of my talk from the Weird DH panel organized by Mark Sample at the 2016 MLA conference in Austin Texas. Other speakers on the panel included Jeremy Justus, Micki Kaufman, and Kim Knight.

***

Speculative Data: Post-Empirical Approaches to the “Datafication” of Affect and Activity

Shane Denson, Duke University

A common critique of the digital humanities questions the relevance (or propriety) of quantitative, data-based methods for the study of literature and culture; in its most extreme form, this type of criticism insinuates a complicity between DH and the neoliberal techno-culture that turns all human activity, if not all of life itself, into “big data” to be mined for profit. Now, it may sound from this description that I am simply setting up a strawman to knock down, so I should admit up front that I am not wholly unsympathetic to the critique of datafication. But I do want to complicate things a bit. Specifically, I want to draw on recent reconceptions of DH as “deformed humanities” – as an aesthetically and politically invested field of “deformance”-based practice – and describe some ways in which a decidedly “weird” DH can avail itself of data collection in order to interrogate and critique “datafication” itself.

SpeculativeData-jpg.002

My focus is on work conducted in and around Duke University’s S-1: Speculative Sensation Lab, where literary scholars, media theorists, artists, and “makers” of all sorts collaborate on projects that blur the boundaries between art and digital scholarship. The S-1 Lab, co-directed by Mark Hansen and Mark Olson, experiments with biometric and environmental sensing technologies to expand our access to sensory experience beyond the five senses. Much of our work involves making “things to think with,” i.e. experimental “set-ups” designed to generate theoretical and aesthetic insight and to focus our mediated sensory apparatus on the conditions of mediation itself. Harnessing digital technologies for the work of media theory, this experimentation can rightly be classed, alongside such practices as “critical making,” in the broad space of the digital humanities. But due to their emphatically self-reflexive nature, these experiments challenge borders between theory and practice, scholarship and art, and must therefore be qualified, following Mark Sample, as decidedly “weird DH.”

SpeculativeData-jpg.003.jpeg

One such project, Manifest Data, uses a piece of “benevolent spyware” that collects and parses data about personal Internet usage in such a way as to produce 3D-printable sculptural objects, thus giving form to data and reclaiming its personal value from corporate cooptation. In a way that is both symbolic and material, this project counters the invisibility and “naturalness” of mechanisms by which companies like Google and Facebook expropriate value from the data we produce. Through a series of translations between the digital and the physical—through a multi-stage process of collecting, sculpting, resculpting, and manifesting data in virtual, physical, and augmented spaces—the project highlights the materiality of the interface between human and nonhuman agencies in an increasingly datafied field of activity. (If you’re interested in this project, which involves “data portraits” based on users’ online activity and even some weird data-driven garden gnomes designed to dispel the bad spirits of digital capital, you can read more about it in the latest issue of Hyperrhiz.)

SpeculativeData-jpg.004

Another ongoing project, about which I will say more in a moment, uses data collected through (scientifically questionable) biofeedback devices to perform realtime collective transformations of audiovisual materials, opening theoretical notions of what Steven Shaviro calls “post-cinematic affect” to robustly material, media-archaeological, and aesthetic investigations.

SpeculativeData-jpg.005

These and other projects, I contend, point the way towards a truly “weird DH” that is reflexive enough to suspect its own data-driven methods but not paralyzed into inactivity.

Weird DH and/as Digital Critical (Media) Studies:

So I’m trying to position these projects as a form of weird digital critical (media) studies, designed to enact and reflect (in increasingly self-reflexive ways) on the use of digital tools and processes for the interrogation of the material, cultural, and medial parameters of life in digital environments.

SpeculativeData-jpg.006

Using digital techniques to reflect on the affordances and limitations of digital media and interfaces, these projects are close in spirit to new media art, but they are also apposite with practices and theories of “digital rhetoric,” as described by Doug Eyman, with Gregory Ulman’s “electracy,” or with Casey Boyle’s posthuman rhetoric of multistability, which celebrates the rhetorical affordances of digital glitches in exposing the affordances and limitations of computational media in the broader realm of an interagential relational field that includes both humans and nonhumans. In short, these projects enact what we might call, following Stanley Cavell, the “automatisms” of digital media – the generative affordances and limitations that are constantly produced, reproduced, and potentially transformed or “deformed” in creative engagements with media. Digital tools are used in such a way as to problematize their very instrumentality, hence moving towards a post-empirical or post-positivistic form of datafication as much as towards a post-instrumental digitality.

SpeculativeData-jpg.007

Algorithmic Nickelodeon / Datafied Attention:

My key example is a project tentatively called the “algorithmic nickelodeon.” Here we use consumer-grade EEG headsets to interrogate the media-technical construction and capture of human attention, and thus to complicate datafication by subjecting it to self-reflexive, speculative, and media-archaeological operations. The devices in question cost about $100 and are marketed as tools for improving concentration, attention, and memory. The headset measures a variety of brainwave activity and, by means of a proprietary algorithm, computes values for “attention” and “meditation” that can be tracked and, with the help of software applications, trained and supposedly optimized. In the S-1 Lab, we have sought to tap into these processes in order not just to criticize the scientifically dubious nature of these claims but rather to probe and better understand the nature of the automatisms and interfaces taking place here and in media of attention more generally. Specifically, we have designed a film- and media-theoretical application of the apparatus, which allows us to think early and contemporary moving images together, to conceive pre- and post-cinema in terms of their common deviations from the attention economy of classical cinema, and to reflect more broadly on the technological-material reorganizations of attention involved in media change. This is an emphatically experimental (that is, speculative, post-positivistic) application, and it involves a sort of post-cinematic reenactment of early film’s viewing situations in the context of traveling shows, vaudeville theaters, and nickelodeons. With the help of a Python script written by lab member Luke Caldwell, a group of viewers wearing the Neurosky EEG devices influence the playback of video clips in real time, for example changing the speed of a video or the size of the projected image in response to changes in attention as registered through brain-wave activity.

At the center of the experimentation is the fact of “time-axis manipulation,” which Friedrich Kittler highlights as one of the truly novel affordances of technical media, like the phonograph and cinema, that arose around 1900 and marked, for him, a radical departure from the symbolic realms of pre-technical arts and literature. Now it became possible to inscribe “reality itself,” or to record a spectrum of frequencies (like sound and light) directly, unfiltered through alphabetic writing; and it became possible as well to manipulate the speed or even playback direction of this reality.

SpeculativeData-jpg.009

Recall that the cinema’s standard of 24 fps only solidified and became obligatory with the introduction of sound, as a solution to a concrete problem introduced by the addition of a sonic register to filmic images. Before the late 1920s, and especially in the first two decades of film, there was a great deal of variability in projection speed, and this was “a feature, not a bug” of the early cinematic setup. Kittler writes: “standardization is always upper management’s escape from technological possibilities. In serious matters such as test procedures or mass entertainment, TAM [time-axis manipulation] remains triumphant. [….] frequency modulation is indeed the technological correlative of attention” (Gramophone Film Typewriter 34-35). Kittler’s pomp aside, his statement highlights a significant fact about the early film experience: Early projectionists, who were simultaneously film editors and entertainers in their own right, would modulate the speed of their hand-cranked apparatuses in response to their audience’s interest and attention. If the audience was bored by a plodding bit of exposition, the projectionist could speed it up to get to a more exciting part of the movie, for example. Crucially, though: the early projectionist could only respond to the outward signs of the audience’s interest, excitement, or attention – as embodied, for example, in a yawn, a boo, or a cheer.

SpeculativeData-jpg.010

But with the help of an EEG, we can read human attention – or some construction of “attention” – directly, even in cases where there is no outward or voluntary expression of it, and even without its conscious registration. By correlating the speed of projection to these inward and involuntary movements of the audience’s neurological apparatus, such that low attention levels cause the images to speed up or slow down, attention is rendered visible and, to a certain extent, opened to conscious and collective efforts to manipulate it and the frequency of images now indexed to it.

According to Hugo Münsterberg, who wrote one of the first book-length works of film theory in 1916, cinema’s images anyway embody, externalize, and make visible the faculties of human psychology; “attention,” for example, is said to be embodied by the close-up. With our EEG setup, we can literalize Münsterberg’s claim by correlating higher attention levels with a greater zoom factor applied to the projected image. If the audience pays attention, the image grows; if attention flags, the image shrinks. But this literalization raises more questions than it answers, it would seem. On the one hand, it participates in a process of “datafication,” turning brain wave patterns into a stream of data called “attention,” but whose relation to attention in ordinary senses is altogether unclear. But this datafication simultaneously opens up a space of affective or aesthetic experience in which the problematic nature of the experimental “set-up” announces itself to us in a self-reflexive doubling: we realize suddenly that “it’s a setup”; “we’ve been framed” – first by the cinema’s construction of attentive spectators and now by this digital apparatus that treats attention as an algorithmically computed value.

So in a way, the apparatus is a pedagogical/didactic tool: it not only allows us to reenact (in a highly transformed manner) the experience of early cinema, but it also helps us to think about the construction of “attention” itself in technical apparatuses both then and now. In addition to this function, it also generates a lot of data that can indeed be subjected to statistical analysis, correlation, and visualization, and that might be marshaled in arguments about the comparative medial impacts or effects of various media regimes. Our point, however, remains more critical, and highly dubious of any positivistic understanding of this data. The technocrats of the advertising industry, the true inheritors of Münsterberg the industrial psychologist, are anyway much more effective at instrumentalizing attention and reducing it to a psychotechnical variable. With a sufficiently “weird” DH approach, we hope to stimulate a more speculative, non-positivistic, and hence post-empirical relation to such datafication. Remitting contemporary attention procedures to the early establishment of what Kittler refers to as the “link between physiology and technology” (73) upon which modern entertainment media are built, this weird DH aims not only to explore the current transformations of affect, attention, and agency – that is, to study their reconfigurations – but also potentially to empower media users to influence such configuration, if only on a small scale, rather than leave it completely up to the technocrats.

Speculative Data #WeirdDH #MLA16 #S107

storify-mla16

The “Weird DH” panel at MLA 2016 in Austin, chaired by Mark Sample, was great fun, and it generated quite a bit of discussion, both online and off. Luckily, Eileen Clancy preserved the twitter discussion in a Storify (https://storify.com/clancynewyork/weird-dh), so in case you couldn’t make it out last week, you can still catch up on some of the topics and reactions to the talks by Jeremy Justus, Micki Kaufman, Kim Knight, and myself.

(If I get around to it, I will also be posting the full text of my talk very soon.)

Conversations in the Digital Humanities at Duke

page-events-dhduke

Today, Oct. 2, 2015, the Franklin Humanities Institute, the Wired! Lab, the PhD Lab in Digital Knowledge, and HASTAC@Duke will be presenting “Conversations in the Digital Humanities,” the inaugural event of the new Digital Humanities Initiative at Duke University. More information about the event, in which I will be participating alongside colleagues from the S-1: Speculative Sensation Lab, can be found on the FHI website.

Also, all of the 10-minute “lightning talks” will be live-streamed. The first block of sessions, from 2:15-3:45pm EST, will be streamed here, and the second block, from 4:00-5:40pm, will be viewable here. (Apparently, the videos will be archived and available after the fact as well.)

Here is the complete schedule:

2:00 – 2:15
Welcome and Introduction to Digital Humanities Initiative

2:15 – 3:45 
Session 1 (10 minutes per talk)

  1. Project Vox (Andrew Janiak, and Liz Milewicz)
  2. NC Jukebox (Trudi Abel, Victoria Szabo)
  3. Visualizing Cultures: The Shiseido Project (Gennifer Weisenfeld)
  4. Going Global in Mughal India (Sumathi Ramaswamy)
  5. Israel’s Occupation in the Digital Age (Rebecca Stein)
  6. Digital Athens: Archaeology meets ArcGIS (Tim Shea, Sheila Dillon)
  7. Early Medieval Networks (J. Clare Woods)

3:45 – 4:00
Coffee Break

4:00 – 5:40 
Session 2 (10 minutes per talk)

  1. Painting the Apostles – A Case Study in “The Lives of Things” (Mark Olson, Mariano Tepper, and Caroline Bruzelius)
  2. Digital Archaeology: From the Field to Virtual Reality (Maurizio Forte)
  3. The Memory Project (Luo Zhou)
  4. Veoveo, children at play (Raquel Salvatella de Prada)
  5. “Things to Think With”: Weird DH, Data, and Experimental Media Theory (S-1 Lab)
  6. s_traits, Generative Authorship and the Emergence Lab (Bill Seaman and John Supko)
  7. Found Objects and Fireflies (Scott Lindroth)
  8. Project Provoke (Mary Caton Lingold and others)

5:40 – 6:00 
Reception