Notes and Impressions on the potential of simulative technologies in Social Research
This project is best accessed via a computer browser
if you are on mobile turning desktop view on might help a little
The pretext :
“The myth of Total Cinema”
The term emerged in the mid 1940s, coined by novelist René Barjavel and popularized by film theorist André Bazin and it expressed an abstraction regarding purpose of cinema, as both authors believed that the idea of cinema has not been realized yet (Leotta, 2018). Adopting an idealist approach Bazin expressed that the material conditions of what we came to call cinema preexisted its invention and that the spirit that is guiding the this evolution is that of “a total and complete representation of reality […] the reconstruction of a perfect illusion of the outside world in sound, color, and relief.”; hence the concept of the “myth” as Bazin is relating it to the myth of Icarus (Bazin, 2005).
“This evolution will be complete when it is able to offer us characters in full relief, in full colour, and even perhaps whose perfume we can detect; a time when these characters will be freed from the screens and the darkness of the film theatres to step out into the city streets and the private quarters of their audiences.”
René Barjavel in (Leotta, 2018)
André Bazin
The concept of a cinematic totality reappeared the last decade with technological innovations in VR [Virtual Reality] which appart from visual simulation, have started to expand to other sensorial realms, such us motion capturing, haptic feedback and even smell.
Apart from the advancements in the scope of experiential agency, innovations in photogrammetry, Computer-generated imagery (CGI) rendering, and omnidirectional cinematography / sound design are elevating the virtual worlds.
Two processes of photogrammetry.
Photography (top)
LiDAR – Light Detection and Ranging(left)
There are still great leaps to be covered in order to get to the totality of the cinematic experience, with one of the greatest challenges being the capturing of a continuum of time instead of its stasis. For the time being these limitations can only be overcomed in simulations where the majority of the environment is artistically fabricated ( e.g.. Video Games ); a field with its own limitations. Nevertheless, the above-mentioned innovations are already shaping our world with their applications ranging from medicine to architecture, forensics, entertainment, communication, and the list goes on.
So, where do the social sciences stand in this emerging field ?
Simulating
the
Social
[ Between ethnographic immersion and aestheticization ]
The greatest promise that the myth of Total Cinema -and subsequently these technologies- is immersion. A film without frame. An experience with greater freedom of interaction.
Does that mean the end of authorship?
Does that mean a creation without positionality?
Would that be an embracement of relativism?
Can there be a direct simulation without cultural mediation?
What are the politics of a simulation?
These are some of the questions that stimulate a cultural / societal perspective on these matters. Questions in the realm of myth and fiction. Social Science Fiction if you may. But in Bazinian terms, fiction is not separated from reality, fiction is the driving force of material creativity and we should take it seriously.
But let’s betray Bazin for now. Let’s be marxists and observe what’s emerging through the current material circumstances.
Produced for the United Nations, Clouds Over Sidra is a stationary VR story of a 12-year old girl, Sidra at the Za’atari camp in Jordan.
[Use the mouse to move the perspective around]
Coral Compass: Fighting Climate Change in Palau – Stationary VR experience created by The Virtual Human Interaction Lab at Stanford University
“See and experience how Palau is adapting to climate change to combat its effects on their reefs and economy.”
[Use the mouse to move the perspective around]
This time The Virtual Human Interaction Lab created a semi-interactive (stationary but with hand control) VR experience which attempts to simulate homelessness.
[the “game” can be downloaded for free on Steam but requires a VR headset & controllers. Here you can watch a playthrough.]
3D scanned exibits via photogrammetry from the collection of the Peabody Museum of Archaeology and Ethnology at Harvard University.
But lets step back a briefly from instances where simulative technologies are put in use consciously in order to participate in social issues and academic discourse; and wonder around two “low tech” examples of blockbuster video games.
The Grand Theft Auto (GTA) series, is one of the most successful videogame franchises of all time. Rockstar the company behind the series, has managed multiple times to break industry records on production budgets with its latest game costing almost 200 million dollars. The GTA games are set in an open-world, experienced through a 3rd-person perspective, and since their initial release they have caused major controversies, due to their emphasis of simulating (mostly racialized) criminal activity. GTA: San Andreas follows the life of a Black American protagonist, who left his city to seek a different life but is dragged back to the gang violence reality of Los Santos (Los Angeles) after the unexpected murder of his mother. The game is inspired by a series of real-life events such us the crack epidemic, police corruption in anti-gang units and the 1992 Los Angeles Riots.
Also a 3rd person , open world game the Assassin’s Creed series, portrays an imagined ancient conflict spanning millennia. On one side are the Order of Assassins, champions of peace and personal freedom, while on the other stand the Knights Templar, advocates for peace achieved through structure and authority. Within the series, elements of historical fiction, science fiction, and fabricated personas are seamlessly woven into actual historical events and the lives of prominent figures. Each game has as extensive log which provides contexts for the real live events that influenced the game’s plot. In the third installments of the series, you play as an a half-British, half-Mohawk protagonist Ratonhnhaké:ton who becomes a central figure in the American Revolution and the subsequent Revolutionary War as he helps the Patriot cause with the goal of protecting his people’s lands from incursions and preventing the Templars from taking control of the young United States.
Although many game developers avoid the analytic disclosure of their world-making process, there are prime examples that showcase dynamic involvement of people associated with the social sciences and humanities.
Maxime Durant a historian has been employed by Ubisoft (the main production company behind Assassins’ Creed) for the past 13 years in credited as World Design Director and Production Cordinator.
Kate Edwards a geographer, has been cultural and political consultant in the game industry since 2005 and has been included in Futures. “I’ve been in the industry for 27 years and there’s a reason why I’m vocal: it’s because I want other women, especially young women, who are considering gaming as a career, who see all the bad stuff online, to join gaming.”
But if social scientists are consulting game developers to create more immersive worlds, should game developers consult social scientists to create more immersive research ?
Huni Kuin: Yube Baitana is a side-scrolling, platformer video game and its a result of an interdisciplinary project involving anthropologists, programmers and artists from São Paulo (SP) and the Huni Kuin (Kaxinawa) people of Rio Jordão (AC).
“A key part of the game’s creation methodology was the four ethnographic incursions to the Huni Kuin villages in Acre (in the Kaxinawa indige nous lands Alto Rio Jordão, Baixo Rio Jordão and Seringal Independência), where drawing, singing and storytelling workshops were held in order to elaborate the thematic proposal of the game, as well as visual and narrative content production for each level” (GUILHERME PINHO MENESES, 2017)
[ The game is expected to be release in Q3 2024. A free demo is available o steam ]
Never Alone is also a side-scrolling platform adventure made in partnership between the Cook Inlet Tribal Council in Alaska. Unlike the academic origin of Huni Kuin, never alone is a comercial product is a commercial that acomplieshed a major success and was released on all the major platforms.
Bonus : Gaming ethnography | Undergrad Sophia Rosenberg
Rosewood: An Interactive History is a project by Edward González-Tennant Assistant Professor in the Department of Anthropology at the University of Texas Rio Grande Valley. On this ongoing research (est. 2005) González uses gaming technology to explore Rosewood, Florida; an African American town destroyed during the 1923 race riot. González incorporates a plethora of technologies, such as photogrammetry and 3D modeling, in order to construct a virtual representation of the town and its erased history after the massacre.
“
“When I began this research in 2005 I expected to develop a traditional archaeological project, complete with large-scale excavations. However, my growing network of collaborators felt this would add little to their understanding of Rosewood.”
-González, 2015
“The appearance of these structures is based on oral history accounts, property descriptions, and the documentation of extant historic structures.”
-González, 2015
What is the
meaning of all this ?
[an outcry for interdisciplinarity]
Do we need a simulation in order to feel empathy for the homeless or the refugee?
Where does the constructive immersion ends and the aestheticization starts?
If filming is can be a research tool how do video games differentiate?
How much are we willing to sacrifice our academism for the sake of accessibility?
How do we produce interactive experiences that are reflexive and accountable?
This multimodular journey is a experimental attempt to traverse through the narrow corridors between applied ethnographic research and realm of simulative technologies. It acts as a draft blueprint for reaserch, resulted out of personal experimentation and passion in the fields of Visual Anthropology and Science and Technology Studies. As one of the main concerns in ethnography is the power relations between the reasercher and the reaserched and the resulted mediated material; I was fascinated by Bazin’s myth and wanted to investigate the societal, political and academic ramifications of such concept. As Westmoreland (2022) points out, ethnographic research at its essence was always a multinodular approach despite the textual sovereignty of academia. Visual ethnography was a catalytic addition to social research as it embraced the evermore fragmented process of the establishment of subjectivity and culture in general. Our challenge now is to expand on this tradition by discovering the different the means which could enable our understanding of this diverse experience. I would argue that this can only be attempt via interdisciplinary projects.”While the polysemic affordances of multimodality offer visual anthropology a more expansive nomenclature, multimodality is not reducible to any one subdiscipline but rather signals a much broader invitation” (Westmoreland, 2022).
Homo Universalis is dead.
The knowledge of our times is decentralized is cannot be accumulated in one particular point of reference.
Any references not mentioned below are linked to their online source on the text
References
Bazin, A. (2019). THE MYTH OF TOTAL CINEMA. In What Is Cinema? Volume I (pp. 17–22). Berkeley: University of California Press
Leotta, A. (2018). Total cinema: René Barjavel and the future forms of film. Screen (London), 59(3), 372–380
Westmoreland, M. R. (2022). Multimodality: Reshaping Anthropology. Annual Review of Anthropology, 51(1), 173–194
Meneses, G.P., & Baitana, Y. (2017). KNOWLEDGE IN PLAY: THE CREATION OF THE VIDEO GAME HUNI KUIN:.
González E.,(2015) “Resurrecting Rosewood: New Heritage as Applied Visual Anthropology” in Participatory Visual and Digital Research in Action, Aline Gubrium, Krista Harper, and Marty Otañez, eds., pp. 163-177.
Additional Sources
Rosewood: An Interactive History – https://www.digital-heritage.net/port/raih/
VISUAL DOCUMENTATION LAB – http://atokovinine.people.ua.edu/3d-lab.html
MIT Open Documentary Lab – https://docubase.mit.edu/
Virtual Human Interaction Lab – https://stanfordvr.com/#aboutus
Influences
González E(2017),The Virtual Museum of Human Evolution: Simulation and Physical Anthropology in GAMING ANTHROPOLOGY: A SOURCEBOOK FROM # ANTHROPOLOGYCON, pp. 26-30
DUMIT J. (2017), Game Design as STS Research in in GAMING ANTHROPOLOGY: A SOURCEBOOK FROM # ANTHROPOLOGYCON, pp. 31-38
DeVane, B., & Squire, K. D. (2008). The Meaning of Race and Violence in Grand Theft Auto: San Andreas. Games and Culture, 3(3-4), 264–285
Russell, C. (1999). Experimental Ethnography : The Work of Film in the Age of Video. Durham, NC, USA: Duke University Press