This is a guest post from FFDW project partner Starling Lab, written by Lindsay Walker and edited by Adam Rose.

The internet and related technologies have brought about a new structure to the “reality” that we perceive –  and the way we interact with that reality. The internet enables people to instantly find information, communicate, and even publish on a global scale. New opportunities and challenges are constantly arising as humankind continues to develop technologies that empower worldwide connection. Starling Lab aims to address challenges related to trust in digital records.

Our digital age has brought about, in a matter of decades, a way of interacting and recording our experiences that has fundamentally shifted our world on a global scale. The influence that the world wide web, social media, and AI now have on our values, beliefs, and politics has become a powerful and important global issue. Starling Lab’s work aims to have a positive impact by developing technology and methodology to authenticate and protect digital records.

Capture, Store, Verify

Starling Lab, co-founded by Stanford University’s Department of Electrical Engineering and USC Shoah Foundation, develops technology and protocols for digital records that can establish provenance and authenticity. We use a framework of Capture, Store, Verify with fellows and collaborators in journalism, law, history, and technology. The purpose of our work is to establish trust in digital assets such as photographs, websites, audio, and video.

Starling Lab does this work while supporting the ethical use of Web3 technologies to empower historians, lawyers, and journalists to use cryptography and decentralized systems to protect the authenticity and provenance of digital assets. We do this in a manner that respects privacy and ensures the preservation of important digital assets over time.

Generative AI is enabling more convincing deepfakes, such as this video of the U.S. President declaring World War 3. It is increasingly important to establish ways we can create content that the public can trust. There is a clear need to develop tools and protocols that can help establish authenticity starting from the moment a digital asset is captured or created, and in a way that is provable and reliable. Doing this as close to the source as possible establishes a verifiable root of trust. This work continues into storing, distributing, and even publishing the authenticity work to make it available to the world.

It takes technological innovation to capture, store, and eventually publish these digital assets in a verifiable manner. We extend this innovation into collaborations on projects and case studies with experts in a range of different domains of digital media and preservation to develop solutions for real-world needs and use cases for data provenance and authenticity.

The Work

With the support of Filecoin Foundation for the Decentralized Web (FFDW),Starling Lab has worked with many different organizations and fellows on data integrity projects across the domains of history, journalism, and law.

The first phase of the Starling Lab Framework is developing technology and methodologies to capture photos, websites, audio, video, and other data in a way that allows us to create a verifiable chain of custody.

The next step in our framework is storing the digital media and its metadata in manner that creates a lineage of edits and changes, associating information such as the location of capture or creator with those digital assets, as well as providing a way to capture statements about the context or reliability of an asset.

Once information is captured and stored, we make it possible to analyze and audit the information collected by experimenting with verifiable sharing and distribution technology, content management systems, and journalism publications. We implement tools and resources that allow those consuming media –  be it webpages, photographs, or other kinds of digital data – to view and evaluate authenticity information. This empowers users to draw their own conclusions about the veracity and authenticity of that media.

Authenticity By Design

Our approach is inspired by the philosophy of Privacy-by-Design and the Fair Information Practice Principles. For decades, organizations have recognized that the best way to ensure privacy – for their customers or citizens – is to prioritize it in every phase of their activities.

Authenticity-by-Design calls for a similar approach to information integrity to ensure provenance data is created and preserved throughout the entire lifecycle of a digital media asset. This is a crucial building block underpinning the Starling Framework for Data Integrity. The principles include integrity, privacy, verifiability, persistence, and accountability.

Starling Lab believes that every piece of digital content should include a signal of its trustworthiness, be embedded or displayed with information that allows audiences to judge the trustworthiness of that content, and be preserved with redundant copies in resilient systems.

Any piece of digital content we publish has a digitally-signed and tamper-evident data structure that can be validated. We also take care to ensure creators and publishers of digital content are informed of how an asset could be misused and empowered to mitigate harm by controlling which provenance data is made public.

Our Framework in Action

Web3 technology offers a number of powerful tools to help establish and preserve the integrity of digital assets. These include content-addressed peer-to-peer systems, blockchain registration, and cryptographic security methods. Using them across each step of capture, store and verify can improve trust and provenance for digital records. Establishing and documenting the origins of each asset creates a universe of context and certainty of information.

Capture

Following Starling Lab's approach of Authenticity-by-Design, we look for ways to secure metadata from the moment a digital asset is created. We focus on information that can establish provenance such as date- and timestamps, as well as GPS coordinates and device metadata. We recognize the importance of privacy and put creators in control of which data is released.

Project Dokaz, the Ukrainian word for ‘Proof,’ is a set of accountability efforts in Ukraine that applies our work with especially vulnerable digital records. Starling Lab and our collaborator Hala Systems filed submissions to the Office of the Prosecutor (OTP) of the International Criminal Court (ICC) about alleged war crimes by Russia in Ukraine, detailing our methodologies to establish credibility of the evidence. A similar submission to the United Nations was cited by the Human Rights Council as an emerging good practice.

In 2023, the Atlantic Council’s DFRLab published an extensive report called Narrative Warfare, tracking Russia’s misinformation campaign in the immediate buildup to the 2022 invasion. Starling supported this effort by authenticating and preserving over 10,000 web pages that were at risk of disappearing or “link rot,” following attention brought by the report.

Project Dokaz has prioritized crimes against children and the right to education, but may collect information about other incidents. Currently, Starling Lab is working on expanding the types of evidence collected, including testimonial evidence.

Store

Digital assets decay, and the technology used to publish and store these assets is constantly evolving. Even with the best of intentions, some of humanity’s most vital digital records have been lost due to hardware failure, media obsolescence, human error, malicious attacks, natural disasters, or economic collapse. In other words, digitization is not preservation. In order to create effective and verifiable digital information that can be used across domains, we store and establish immutable records using Web3 tools, creating metadata using standards such as C2PA.

USC Shoah Foundation, dedicated to preserving the memories of genocide survivors, has a collection of 56,000 testimonies as part of its Visual History Archive. Starling Lab worked to upload these interviews from survivors and witnesses of 14 genocides and episodes of mass violence to Filecoin’s decentralized storage system.

The Starling Integrity Pipeline is a storage workflow that processes assets such as photos, videos, and documents, as well as the authenticity metadata associated with them. The pipeline helps create verifiable attestations about this media and preserve them in several different systems. The pipeline includes a processor and a backend which has automation and protocols to encrypt data, add Authsign certificates, add additional metadata, and verify that root of trust data is incorporated. These records and assets are then placed in distributed storage like IPFS and Storj and archived on Filecoin.

Verify

Starling Lab embraces a distinctly human and digital approach to verification. Our workflows capture the attestations of both experts and the people who were there when an asset was created, allowing audiences a better opportunity to evaluate the evidence for themselves.

A collaboration between Starling Lab and Numbers Protocol with the Mom I See War collections demonstrates the use of Web3 technologies including IPFS, Filecoin, and registration on the NEAR blockchain. Explore the collection using Numbers Explorer to see the archive and registrations, as well as the images injected with C2PA metadata, which enables tracking changes and establishing the provenance if edits are made with Photoshop.

Another highly impactful project by Starling Lab, which received a National Magazine Award and an Emmy Nomination, is The DJ and the War Crimes published by Rolling Stone. To capture, store, and verify photos, we used a ground-breaking cryptographic image workflow that was developed specifically for this story. Over a dozen advanced technologies were used to help journalists address the challenges of disinformation in our time.

To process the records for the Rolling Stone archive, we used a variety of techniques to ingest photos, videos, webarchives, and documents. Highlights include the open source Starling Capture App developed by Numbers Protocol, paired with a professional Canon camera to take and scan photos, as well as ProofMode by the Guardian Project on an Android device to take new photos and videos. WebRecorder tools were used to preserve complete archives of web pages using certificates to sign these assets.

By registering records of the files in the Rolling Stone articles and their content on several blockchains including Avalanche and ISCN (Likecoin), Starling Lab created immutable public records that experts and consumers can view, verify, and use to draw their own conclusions.

Notably, some of the files are encrypted to keep parts of the investigation private. Redactions were done on these documents, in collaboration with Stanford’s Electrical Engineering Department, using Zero Knowledge Proofs. The algorithm developed for this project proves mathematically that the only changes made to the published versions of documents were the application of black boxes over sensitive personal details.

Conclusion

Starling Lab is constantly innovating on the capture, storage, and verification of media and digital information. This includes projects bringing those approaches to real-world workflows, like a recent collaboration with Canon and Reuters. As technology reshapes the way we communicate, collaborate, and inform, Starling Lab strives to keep pace with the technology and authentication methods that can enable us to verify and trust digital information.

Connect with Starling Lab on LinkedIn to stay up to date about new projects and announcements, find our open-source tools on Github, and learn more about our Lab’s latest collaborations and innovations on our website at https://www.starlinglab.org/.