Evidence of Digital Interaction in Physical Space
This blog post teaches how to make an evidence-rich record of mixed reality. Mixed reality is like virtual or augmented reality, but doesn’t necessarily involve a headset. It shows information from both the real world and the cyber world (e.g., "nearables," wearable computers or SCADA devices). The information in a mixed reality environment can be much more complex that what a user perceives through a virtual reality headset.
The Internet of Things (IoT) Creates a Mixed Reality.
In the video below the mixed reality involves interaction among a Bluetooth location Tile, the apps on a smartphone and the cameras and microphone on the phone. As the video is made, the phone is physically moving from one place to another.
 |
| Internet of Things - Attached to pet cat |
Details of the interaction are memorialized in a video that shows:
- images and sounds from the real, physical world;
- activity happening on or through the phone;
- sounds and Bluetooth signals emitted from the tracking Tile (which is attached to a cat) when the Tile is prompted by an app on the phone;
- distinctive visual change in the Tile app as the phone draws nearer to the physical location of the cat:
- circle displayed in the app changes from gray, to dotted green to solid green
- then the tile icon in the app swings back and forth to show the physical Tile is emitting sounds that can be heard through the air (You can actually hear the sound from the Tile as it is detected by the microphone on the smartphone.)
The video includes narration from an eyewitness -- the “investigator” -- who explains what is happening in real time.
The narrated explanation helps the observer – such as a judge or jury who watches the video in the future – understand and believe the evidence so that the observer can reach legal conclusions. (Examples of legal conclusions are that a party is guilty, or innocent, or liable, or trespassing or in compliance with a regulation.)
Notice that the sound of the narrator's voice changes as he walks with the phone. The phone's microphone picks up an echo as the narrator walks through a narrow space (a stairwell). Subtle details like this could have forensic significance when the video is analyzed later. They help to show whether the video is fake or authentic.
The Video Records Images from Both the Front Camera and the Back Camera on the Phone.
In parts of the video, the investigator appears on the left side. When the investigator appears, the investigator is being recorded with the front-facing camera on the phone. The right side of the video shows what the investigator sees and records with the back-facing camera on the phone.The narrated explanation helps the observer – such as a judge or jury who watches the video in the future – understand and believe the evidence so that the observer can reach legal conclusions. (Examples of legal conclusions are that a party is guilty, or innocent, or liable, or trespassing or in compliance with a regulation.)
Notice that the sound of the narrator's voice changes as he walks with the phone. The phone's microphone picks up an echo as the narrator walks through a narrow space (a stairwell). Subtle details like this could have forensic significance when the video is analyzed later. They help to show whether the video is fake or authentic.
A video record like this might be valuable in resolving:
- a lawsuit
- a tax audit
- a police investigation
- a child custody dispute
- a dispute over assets in probate court
- a response to an information security incident
The video reliably captures facts as they appear at the time. It captures the facts in chronological sequence. The video is a version of "screencast" evidence record I have explained elsewhere.
Mixed Reality Is Here Only Momentarily.
The facts captured in a video like this might be ephemeral. They might not be reproducible later. The digital world is in constant flux. For example, the Tile might behave a certain way at the time the video is made, but behave a different way an hour later due to an update to the software that runs the Tile or the app that controls it on the phone.
The investigator lends credibility to the video record by ending his narration with a legally binding statement of authentication: “I Ben Wright hereby sign and affirm this video as my official work.” He concludes by stating date and time with his voice and his lips. That date/time statement can be linked with related representations of the date and time, including the time displayed on the screen of the phone itself in the final moments of the video. The representations of date and time make it harder for a fraudster to counterfeit or manipulate the video later.
Trustworthiness Depends on the Investigator’s Credibility.
Obviously the investigator could fabricate this video, just as other eyewitnesses could fabricate their testimony about what they saw. But if the investigator has a good reputation, then the observer of the video (judge or jury) has more reason to believe what is depicted in the video.
The video can serve as evidence of what happened, even if the investigator is not available later to vouch for it.
Legal records like this video might be needed in court many years after their original creation. Therefore the multitude of visual and auditory details captured in the video, together with the voice authentication stated by the investigator, can be invaluable to a court that is trying to understand and evaluate what happened long ago.
Legal records like this video might be needed in court many years after their original creation. Therefore the multitude of visual and auditory details captured in the video, together with the voice authentication stated by the investigator, can be invaluable to a court that is trying to understand and evaluate what happened long ago.
Video is Efficient Tool for Professional Investigator.
Historically a professional investigator made records by snapping a few photographs and writing a text report. But to write a report takes a long time. This video captures a great deal of compelling evidence in a short time.
Notice that the end of the video records details about how the video was made. For example it shows the video was captured with the AZ Screen Recorder App. Details like that might help answer questions by a judge if the video were used in court.
I invite your comments.
Mixed Reality is Rapidly Growing More Common.
The modern world sports a spellbinding array of digital devices and sensors that can detect and transmit information useful to an investigator like a police officer. Mixed reality devices include;- drones
- security cameras and microphones
- robots
- telepresence or video conference systems
- telemedicine monitors
- automotive infotainment systems
- vehicle navigation displays
- Fitbits
 |
| Fitbit |
The backup camera/sensor on a car begets a mixed reality.
The driver sees a video image from the camera. But the driver experiences much more than just a video image.
Superimposed on the image are colored guidelines. Plus the system, which includes multiple cameras and sensors, presents a simulated image of what the car and its surroundings look like from 20 feet above! (Cool)
The cameras/sensors may emit audio if the car approaches danger. Moreover, the sensors may give the driver haptic feedback through the driver's seat. All of this "reality" transpires in a physical space where the driver also directly hears, sees and feels what is happening in and around the car.
 |
| Mixed Reality for Motorist |
The cameras/sensors may emit audio if the car approaches danger. Moreover, the sensors may give the driver haptic feedback through the driver's seat. All of this "reality" transpires in a physical space where the driver also directly hears, sees and feels what is happening in and around the car.
I invite your comments.
Related Blog Posts:
No comments:
Post a Comment