How Much Digital Evidence Is Enough?

Alternatively, how credible are the electronic facts uncovered by a forensic expert?


The short answer is nobody really knows.

Bitcoin is a Morass of Evidence.


Bitcoin investigators are digging through mountains of digital evidence to assess fraud, deception, ownership, security breaches and the identity of Bitcoin's inventor(s).  They find algorithms, digital signatures,
Attribution
snippets of meta data, and messages that purport to come from significant e-mail addresses. "For the Bitcoin Sleuths, Curiouser and Curiouser," Wall Street Journal, March 8 - 9, 2014.  To what extent do these snippets of evidence prove anything? Which evidence is authentic and which is not?

Stay tuned, as the debate among the investigators over-boils.

Intriguingly, Bitcoin's self-appointed investigators find (and publish) a spreadsheet that some claim to be evidence of fraud at a Bitcoin exchange named Mt. Gox, which has filed for bankruptcy in US court.


What Constitutes Probative Evidence of Something?



The larger question is this: To what extent can evidence from text messages or electronic mail or documents stored on a hard drive be believed?

Electronic evidence can be forged; it can be tampered.

But the same was true for paper or other physical evidence on which law has relied for centuries.  Also, testimony from witnesses – a mainstay source of courtroom evidence -- can be faulty.

Evidence Is in the Eyes of the Beholder.


Many legal authorities evaluate evidence.  Different courts evaluate evidence under different standards.  Commonly, a criminal court evaluates evidence under the standard of "proof beyond a reasonable doubt."  That is a high standard.

By contrast, in many civil courts the standard is "preponderance of the evidence."  That is a lower standard.

Yet courts of law are not the only authorities that evaluate evidence.  Other example authorities include an auditor, a prosecutor, a regulatory agency, or simply the court of public opinion. In the Bitcoin investigations, the "authority" who evaluates evidence may be the world-wide community of Bitcoin users, investors and enthusiasts.

Ultimately All Authorities Are Human, and Fallible.


Commonly the evidence available to an authority is imperfect and incomplete.  The authority commonly determines that some evidence is more credible than contrary evidence. This determination might be made on the basis of logic, science, intuition, best guess or the authority's interpretation of what an expert tried to say.

The Outcome of Evidence Disputes is Often More Art Than Science.


It is hard to predict in advance what evidence a legal authority will believe and not believe.  Often, in a criminal court for instance, the primary authority for evaluating evidence is a humble jury of 6 or 12 common people.

Judges, juries and other authorities often do not have much technical expertise.  When it comes to computer evidence, often they must rely upon testimony and guidance from experts.

However, when evaluating complex evidence, even experts can disagree.  Smart forensic experts can see the same evidence and evaluate it differently.

The Quality of Advocacy Counts.


Also, a factor in determining the credibility of electronic evidence is the quality of the lawyer (that is, the advocate) who advances a particular interpretation of the evidence.  A talented lawyer will, depending on the lawyer's objective:

  • explain evidence well, or
  • raise doubts about the evidence, or 
  • cause the evidence to be excluded from consideration altogether.  
A less-talented lawyer will not be able to explain the evidence or will leave the fact-finding authority confused about it.

Is the Expert Qualified Enough and Humble Enough?


Increasingly, legal and financial evidence comes from new electronic sources such as social media, mobile devices, cloud computing and virtual currency communities.  Our ability to fully understand this evidence lags behind.  For this type of evidence, nuances and misunderstanding are common. The need for qualified forensic investigators swells.

A good investigative expert understands how to weigh evidence and how to separate strong evidence from weak evidence.  Such an expert is able to separate emotions from logic.  Such an expert is also able to set his or her ego aside and acknowledge when he or she does not know something or have enough data to state an opinion.

For an example of a case where an expert should have been more humble, see Stephen Mason's critique of a police officer's testimony regarding pornography on a teacher's computer: State of Connecticut v. Julie Amero (Mason argues court failed to understand how malware works; was too ready to believe faulty police work).

The training, experience and reputation of an investigator are all relevant to assigning weight to any conclusions drawn by the investigator from the evidence.

Another factor that is relevant in understanding evidence is whether the investigator is biased.  Bias can come from background, conflict of interest or professional disposition.

Look for Corroboration.


One technique for improving the quality of evidence is to corroborate it.  Corroboration means getting similar evidence from more than one source.  For example, if the time stamp for a photograph on a smart phone is approximately the same as the time stamp connected to the same photograph in Facebook, then evidence of the time of the photograph is stronger.

Cross-examination Compels Accuracy.


In the courtroom, a powerful technique to evaluate evidence is cross-examination.  Cross-examination is a time-honored process for forcing a witness -- such as a digital forensics expert -- to explain himself carefully and to admit any of his shortcomings.

In cross-examination an expert witness must answer hostile questions under oath; if she lies, she could be punished (e.g., fine, loss of license, jail-time, embarrassment).

However, cross-examination of a computer expert can fall short because very few lawyers know how to execute it masterfully. Very few top-flight trial lawyers possess a deep understanding of computer forensics and technology.

Evidence Floats in the Cloud.


Sometimes, such as in cloud computing, the investigator does not have direct access to the hardware that stores digital evidence.  The investigator is only able to see the evidence temporarily, through a software client such as a web browser or a mobile app.  Use of that evidence may require eye witness memory and testimony by the investigator.  In such a case, the investigator may be wise to print what she sees or record it as a video.  See discussion of example videos:


Legal Fact-finding Is a Form of Theater (and That's Not Necessarily Bad or Wrong).


Digital forensic dispute guru Craig Ball publishes a priceless guide for forensic experts who testify as witnesses in court. His guide and his experiences teach a profound lesson about the use of computer forensics in law.

The lesson is that the process for articulating and evaluating the true facts in a case can influence the outcome as much as the facts themselves.

In other words, the following factors in combination can have a heavy impact on the final interpretation of electronic evidence by a judge, a jury or other legal authority:

  • the skill of the lawyers, 
  • the demeanor of the expert witness, 
  • the clothes the expert wears in the witness stand (!), 
  • the expert’s advance preparation, and 
  • many other aspects of courtroom theater and procedure

Justice is Not Inexpensive.

All of the above leads to a philosophical observation: Our justice system is underpinned by checks and balances called due process of law. These checks and balances try to prevent the abuses of civil and property rights and to prevent hasty rushes to judgment.

Owing to these checks and balances, getting to the truth in our justice system is hard work and imprecise work. It requires a lot of resources -- such as the time of people like judges, juries, courtroom staff, lawyers, experts and so on.

Getting to the truth is commonly expensive. It commonly costs the government a lot of money to run a trial, especially a jury trial.

Skilled lawyers and skilled experts are expensive. They are scarce commodities.

A non-lawyer can sometimes believe that the truth in his/her own case is abundantly obvious. I've encountered people who believe the legal system can, will and should just swiftly force out all of the evidence from the computers (deleted records, metadata, yada yada) and declare what the truth is . . . just like a 60-minute TV drama.

That belief is naive.

Postscript - How to Present Evidence to a Deciding Authority.


Computer technology can be used in novel ways to explain or present evidence to legal authorities. Two examples illustrate how increasingly inexpensive technology can help to put evidence into a context:

1. Shooting simulator: Grand jurors in Harris County, Texas, are offered training in a computer-driven shooting simulator. The simulator is a type of video game that allows the juror to virtually experience the emotions of a person like a police officer who suddenly must decide whether and how to use deadly force (i.e., shoot someone). In Texas it is the role of grand juries to decide whether police officers should be indicted criminally in cases where excessive force was allegedly used. The purpose of the simulator is to provide jurors experience in evaluating the evidence they will be presented later in particular excessive force cases. The simulator serves as a substitute for training that might be delivered in lecture format by a police officer. Critics argue the simulator training causes grand jurors to be biased in favor of police in their future evaluation of evidence.

2. Sentencing Mitigation Videos: After a court has determined that a defendant is guilty of a crime, the judge may hear evidence regarding how harshly to sentence the defendant. Traditionally this evidence included letters and testimony from family, friends and victims. A new form of evidence is a prerecorded video, often created on behalf of the defendant. It provides a controlled, edited, visual format for one side or the other to present evidence on why the sentence should be harsh or lenient. Here is a demo:

The demo above uses a videographer who interviews the witnesses in person. In principle this type of video could be created at much lower cost using webcams and an interviewer who works from a remote location.

The two examples above illustrate how different technological formats for delivering evidence can have subtly different impact on a decision-making authority.