"Expert" claiming proprietary work product on simulation files?

Topics Related to Analysis of Motor Vehicle Collisions
Post Reply
MSI
Site Admin
Posts: 1720
Joined: Thu Jun 18, 2009 12:37 pm

"Expert" claiming proprietary work product on simulation files?

Post by MSI »

For latest update on this topic Click here or browse to bottom of topic. Have added a copy of a court order.

Now that a certain case in a certain jurisdiction is over and out i can pose this question:
(and without naming an expert or jurisdiction)
I ask the following:
  • Have any of you encountered an expert who used a simulation to simulate an accident and/or the EDR recorded steer and speeds, etc and then maybe also had a 3D survey prepared and then...
    claimed that all the simulation files and survey files were proprietary?
    In other words they would ONLY provide limited outputs and a printed 500+ PDF of the survey (which had arbitrary line breaks so it could NOT be used when saved as text)
In rebuttal to our motion to produce the simulation files they used arguments like:
  • “Procure the simulation at great expense to Plaintiffs’
  • “providing the data in the format requested by Defendant leaves the data subject to manipulation, misuse, and abuse”
And we countered with documenting and demonstrating the importance of simulation verification (we may post up redacted filings)
However
The judge did not force the production of any more documents and left the argument for trial arguments.

Fortunately in the case even without the aforementioned input files we scared them enough for them to realize that this might not play well with the jury.
  • When we deposed their "expert" many question we provided to legal counsel demonstrated the experts lack of foundation on the underlying computer code they used and the importance of a full open exchange of ALL inputs for verification of simulation analysis (we asked questions that demonstrated WHY full open exchange was important)
  • When we were deposed we demonstrated a knowledge of the science behind the computer code (we know them all!) and clearly listed the many reasons why one must NOT be allowed to present anything without the other side being provided a chance to verify:
    • The inputs produced were proper and complete (why electronic form is essential)
    • That when the inputs were rerun with the computer program they produced the same outputs/video
    • That the computer program was being applied properly
    • That the results did not represent a sensitivity or anomaly of the specific computer program being applied
  • At trial counsel withdrew the "expert" and did not use their simulation video.
    • Since i was a rebuttal witness, this avoided me being called as a rebuttal witness
MAIN QUESTIONS ARE:
  • Have any of you encountered this in any case? and if so, can you provide details?
  • Also if any of you are aware of any case law related to this type of situation?

In Federal court it generally is required that experts exchange files so that the opposing expert can re-run and verify the simulation and test sensitivities.
Is anyone aware of specific rulings related to 'simulation input files' in electronic format?
  • In the case we were involved in they got the judge to believe the printouts were enough
    sometimes in some instances printed output may be adequate
    however
    In these modern times, with 3D inputs, 3D surveys and many options for importing the 3D survey data into the 3D software, printouts ARE NOT adequate to allow verifying the results of their simulation
In the state jurisdiction they allowed them to hide the electronic files
Curious if others have encountered this gamesmanship.
Our interest is so we can publish to the web any related case law rulings to assist others in the future to avoid having to tap dance in this type of crap...

Please respond to this thread or email us at forum@mchenrysoftware.com. All emailed responses will be kept confidential unless you say it is ok to identify you.

Brian McHenry
NEW!! LEASE pricing of msmac3D Software!!
FOR MORE TOPICS see:Forum Index & McHenrySoftware.com
Question? Comment? Please email us (all communications considered confidential)
(c)McHenry Software, Inc ALL Rights Reserved.
MSI
Site Admin
Posts: 1720
Joined: Thu Jun 18, 2009 12:37 pm

Re: "Expert" claiming proprietary work product on simulation files??

Post by MSI »

Feb 18, 2018: Have had a few responses so far saying a lot of folks are trying to hide their simulations behind the 'printed output adequate' scenario which is ridiculous in our cloud scan world where billions of points have to be transformed into simulation surfaces (for HVE, PC-CRASH and all the rest!) and all inputs ARE NOT included in the printouts!
Main point is DEMAND they provide electronic versions of all the inputs files (simulation and raw survey files) so you can re-import, rerun and verify their results so that you KNOW that the inputs they provided give the results they relied on!
we will post more soon and hopefully some case law supporting this requirement!
and from a paper a while back...
  • SAE paper number 980018, Documenting Scientific Visualizations and Computer Animations used in Collision Reconstruction Presentations which includes as the Summary:
    • “This paper has presented a proposed standard for documenting computer generated images, animations, scientific visualizations, etc. The basic standard is that any still images or videos should be documented such that any qualified analyst can reproduce them. This is the requirement for the scientific community in general and should be adopted in the crash reconstruction community. It is important to note that this standard does not refer to any method of generating these images or videos. There is no implication that any one method or any one program is superior to others. This standard addresses only the images and videos and does not address the analysis or opinions being expressed by analyst. However, the only way to fully understand the analysis being presented or discussed is to have the ability to duplicate the images or video being presented”
NEW!! LEASE pricing of msmac3D Software!!
FOR MORE TOPICS see:Forum Index & McHenrySoftware.com
Question? Comment? Please email us (all communications considered confidential)
(c)McHenry Software, Inc ALL Rights Reserved.
MSI
Site Admin
Posts: 1720
Joined: Thu Jun 18, 2009 12:37 pm

Re: "Expert" claiming proprietary work product on simulation files??

Post by MSI »

Here's some information related to the topic from the American Bar Association
    • A missing element in the ABA document is that any party who uses a computer simulation should be required to produce all raw information(survey or cloud scan data, vehicle specification data, etc) and all simulation inputs and outputs IN NATIVE ELECTRONIC FORMAT FOR THE PROGRAM(s) BEING USED to permit ALL PARTIES to
      • (1) verify that the information was properly input/entered into the computer program and that
        (2) the inputs provided produce the results cited by the expert, and that
        (3) the analysis is not extremely sensitive to any arbitrary or subjective inputs used by the expert.
      This should be a requirement for any and all courts which allow computer animation or simulation.
      This has been required in the courts in our 40 years experience in the courts.
      Only recently have we seen or heard of some experts trying to hide their animation and simulation analyses behind 'proprietary' arguments and/or where they will only provide incomplete printed disclosures of inputs and/or outputs.
The ABA document includes:
  • H. Computer Animations and Computer Simulations.
    Computer animations and computer simulations also raise unique evidentiary issues.(ref 60) Because of the persuasive power of demonstrative evidence, such as animations and simulations, courts are obligated to make a thorough foundational inquiry into its reliability before admitting it, giving the potential that it may mislead, confuse, divert or otherwise prejudice purposes of a trial if not reliable.(ref 61) The court in Sayles explained the difference between computer animations and computer simulations as follows:
    • Computer generated evidence is an increasingly common form of demonstrative evidence. If the purpose of the computer evidence is to illustrate and explain a witness’ testimony, courts usually refer to the evidence as an animation. In contrast, a simulation is based on scientific or physical principles and data entered into a computer, which is programmed to analyze the data and draw a conclusion from it, and courts generally require proof to show the validity of the science before the simulation evidence is admitted.(Ref 62)
    The Lorraine court reviewed the cases and observed that courts have generally allowed the admission of computer animations if authenticated by testimony of a witness with personal knowledge of the content of the animation, upon a showing that it fairly and adequately portrays the facts and that it will help illustrate the testimony given in the case.63 In Friend v. Time Mfg. Co., 2006 WL 2135807 at *7 (D. Ariz. July 28, 2006), the court held that, at a minimum, with respect to animations, the proponent must show the computer simulation fairly and accurately depicts what it represents, whether through the computer expert who prepared it or some other witness who is qualified to so testify. The opposing party must then be afforded an opportunity for cross-examination.
    On the other hand, computer simulations are treated as a form of scientific evidence offered for a substantive, rather than demonstrative, purpose.(ref 64) Courts often treat such simulations like other scientific tests and condition admissibility upon showings that
    • (1) the computer is functioning properly;
      (2) the input and underlying equations are sufficiently complete and accurate (and disclosed to the opposing party so that they may challenge them); and
      (3) the program is generally accepted by the appropriate community of scientists. (Ref 65)
    The State v. Swinton case, supra, adopted the Commercial Union case but added that the key to authenticating computer simulations is to determine their reliability. The court noted that the problems that could arise with such evidence include
    • (1) the underlying information itself could be unreliable;
      (2) the entry of the information in the computer could be erroneous;
      (3) the computer hardware could be unreliable;
      (4) the computer software programs could be unreliable;
      (5) the execution of the instructions which transforms the information in some way – for example, by calculating numbers, sorting names or storing information or retrieving it later – could be unreliable;
      (6) the output of the computer, such as the printout transcript or graphics, could be flawed;
      (7) the security system used to control access to the computer could be compromised; and
      (8) the user of the system could make errors. (Ref 66)
    References:
    • 60. See Imwinkelried, Evidentiary Foundations at § 4.09[4][a].
      61. See, e.g., Taylor v. United States, 759 A.2d 604, 608 (D.C. 2000); Hanon, Computer Generated Evidence: Testing the Envelope, 63 DEF. COUNS. J. 353, 361 (1996).
      62. State v. Sayles, 662 N.W.2d 1, 9 (Iowa 2003).
      63. 241 F.R.D. at 559-60 (citing cases).
      64. Weinstein at § 900.03[1]; Imwinkelried, Evidentiary Foundations at § 4.09[5][a], [c].
      65. See Commercial Union Ins. Co. v. Boston Edison Co., 591 N.E.2d 165, 168 (Mass. 1992).
      66. 847 A.2d 921, 942-43.
NEW!! LEASE pricing of msmac3D Software!!
FOR MORE TOPICS see:Forum Index & McHenrySoftware.com
Question? Comment? Please email us (all communications considered confidential)
(c)McHenry Software, Inc ALL Rights Reserved.
MSI
Site Admin
Posts: 1720
Joined: Thu Jun 18, 2009 12:37 pm

Re: "Expert" claiming proprietary work product on simulation files?

Post by MSI »

I realized I hadn't updated this post so will summarize the many responses:
    • NOTE this topic applies to any and all simulation programs used to support expert testimony.
      I have changed the identification of simulation programs to ZZZ, and also did not mention companies so those were changed to XXXX and YYYY
Then here are a few more comments received:
  • Main response was shock that a judge would allow a 'secret' weapon and not require all data and information (scans, input files, etc) to be provided to the other side of a litigation proceeding since only with full disclosure can there be verification that the inputs listed produce the results discussed/displayed/relied on by the expert. It MUST be a requirement!
    • Another was that they currently have an influx of these type of situations. There must be something out there now that these guys are realizing they can hide their work. Mine all involve the bigger engineering firms like XXXX and YYYY playing games.
      • Another had a case where XXXX did a ZZZ simulation, but would not produce anything other than PDFs.
        • In that case it had nothing to do with legal counsel trying to interfere with me getting materials it was simply them not wanting to give it up. Probably due to problems with their simulations. In the end on that case was settled rather than continue with Discovery or head to trial. That surprised me.
      • Another mentioned new ones they have are with YYYY. Your email hits home because that’s about exactly what they said on this one.
        • They said since I have the FARO scans, I should be able to create the surface.
          • As you know ZZZ does not run on a cloud. They provided the accident history report and said this in conjunction with their output videos should be sufficient for me to recreate their simulation. That is ridiculous.
            They have multiple friction services they have lots of geometry associated with the surface numerous vehicles and the associated driver and puts all absent from the production.
          • Currently I’m trying to figure out the best approach, but I think because we are in federal court is to do a Daubert challenge. Their work in its current state is not reproducible. This will put the burden of proof back on them to show that their methods can in fact be reproduced. I hope this works.
            I don’t understand the crazy gamesmanship, as it is my understanding in our profession is that we explain what we do, what our opinions are, and it is what it is. We don’t skew facts. However I don’t have the overhead those huge firms do.
            In this case the PhD expert for YYYY actually testified that this would take great effort’s and an extremely long time to be able to produce the ZZZ file I was interested in. I explained to my attorney that you simply click on the file and drag it to a jump drive.In these cases what I typically ask for is the ZZZ file, the surface file, and any vehicles they built in the case. I ask that simulation be executed prior to saving.
      • Another one said they will not even provide their scans, as they say we have the ability to do this work ourselves.
      One idea floated by a respondent was "get a dozen affidavits by various people in the industry and submit them in challenge what materials should be produced and that the materials provided are insufficient to re-construct their work product"

      I am now just reminded of the case we had in California where the expert did not preserve the simulation inputs he relied on for his initial opinion.
      • The judge would not allow him to testify.
        That is a solid reinforcement that experts MUST 'preserve and produce all materials relied on for an opinion"
        Here is the document:
        • Court Order Precluding Expert Testimony Related to Computer Simulation
          • Starts on page 3 (other items in the court order were unrelated to the expert and computer simulation)
            • Note support for the court order rested with the repeated requests/demands/specifications of the computer simulation files needed.
              BE SURE to create a paper trail requesting simulation input files and supporting files (scans, surveys, etc.)
              Then when experts/legal firms repeatedly refuse to provide documents/files in a timely manner. you have supporting documentation of requests and justification such that a judge may move to preclude the simulation work and any opinions based on the simulation work
      • In a couple other cases over the years (i will try to retrieve the filings and post up), we fought to get the actual inputs IN ELECTRONIC FORM since the results presented in video form didn't match the printed inputs (since printouts do not have all the inputs!)
        Things didn't add up and we finally were provided the input files.
        • In once case we found 'hidden friction zones' and other changes in the inputs
        • In another case the expert either intentionally or by accident changed the inputs
          • with most programs when you change inputs the programs saves ONLY the changed inputs...
          MAIN point is if you create a video, write a report or base an opinion on a simulation SAVE THE COMPLETE INPUTS!)
        Main Point: Be sure to preserve your complete input file in electronic form for any runs on which a report, opinion or video is based
      Please contact us with any additional information, court ruling, etc related to production of materials related to computer simulations of crashes.
      Please respond to this thread or email us at forum@mchenrysoftware.com. Your emails and discussions are considered confidential.
      NEW!! LEASE pricing of msmac3D Software!!
      FOR MORE TOPICS see:Forum Index & McHenrySoftware.com
      Question? Comment? Please email us (all communications considered confidential)
      (c)McHenry Software, Inc ALL Rights Reserved.
      Post Reply