BenefitsAddress Replication Crisis challenges

Eliminate local variants with cloud-based solutions

When Amgen tested 53 landmark studies in hematology or oncology, only 6 (11%) could be replicated and reagents were the #2 concern.
Nature 2012 DOI: 10.1038/483531a
When Bayer repeated 12 studies with identical model systems, only 1 (8%) could be replicated.
Nature Reviews Drug Discovery 2011 DOI: 10.1038/nrd3439-c1
Approximately half of all biological research is irreproducible.
Nature 2015
Two-thirds of respondents to a Nature survey felt that selective reporting of results contributes to poor data reproducibility.
Nature 2018 DOI: 10.1038/d41586-018-04590-7
The Cancer Biology Reproducibility Project aiming to repeat key experiments from 50 high profile cancer papers stopped at 18 because of difficulties knowing what was done in original experiments.
Nature 2015 DOI: 10.1038/nature.2015.18938
Reduce Variability
01 / 05

Each researcher and technician interprets a standard operating procedure (SOP) slightly differently, leading to variations in the end product.

  • With Revo
  • Without Revo

Revo executes the exact same procedure from the cloud every time, imposing complete global uniformity.

The LabMinds AI will help you reliably hit your targets. The best practices will be logically applied to each solution in exactly the same way, limiting variation. Furthermore, the AI proactively looks for anomalies indicative of problems in sensors or input materials, providing the consistency that gives the end user complete peace of mind.

Show More

SOP writing is a tough balance between being readable and explicit, which is instantly hampered by remarkably non-scientific best practices starting with %w/v, solutions with over 100% v/v content etc.

Even if it were possible to fight to clarity through all this, enforcing the SOPs would require constant diligence that seems impractical to try and impose.

Show More

Did you know?
Variation in starting materials is central to the reproducibility crisis that’s causing billions in wasted R&D every year.

  • Failure to reproduce another’s experimental data is reported by >70% of scientists, while more than half of scientists have been unable to reproduce their own findings. (Nature, 2016 DOI: 10.1038/533452a)
  • Materials have been estimated as the primary source of irreproducibility at 36%, ahead of study design (28%) and data analysis (26%)? (Nature, 2015)
  • Irreproducible Biology Research costs put at $28 billion per year (Nature, 2015)
Sensor Inconsistency
02 / 05

Experienced laboratory staff know exactly how far they can trust a pH meter, which is to say not very far.

Inside of the Revo
Front of the Revo

Sensor challenges

  • Audit trail is mandatory
  • pH calibration false positives are a thing
  • Context for probe insults needs to be stored
  • Some pH standards can go off in 2 weeks
  • With Revo
  • Without Revo

The performance of key sensors and input mechanisms is validated by automatically calibrating them several times a week and comparing the results to their historical trends and specifications. The AI can also trigger additional calibrations if anything looks suspicious during normal operation.

Data from every solution prepared by every Revo are sent to LabMinds where a Deep Learning System does the hard work of digesting and analyzing it. Sensors and inputs with anomalies are identified, and appropriate actions can be taken.

Show More

Sensors – particularly the pH probe – are key factors when addressing buffer replication issues. For instance, around 1 in 200 pH calibrations is 0.1-0.3 units off target. This can be caused by “insults” from materials such as Tris, histidine, or even open air that lead to incorrect results and require prolonged recovery times.

To avoid these problems, we advise collecting raw data from daily calibrations for a minimum of a month and assessing the deviation from expectations.

Show More

Did you know?
pH probes

  • Have an error stack of 0.09 units according to Mettler-Toledo, which means that by adjusting pH, you could end up with solutions 0.18 units apart due to the probe alone
  • Can be considered “successfully” calibrated with a pH more than 0.2 units off their target
  • Can be insulted by certain materials (i.e. Tris) leading to errors of up to 0.4 units
Reagent Inconsistency
03 / 05

Reagents not matching their labels was the biggest surprise during the development process of the Revo. Anhydrous chemicals stored improperly can contain up to 8% w/w water, resulting in a dramatic difference in molarity.

  • With Revo
  • Without Revo

The AI can spot problems caused by changes in a chemical. Let’s use the example of a changed Tris HCl and take a human and an AI perspective to the next day’s solution.

Human: your pH was 7.75, rather than the theoretical 7.7, and the conductivity was 5% lower than usual. This seems acceptable.

The Revo: Tris HCl dosed 40% slower than usual, conductivity is 3 standard deviations low, and pH is 2 standard deviations high. An alarm is raised.

A standard going off is even more easily caught – a pH 10 standard that went off 2 weeks before will be >10 standard deviations out of specification.

Show More

Statistics need to be kept on every solution made, so that anomalies can be detected by comparing new solutions to the most recent solutions and reference data. This involves a tremendous amount of work.

Quality control of the data is also vital. Conclusions and actions based on bad data is a nightmare scenario for pharmaceutical scientists, directors, and executives. We’ve learned these lessons the hard way, having had bad data sneak into the database in the early stages of developing Revo.

Show More

Did you know?
Chemical variability is a reality

  • pH 10 buffer standards begin deteriorating ~2 weeks after the container is opened and can reach a pH as low as 9.8 by week 4
  • Even chemicals from premium producers can have up to 8% extra water weight, with the highest LabMinds has ever directly encountered at ~4.5%
Expired Materials
04 / 05

Laboratories are always either running out of a chemical that will cause problems, or more often throwing away chemicals that have gone bad – hopefully before they were used in an important study or manufacturing run.

  • With Revo
  • Without Revo

The levels and usage rate of chemicals and other consumables are monitored continuously, with replacements shipped 2-4 weeks before the end user is likely to run out. This guarantees minimal on-site inventory, a constant supply of fresh chemicals, and no concerns of running out of material during a critical project.

Show More

Inventory checks are frequent to avoid shortages and confirm that materials have not expired or gone bad. Large quantities are often purchased to reduce the overheads due to frequent ordering and inventory checks, but this results in considerable waste that is often anything but nature-friendly.

Show More

Did you know?
Laboratory waste is abundant

Almost 35% of all laboratory chemicals are discarded before use (Dassault whitepaper available here.)

Stock solutions can hang around well past their use-by date. In 2016 we found a solution from 1997 that hadn’t been used in the last decade. Can you beat 19 years by searching your lab’s stocks?

Traceability Problems
05 / 05
Using the Labminds interface

Challenges with traceability

  • Unpopular activity
  • Container history is ignored
  • Downstream integration is poor
  • Upstream integration is poor
  • With Revo
  • Without Revo

Each chemical container is given its own LabMinds ID to guarantee that we know exactly what was used and where. If any problems manifest even years later, it will be simple to trace and flag any study or product that used the potentially problematic reagent.

There are also features in development that allows tracking the solutions to the downstream systems, or even specific inlets of downstream systems.

Show More

All chemical containers should be labeled with unique QR codes and IDs upon their arrival at the company site, after which these should be referenced to in any ELN or equivalent when a solution is prepared, as well as when the chemical is discarded.

Coming up with such a process would be painful, and actively enforcing would be much more so.

Show More

Automated solution preparation – it’s about time! More time to think and plan. Less time spent calibrating and worrying.