Cancer Research UK’s Trailblazer tool has enabled volunteers to distinguish between cancer cells and healthy cells in tissue samples with 95% accuracy and only 25 minutes of training. And, for the first time, citizen scientists were able to detect clinically significant features of cancer cells with 90% accuracy. SciFabric’s James Doherty meets Andy Paterson, Digital Solutions Architect at Cancer Research UK and the technical lead on the Trailblazer project.
No, Trailblazer is actually Cancer Research UK’s fifth citizen science project.
Each of our projects prior to Trailblazer had proven an important assumption: Our first project, Cell Slider,
demonstrated the public’s ability to do accurate scientific work with the right training.
And our mobile games Genes in Space and Reverse the Odds showed that we could engage the huge numbers of people needed to accelerate research.
One of the things our citizen scientists struggled with in Cell Slider was distinguishing between different types of cells in the images. This meant that in some cases they overestimated the number of cancer cells. Trailblazer aimed to find out whether, with enhanced training and providing more feedback to citizens, it would be possible to boost accuracy and enable people to analyse more complicated samples. This could potentially really save our researchers’ time.
The work we have done on Trailblazer covers a number of different tumour types including bladder, oesophageal and lung cancer. We wanted to come up with a general approach rather than something specific to a particular type of cancer.
We had learned that we needed to experiment with different approaches to user interface and tutorials to find an effective way to achieve the best scientific results. We chose a ‘lean startup’ method of working: identifying assumptions and hypotheses that were fundamental to our success and then focused on getting experimental data to verify (or disprove) those assumptions. To do this we needed to work in a very dynamic and agile way, releasing new versions frequently and analysing results as quickly as possible. We chose to use PyBossa and host Trailblazer ourselves as this provided a way of developing citizen science software in this fast, agile way.
We developed the initial release in about eight weeks. However, we have had 15 releases in total over a six-month development period. We were pleasantly surprised how much we managed to achieve by ourselves. The few questions and minor problems we encountered were quickly resolved by SciFabric.
Through our citizen science products, we have engaged with over half a million people in 182 countries, obtaining over 11 million contributions. For Trailblazer, we worked with smaller groups and had around 1000 volunteers.
The programme has achieved a huge amount over the last three years and made a real impact for scientists. We’ve published a peer-reviewed paper on our work on pathology and presented at international conferences. The team will be working hard to publish more scientific papers and to make the data contributions provided by our citizen scientists available for anyone to study. Citizen science has enabled the world’s largest predictive study in invasive bladder cancer.
Before Trailblazer, citizen science was to some extent an unproven method within the field of cancer research. We knew the public wanted to be involved, and that they could accurately carry out analysis tasks in a limited area. Trailblazer confirmed that citizen scientists could play an important role in accelerating research across a wide range of cancer types, including those previously considered difficult to analyse. The levels of accuracy achieved have shown that citizen science can be a reliable, accurate and efficient method for biomedical research. Most excitingly, we hope that the results from projects such as Trailblazer can one day be used to help to train algorithms to automatically detect cancer cells.
The main area of relevance for citizen science is in research. It is not an appropriate tool for clinical diagnosis where you need a quick and reliable expert opinion. We’re not attempting to replace pathologists. But it is in the area of research – where there is large volumes of data in which researchers want to identify trends and patterns – that the potential for citizen science truly lies.