Patients Participate!

As part of the process of wrapping up projects, this post sums up some of the outputs of the Patients Participate! project which ran during 2011-2012, and was funded by JISC.  Further information about the project and partners can be found in the links for this blog site and in the blog posts.

Two of the key outputs from the project were two events, a workshop and the  launch of the project report at the Talk Science event.

Workshop links: Programme Presentations Report

TalkScience links: Event Page YouTube recording

The project also produced guidance for academics (Briefing paper on citizen science, How To Guide on Writing Lay Summaries) and guidance for patients.

Two other key publications are the case study report and the project summary report.  Other reviews include the analysis of citizen science platforms, a literature review on usability, activity by medical charities and some scenarios for writing lay summaries.

All the deliverables and dissemination activities can be found in the respective pages on this blog site.  The project thanks all those who engaged with the project through our events and other channels.  Each of the project partners maintains an interest in this are and looks forward to future activities to become involved in.

Last week I received my supply of printed copies of the Digital Curation Centre (DCC) Briefing Paper on Citizen Science, produced by the Patients Participate! project. Order your copies through the DCC by emailing info@dcc.ac.uk or print off your own using the downloadable pdf from the website. We will be really pleased to receive any feedback on the publication or how you have used it.

The briefing paper from the Patients Participate! project has now been published on the Digital Curation Centre website as part of their briefing paper series. The Citizen Science briefing paper provides an introduction to citizen science, presents benefits, and lists issues to be considered when planning citizen science activities. It is available both as a webpage and as a downloadable pdf.

This post gives a brief overview of a selection of systems that are used to support citizen science or crowd sourcing activities.  The examples are selected to illustrate a range of activities and systems and provide an idea of the functionality supported. The systems are Galaxy Zoo, PatientsLikeMe, Acawiki, CoCoCo, Hyperwords and WikiMedia Medicine Portal.

Galaxy Zoo. http://www.galaxyzoo.org/

Who is behind this initiative?

Galaxy Zoo started in June 2007, with Chris Lintott as principal investigator, and asked users to look at images of galaxies from the Sloan Digital Sky Survey (SDSS). The team consists of astronomers and developers in different universities in the UK and the US, with support from teams in several other countries. http://www.galaxyzoo.org/team

What is it for?

Galaxy Zoo is an initiative in which volunteers classify images of galaxies. Volunteers provide descriptions of the galaxies helping to build a database of characteristics of galaxies (e.g. shape and direction of rotation). This can lead to new discoveries, for example showing that although red galaxies were thought to be elliptical, several of them are spiral. Some new objects, such as the voorwerp, have also been found. Results have been published in peer-reviewed journals.

How does the crowd participate?

The users examine the pictures and click on buttons to select the properties of galaxies, from a selection of characteristics. Before getting started the volunteers follow a simple on-line tutorial on galaxy types and can have a go at trying to classify the galaxies. They are then tested on what they have learned to check that their classifications are reliable, and can then move on to making classifying galaxies.

Users are supported by these other features:

1. a blog (www.galaxyzooblog.org) where the research team writes about the research that they are conducting, and volunteers ask questions and receive answers in the comments section.
2. a forum (www.galaxyzooforum.org) where volunteers communicate with one another. The volunteers can share images of beautiful galaxies that they have seen while classifying, discuss Galaxy Zoo research projects, ask and answer questions about astronomy and coordinate meetups between volunteers in real life. Scientists also contribute to the forum.

The volunteers have also taken the initiative to identify research questions for themselves and get organised to try and answer them.

References:
Raddick, M. J., Bracey, G., Carney, K., Gyuk, G., Borne, K., Wallin, J., Jacoby, S., et al. (2009). Citizen Science: Status and Research Directions for the Coming Decade. AGB Stars and Related Phenomena Aastro2010 The Astronomy and Astrophysics Decadal Survey, 2010, 46P. http://www8.nationalacademies.org/astro2010/DetailFileDisplay.aspx?id=454

Raddick, M. J., Bracey, G., Gay, P. L., Lintott, C. J., Murray, P., Schawinski, K., Szalay, A. S., et al. (2009). Galaxy Zoo: Exploring the Motivations of Citizen Science Volunteers. Astronomy Education Review, 9(1), 15. AAS. http://arxiv.org/abs/0909.2925

PatientsLikeMe http://www.patientslikeme.com

Who is behind this initiative?

Patients like me was founded in 2004. The founders were motivated by the diagnosis of a friend/family member who was diagnosed with an illness. PatientsLikeMe operates as a for-profit company but claims that its mission is not ‘just for profit’. They have four core values by which to operate, and form partnerships with not for profit research and industry.  There is no advertising.  The business model is based around selling anonymised aggregated data and collaboration with industry and researchers to carry out market research and promote clinical trials.

What is it for?

The site has been built around the assumption that patients have a primary question “Given my current situation, what is the best outcome I can expect to achieve and how do I get there?” The site features and tools are then designed around being able to answer the different parts of that question e.g. a patient can match their “current situation” in terms of genetics, disease characteristics etc. The site claims that it is able to act as a a barometer of patient perceptions (e.g. their response to news about adverse effects of a drug) by monitoring feeling in the discussion forums, and suggests that the site is more effective at measuring the rates of side effects, which are reported at a higher % through the site compared to the percentages reported in clinical trials.

PatientsLikeMe has chosen to prioritise enabling peer-interaction over providing medical advice through the site, thus allowing the patient-patient exchanges to be the source of information derived from their own personal experience, using personal pages and forums.

How does the crowd participate?

PatientsLikeMe has 40,000 patients organised around disease communities. Patients can anonymously share their personal treatment, symptom, progression and outcome data, and participation is free. Patients can also participate in experiments, and are involved in collecting adverse effect data. Research has been carried out on the outcomes for patients who share health data. The benefits identified include finding other patients who understand the condition and treatments, and making decisions on medications.

Reference:
Brownstein, C. A., Brownstein, J. S., Williams, D. S., Wicks, P., & Heywood, J. A. (2009). The power of social networking in medicine. Nature Biotechnology, 27(10), 888-890. Nature Publishing Group. http://www.ncbi.nlm.nih.gov/pubmed/19816437
Wicks P, Massagli M, Frost J, Brownstein C, Okun S, Vaughan T, Bradley R, Heywood J.
J Med Internet Res. 2010 Jun 14;12(2):e19. http://www.ncbi.nlm.nih.gov/pubmed

Acawiki http://acawiki.org/

Who is behind this initiative?

Neeru Paharia is acknowledged as the founder of acawiki.org, with a small group of volunteers to help drive the initiative. It is run by a Californian non-profit organisation (AcaWiki Inc) and received a start-up grant from Hewlett Packard.

What is it for?

It is a site run on a wiki platform, that invites summaries of academic articles to make them more accessible. “AcaWiki is building a community of scholars, graduate students, and bloggers to write summaries of papers that are of the most general interest to the public”. The aims of the site are to “make academic scholarship widely available” and to “make it accessible to a wide audience” and generally increase public engagement with scholarship. As of October 2011 it has over 860 summaries written in areas like economics, psychology, sociology, business and computer science.

How does the crowd participate?

Users write summaries of academic articles following the posting guidelines. The users can also contribute ideas to help the site grow – for example a project to summarise the top papers in each academic discipline was suggested by one of the community. http://acawiki.org/AcaWiki:Top_100_Papers

Reference: http://acawiki.org/AcaWiki:FAQ

CoCoCo http://projects.oucs.ox.ac.uk/runcoco/resources/cococo.html

Who is behind this initiative?

CoCoCo (Community Contributed Collection) software was developed for the Great War Archive project by Oxford University and was used for the Great War Archive Initiative the Woruldhord project, and the Europeana Erster Weltkrieg Project.

What is it for?

CoCoCo (Community Contributed Collection) is a Ruby on Rails Web application used to collect and catalogue content contributed by website users. That content can be either textual or consist of uploaded files with metadata collected via form fields. The quality assessment of contributions is performed by administrators by adding metadata and marking suitable items as “approved”. The software is available as open source and there is documentation available to help the various users and tasks: for contributors to make and review submissions, and for cataloguers and administrators to help set up an instance of the software, define and configure metadata and set collection parameters, and to edit interfaces and set up a site.

How does the crowd participate?

Contributors fill in a form and optionally can add images, sound files or text documents to their submissions. The participants provide content for the collections being managed through CoCoCo. The RunCoCo team have written a report offering advice on how the public has been engaged in the projects using the CoCoCo software, including using events to meet volunteers who bring materials for the collection, using Flickr groups and other social media like blogs and discussion fora to build the community. One of the collections built using the software (Woruldhord) ran only for a limited time and now offers the resources collected through a website – the collection relied on online methods of communication only.

Reference:
RunCoCo Project. (2011) Final Report: How To Run A Community Collection Online. http://projects.oucs.ox.ac.uk/runcoco/resources/RunCoCo_Report.pdf.

Hyperwords : http://www.hyperwords.net/

Who is behind this initiative?

Hyperwords is made available by the Hyperwords Company which is run by Frode Hegland, a London-based Norwegian philosopher. The company has an advisory board and development teams in Europe. http://www.hyperwords.net/about_us.html A ‘liquid information’ philosophy underlines the design ideas, and is inspired by the work of Doug Engelbart.

What is it for?

Hyperwords makes the words within text interactive. It is available as a browser plug-in (and other solutions). An icon appears next to the words, which can then be clicked to issue a command. The command can bring up a minibrowser with the results of the command. The command depends on the text to which it is associated. The examples given include a ‘reference’ command for Barack Obama which can bring up information e.g. from Wikipedia about Barack Obama. For currency values, the commands could be used for currency conversions.

How does the crowd participate?

The software can be used to help users to connect to information provided by others (e.g. explanation of medical terms) and also to share information through social media.

Wikipedia Medicine Portal: http://en.wikipedia.org/wiki/Portal:Medicine

Who is behind this initiative?

The site is run as a wikipedia portal. As with the rest of wikipedia, portals can be edited by anyone and are the work of the community that engages with them. Some of the backgrounds for contributors to the Medicine Portal section are listed as a retired academic and a student, and include users who appear to have a medical or scientific background (among others).

What is it for?

This is a wiki-based portal to various Medicine-related topics, and includes links to articles, images, quotations and news, as well other resources, and is part of the numerous wikipedia portals to different topics. Information related to medicine is presented at this site. Various topics are addressed, including diseases, history, areas of medicine, equipment and treatment. Portals in wikipedia are intended to aid navigation by acting as main pages for topic areas. They comply with wikipedia’s policies. http://en.wikipedia.org/wiki/Wikipedia:Portal

How does the crowd participate?

The contributors to wikipedia participate by adding information, editing existing information, creating new sections and raising queries and discussion points on the information presented. There is a manual of style which suggests the correct way of writing about medicine in wikipedia, for example use of correct nomenclature for disease and drugs. http://en.wikipedia.org/wiki/Wikipedia:Manual_of_Style_(medicine-related_articles). A discussion section http://en.wikipedia.org/wiki/Portal_talk:Medicine is used for discussions around writing on medical topics in wikipedia. The lists of contributors can be viewed through links, and contributors can write their profile with information about their background.

The Association of Medical Research Charities (AMRC) and our 127 member charities provide a direct link to the broader citizen-patient community. AMRC member charities funded over £1 billion for research in 2009/10 – carried out mostly in the university sector.

Our involvement in Patients Participate! builds on a project called ‘Natural Ground’, which we carried out in 2008/09, where we examined how our member charities involve patients and the public in research.

Many of those who support and donate to charities have a deep and ongoing interest in how those funds are used. They want to know how the research charities support increases our understanding, contributes to better treatments and improves outcomes for patients. But it’s challenging to communicate highly complex scientific information in an engaging way. Information about health and medical research may be more available than ever before, but this does not always help patients or the public make sense of scientific research.

We are looking forward to finding out how patient and public involvement in medical research has evolved and to explore the possibility of applying citizen science concepts to make biomedical research information more accessible to lay audiences.

The Patients Participate! project has released two deliverables for open community review. The first is a guide to writing Lay Summaries, and forms part of the DCC series of How To guides, pitched at an intermediate level, offering practical advice.

We’re currently seeking comments on the draft of the ‘How To write a Lay Summary’. The guide is 11 pages long and is available through the A.nnotate system. A.nnotate supports interactive commenting, and the paper can be navigated through the page index which appears after hovering with the mouse underneath the pdf link at the top left. Alternatively you can download the pdf and comment by email. We are hoping to obtain comments from the wider community, so you are invited to share the link with any colleagues who might be interested.

The second draft released for comment is a Briefing Paper on Citizen Science, also being released as part of the DCC briefing paper series. The briefing paper is only four pages long, and comments can be made on the draft via the A.nnotate system. The briefing paper is pitched at a beginner’s ‘awareness’ level.

Tomorrow the Patients Participate! project will be speaking at the 6th Bloomsbury Conference which has the theme of Scholarly outreach, impact and outcomes. There is a wide-ranging programme from discussions of open access, persepectives on impact and public engagement. A link to the slides will be added once the conference website will be updated. An overview of the findings of Patients Participate! will be presented. Citizen science has already featured in some of this morning’s talks.

I’ve just heard about the Open Research Reports initiative, an idea to make disease research as openly accessible as possible to the people that really need it.

Open Research Reports aim to pull together the corpus of open access literature on a certain disease and also create a mechanism of extracting the important information from closed access papers to generate an open summary.

The idea will be discussed at the Open Science Summit, with a Semantic Web Applications and Tools for Life Sciences Hackathon in December. Links to more information and registration are available in a blog post

A comment on that blog post also brings to light another related initiative, described as a repository of evolving reviews on research topics on RocketHub linked to the SciFund Challenge.

In preparation for the workshop, the project constructed a number of scenarios to help think through examples of how web-based, biomedical information resources, technologies and communities could support translation of emerging research findings into plain English. These scenarios were not intended as suggested services, but rather the aim was to stimulate group discussion and invite feedback. These scenarios are completely hypothetical and are not trying to illustrate how a specific service should be built. They are used to draw out questions and issues around the idea of producing lay-summaries through a crowd-sourcing approach, and this is the spirit in which they are being shared, together with some of the questions raised.

Scenario 1: A patient writes a lay summary which is then published and shared

A patient who has been writing lay summaries for six months logs into the literature review system to look for articles that he can write about, in order to share them with other patients. The patient consults the queue of reviews suggested to him on his home page. These have been selected by the system by matching new articles to a list of topics of interest that the patient had written in his profile. The patient selects one of the papers, a report on a trial of a new drug, by reading the abstract, and then follows a link to read the full paper. Having read the paper, the patient then fills in a form which suggests questions and sections that help to summarise the research. Other tools, like a glossary, are available to help. The patient completes the summary, it is marked as finished and then becomes public. The patient reflects on the outcome of this new research and makes a mental note to ask his consultant about the new medications at his next treatment review. Before logging out, the patient spots a new message in his notification area, which advertises the next training session for writers of lay summaries. The patient remembers the last training session where he met other lay summary writers that he now keeps in touch with through the discussion forum, and he considers signing up.

Questions

How should information (lay summaries) be generated, are there accepted methodologies and best practice in this area?
Would this necessitate training and support?
Who should write lay summaries? What are the risks of blurring the division between the amateur and the professional?
What training is needed, and who is going to provide training? How are lay summary writers recruited and trained? (e.g. training methods: open days, shadowing researchers, specific training, handbooks, joint training; recruiting methods: through service users panels, advertising in national newspapers, through charity websites and forums, using links with service managers, websites dedicated to Public Patient Involvement – PPI)
What methods would be most suitable?
What features and tools are needed in a system supporting the writing of lay summaries?
How much guidance would be required to write a lay summary?
Could this guidance be in the form of questions or sections to be filled in about the research?
How are writers matched up with the articles that they summarise?
To what extent are authentication and authorisation systems required? Is there a role for anonymity, and how could this impact trust and quality?
What about issues of privacy?
Is a review process for lay summaries needed before it is shared with others? Who is responsible for editorial control and how is quality maintained? Would this type of activity need to be moderated in some way?
What other benefits could patient writers of lay summaries gain from becoming involved in the activity e.g. gaining of new skills through the training, being better acquainted with the literature,improved understanding, increased interest in research, support gained through building up social networks and other social contact, confidence to participate in other forms of on-line activities like discussions, empowerment through being active participants, inclusivity.
Could lay summaries be used to help connect users with shared interests, and how would they integrate with other online systems that patients use socially?
Where and in which communities do patients and potential lay summary writers already have an on-line presence, how can these be harnessed to build a community around the lay summary writing process?

Scenario 2: A patient advocate reads lay summaries and rates the content.

A patient advocate wants to look at recent research in order to stay up to date with developments in his disease area. Later in the week he will be delivering a talk at an event organised by the charity that he supports to an audience of patients, and his diary for that week also includes a funding review meeting where the research areas for the next call for projects will be discussed. He carries out a search on PubMedCentral using the key words that he has learnt produce the best results for the most relevant research, and asks for items added in the last six months. When the results are returned he sorts them so that the ones flagged as having lay summaries are listed first. He reads through the lay summaries, and for each one that he reads, he provides a rating on a scale which asks (1) how useful the summary was to him and (2) was it easy to understand. He then looks at the next batch of results that do not have any lay summaries. Two of them catch his eye and although he is now running out of time and is almost late for his next appointment, he clicks on the button that pushes the two studies up in the queue for the articles with most requests for lay summaries.

Questions

Who will be reading lay summaries?
What motivates different types of user to look for biomedical information?
How do different communities decide what sources are ‘trusted’ and how is quality decided? What is the definition of quality? (e.g. accurately reflects the research, language is understandable by the proposed readers).
How can charities make use of lay summaries?
What could be the impact of availability of summaries on research funding decisions?
How will lay summaries be discovered?
How can they be presented alongside the original article?
What features are needed in systems providing lay summaries? (e.g. ability to rate summaries, different ways of searching and ranking, how to prioritise articles to be summarised)
Should lay summaries be rated? What sort of ratings would be useful?
Who will be evaluating lay summaries, and how?
What are the key factors for the way biomedical information is presented that enable easy comprehension by a range of people? And what would motivate them to revisit one information resource over another?
What would be the strongest motivators for different groups to produce this type of information (incentives eg credit/visibility or sanctions/mandates)?

Scenario 3: An academic reviews lay summaries written for papers she has published.

An academic wants to find out if any lay summaries have been published for her most recent work in order to evaluate the lay review and make a note of it. The academic checks her inbox and sees that she has received an email notification of new reviews. She uses the links which lead her to the lay summaries. She logs in and identifies herself as the author of the paper; there is a box which allows her to mark that the lay summary is considered to be an accurate representation of the research. She notices that some of the reviews have been rated as 5* for their usefulness and that the lay summaries have been read over 200 times. She makes a mental note to mention this as a measure of the impact of her work in her upcoming staff development review meeting with the Head of Research in her department. Before logging out she notices that one of the summaries has a flag that shows that there are questions posed by patients in the discussion group relevant to that research article. The lay summary had been written by TrustMeI’mAPatient (real life name: Paul), one of the most active of the lay summary writers, whose profile shows that his summaries have high ratings for both usefulness and quality. A link takes her to a discussion group and she engages with the thread, answering some of the questions asked about details of her research, and suggesting some other articles that the patients may also wish to read. While still on the system she scans to see which other lay summaries have been written in the her research area, and finds it interesting to note some trends in the most popular articles, and thinks there may be a link with the main areas of interest highlighted in the latest call from one of the charities that funds her research.

Questions

Who should write and review lay summaries?
What procedures to ensure quality should be put in place?
How do users of lay summaries make judgments of quality and trust based on the presentation of lay summaries, and issues such as readibility and accuracy?
How are academics included in the work flow of lay summary production? Which aspects of the workflows of academics does the lay summary system need to fit in with?
What would be the impact on the work load of academics who need to write or review them?
What feedback do researchers want on the lay summaries related to their research?
What are the effects on impact of the research? What could be the impact on making research better-known and better-used by different stakeholders?
Could lay summaries lead to better quality research? Or have an impact on research ethical practice?
Do researchers think that their research will have more impact if it is made available to patients as lay summaries?
What concerns could academics have around the impact that lay summaries have on their chances of getting further funding for research?
What could the writing and reading of lay summaries tell us about which research is important to patients?
How could statistics on lay summaries be used to provide feedback to academics on the impact of their work?
What role could lay summaries play in improving communication between researchers and patients? How might this scenario effect researcher-patient relationships?
What opportunities do lay summaries present for researchers to engage with the patients that their research affects? What could be the positive consequences e.g. the views of researchers could change as a result of engaging with patients, they could gain a clearer understanding of patient views or be more motivated.
What could be the impact of lay summaries on the directions of future funding of research?
What assumptions do we need to question?

Scenario 4: A charity notifies its members of new research in their field

A charity communications officer is reviewing the ‘latest research’ newsfeed that is displayed on the main home page of the research charity website. He is pleased to find notification about several new lay summaries all in the disease area that is of interest to the members of the charity, and that some of them are derived from some papers that were recently published by research teams funded by the charity. The charity communications officer remembers submitting those articles to the lay summary writing system so that they would be suggested to the lay summary writers for review. He clicks the button that will publish these new reviews in the newstream on the charity website. Before finishing for the day, he places the advert in the newspaper that advertises the next round of training and recruitment for lay summary writers.

Questions

How do lay summaries fit into the other channels of communications that charities use?
What sort of impact could they have e.g. drive traffic to charity websites, increased demand and requests for further information or assistance.
What other systems used by charities do the lay summaries need to integrate with?
How are lay summary reviewers selected? What involvement would be required from charities?
How are articled prioritised for writing lay summaries and what would be the role for charities?
What could be the impact on charities in making the research that they fund better known and more accessible?
What is the benefit of bringing in new voices to the conversations around research?
Would lay summaries help patients to be in a better position to contribute to shaping research strategy and taking part in activities such as peer review, areas in which charities are already involving patients.
Are there models and experience that would help inform charities on how to take this forward?
Where could lay summaries make a greatest difference to the current activities of charities?
What will other internal and external stakeholders of charities think, and how could they be affected?

Scenario 5: Tool builders carry out a user focus group meeting

The web team for the lay review system meet for their weekly planning meeting. Some of them have been looking at the system logs and they have noticed a trend that the lay summaries that are associated with articles which are freely available are more frequently read and rated than those which are behind subscription firewalls. The logs also provide some insight into areas of the website that are well-used and others that appear underused, and some frequently-occuring user errors are also reported. In addition, an email has been received by the web-team from an irate academic who complains that a lay summary of one his articles is incorrect. A decision is made that the summary in question will temporarily be made invisible pending review; this email needs to be passed on to the other teams as the web team are not qualified to judge issues of content quality; however a note is made that perhaps the system requires a feature to help facilitate communication to help resolve such issues as the web team is not the best team to be contacted regarding this. Two team members also reports on a focus group held with lay summary reviewers who provided feedback on some aspects of the user interface and made suggestions for improvement. The focus group participants had also commented on the form that is used to guide the writing of the lay summary as they think it needs some changes. The leader of the team adds this to the list of items that need to be passed on to other teams in the charity who will be able to review the summary-writing form with researchers and lay summary writers. The team discuss dates to plan the next user focus group meeting which will be needed before any of the changes to the interface and the form can be made live.

Questions

How are issues of quality of lay summaries dealt with?
How do we ensure quality of information and how do we avoid propagating incorrect information? (Editorial control and approvals process)
What are the different responsibilities of publishers, charities, service providers? What new roles may be required to manage a lay summary system?
How will the systems be evaluated? What evaluation is needed? (functionality, usability, impact) How can the impact be measured? Need to consider impact on patients, researchers, charities/funders…. how can evidence be collected?
What features are needed in the lay summary system to enable users from all stakeholder groups to provide feedback?
What would make a lay-summary system easy to use?
What is the impact of open access and pay walls?
How could the different communities work together eg patients, charities and researchers in enabling access to and facilitating translation of biomedical information?
What are the different interface design and usability issues associated with the different levels of knowledge, expectation and need?
Who owns the IPR over the summaries? Should they be open access? What are the rights over their re-use? (by charities, researchers). What sort of access-control should be applied? (e.g. registration only, pay-only)

The project with the closest idea to that being investigated in Patients Participate! that we have identified so far is the acawiki.org site. It is a general site run on a wiki platform, that invites summaries of academic articles to make them more accessible. “AcaWiki is building a community of scholars, graduate students, and bloggers to write summaries of papers that are of the most general interest to the public”

Acawiki recently announceda summer project to gather summaries for the top 100 academic papers around the world. They are calling for volunteers to help them reach this target. Can you help?