In preparation for the workshop, the project constructed a number of scenarios to help think through examples of how web-based, biomedical information resources, technologies and communities could support translation of emerging research findings into plain English. These scenarios were not intended as suggested services, but rather the aim was to stimulate group discussion and invite feedback. These scenarios are completely hypothetical and are not trying to illustrate how a specific service should be built. They are used to draw out questions and issues around the idea of producing lay-summaries through a crowd-sourcing approach, and this is the spirit in which they are being shared, together with some of the questions raised.
Scenario 1: A patient writes a lay summary which is then published and shared
A patient who has been writing lay summaries for six months logs into the literature review system to look for articles that he can write about, in order to share them with other patients. The patient consults the queue of reviews suggested to him on his home page. These have been selected by the system by matching new articles to a list of topics of interest that the patient had written in his profile. The patient selects one of the papers, a report on a trial of a new drug, by reading the abstract, and then follows a link to read the full paper. Having read the paper, the patient then fills in a form which suggests questions and sections that help to summarise the research. Other tools, like a glossary, are available to help. The patient completes the summary, it is marked as finished and then becomes public. The patient reflects on the outcome of this new research and makes a mental note to ask his consultant about the new medications at his next treatment review. Before logging out, the patient spots a new message in his notification area, which advertises the next training session for writers of lay summaries. The patient remembers the last training session where he met other lay summary writers that he now keeps in touch with through the discussion forum, and he considers signing up.
Questions
How should information (lay summaries) be generated, are there accepted methodologies and best practice in this area?
Would this necessitate training and support?
Who should write lay summaries? What are the risks of blurring the division between the amateur and the professional?
What training is needed, and who is going to provide training? How are lay summary writers recruited and trained? (e.g. training methods: open days, shadowing researchers, specific training, handbooks, joint training; recruiting methods: through service users panels, advertising in national newspapers, through charity websites and forums, using links with service managers, websites dedicated to Public Patient Involvement – PPI)
What methods would be most suitable?
What features and tools are needed in a system supporting the writing of lay summaries?
How much guidance would be required to write a lay summary?
Could this guidance be in the form of questions or sections to be filled in about the research?
How are writers matched up with the articles that they summarise?
To what extent are authentication and authorisation systems required? Is there a role for anonymity, and how could this impact trust and quality?
What about issues of privacy?
Is a review process for lay summaries needed before it is shared with others? Who is responsible for editorial control and how is quality maintained? Would this type of activity need to be moderated in some way?
What other benefits could patient writers of lay summaries gain from becoming involved in the activity e.g. gaining of new skills through the training, being better acquainted with the literature,improved understanding, increased interest in research, support gained through building up social networks and other social contact, confidence to participate in other forms of on-line activities like discussions, empowerment through being active participants, inclusivity.
Could lay summaries be used to help connect users with shared interests, and how would they integrate with other online systems that patients use socially?
Where and in which communities do patients and potential lay summary writers already have an on-line presence, how can these be harnessed to build a community around the lay summary writing process?
Scenario 2: A patient advocate reads lay summaries and rates the content.
A patient advocate wants to look at recent research in order to stay up to date with developments in his disease area. Later in the week he will be delivering a talk at an event organised by the charity that he supports to an audience of patients, and his diary for that week also includes a funding review meeting where the research areas for the next call for projects will be discussed. He carries out a search on PubMedCentral using the key words that he has learnt produce the best results for the most relevant research, and asks for items added in the last six months. When the results are returned he sorts them so that the ones flagged as having lay summaries are listed first. He reads through the lay summaries, and for each one that he reads, he provides a rating on a scale which asks (1) how useful the summary was to him and (2) was it easy to understand. He then looks at the next batch of results that do not have any lay summaries. Two of them catch his eye and although he is now running out of time and is almost late for his next appointment, he clicks on the button that pushes the two studies up in the queue for the articles with most requests for lay summaries.
Questions
Who will be reading lay summaries?
What motivates different types of user to look for biomedical information?
How do different communities decide what sources are ‘trusted’ and how is quality decided? What is the definition of quality? (e.g. accurately reflects the research, language is understandable by the proposed readers).
How can charities make use of lay summaries?
What could be the impact of availability of summaries on research funding decisions?
How will lay summaries be discovered?
How can they be presented alongside the original article?
What features are needed in systems providing lay summaries? (e.g. ability to rate summaries, different ways of searching and ranking, how to prioritise articles to be summarised)
Should lay summaries be rated? What sort of ratings would be useful?
Who will be evaluating lay summaries, and how?
What are the key factors for the way biomedical information is presented that enable easy comprehension by a range of people? And what would motivate them to revisit one information resource over another?
What would be the strongest motivators for different groups to produce this type of information (incentives eg credit/visibility or sanctions/mandates)?
Scenario 3: An academic reviews lay summaries written for papers she has published.
An academic wants to find out if any lay summaries have been published for her most recent work in order to evaluate the lay review and make a note of it. The academic checks her inbox and sees that she has received an email notification of new reviews. She uses the links which lead her to the lay summaries. She logs in and identifies herself as the author of the paper; there is a box which allows her to mark that the lay summary is considered to be an accurate representation of the research. She notices that some of the reviews have been rated as 5* for their usefulness and that the lay summaries have been read over 200 times. She makes a mental note to mention this as a measure of the impact of her work in her upcoming staff development review meeting with the Head of Research in her department. Before logging out she notices that one of the summaries has a flag that shows that there are questions posed by patients in the discussion group relevant to that research article. The lay summary had been written by TrustMeI’mAPatient (real life name: Paul), one of the most active of the lay summary writers, whose profile shows that his summaries have high ratings for both usefulness and quality. A link takes her to a discussion group and she engages with the thread, answering some of the questions asked about details of her research, and suggesting some other articles that the patients may also wish to read. While still on the system she scans to see which other lay summaries have been written in the her research area, and finds it interesting to note some trends in the most popular articles, and thinks there may be a link with the main areas of interest highlighted in the latest call from one of the charities that funds her research.
Questions
Who should write and review lay summaries?
What procedures to ensure quality should be put in place?
How do users of lay summaries make judgments of quality and trust based on the presentation of lay summaries, and issues such as readibility and accuracy?
How are academics included in the work flow of lay summary production? Which aspects of the workflows of academics does the lay summary system need to fit in with?
What would be the impact on the work load of academics who need to write or review them?
What feedback do researchers want on the lay summaries related to their research?
What are the effects on impact of the research? What could be the impact on making research better-known and better-used by different stakeholders?
Could lay summaries lead to better quality research? Or have an impact on research ethical practice?
Do researchers think that their research will have more impact if it is made available to patients as lay summaries?
What concerns could academics have around the impact that lay summaries have on their chances of getting further funding for research?
What could the writing and reading of lay summaries tell us about which research is important to patients?
How could statistics on lay summaries be used to provide feedback to academics on the impact of their work?
What role could lay summaries play in improving communication between researchers and patients? How might this scenario effect researcher-patient relationships?
What opportunities do lay summaries present for researchers to engage with the patients that their research affects? What could be the positive consequences e.g. the views of researchers could change as a result of engaging with patients, they could gain a clearer understanding of patient views or be more motivated.
What could be the impact of lay summaries on the directions of future funding of research?
What assumptions do we need to question?
Scenario 4: A charity notifies its members of new research in their field
A charity communications officer is reviewing the ‘latest research’ newsfeed that is displayed on the main home page of the research charity website. He is pleased to find notification about several new lay summaries all in the disease area that is of interest to the members of the charity, and that some of them are derived from some papers that were recently published by research teams funded by the charity. The charity communications officer remembers submitting those articles to the lay summary writing system so that they would be suggested to the lay summary writers for review. He clicks the button that will publish these new reviews in the newstream on the charity website. Before finishing for the day, he places the advert in the newspaper that advertises the next round of training and recruitment for lay summary writers.
Questions
How do lay summaries fit into the other channels of communications that charities use?
What sort of impact could they have e.g. drive traffic to charity websites, increased demand and requests for further information or assistance.
What other systems used by charities do the lay summaries need to integrate with?
How are lay summary reviewers selected? What involvement would be required from charities?
How are articled prioritised for writing lay summaries and what would be the role for charities?
What could be the impact on charities in making the research that they fund better known and more accessible?
What is the benefit of bringing in new voices to the conversations around research?
Would lay summaries help patients to be in a better position to contribute to shaping research strategy and taking part in activities such as peer review, areas in which charities are already involving patients.
Are there models and experience that would help inform charities on how to take this forward?
Where could lay summaries make a greatest difference to the current activities of charities?
What will other internal and external stakeholders of charities think, and how could they be affected?
Scenario 5: Tool builders carry out a user focus group meeting
The web team for the lay review system meet for their weekly planning meeting. Some of them have been looking at the system logs and they have noticed a trend that the lay summaries that are associated with articles which are freely available are more frequently read and rated than those which are behind subscription firewalls. The logs also provide some insight into areas of the website that are well-used and others that appear underused, and some frequently-occuring user errors are also reported. In addition, an email has been received by the web-team from an irate academic who complains that a lay summary of one his articles is incorrect. A decision is made that the summary in question will temporarily be made invisible pending review; this email needs to be passed on to the other teams as the web team are not qualified to judge issues of content quality; however a note is made that perhaps the system requires a feature to help facilitate communication to help resolve such issues as the web team is not the best team to be contacted regarding this. Two team members also reports on a focus group held with lay summary reviewers who provided feedback on some aspects of the user interface and made suggestions for improvement. The focus group participants had also commented on the form that is used to guide the writing of the lay summary as they think it needs some changes. The leader of the team adds this to the list of items that need to be passed on to other teams in the charity who will be able to review the summary-writing form with researchers and lay summary writers. The team discuss dates to plan the next user focus group meeting which will be needed before any of the changes to the interface and the form can be made live.
Questions
How are issues of quality of lay summaries dealt with?
How do we ensure quality of information and how do we avoid propagating incorrect information? (Editorial control and approvals process)
What are the different responsibilities of publishers, charities, service providers? What new roles may be required to manage a lay summary system?
How will the systems be evaluated? What evaluation is needed? (functionality, usability, impact) How can the impact be measured? Need to consider impact on patients, researchers, charities/funders…. how can evidence be collected?
What features are needed in the lay summary system to enable users from all stakeholder groups to provide feedback?
What would make a lay-summary system easy to use?
What is the impact of open access and pay walls?
How could the different communities work together eg patients, charities and researchers in enabling access to and facilitating translation of biomedical information?
What are the different interface design and usability issues associated with the different levels of knowledge, expectation and need?
Who owns the IPR over the summaries? Should they be open access? What are the rights over their re-use? (by charities, researchers). What sort of access-control should be applied? (e.g. registration only, pay-only)
