Product Details

Author Information

Corresponding Author
Kelly Washburn, MPH
Institute for Community Health
163 Gore Street
Cambridge, MA 02141
United States
p: 617-499-6680
kwashburn@challiance.org

Authors (listed in order of authorship):
Kelly Washburn, MPH
Institute for Community Health

Leah Zallman, MD, MPH
Institute for Community Health

Karen Hacker, MD, MPH
Institute for Community Health

Nazmim Bhuiya, MPH
Institute for Community Health

Shalini Tendulkar, ScD, ScM
Institute for Community Health

Monica Demasi, MD
Cambridge Health Alliance

Ruth Hertzman-Miller, MD, MPH
Joslin Diabetes Center

Rob Meyer, MD
Cambridge Health Alliance

Product Description and Application Narrative Submitted by Corresponding Author

What general topics does your product address?

Allied Health, Medicine, Public Health

What specific topics does your product address?

Interdisciplinary collaboration, Partnership building , Research ethics, Workforce development, Community-based participatory research

Does your product focus on a specific population(s)?

N/A

What methodological approaches were used in the development of your product, or are discussed in your product?

Community-academic partnership, Community-based participatory research

What resource type(s) best describe(s) your product?

Lecture/presentation, Training material

Application Narrative

1. Please provide a 1600 character abstract describing your product, its intended use and the audiences for which it would be appropriate.*

Primary care providers, staff, and administrators in community health centers (CHC) have the opportunity to develop research questions relevant to the populations they serve as well as to translate evidence into practice in order to improve the health of their communities. Often, they lack the necessary knowledge and skills to effectively engage in research in their health centers. Available free of charge on the internet, this toolkit contains audio-recorded modules with accompanying PDFs to provide CHC providers and staff with the basic foundation of knowledge, skills, and tools to engage in research. Accompanying the toolkit are three worksheets designed to assist the CHCs as they navigate developing grants, collaborating with external researchers, and utilizing their electronic health record system for research. The toolkit is intended for providers, staff, and administrators at CHCs who are initiating research and/or collaborating with external researchers.

2. What are the goals of the product?

The toolkit aims to equip providers, staff and administrators in CHCs with the basic knowledge needed to engage in translational research. Translational research facilitates the implementation of findings from basic science to practical applications; by definition, translational research requires health centers (such as CHCs) to engage in research in order to test effective implementation practices in their specific populations. However, CHC administrators and providers often lack the knowledge and guidance to participate in translational research (1,2). This toolkit provides CHC clinical and administrative staff with the basic foundation of knowledge, skills and tools to engage effectively in research, both as lead researchers and with external research partners. The online toolkit is comprised of (1) eight audio-recorded modules with discussion questions at the end of each module, (2) eight accompanying PDFs that emphasize educational concepts and (3) three worksheets that provide a roadmap for CHCs as they navigate grant development, collaboration with external researchers, and utilizing the electronic health record for research.

3. Who are the intended audiences or expected users of the product?

The toolkit is intended for providers, staff, and administrators at CHCs who are initiating research or intending to collaborate with external researchers. Initiation of research stems from several ways: an institution responding to an outside request for collaboration, an organization perceiving an unfilled need for information, an individual employee expressing interest, or administration expressing priorities. While the toolkit was developed specifically for CHCs, the concepts covered are applicable to other settings such as other healthcare institutions (e.g. non-community health center practice sites, hospital settings), community-based healthcare agencies (e.g. visiting nurses associations, social work providers), and public health entities (e.g. departments of public health).

4. Please provide any special instructions for successful use of the product, if necessary. If your product has been previously published, please provide the appropriate citation below.

The toolkit is designed to be utilized as a complete package; however, modules can be delivered individually. The “Grant Development Worksheet” prompts users to consider details necessary in writing a grant. The “Collaborating with External Researchers Worksheet” leads CHCs through initial discussions with an external researcher focusing on specifics of the project and impact across the CHC. The “Data Request Form” guides discussions between the researcher and data analyst when requesting data from the EMR to determine specific data needed. Thank you to Dr LeRoi Hicks, Dr Jonathan Finkelstein, and Dr Jocelyn Chu for their contributions. Support came from Harvard Catalyst, The Harvard CTSC and financial contributions from Harvard University and its affiliated academic health care centers.

5. Please describe how your product or the project that resulted in the product builds on a relevant field, discipline or prior work. You may cite the literature and provide a bibliography in the next question if appropriate.

The Institute of Medicine advocates the concept of “a learning health system—in which science, informatics, incentives, and culture are aligned for continuous improvement and innovation,”. (1) CHCs are well-placed to become learning health systems, but need additional support to do so. Although in theory, CHCs are often partnered with academic institutions, in practice, there is often a disconnect between the academic and the community-centered approach.(2) CHCs have the unique opportunity to develop new research questions relevant to their populations and subsequently initiate evidence-based research in their health facilities.

While CHC providers and administrators express high levels of interest in conducting research at their centers (3,4) and in collaboration with external research partners; (5) CHC providers, staff and leadership often lack the knowledge and skills needed to conduct research.(3,4) Training CHC staff in all aspects of the research process would enable them to undertake research efforts as part of a team.(5) Research training should include research techniques (such as study design, tool development) as well as writing, dissemination, and working with collaborators. (5)

To our knowledge there is no publicly available research training that can be directly conducted within the CHC. Research trainings have largely taken the form of conferences (6, 7); knowledge gained is largely limited to those who attend . Practice-based research networks share opportunities for research and collaboration but are also limited to those participating clinics.(5) The National Association of Community Health Centers does provide a free online comprehensive catalog of existing research trainings (9) However, it was not designed as a step-by-step training mechanism and does not contain a curriculum that guides users through each stage of the research process. Obtaining training on each stage of research requires review of all resources, which would be too-time consuming for most CHCs. Consequently, there is no one publicly available resource that provides training from the start to finish of the process in a concise enough form as to be useful to CHC staff. (9) This toolkit addresses this lack of publicly available comprehensive research training at CHCs.

6. Please provide a bibliography for work cited above or in other parts of this application. Provide full references, in the order sited in the text (i.e. according to number order). .

1. Institute of Medicine. The Learning Health System and its Innovation Collaboratives: Update Report. Washington, D.C., 2011. Accessed online 4/1/13 at http://www.iom.edu/Activities/Quality/.

2. Tyus NC, Gibbons MC, Robinson KA, Twose C, Guyer B. In the shadow of academic medical centers: a systematic review of urban health research in Baltimore City. J Community Health. 2010 Aug;35(4):433-52.

3. Hacker, K., Bhuiya, N., et al. Assessing Research Interest and Capacity in Community Health Centers. Clinical and Translational Science. Accepted for Publication.

4. Zallman, L., Tendulkar, S., et al. Provider’s perspectives on building research and quality improvement capacity in primary care: a strategy to improve workforce satisfaction. Clinical and Translational Science. Accepted for Publication.

5. Del Mar C, Askew D. Building family/general practice research capacity. Ann Fam Med. 2004 May 26;2 Suppl 2:S35-40

6. Ried, K., B. D. Montgomery, et al. (2008). "General practice research training: impact of the Australian Registrar Research Workshop on research skills, confidence, interest and involvement of participants, 2002-2006." Family practice 25(2): 119-26.

7. Westfall JM, Ingram B, et al. (2012). “Engaging communities in education and research: PBRNs, AHEC, and CTSA.” Clin Transl Sci. 2012 Jun;5(3):250-8. doi: 10.1111/j.1752-8062.2011.00389.x.

8. Wallerstein N, Duran B. Community-based participatory research contributions to intervention research: the intersection of science and practice to improve health equity. Am J Public Health 2010;100(S1):S40-S46.

9. Research Training Catalog. Retrieved from http://eclinician.org/NACHC/. Accessed on 4/1/2013.

7. Please describe the project or body of work from which the submitted product developed. Describe the ways that community and academic/institutional expertise contributed to the project. Pay particular attention to demonstrating the quality or rigor of the work:

• For research-related work, describe (if relevant) study aims, design, sample, measurement instruments, and analysis and interpretation. Discuss how you verified the accuracy of your data.
• For education-related work, describe (if relevant) any needs assessment conducted, learning objectives, educational strategies incorporated, and evaluation of learning.
• For other types of work, discuss how the project was developed and reasons for the methodological choices made.

In 2008, Harvard Medical School received a Clinical Translational Science Award (CTSA) from the National Institutes for Health. The focus of Harvard Catalyst, Harvard’s CTSA, is to improve population health by promoting the translation of clinical research into practice. A key program of Harvard Catalyst is the Community Health Innovation and Research Program (HC-CHIRP). One of HC-CHIRP’s goals is to improve population health by engaging communities to implement evidence-based practices in their organizations; a key mechanism is the Safety Net Research Infrastructure Initiative (SNRII).

The SNRII aims to build safety net health centers’ research infrastructure by engaging safety net clinic staff. The SNRII represents a partnership between a community-based research organization (the Institute for Community Health), an academic institution (Harvard Medical School), an integrated community health system (Cambridge Health Alliance or CHA) and three CHCs in the Boston area, representing the community partners. Staff from the health centers, referred to as ‘champions’, were members of the community SNRII collaboration. During its initial year, the SNRII was guided by a core team comprised of academic partners (including authors ST, NB, KH), community partners including three clinician ‘champions’ at CHA (hereafter ‘CHA champions’: authors MD, RH, RM), medical directors, informational technology (IT) staff and ambulatory leadership at CHA. In addition to the previously mentioned partners, two other academic partners from different academic affiliations provided expertise on the following topics: distinguishing quality improvement vs. research, building research infrastructure, data access and utilization, study design and methods, and dissemination. In the subsequent years, the core team (hereafter core team II) consisted of academic partners (including authors ST, NB, KH, LZ, KW) who worked closely with community partners including primary care providers and a quality improvement specialist at three additional MA-based CHCs (hereafter ‘CHC champions’). Throughout this effort, the core team and community partners included clinicians from three medical specialties including pediatrics, adult, and family medicine. Together, these academic and community partners engaged in a collaborative process to build research infrastructure within their health centers.

The SNRII provided the champions with protected time to engage in activities and research training --through monthly learning sessions--, and mentors to provide guidance. The learning sessions were designed to provide practical research skills to the champions; the topics were chosen through a collaborative process in which the champions identified subject areas with which they felt they needed assistance.

During its initial year, the core team developed a survey that assessed safety net providers’ perspectives on involvement in research and quality improvement. The survey was administered to primary care providers at CHA and the three CHCs participating in the SNRII. The survey revealed high levels of interest in conducting research and/or quality improvement (QI), but little prior research training or experience. (4) Around the same time, Harvard Catalyst collaborated with the Massachusetts League of Community Health Centers to administer an analogous survey, aimed to understand CHC Medical Directors’ perspectives on research and/or QI on an organizational level across CHCs in Massachusetts. Similarly, the survey revealed that Medical Directors at CHCs perceived high levels of interest in research and/or QI at their CHCs.(3) Both surveys reported lack of research skills and knowledge and financial support as the major barriers to engaging in such efforts.(3,4)

8. Please describe the process of developing the product, including the ways that community and academic/institutional expertise were integrated in the development of this product.

The core team developed the toolkit in response to the need for research training identified in our prior work (3,4), Throughout this work, they used community-based participatory research (CBPR) principles, a collaborative approach that involves all partners equally in the research process with the goal of combining knowledge and action to improve community health outcomes and eliminate health disparities. (6) The core team chose to use the CBPR approach because they believed that when a collaborative approach is applied, findings from the research are more likely to be translated into action. (8)
The core team identified a need for practical, basic tools for CHCs that wanted to effectively participate in research. Through a consensus building process, the core team developed a toolkit that would be comprised of lectures and identified key topics areas or ‘modules’. Three of these modules were adapted from previous lectures that core team members had presented during learning sessions. The remainder of the modules were developed by the academic partners of the core team; they drew on the knowledge they had gained from their extensive work with the champions and reviewed the literature. Associated with each of the eight modules are 3-5 learning objectives. These learning objectives provide participants the intended goals of each module. Each of the learning objectives tie back into the overarching goal of the toolkit, which is to provide CHCs the elements needed to build primary care research infrastructure.
The core team then modified the toolkit based on extensive feedback from program staff, champions in the SNRII and other CHC providers. In order to obtain feedback from a broader range of providers aside from the champions, the core team presented the toolkit to primary care providers at CHA. This process began with a kick-off event in which all primary care providers at CHA were invited to learn about the importance of conducting research and to network with other primary care providers who are already engaging in research. At this meeting, providers were invited to form the Primary Care Research Network, a network of providers at CHA with an interest in conducting quality improvement or research. The Primary Care Research Network, comprised of 30 primary care providers, was invited to three presentations that would later form the foundation for three of the toolkit modules. After each presentation, attendees provided feedback through written evaluations. Through this process, the core team modified the toolkit to include a module on Institutional Review Boards and modified the three presentations in order to ensure the potential limitations of the product were minimized.

As the champions in the initiative engaged in research, they identified a need for guidance in several areas including “external research relationships”, “grant development” and Data extraction. Each of these areas was identified as a critical point in the research engagement process.

Initially, one of the champions (RM) was approached by an outside researcher wanting to do research at their health center. This brought up a host of issues that were discussed among the core team. To meet this need, core team II developed the “Collaborating with External Researchers Worksheet” to guide the discussion with the external researcher.

The second tool focused on grant development which was developed following the experience of one of the CHC champions as they collaborated on a pilot grant with an external researcher. Core team II developed the “Grant Development Worksheet” to address this need. Once the grant was submitted, core team II sat down with the champions to reflect on the process of developing the grant. The champions identified questions they wish they had known or thought about before the process started. These questions formed the basis of the worksheet, which the champions then pilot tested.

The third tool called the “Data Consultation Form” was developed in response to the core team’s experience that clinical data requests utilizing the Electronic Medical Record (EMR) system for research at CHCs were often incomplete. A pre-existing form, developed by the IT staff member of the core team, used at ICH was piloted with five champions - the champions were invited to use the form in their learning session homework assignments and in preparing for a pilot grant submission. The core team II then elicited oral feedback from the champions and presented the form to a separate group of fifteen primary care providers conducting research and/or quality improvement projects. Based on feedback from these providers, the form was extensively revised. The form was then piloted with two CHC champions, and again extensive revisions were made.

9. Please discuss the significance and impact of your product. In your response, discuss ways your product has added to existing knowledge and benefited the community; ways others may have utilized your product; and any relevant evaluation data about impact, if available. If the impact of the product is not yet known, discuss its potential significance.

The toolkit is available online free of charge on the Harvard Catalyst website; a link to the Harvard Catalyst website is available on the National Association of Community Health Centers (NACHC) website. (9) The toolkit modules provide basic research concepts, illustrated through examples drawn from CHCs, geared specifically to CHCs. The accompanying worksheets guide the CHCs in key interactive aspects of engaging in research (grant development, collaboration, data request). To our knowledge, these concepts have never been outlined in a free, practical, accessible medium geared toward CHCs. There has been no concentrated dissemination effort since the launch of the toolkit on the Catalyst website. With that stated, from the June 2012 launch on the Catalyst website and February 2013, the toolkit received over 300 unique views or roughly 35 views per month. This number may increase with a more concentrated effort to disseminate the webpage. The consistent interest in the toolkit confirms a need for a practical free tool such as ours. In addition, the SNRII champions have used the worksheets to guide conversations with external researchers; several of these conversations have led to collaborations with increased confidence on the part of the champions that they understood the details of the study and the impact on clinic staff, patients and their own careers. Several of champions have used the data consultation form to guide discussions with their data analysts when seeking data for grant writing or research purposes.

10. Please describe why you chose the presentation format you did.

We chose this format for several reasons. We wanted a medium that would be accessible to large audiences (i.e. available through a website). We also wanted a format that was both logistically simple for community health centers to use and available at no cost to the health centers. CHCs typically do not have sufficient funds to purchase educational materials for their staff. By making the toolkit free and easy to access, more opportunities are given to health center staff to digest the different topics. By pre-recording the toolkit lectures, we were able to provide learning opportunities at a relatively low cost to Catalyst and free to providers and health centers. We chose to provide PDF versions to accompany the audio-recorded lectures so that learners could easily interact with the material (i.e. take notes) as they listened to the lectures. The PDFs are also available online as a reference and for consumers to view the content if they are unable to listen to the module. Finally, we chose to use worksheets to address topic areas in which providers engage with other members of the research team (external collaborators, grant partners, data analysts). We chose worksheets because they allow providers to actively engage with the material through answering questions and simultaneously gain and record information.

11. Please reflect on the strengths and limitations of your product. In what ways did community and academic/institutional collaborators provide feedback and how was such feedback used? Include relevant evaluation data about strengths and limitations if available.

The main strength of the toolkit lies in its accessibility to a broad population of health center researchers. The toolkit is free for anybody to use and is easily found through the Harvard Catalyst and National Association of Community Health Centers websites. It does not assume the participant has any prior research knowledge, meaning a user with little or no experience in conducting research is able to use the toolkit to begin the process. It is therefore easy to understand and provides real-world examples to help answer questions. One of the key causes of its accessibility is the collaborative process which informed the development. Based on extensive feedback from program staff, the champions in the SNRII and other CHC providers, we made improvements, such as adding in the IRB module, mentioned earlier, and modifying the data request worksheet based on the feedback of the pilot test, in order to ensure the potential limitations of the product were minimized.
Despite its wide reach and accessibility, the toolkit has a few limiting features. The toolkit is not generalizable to institutions already engaged in large-scale research efforts or to providers engaging in quality improvement efforts. In order to maintain low implementation costs, we were unable to include a mechanism for participants to interact with the lecturers (such as to ask questions). We have attempted to minimize this limitation by anticipating questions and providing links to other resources that address those issues. Finally, the toolkit has no self-assessment for the participant to determine the topics they mastered or areas of improvement.

12. Please describe ways that the project resulting in the product involved collaboration that embodied principles of mutual respect, shared work and shared credit. If different, describe ways that the product itself involved collaboration that embodied principles of mutual respect, shared work and shared credit. Have all collaborators on the product been notified of and approved submission of the product to CES4Health.info? If not, why not? Please indicate whether the project resulting in the product was approved by an Institutional Review Board (IRB) and/or community-based review mechanism, if applicable, and provide the name(s) of the IRB/mechanism.

HC-CHIRP, the program the SNRII sits under, builds relationships and collaboration with community partners. One of its main aims is to increase participation with community partners in developing and engaging in research. For the past three years, the SNRII has collaborated with community partners in health centers in research training. At the beginning of the funding year, staff from the initiative, along with the multi-disciplinary core team, developed topics and trainings believed to be beneficial to the participants. Due to the varied disciplines of each member of the development team, each member was able to use their expertise and knowledge to share in creating a well-rounded product. From the beginning of the initiative, staff used a participatory approach to solicit feedback from the community partners, known as champions, on topics and skills they wanted to develop or improve in order to be successful at engaging in research. That feedback influenced the discussions and trainings provided to the champions with the overarching goal of implementing into their practices. Involving the champions in the process showed a respect of their knowledge and experiences and they offer valuable insight into their health centers. Modules of the toolkit and all three accompanying worksheets are direct results from the collaboration with the champions to address their needs.

Key collaborators who helped develop the toolkit are listed as authors on this submission. Therefore, they reviewed and approved the submission to CES4Health.info.