Product Details

Author Information

Corresponding Author
Alicia Hibbert
University of Alberta
2-410 Enterprise Square
10230 Jasper Ave NW
Edmonton, AB T5J 4P6
Canada
p: 780-248-1194
ahibbert@ualberta.ca

Authors (listed in order of authorship):
Fay Fletcher
University of Alberta

Lola Baydala

Fiona Robertson

Product Description and Application Narrative Submitted by Corresponding Author

What general topics does your product address?

Public Health, Social & Behavioral Sciences

What specific topics does your product address?

Community assessment, Community engagement, Interdisciplinary collaboration, Partnership building , Program evaluation, Community-based participatory research

Does your product focus on a specific population(s)?

Indigenous/Aboriginal

What methodological approaches were used in the development of your product, or are discussed in your product?

Community needs assessment, Community-based participatory research , Focus group , Qualitative research

What resource type(s) best describe(s) your product?

Evaluation tool

Application Narrative

1. Please provide a 1600 character abstract describing your product, its intended use and the audiences for which it would be appropriate.*

This product is comprised of three tools aimed at collecting input from community members participating in a health-focused community based participatory research project. Two were inspired by The Evaluator’s Cookbook1; permission was received by the publisher to share these tools in this publication. The third activity is an original creation. It was developed after implementing the first two activities in a cross cultural context and was initially used with First Nations children.

These activities were piloted with a range of ages. Focus group participants were community stakeholders or direct program recipients. Participants responded positively to these activities, especially in cross-cultural projects where history, language, age, or education may create barriers between facilitators and participants.

The activities create a non-threatening environment that fosters engagement and participation in data collection processes typically burdened by power differentials and language or cultural barriers. This may include children and youth, seniors, those for whom English is a second language, or other vulnerable populations who have difficulty expressing their opinions to facilitators they perceive as representatives of the dominant culture.

These activities exemplify the principles of participatory evaluation; the tools were refined and created in collaboration with community members. The approach in the process and activities ensures that data collection is ethical, respectful of the research relationship, and mindful of the information collected through the activities.

2. What are the goals of the product?

Creative activities allow for visual and physical engagement; active involvement increases investment as participants remain in control of their decisions2,3,4. In seeing immediate visual results from the focus group activities, participants feel they were successfully involved in an important project and have the power to tell their stories5,6.

Based on our experiences implementing the focus group activities, participants were able to think by doing, which stimulated discussion. Rather than directing questions at individual participants, as can occur in the one-to-one interview7, participants were able to reflect and respond to others. Researchers have employed photovoice methodologies to achieve similar ends, allowing rapport to build with child, youth, or adult participants8,9. Discussions are complemented by visual activities because they generate different ideas than verbal or written response alone10,11.

The goals of the product are to:

1. Foster participatory evaluation by:
• building empowerment, increasing engagement, and engendering trust
• incorporating community priorities, including information they hope to gather through the evaluation process
• identifying processes that encourage participation in the data collection design and process
• contributing to relationship building as activities show preparation and commitment from academics and stimulate self-directed communication among participants. Where recording may prove difficult – with children and seniors – activities produce a lasting visual result
• having participants come to consensus on the final appearance of each activity poster, so that participants have greater investment in the discussion

2. Increase engagement of participants through:
• active participation
• creative activities that foster visual and kinetic engagement in the focus group; participants think through doing which stimulates discussion and reduces questions directed toward individuals.

3. Create visual archives that are meaningful to participants and useful for knowledge dissemination activities with community stakeholders.

3. Who are the intended audiences or expected users of the product?

Focus group facilitators working with:
• Children – provide insight into a child’s lived experience12,13
• Social justice issues – engage participants in sensitive topics14,15
• Marginalized groups – build positive anticipation for health interventions with those for whom a direct interview is outside of their sociolinguistic norms16,17,18
• A variety of groups – the tools use facilitation techniques, creating a smooth process for facilitators and participants alike

Conducting research on sensitive topics requires an open and trusting environment. This problem is more pronounced when working with Aboriginal populations, immigrant populations, or children. These activities allow facilitators to foster participation in an informal environment, reducing barriers to participation.

4. Please provide any special instructions for successful use of the product, if necessary. If your product has been previously published, please provide the appropriate citation below.

These tools can be used in focus groups or for a decision making process. Focus groups identify issues with participants (skip #1-2 below). The decision making process assesses which pre-identified issues are the most pressing for participants (skip 3c below).
1. Identify concepts with community stakeholders outside of the group setting.
2. Validate concepts among community partners.
3. Prepare facilitators to work with small groups to:
a. Describe the purpose of the activity.
b. Setup recorders. If you have an extra facilitator, take notes.
c. Brainstorm the concepts for each activity in small groups. Write these on sticky notes.
d. Read all post-it notes aloud; assist in placement.
e. Encourage discussion by asking questions.
f. Photograph the results.

5. Please describe how your product or the project that resulted in the product builds on a relevant field, discipline or prior work. You may cite the literature and provide a bibliography in the next question if appropriate.

While conducting a needs assessment at Buffalo Lake Metis Settlement, the diversity of participants’ age and formal education made written surveys and direct interviews challenging. Health-related topics, such as substance abuse and violence prevention, made it crucial to maintain an environment of trust. The team was motivated to be creative in focus group strategies due to experience with traditional focus groups with children in a First Nations community. Drawing on this experience, the team implemented the Target and ThermoScale activities, inspired by The Evaluator’s Cookbook19. Previous work with First Nations children, combined with the needs assessment at Buffalo Lake Metis Settlement, led to the original creation of the third activity, Two Circles.

Empowerment
Research shows that visual methods empower participants20,21. Conducting focus groups with Aboriginal participants is difficult if the researcher is perceived as representative of the dominant culture. In our approach, we aim for shared power through reciprocal exchange, similar to Swartz22. The product allows participants to visualize the results of their choices immediately. As Coad23 attests, the discussion that takes place during the activity provides qualitative data resulting in visual output.

Engagement
Researchers have a responsibility to ensure that participants are treated ethically and equitably; this will result in an increase in productive engagement24,25,26. Engagement is also improved through modes of communication that are comfortable for participants, whether they are children, seniors, or minorities. These tools foster an informal tone that would otherwise be inhibited by the use of structured questions. No reading is required, eliminating literacy-based exclusion. Completing the activity encourages participants to consider their responses, to better express their opinions, and allows researchers to “literally see what participants are talking about”27.

Trust
Substance abuse and violence are sensitive topics that require an atmosphere in which people feel safe to engage. Age and education imbalances create anxieties about public speaking for participants and junior researchers. We flattened power gradients by incorporating these activities; participants and researchers were involved, hands-on, throughout discussions of the visuals. Participants felt their input was valued.

6. Please provide a bibliography for work cited above or in other parts of this application. Provide full references, in the order sited in the text (i.e. according to number order). .

1. McCabe A, Horsley K. The Evaluator’s Cookbook. New York: Routledge; 2008.
2. Crishna B. Participatory evaluation (I) – sharing lessons from fieldwork in Asia. Child: Care, Health and Development 2006;33(3):217-223.
3. Gittelsohn J, Evans M, Helitzer D, Anliker J, Story M, Metcalfe L, Davis S, Iron Cloud P. Formative research in a school-based obesity prevention program for Native American school children (Pathways). Health Education Research: Theory & Practice 1998;13(2):251-265.
4. Higgins JW, Reed N. The Girl Power project - recreation, BC health goals, and social capital. Canadian Journal of Public Health 2001;92(6):448–452.
5. Davis Halifax NV, Yurichuk F, Meeks J, Khandor E. Photovoice in a Toronto community partnership: Exploring the social determinants of health with homeless people. Progress in Community Health Partnerships: Research, Education, and Action 2008;2(2):129-136.
6. Jackson SF. A participatory group process to analyze qualitative data. Progress in Community Health Partnerships: Research, Education, and Action 2008;2(2):161-170.
7. Gittselsohn et al., Health Education Research.
8. Epstein I, Stevens B, McKeever P, Baruchel S. Photo elicitation interview (PEI): using photos to elicit children’s perspectives. International Journal of Qualitative Methods 2006;5(3):1-11.
9. Foster-Fishman P, Nowell B, Deacon Z, Nievar MA, McCann P. Using methods that matter: The impact of reflection, dialogue, and voice. American Journal of Community Psychology 2005;36(3-4):275-291.
10. Barbyshire P, MacDougall C, Schiller W. Multiple methods in qualitative research with children: more insight or just more? Qualitative Research 2005;5(4):417-436.
11. Gittelsohn et al., Health Education Research.
12. Barbyshire et al., Qualitative Research.
13. Milton B, Woods SE, Dugdill L, Porcellato L, Springett RJ. Starting young? Children’s experiences of trying smoking during pre-adolescence. Health Education Research 2008;23(2):298-309.
14. Davis Halifax et al., Progress in Community Health Partnerships.
15. Foster-Fishman et al., American Journal of Community Psychology.
16. Gittselsohn et al., Health Education Research.
17. Epstein et al., International Journal of Qualitative Methods.
18. Jackson, Progress in Community Health Partnerships.
19. McCabe and Horsley, p 18-19, 47.
20. White A, Busine N, Caprena-Méndez F, Laoire CN. Using visual methodologies to explore contemporary Irish childhoods. Qualitative Research 2010;10:143-158.
21. Brazg T, Bekemeier B, Spigner C, Huebner CE. Our Community in Focus: The Use of Photovoice for Youth-Driven Substance Abuse Assessment and Health Promotion. Health Promotion Practice 2011; 12(4):502-511.

22. Swartz S. Going deep and giving back: Exceeding ethical expectations when working amongst vulnerable youth. Qualitative Research 2011;11(1):47-68.
23. Coad J. Using arts-based techniques in engaging children and young people in health care consultations and/or research. Journal of Research in Nursing 2007;12:487-497.
24. White et al., Qualitative Research.
25. Davis Halifax et al., Progress in Community Health Partnerships.
26. Hill M. Children’s Voices on Ways of Having a Voice: Children’s and young people’s perspectives on methods used in research and consultation. Childhood 2006;13:69-89.
27. Liebenberg L. The visual image as discussion point: Increasing validity in boundary crossing research. Qualitative Research 2009;9(4): 444.
28. McCabe and Horsley, p 18-19, 47.
29. Plested BA, Edwards RW, Jumper-Thurman P. Community Readiness: A Handbook for Successful Change. Fort Collins, CO: Tri-Ethnic Center for Prevention Research; 2006.
30. Botvin GJ, Griffin KW, Nichols TR. Preventing youth violence and delinquency through a universal school-based prevention approach. Prevention Science 2006;7:403-408.
31. Coad, 2007, Journal of Research in Nursing.
32. McCabe and Horsley, p 18-19, 47.
33. Plested et al.
34. Botvin et al.
35. Plested et al.
36. Plested et al., p 14.
37. McCabe and Horsley, p 18-19.
38. Graphics for Evaluation Frameworks and Reports: Program Evaluation Models. Johnston Research Inc. [cited September 25, 2012]. Available from http://johnstonresearch.ca/approach/graphics-for-evaluation-frameworks-and-reports/
39. Self-Study Modules on Tuberculosis. Centers for Disease Control and Prevention. Module 6: Contact Investigations for Tuberculosis Reading Material [cited September 25, 2012]. Available from http://www.cdc.gov/tb/education/ssmodules/module6/ss6decision.htm
40. The Effective Management of Cultural Diversity. Gardenswartz L, Rowe A. Part II: Understanding the Role of Cultural Diversity in the Workplace [cited September 25, 2012] Page 37. Available from http://www.sagepub.com/upm-data/26078_pt2.pdf
41. About Us. California Department of Education, Child Development Division. Desired Results for Children and Families [cited September 25, 2012]. Available from http://www.wested.org/desiredresults/training/aboutus.htm
42. Concentric Circles of Knowledge. Kaplan S. Tempo [serial online] Spring 2002 [cited September 25, 2012] Volume XXII Issue 2. Available from http://www.txgifted.org/files/tempo/2002-2.pdf
43. McCabe and Horsley, p 47.
44. Visual Materials and Advertisements. United Way. [cited September 25, 2012]. Available from http://www.unitedwaytoronto.com/campaign/VisualMaterials.php
45. Wilcox C ,Sigelman L, and Cook E. Some like it hot: Individual differences in responses to group feeling thermometers. Public Opinion Quarterly 1989;53:246–257.
46. McCabe and Horsley, p 47.
47. Plested et al.
48. McCabe and Horsley, p 47.
49. McCabe and Horsley, p 18-19.
50. Baydala L, Letendre S, Ruttan L, Worrell S, Fletcher F, Letendre L, Schramm T. “Why do I need to sign it?” Child Assent in School-Based Prevention Research in a First Nation Community. First Peoples Child & Family Review 2011;6(1):99-113.
51. Baydala L, Worrell S, Fletcher F. FASD: A preconception prevention initiative. In Riley EP, Clarren S, Weinberg J, Jonsson E, editors. Fetal Alcohol Spectrum Disorder: Management and Policy Perspectives of FASD. Weinheim, Germany: Wiley-Blackwell; 2011. p. 151-160.
52. Fletcher F, McKennitt D, Baydala L. Community Capacity Building: An Aboriginal Exploratory Case Study. Pimatisiwin: A Journal of Indigenous and Aboriginal Community Health 2008;5(2):9-24.

7. Please describe the project or body of work from which the submitted product developed. Describe the ways that community and academic/institutional expertise contributed to the project. Pay particular attention to demonstrating the quality or rigor of the work:

• For research-related work, describe (if relevant) study aims, design, sample, measurement instruments, and analysis and interpretation. Discuss how you verified the accuracy of your data.
• For education-related work, describe (if relevant) any needs assessment conducted, learning objectives, educational strategies incorporated, and evaluation of learning.
• For other types of work, discuss how the project was developed and reasons for the methodological choices made.

The use of the Target and ThermoScale activities and the development of the Two Circles activity are the culmination of many years of qualitative data collection with children, youth, and seniors in First Nations in Canada. Work began in Alexis (project 2 below) 7 years ago, informing the decision to adapt the Target and ThermoScale at Buffalo Lake Metis Settlement. This subsequently informed the Two Circle activity at Alexis for post-evaluations. Cognizant of the limitations of traditionally designed focus groups, two activities inspired by the Evaluator’s Cookbook28 were implemented and a new activity was developed in collaboration with community members employed as research assistants, educators, or advisors.
Community Partners and Roles
1. Buffalo Lake Metis Settlement: Life Skills Journey
Partner and Project: The community based participatory research project that led to the implementation of the ThermoScale and Target activities began in 2009 with Buffalo Lake Metis Settlement (BLMS) of Alberta, Canada. The first two years were dedicated to building a community–university partnership that would lead to the ground-up development of a community-specific substance abuse and violence prevention program for children. This included exploring community readiness for a life skills and substance abuse prevention program using Community Readiness: A Handbook for Successful Change29. BLMS Council first approved the research project through a Council motion on June 22, 2010, and its continuation through a subsequent motion on April 12, 2011. Using focus groups to consult with a diverse sample of community representatives, the project’s end goal is to address health issues that Metis face by identifying and strengthening local resiliency and protective factors, such as cultural awareness and self efficacy.
Roles: BLMS residents who served on the project advisory committee played a very significant role in defining the key words and providing input into the visuals that would be used in the activities. Advisory committee members also advised on the make-up of the focus groups (ie: elders, youth, community representatives) and recommended participants who would have knowledge while taking into consideration the diversity of the community.
2. Alexis Nakota Sioux Nation (ANSN) Nimi Icinohabi Life Skills Program
Partner and project: In response to substance abuse within their community, ANSN invited University of Alberta researchers to collaborate on the development and implementation of a school-based substance abuse prevention program in 2004. After a literature review of evidence-based programs, the team selected and culturally adapted the Life Skills Training (LST) program30. Renamed Nimi Icinohabi, the culturally adapted program was piloted to determine the feasibility of delivering the full program. Results of the feasibility study were positive and led to full implementation and evaluation of the Nimi Icinohabi program. The focus group activities, including the pilot of the original Two Circles activity, were used for the third year of evaluation in March of 2011.
Roles: Having experienced success in using the tools with BLMS focus groups, we presented the tools to Sherry Letendre, educator and research assistant with the Alexis Nakota Sioux Nation. She advised on revisions to the tools (new images and key words/concepts) and used these with Nimi Icinohabi participants.

Academic Partners and Roles
In both the projects described above, Associate Professors from the University of Alberta (Fletcher and Baydala) and academic research team members were responsible for making sure that Settlement and Nation approval for research was granted, ensuring institutional ethics was attained, providing a framework for program development and adaptation. Academic research team members also facilitated focus groups, recorded activity visuals and comments, and presented analysis and findings back to community for validation and input.

8. Please describe the process of developing the product, including the ways that community and academic/institutional expertise were integrated in the development of this product.

Researchers used two activities to assess the readiness of Buffalo Lake Metis Settlement (BLMS) to participate in a community based participatory research project. One of these activities was also used to evaluate the learning impacts of a life skills program for children in Alexis Nakota Sioux Nation (ANSN). In addition, one original activity was developed for use with ANSN. The research team decided to use visual activities, anticipating that tailoring the directions to each audience should result in more effective data collection. Previous focus groups with children were relatively unsuccessful, possibly due to discomfort and multi-dimensional power dynamics.
Some of the seniors at BLMS spoke Cree as their first language, making it difficult to express abstract concepts in English. Interviews prove difficult as these seniors do not generally consent to voice recordings. A culture of silence can impede discussions on sensitive topics. Activities that allow for a visual record are strategic means for overcoming language difficulties31. Reading comprehension levels also varied within all of the groups and the first concern was to enable everyone to participate. In activities led by a facilitator, those otherwise restricted by their reading level can participate freely. To complement the activities, facilitators prompted participants to describe their responses. McCabe and Horsley’s32 Thermoevaluator and Target activities acted as a starting point, inspiring the Target and TermoScale activities, along with developing one additional activity to suit the communities’ needs. In both cases, ANSN and BLMS members’ input was crucial to the content and design of the activities.
Focus group themes were based on the community readiness handbook prepared by the Tri-Ethnic Center for Prevention Research33. This model was chosen for its focus on culturally-specific programming. Content for these activities was derived from previous experience at ANSN, with advisement from community members at BLMS.

Target Activity
This activity, used at BLMS, allowed participants to prioritize issues of concern in their community by placing pre-written notes on a target made of concentric circles. The research team discussed a list of issues with community partners to ensure their relevance. The initial list of issues (including alcohol and drug use, smoking, decision making, and dealing with anger) was adopted from life skills training and substance abuse prevention programs34. The list was revised as focus groups progressed to include new issues added by participants (gambling, grief, gender roles).
The secondary goal for this tool was to determine what community members felt was an appropriate age for intervention on each issue. This resulted in critical data for future program development, as participants often suggested intervention ages younger than facilitators anticipated. We specified that we were looking for the earliest age for intervention rather than the age when the knowledge would be applied.
When compared to traditional focus group questioning, these tools provided a much more comfortable atmosphere for feedback from community members.

Two Circles Activity
With ANSN, this activity was used to encourage children to share what they had learned about their culture and favourite topics. A list of teachings was provided by the program facilitator while developing the tool. Children placed the teachings they recalled learning inside a large circle in the form of a medicine wheel. They then moved what they most enjoyed learning about to a smaller circle, to indicate their desire to learn more about that topic.
This activity was an original creation of the research team. The academic team considered the basic question we wanted to ask the students; in our experience, yes/no questions have elicited the most responses from children. The original concept was a large circle and a small circle. However, the community facilitator asked that the circle be a medicine wheel. This alteration enhanced the adaptive potential of the two blank circles - any type of representative image might be placed in the circles to give the activity a familiar visual design for participants.

ThermoScale Activity
With BLMS, community partners approved a list of potential assets for use in the activity. This list was adapted from the Tri-Ethnic Center for Prevention Research publication entitled Community readiness: a handbook for successful change35, which provides a series of questions to illustrate the character of a community. These questions were adapted into singular concepts, or groups of concepts. For example, instead of asking, “Do efforts that address this issue have a broad base of volunteers?”36, the research team offered “volunteers, mentors, and role models” as potential community assets to consider. In this process, complex questions requiring greater comprehension skills were distilled into simple concepts for participants to discuss in their own terms. After placing each asset, participants were encouraged to add assets that facilitators had not included, further revealing facets of their community.
At ANSN, children in grades 5 and 9 talked about their favourite lessons. The program facilitator provided the artwork from the life skills manual in the hope that the visual images would trigger students’ memories. The Stoney language titles of the modules were included along with the English language titles and students were encouraged to teach facilitators the proper pronunciation of the Stoney words.

9. Please discuss the significance and impact of your product. In your response, discuss ways your product has added to existing knowledge and benefited the community; ways others may have utilized your product; and any relevant evaluation data about impact, if available. If the impact of the product is not yet known, discuss its potential significance.

In 2011, we presented these tools in a poster at the National Outreach Scholarship Conference. We received requests from attendees to share the tools for use with a variety of stakeholders, including work with rural communities on disaster preparedness.

Target Activity

The original tool in The Evaluator’s Cookbook37 uses 5 concentric circles to gather quantitative data. Target, or bullseye, visuals are used in evaluation with Aboriginal people as they reference circle theories38 and in epidemiology to measure risk of exposure to infection39. They are used to indicate degree of influence through the relationship of concepts to an individual (central circle) as opposed to the family or community (outer circles)40,41 or to reference a standard (inner circle) as compared to alternate ways of categorizing knowledge (outer circles)42. We simplified the 5 circle formula to 3 circles. We asked participants to identify life skills topics aimed at children and youth as critical, serious, or important.

The Target allowed participants to identify pertinent issues. The imagery was understood by all community members, youth and seniors alike; participants used the circles to compare the importance of topics. New topics were added by community members at BLMS, supporting the overall accessibility of the tool. The results of this focus group process allowed for strategic decisions about our research aims, including topics for life skills program development at BLMS, but more specifically the ages most in need. In asking a second question around age, we discovered a need to target a younger audience across all of the suggested topics, including drug prevention.

ThermoScale Activity

This tool, from the The Evaluator’s Cookbook43, is structured as a thermometer. The thermometer image is used to measure success/benchmarks in marketing campaigns (ie: the United Way)44 as well as in program evaluation45. The hot end of the thermometer represents positive or prominent while the cold end represents negative or absent. The ThermoScale organizes concepts on a fluid, relational scale, offering an immediate representation of gradations. This activity acted as an icebreaker as the subject matter was not as sensitive as in other activities.

In The Evaluator’s Cookbook, this tool was used to gain feedback from children on their school lessons46. For the post-evaluation at ANSN, children used the ThermoScale to discuss their life skills program preferences. In both settings, facilitators prompted discussions about how topics related to one another on the ThermoScale.



At BLMS, researchers pared down complex community needs assessment questions47 into singular concepts; participants then used the ThermoScale to discuss community strengths and weaknesses. These activities expand significantly on the tool’s initial uses48. The cultural understanding of Metis Settlements that arose from this activity was critical for researchers, leading to the rejection of a pan-Aboriginal approach in all aspects of our research. Metis Settlements are unique within the cultural concept of Metis, and each Settlement may be unique from the other Settlements.

Two Circles Activity

In this activity, children answered “yes or no” questions. They were asked two questions - an initial query and a follow-up.

This activity provided feedback for the life skills program. The facilitator provided Stoney concepts to explore the impact of cultural teachings. Placing concepts in the large circle indicated that participants had internalized the concept. Children then moved notes from the first circle to the smaller circle if they wanted to learn more about it, revealing the students’ areas of interest for future program planning. The large circle appeared as a medicine wheel to integrate a cultural motif. This adaptation could be suited to any cultural group. Elsewhere, this activity was adapted for a Plains Cree group using the life skills program logo instead of the medicine wheel.

10. Please describe why you chose the presentation format you did.

These tools are in PDF so they can be easily viewed and printed to a variety of sizes. A need was identified during a presentation of these activities in October 2011 at the National Outreach Scholarship Conference.
We used these activities because the standard focus group format was inappropriate for the community in which we conducted our needs assessment and did not yield quality information. In contrast, activities allowed facilitators to foster engaged participation that was lacking in previous focus groups and evaluations.
The Target and ThermoScale were chosen as both tools allow for a variety of information to be sorted and ranked according to our requirements. These tools answer questions that would be difficult to address through discussion alone. Our focus groups addressed alcohol and drug attitudes, perceptions of violence, community shortcomings, and youth delinquency - issues that many are uncomfortable discussing. The tools’ colourful, playful graphics keep topics from becoming personal. Both concepts were readily understood by all participants, including young children and seniors whose first language is not English.
The Two Circles activity was conceived as a means for discussing cultural knowledge acquired by school-aged children. It can be used to answer any two related yes-or-no questions, in a successive order. The simple format allows for wide variations in use.
For each tool, we used post-it notes for movement and adaptability. With simple supplies, the potential flexibility of these activities is limitless.

11. Please reflect on the strengths and limitations of your product. In what ways did community and academic/institutional collaborators provide feedback and how was such feedback used? Include relevant evaluation data about strengths and limitations if available.

Strengths

Overall, these three tools are adaptable for many types of content. The ThermoScale and the Target are useful for ranking information on a relational scale. The Two Circles activity allows for basic yes or no questions. All of these activities rely on visual interaction that feels simpler and less invasive to most participants than traditional, question-based focus group techniques. During the activities, participants willingly responded to relevant questions and explained their choices without reservation.

Target
• We were able to address two questions by layering smaller post-its over the larger ones or by writing on the original notes
• The design is a familiar concept to most people; the closer the issue is placed to the centre of the target, the more important it is to participants
• Can be used for program evaluation, comparing different areas within a program, intervention, or organization
• Useful for ranking information into separate categories
• The target as a visual is immediately accessible for a variety of stakeholders

ThermoScale
• Highly adaptable
• Allows for easy visualization of normally abstract concepts and how they relate to one another
• Converts well to quantitative data when marks are identified to break post-it notes into percentiles
• The thermometer visual was familiar to the entire age range of participants
• This tool allows participants to express their perceptions about a topic
• Participants can identify perceived weaknesses themselves rather than an outsider (facilitator) doing so

Two Circles
• Allowed us to address two questions, by moving post-it notes first to the larger circle and then to the smaller circle
• Contains inherent potential for a huge variety of design possibilities. Does not need to be a medicine wheel, could be any relevant graphic within the circles


Limitations

The main limitation of all these tools is their inability to address complex and nuanced questions. They were designed to achieve basic answers for basic questions, with only one or at most two questions for each activity. Often, they served to open up a larger discussion that would otherwise be difficult for a facilitator to prompt, making audio recording a major advantage. These activities are not intended to replace traditional, long-form focus group questions in situations where that format is applicable.

Target
• In order to quantify in a meaningful way, more rings would be needed, which adds complexity. The original Target from The Evaluator’s Cookbook49 had a more complex design for this purpose.
• In some cases, participants placed every post-it note in the centre-most ring, indicating that every issue was of critical importance to them. In our experience, this type of statement is still valuable but other types of content could conceivably be limited by this kind of participant input. One way to address this would be to create greater distinction and disparity between the meanings of each concentric circle, encouraging participants to think about relative differences between issues.
ThermoScale
• It was suggested to us that the thermometer visual may not be familiar to all cultural groups. We did not run into this issue, but it could be overcome through review and validation by community partners.

Two Circles
• A limitation of this tool is its reliance on yes or no questions. The tool is most effective when used to address a question that can only be answered with a positive or negative. The second, smaller circle presents the opportunity to further qualify that simple question. Our main purpose for this tool was to promote recollection and discussion. The simplicity of the tool served the participants (children) well and acted as an icebreaker for more complex discussions. This tool is not designed to produce critical reflection on sophisticated questions but to facilitate discussion on sensitive topics.

12. Please describe ways that the project resulting in the product involved collaboration that embodied principles of mutual respect, shared work and shared credit. If different, describe ways that the product itself involved collaboration that embodied principles of mutual respect, shared work and shared credit. Have all collaborators on the product been notified of and approved submission of the product to CES4Health.info? If not, why not? Please indicate whether the project resulting in the product was approved by an Institutional Review Board (IRB) and/or community-based review mechanism, if applicable, and provide the name(s) of the IRB/mechanism.

These products were motivated by a desire to find ways to encourage discussion with youth and elders. For these groups, power dynamics and literacy were obstacles to effective data collection; community based participatory research (CBPR) principles are foundational to our process and relationships with community partners; the principles of equity and collaboration drive our research questions and methods. The mandate of this research team is to conduct ethically grounded research. The goal of these particular research projects is to create culturally supported life skills programs for children and youth in partnership with communities. These programs are intended to increase drug and alcohol awareness and refusal skills, build identity and self-esteem, and teach skills to cope with anger and anxiety. As we work with Aboriginal communities, culturally distinct from one another as well as the academic researchers, it is imperative that we foster an environment of trust, exchange, and co-learning so that all parties may grow in understanding.
The founding members of this research team have been conducting CBPR with the Alexis Nakota Sioux Nation (ANSN) since 2004. As such they have established a partnership based on mutual respect and shared knowledge. This collaborative team has produced several peer-reviewed publications50,51,52, and presented papers and posters at multiple conferences. This long-term project with ANSN, as well as another with the four Maskwacis Cree Nations (Samson, Louis Bull, Ermineskin, and Montana), laid the foundation for a new project with Buffalo Lake Metis Settlement (BLMS). The activities under discussion were used to conduct a community readiness assessment with BLMS, but the research team was able to begin partnership building months before focus groups were conducted. In this instance, we felt confident in our capacity to consult with community partners in these focus group methods and to incorporate their input.
The idea to use non-traditional focus group methods to increase participant engagement came about as a result of experience conducting structured, interview-based focus groups with First Nations children and adults. Traditional methods do not tend to yield either satisfactory participation or useful information. The idea of focus group activities instead of traditional, structured questioning was met with enthusiasm by all of our community partners, including the life skills facilitator at ANSN. The content and conceptual design was reviewed and approved by community members throughout the creation process. At different stages, community partners contributed content, artwork, design suggestions, and feedback for improvement after facilitation.
These research projects were approved through two of the University of Alberta Research Ethics Boards. One board approved the life skills program pilot at Alexis Nakota Sioux Nation and one board approved the community needs/readiness assessment at BLMS.