Appendix VII

In Practice: How to Think Like NAKFI

What is clear from participant feedback is that the National Academies Keck Futures Initiative (NAKFI) worked because it engaged participants and provided venture science funding. Although the conference location helped create an amazing experience, NAKFI was a state of mind; a state of being. So it was not surprising that people reported wanting more—a chance to continue the magic.

The purpose of this appendix is to provide guidance on “How to think like NAKFI.” It provides information on the Futures Conference Model and how to apply it.

FUTURES MODEL

When NAKFI launched it had a vision and specific goals: to support interdisciplinary collaborative research (IDR) and create systemic change to overcome barriers to interdisciplinary and collaborative research. The Futures Initiative invested time and energy early on to understand what interdisciplinary collaborative research looked like in 2003 and how it could be improved. Early on, efforts were made to identify a common language to describe IDR, which resulted in working definitions for the program that were used to guide the development and evolution of the Futures Model. Today, individuals, institutions, and funders similarly inclined to advance IDR may want to adapt these approaches to their situation. The benefits to applying these approaches are numerous. Beyond the learning and growth opportunities for individuals, institutions can benefit from their role as

leaders in science and IDR. Additionally, the scientific research gains and advancement in understanding of challenges and opportunities facing the world will benefit the wider human community.

The Futures Model that developed and evolved over 15 years had four aspects: Invite, Prepare, Convene and Seed, and Adapt and Evolve. The aspects are described in this section. The next section provides tips on using the Futures Model and the last section offers tools to help implement the model.

Invite

Activities for this aspect included identifying topics or themes; assembling an oversight body to develop key questions for the topic; overseeing the call for conference applicants and selecting attendees; guiding conference planning, including format and preparatory materials; and selecting the grant recipients. Inviting and selecting participants can be a challenge given the need for participants to be qualified experts appropriate to the topic AND to have an interest and willingness to engage in a collaborative, interdisciplinary style conference.

Prepare

Preparation of conference participants was critical to the success of the conference, both for the attendees’ experience and for teeing them up for the grant application process.

Convene and Seed

This aspect involved all of the planning and implementation of the conference event, as well as the Futures grant process.

Adapt and Evolve

Data gathering and evaluation were integral to NAKFI’s ability to adapt and evolve. Input and feedback were sought from everyone involved, including committee and oversight bodies, conference participants, and grant recipients, as well as NAKFI alumni. Surveys and ideas were reviewed to inform the topic selection, conference and grant application and selection processes, conference logistics, and planning for the event and the attendee

experience. The evaluation strategies developed and deployed informed the improvement of the entire program. Individuals, institutions, and funders interested in the Futures Model will want to develop an appropriate evaluation strategy for their efforts. In addition to supporting continuous improvement, the results can be used to explain and support the value proposition associated with implementing a Futures-type model. Leveraging what is learned can support promotional efforts and future activities.

FUTURES CONFERENCE MODEL AND TOOLS

Successful collaborative interdisciplinary science requires getting the right people to engage in the right process. The goal of the Futures Conference Model is to create a venue that allows scientists and non-scientists to cross disciplinary boundaries and have deep conversations about real-world challenges. Before deploying the conference model it is necessary to consider the overall program’s goals. For individuals and institutions who want to create interdisciplinary, collaborative science or experiences for themselves, their teams, students, and institutions, there will be a benefit to working through the planning tool that reflects key lessons gleaned over 15 years, as well as feedback from steering committees and participants.

Over the course of 15 years, a number of tools were developed to support the Futures Model, including assessments to help applicants, attendees, and conference organizers to identify potential gaps in preparedness for an interdisciplinary collaborative experience. Tools provided include:

• Planning Tool for Conference Organizer(s)
• Applying the Futures Conference Model
• Collaborative Readiness Assessment
• Research Orientation Assessment
• Transdisciplinary Orientation Assessment
• Mentoring Guidelines
• Daily Schedule/Expectations of Mentors
• Conference Agenda Example

The assessment tools were developed for NAKFI in consultation with Dan Stokols and based on the work of Stokols and his colleagues (Mâsse et al., 2008). In addition to the tools described below, publications that are relevant to team science are included as part of the References section of this publication.

Planning Tool for Conference Organizer(s)

If you have made it this far into this publication, you:

• clearly have an interest in applying the Futures Conference Model, and
• understand that the model supports interdisciplinary, collaborative science and that inclusive conversation, pluralism of perspective, and the uniting of art and science support success.

This Planning Tool is designed to help focus your efforts. The expected output would be to use the summary of responses to draft a detailed explanation of why, what, who, challenges and solutions, and measures of success that could guide the development of a program and plan for the implementation of a NAKFI-style experience.

Why do you want to do this?

• What problem are you trying to solve?
• What question are you hoping to answer?
• What do you hope to accomplish?
• Why do you want to use the Futures Conference Model?
• How do you envision the Futures Conference Model helping you?

How do you know this is the right problem or question?

• What do you already know and how do you know it?
• What else do you need to know to confirm it is the right problem or question? Where will you get that information? Who can help you?
• How did you or will you confirm this is the right question?

What is the desired outcome or impact?

• If your efforts are successful what will be different?
• Who do you expect to benefit and how?
• How will you measure success?
• What is the value proposition for you, your team, department, organization, funder, and other researchers or participants?

• What experience are you hoping to create?
• What is the look and feel you hope to achieve for participants?

What is the value proposition for doing this? What are the potential benefits, gains, outputs, and outcomes for you, your team, students, and institution? Will a NAKFI experience:

• Advance the work and projects of your team, lab, or department?
• Contribute to your discipline and professional society’s goals and status?
• Enhance your professional network (professional development)?
• Serve as a teaching tool to benefit students (educational)?
• Garner support from stakeholders—internal and external to the institution, including potential supporters/funders?
• Benefit the institution—offer an opportunity for a shared experience within the institution and create process improvement opportunities?

Who do you envision being involved?

• Who are the right people to participate? How will you find them?
• What is the role of your team, department, organization, researchers and creative practitioners from other organizations, and other stakeholders?
• Who else needs to be involved to support implementation?
• How will you explain what is in it for them and garner their support?

Does your team have what it takes to NAKFI? Creating the experience requires:

• Experts relevant to the purpose and goals
• Inclination toward interdisciplinary, e.g., crossing disciplinary boundaries
• Openness to a plurality of ideas
• Willingness to engage in creative exploration
• Familiarity with cross-cutting issues and terminology

Is your team ready? Do a reality check.

• Readiness—assess self, team, and institution’s level of interdisciplinary, collaborative activities to identify areas of focus to support planning and implementation.
• Use self-assessment tools to determine openness to and experience with interdisciplinarity and collaboration.
• Model and challenge team to engage in collaborative behavior and self-reflection.
• Engage team in creative disruption by inviting them to challenge assumptions and incorporate thinking and activities from other disciplines and outside their comfort zone.
• Launch activities to support the above, e.g., create a graffiti wall to share information and capture questions and ideas; incorporate ice breakers into routine meetings to support creativity and engagement; ask non-science departments for ideas and help how to be more creative.

What are the challenges?

• What are the challenges or barriers to success?
• Why is it important to address them?
• How might they be addressed?

Applying the Futures Conference Model

The goal is to host a think-tank style conference that supports idea incubation through boundary crossing, creative thinking, and diverse perspectives. The format, agenda, and management of the experience are important to supporting growth and the development of solutions.

Articulate the Dream and Rally the Troops

• What are the goal(s), purpose, expected outputs/outcomes, and measures of success for the conference?
• Look to a variety of sources and get input from experts to identify possible topics that are timely, pressing, real-world challenges, and relevant to the team.

• Tap into institutional and professional networks for ideas and gather insight from an interdisciplinary and cross-professional group of individuals.
• Select the topic based on where the intersection of science, engineering, medicine, and the arts and humanities will have the greatest potential for innovation. Focus on a single, overarching theme.
• Create hypothesis-driven interdisciplinary research team challenges related to the broader conference theme. Once participants have been selected and invited, ask them to help refine or identify challenge ideas and questions.
• Who can help with planning and finding potential participants? What are the criteria for inclusion, e.g., what types of experts are needed in the room? The ability to promote and support interactivity by attendees is tied to the overall size of the event.
• Draft a statement explaining what is to be accomplished and why, who could benefit and who should be involved, what would be different if successful, and how those involved will know.
• Create a proposal that communicates enthusiasm and can be used to garner stakeholder support and participant interest.
• Determine necessary stakeholder support and organize a team. What resources are available to support the successful launch of event? What is the value proposition to stakeholders or resources gatekeepers? Who can help secure resources, funding, or buy-in?

Attend to Resource and Logistical Details

• Are the necessary resources available, e.g., time, skills, knowledge needed to plan and implement, facility, and funding?
• How will the costs for invited participants be handled? Can funding be obtained to cover all of the costs and thereby create a micro-sabbatical to support an experience where risk taking and ingenuity are encouraged?

Identify, Select, and Engage Participants

• Create an application and selection process that includes assessments of collaborative readiness and inclination toward interdisciplinarity. Cast a wide net for applicants including team, lab, de-

• partment, students, institution, professional society, community, and agencies. Consider including non-STEMM (science, technology, engineering, mathematics, and medicine) scientists from behavioral or social sciences and professionals such as science writers, artists, designers, and members of the public.
• What is the plan for promoting the conference to garner interest?
• What is the WIIFM (“What’s in it for me”) for applicants/invitees?
• How will attendees prepare to ensure their complete immersion in the conference experience? What will be done to equip them with a common language to engage in deep conversations, transcend boundaries, and build bridges? Examples of pre-conference tutorials or preparatory activities include webcast tutorials, podcasts and recorded presentations, suggested readings, and activity challenges where invitees seek out an experience relevant to the challenge topic prior to the conference. These activities can be general or specific to the challenge topic and can address the concepts of interdisciplinary, collaboration, and teamwork (for examples, see NAKFI YouTube Channel, https://www.youtube.com/channel/UCSJX5NYP9UsgH8a-gwRDapg).
• Place is important. A comfortable and beautiful setting is conducive to thought-provoking interactions. What physical space is available that is not just functional but also inspiring?
• Will attendees pre-select or be assigned to a specific group? Group size needs to support interactivity and collaboration but the process should be flexible enough to allow individuals to leave a group to join another or break off to form a new group.
• The agenda should include daily opportunities for full conference sessions and small group sessions to support cross pollination among groups and informal conversations among participants. Progress updates midway through the conference and final presentations by the groups support sharing, exchange, and feedback on ideas.

Let the Magic Happen

• Breakout groups can be charged to delve into one challenge and more than one group may tackle the same challenge.

• Groups should be encouraged to explore the challenge from multiple perspectives rather than trying to solve the problem with existing approaches.
• Direction to groups should include permission to be creative and generate wild ideas.
• Does the schedule allow for small group reporting midway through the conference and time for participants to engage in informal conversations, networking, and “creative engagements” that can activate unexpected ideas?
• Facilitation, mentoring, and coaching may be needed for some groups. See the Tools section for Mentor Guidelines.
• Face time with individuals outside of one’s group is critical for informal conversations and cross pollination. There are multiple ways to drive attendee interaction. Traditional and familiar activities might include poster or panel sessions. Less conventional and unexpected activities might include social gatherings that involve different out-of-the-box experiences or exposure to the creative and performing arts. In addition, travel to and from the venue provides an opportunity for informal conversations that can spark intellectual coordination.
• Providing a mechanism for attendees to connect after the conference allows for ideas to flourish and develop and supports continued interaction and collaboration. What can be done to support continued participant-to-participant engagement, e.g., inviting another to speak on campus, working on a grant proposal, sharing unpublished research findings, conducting email conversations, or collaborating on publications?

Collaborative Readiness Assessment

This series of questions served as tool for applicants to pause and reflect on their interest in participating in NAKFI’s unique conference format.

Interpersonal Collaboration Scale

Please indicate how strongly you agree or disagree with each of the following statements: Scale: Strongly disagree, somewhat disagree, neither agree nor disagree, somewhat agree, strongly agree.

1. I tend to be more productive working on my own research projects than working as a member of a collaborative team.
2. There is so much work to be done within my field that I feel it is important to focus my research efforts with others in my own discipline.
3. The research questions I am often interested in generally do not warrant collaboration from other disciplines.
4. While working on a research project within my discipline, I sometimes feel it is important to seek the perspective of other disciplines when trying to answer particular parts of my research question.
5. Although I rely primarily on knowledge from my primary field of interest, I usually work interactively with colleagues from other disciplines to address a research problem.
6. I believe that the benefits of collaboration among scientists from different disciplines usually outweigh the inconveniences and costs of such work.
7. In my collaborations with others I integrate research methods from different disciplines.
8. In my own work, I typically incorporate perspectives from disciplinary orientations that are different from my own.
9. In my collaborations with others I integrate theories and models from other disciplines.

Research Orientation Assessment

Research Orientation Scale

Please assess the frequency with which you typically engage in each of the activities listed below using the following 7-point scale (Never, Rarely, Once a year, Twice a year, Quarterly, Monthly, Weekly)

1. Read journals or publications outside of your primary field.
2. Attend meetings or conferences outside of your primary field.
3. Participate in working groups or committees with the intent to integrate ideas with other participants.
4. Obtain new insights into your own work through discussion with colleagues who come from different fields or disciplinary orientations.

5. Modify your own work or research agenda as a result of discussions with colleagues who come from different fields or disciplinary orientations.
6. Establish links with colleagues from different fields or disciplinary orientations that have led to or may lead to future collaborative work.

Transdisciplinary Orientation Assessment

Transdisciplinary Orientation Scale

The following items pertain to your thoughts, expectations, and behaviors about your research to date. Please indicate how strongly you agree with each of the following statements.

1. My work to date reflects my openness to diverse disciplinary perspectives when analyzing particular problems.
2. My work to date reflects my interest in learning about disciplinary concepts and theories in addition to the ones I am most familiar with.
3. My work to date reflects my interest in learning about new research methods that are different from the ones I am most familiar with.
4. I would describe myself as someone who values interdisciplinary collaboration.
5. I am willing to invest the time required for learning about fields that are different from my own.
6. I enjoy tackling the challenges posed by working on complex problems, even if doing so requires me to expend extra time and effort.
7. I generally approach problems from a multilevel perspective that encompasses both micro and macro level factors.
8. My projects to date reflect my ability to conceptualize complex problems by identifying various situations specific factors that account for those problems.
9. My work to date reflects my ability to create conceptual frameworks that bridge multiple fields.

10. My work to date reflects my ability to think broadly about complex problems. In my own work, I incorporate perspectives from fields that are different from my own.
11. In my own work, I use methods drawn from more than one discipline rather than relying exclusively on a single disciplinary approach.

Mentoring Guidelines

Characteristics of Ideas

Futures Conferences help participants make potentially significant contributions to issues and questions that are personally and professional important to them. The interdisciplinary nature of the teams leads to energetic conversations and innovative ideas. When remaining in a state of ambiguity becomes uncomfortable, mentors can provide encouragement to resist jumping to a conclusion too early and offer suggestions for getting unstuck.

Giving Feedback to Groups

Group practices that should be DISCOURAGED:

• Playing it safe—The NAKFI idea development process is about asking participants to dream of innovative solutions that could have a big impact in the world. Encourage participants to think “outside the box” and dream big.
• Repeating an existing idea—Innovation is a key part of this process. If you feel that a participant team is not thinking far enough beyond existing ideas in the world, push them further.
• Trying to stick as closely as possible to the seed idea—Seed ideas are meant to grow and develop over time into ideas that the participant team members have developed on their own. They are not “assignments” to be followed; encourage participants to be inspired by seed ideas, but to take them further or into different directions. Final idea pitches that have no relation to the initial seed idea are OK, as long as the teams that developed them are passionate about and committed to those ideas!

Group practices that should be ENCOURAGED:

• “Blue-Sky” ideas—BIG ideas, even if they are way outside the box or could take a long time to realize, are the aim of this process. Encourage participants to really stretch their dreams for the world until they come up with a concept that has the potential for broad-scale impact.
• Cross-disciplinary thinking—Participants have a unique opportunity this week to create with team members from different disciplinary backgrounds. Encourage them to think about how language, techniques, tools, philosophies, etc. from different disciplines might impact their developing ideas.
• Following personal passion—Encourage participants to take feedback into consideration, but to ultimately make their own choices about their idea and how it develops. Participants’ passion for an idea is ultimately the most important thing, even beyond your own belief in the idea or your ability to help the team move that specific idea forward.

The Participant Experience

Participants in each team are dealing with a lot. Not only have they just met each other for the first time, they are also responding to tight deadlines, trying to build group consensus, and getting lots of feedback on their developing team idea within a short period of time. This can at times be a frustrating process. A few things that you can keep in mind as a mentor to support participants in this process include:

• Pitching an idea you care about is hard—Remember that when giving feedback to a group, constructive feedback (feedback that helps teams to move forward, is specific, and points out both pros and cons of an idea) is much more helpful than purely critical feedback.
• Participants are working toward daily benchmarks—Keep an eye on what participants are expected to prepare and hand in each day, and help them stay on track with meeting those benchmarks.
• Participants are likely to hear conflicting feedback—While you might give feedback to a team that you think is right-on, that same team could hear different or conflicting advice from other

• advisers during this process. As a mentor, you can help teams to make decisions about which advice to take, and which to table for consideration later on.
• Every voice is important—If you feel that there are any dominant voices in a particular team, encourage all members of the group to give opinions (without calling out specific team members who might be shy or not participating fully for other reasons).
• Teamwork is key—Often times, group disagreements or interpersonal conflicts arise during the Idea Translation process. You can help teams by encouraging them to build consensus, or by helping them to weigh various options that are up for consideration among the team members.

The Mentor Experience

In addition to helping participants navigate group dynamics issues and connecting them to resources, one of your biggest responsibilities as a mentor is to give feedback to teams about their developing ideas. As you do this, commit yourself to being open to solutions outside of your own discipline. Chances are that even if participants began with a seed idea with which you have great familiarity, at some point the idea will move into an area that is less familiar to you. Let go of any subconscious desire to steer participants toward an idea that is in your own discipline, or an idea about which you feel comfortable advising the team. The most important thing is that participants develop an idea that is innovative, personally-important to the team members, and has the potential for big impact.

Participants should take notes during idea feedback sessions.

When giving feedback to groups, in addition to intuitively advising on the merits of the idea and where to go next, mentors often ask participants about:

• Scale of the idea—Do they perceive the idea being used on a local level? On a national or global level? If developing a product or a service, do they perceive it being used by individual users? By communities or towns? By entire countries?
• User experience—What considerations relate to the end-user’s experience for this idea?

• Product/service/experience—If a group is considering developing a product, is this the right application? Might it be better suited toward a type of service or experience instead?
• Cost-effectiveness—Does it seem that the idea could be done in a cost-effective way? If developing an idea for users in resource-constrained environments, does the approach to the idea take into account these constrained resources (time, money, geographic location, etc.)?
• Innovation—Is the group actively seeking out an idea that makes a significant departure from what’s been done before?
• Ethical considerations—Are there any ethical considerations that the group might not have considered yet?
• Clarity of the idea—Is the essence of the idea coming across? Can a first-time listener easily understand what’s exciting, innovative, and compelling about the idea? Does the potential impact of the idea come across clearly?
• Alternate applications—Are there other potential applications for the idea that the group might not have considered yet?
• Dreaming BIG—Is the team thinking big enough, or going after “low-hanging fruit?”
• Passion—Are team members passionate about the idea? If not, you can usually tell, and it’s a red flag that the team needs to go back to the drawing board.

Common idea challenges that mentors can help address include:

If you see a major flaw in the approach, design, or rationale behind a project idea, you can help by:

• Pointing this out and suggesting ways to alter the approach of the project to better address the problem it is trying to solve.
• Suggesting an alternate problem that would be better served by the idea that the team is developing.

If you know of a project that is very similar to/the same as the idea that participants are pitching, you can help by:

• Suggesting alternate applications for the team’s idea.
• Explaining why the currently-realized innovation hasn’t completely solved the issue it set out to tackle (perhaps it’s too expensive, too slow, not distributed widely enough, etc.). In these cases, it can be good to remind participants that if a perfect solution had already been developed then the problem or opportunity they’re seeking to address probably wouldn’t exist anymore. Many innovations in today’s world have been built on ideas previously developed by others, and whole companies have been created that are focused on doing something cheaper, faster, more efficiently, in a more ethical way, etc. than other companies that have done similar work before.

If you feel that a team’s idea may take many decades—or even lifetimes!—to realize, you can help by:

• Guiding the team to hone in on some first steps that could be achieved in the nearer term—perhaps the next several months, or a year, 3 years, 5 years, etc. Always remind participants that you’re asking them to dream big, and big ideas can take a long time to develop! If they were to find an idea that could be realized in just a few months or a year, they might not be dreaming big enough to make a significant impact on the world, so a far-reaching idea is actually a good thing.
• As participants move toward the final pitch day, you can help teams to drill down to next steps and begin to think about benchmarks that could be achieved within the timeframe of a NAKFI grant, even if participants have a big idea that would take many years or decades to fully realize.

If you feel that participants will not be able to move forward with their idea until they have acquired much more in-depth knowledge, you can help by:

• Suggesting guest experts, institutions, organizations, companies, universities, or research resources that participants can investigate in order to garner some expert knowledge that might not be in your domain.

• If you have a resource or connection that would be helpful to participants, try to make that connection happen!

Daily Schedule/Expectations of Mentors

As a mentor, it can be helpful to understand what is expected of groups each day. While these are only guidelines for groups and the process of developing an idea is not an exact science, you can help groups to keep the end goal in mind by being aware of the benchmarks that will come up this week:

Day 1

During group work time, team members might discuss:

• Key background: Why the particular situation exists
• Need/opportunity: Why the situation presents a problem or creates a need or opportunity, and for whom
• Previous solutions: Why no one has addressed this need or solved this problem before/existing solutions
• Summary of idea: Why group participants personally want to invest more time thinking about, and formulating questions around the problem—or a different problem
• Questions: Attendees develop list of remaining questions, stumbling blocks, etc., for which research and/or group expertise was not available.

Deliverables

1. One-page idea summary—Teams work on/submit one-page idea summaries/first testable hypothesis, and email summaries to assigned mentor prior to retiring for the evening. Summaries to include:
   1. Summary of idea
   2. First testable hypothesis/es: what could be done in a short timeframe with limited seed funding?
   3. Big questions/confusions/stumbling blocks
2. One-minute idea pitch—On Friday morning, each team will provide a one-minute pitch with highlights from their one-page summary

Day 2

During group work time, team members should develop a clear idea that they can really call “their own” (potentially a departure from the starting seed idea). They should also further elements from the submitted one-page proposal.

Deliverable

1. Draft 10-minute idea pitch

Day 3

During group work time, mentors will provide feedback on team idea pitches. This is the final group work time before submitting final pitches to NAKFI staff.

Deliverable

1. Final pitches submitted to registration desk by 12 p.m. (if using a PowerPoint, Keynote, etc.)

Conference Agenda Example from 2016 Futures Conference