Carey Tisdal, Director, Tisdal Consulting

Literature reviews provide useful information for both evaluation and research studies. Yet, frameworks for standard research literature reviews have a poor fit to the practical design problems and ongoing stages of evaluation across the life of an informal learning project evaluation. In this workshop, participants will use a conceptual framework specifically focused on evaluation, increase awareness of accessible sources, practice the process of identifying important concepts to explore, define a search, develop search strategies to identify key references, and consider organizational structure to present their synthesis.

Saul Rockman, President, Rockman Et Al
Jennifer Borland, President, Rockman Et Al
Monnette Fung, President, Rockman Et Al

In this lively and interactive workshop, participants will be introduced to a wide variety of methods for studying web-based visitors in various settings and virtual venues and will discuss unique implications for studying visitors online, as opposed to in-person. The session would be appropriate for museum professionals who do not have extensive evaluation experience as well as those who are more familiar with evaluation practices that might be looking for new ideas or a quick refresher course on opportunities that exist for studying visitors online.

Amy Germuth, President, EvalWorks, LLC.

Increasingly, agencies are being asked to collect information about their activities and results as well as those of projects and agencies they fund. Because these data often are used to make crucial budgeting and planning decisions, the need for high-quality data is critical. This course will introduce participants to the basics of developing data collection instruments that provide measurable evidence of intended outcomes. This course will cover the following topics, with an emphasis on surveying and survey data: types of data (quantitative and qualitative), types of data collection instruments, question development, instrument development, sampling, analysis, and reporting.

Dr. James Kisiel, Associate Professor, California State University, Long Beach,
Dr. Shawn Rowe, Assistant Professor and Learning Specialist, Oregon State University

More and more, researchers and evaluators are turning to naturalistic or qualitative approaches to examine and assess learning in informal learning environments. Video recording is often pointed to as an effective way of documenting just what visitors are doing—yet video recording is also met with concerns and even trepidation. For this day-long workshop, participants will take a closer look at the use of video recording as a method for gathering and analyzing visitor data. This workshop will discuss rationale for using video, as well as the challenges and benefits the resulting data. Participants will also have an opportunity to shoot video and begin some basic analysis of that video as a way to familiarize participants with this approach.

Jennifer Heim, Senior Research & Evaluation Associate, Saint Louis Science Center
Elisa Israel, Research & Evaluation Manager, Saint Louis Science Center

How can museums know what impacts educational programs have on their audiences? Can systems be developed to assess the impacts of multiple, ongoing programs? These questions are crucial to museum evaluators as the informal learning institutions they serve seek to secure funding, inform stakeholders, make decisions, and improve program experiences for their audiences. Through a combination of small-group activities and guided discussion, participants will engage in the process of developing an institution-wide approach to measuring the impact of educational programs. Participants will explore key issues in developing a comprehensive system for assessing impact, as well as gain hands-on experience in creating instruments and procedures to better capture, analyze, and present the experiences of participants in museum programs. Workshop leaders Jenny Heim and Elisa Israel, evaluators at the Saint Louis Science Center, bring their experience of developing and implementing the Science Center’s system for assessing the impact of its educational programs.

Elizabeth Wood

This full-day workshop provides an introduction and overview of using focus groups as an audience-generated inquiry technique. Using hands-on and interactive strategies, participants will review basic concepts and theoretical principles behind the technique, develop skills through practice session with peers, and leave with an action plan to implement the technique into an existing or future evaluation plan.

Steve Yalowitz, Senior Research Associate, Institute for Learning Innovation
Claudia Figuerido, Research Associate, Institute for Learning Innovation

In many museums there is a need to justify the existence of programs and exhibitions, and quantitative data provide a very effective way in which to do this. However, it is not always the case that museum staff or evaluators have a background in statistics and quantitative analysis. This workshop is for museum staff, internal and external evaluators and others who want to actively lead or participate in decision-making and/or conduct the actual quantitative analysis. Using data from various program and exhibit evaluations (including your own if you like) participants will gain knowledge and skills related to frequently used quantitative data analysis techniques appropriate for a variety of types of evaluation questions.

Kerry Bronnenkant, Research Associate, Museum of Science, Boston
Susan Foutz, Research Associate, Institute for Learning Innovation

Increasingly, agencies are being asked to collect information about their activities and results as well as those of projects and agencies they fund. Because these data often are used to make crucial budgeting and planning decisions, the need for high-quality data is critical. This course will introduce participants to the basics of developing data collection instruments that provide measurable evidence of intended outcomes. This course will cover the following topics, with an emphasis on surveying and survey data: types of data (quantitative and qualitative), types of data collection instruments, question development, instrument development, sampling, analysis, and reporting.

Stephanie Downey, Managing Director, New York, Randi Korn & Associates, Inc.

In visitor studies, we are often required to provide measurable, reliable findings to demonstrate impact. Yet, measuring learning that takes place in an informal setting is extremely challenging. The impact an exhibition or program can have on one visitor may vary considerably from its impact on another visitor, and these often subtle differences in impact are difficult to measure through conventional means of evaluation. Through large and small group work, workshop participants will learn to develop and use rubrics, which provide an authentic way to measure the impact of informal educational programs and exhibitions while at the same time, abide by rigorous evaluation standards. Participants will leave with a draft of a rubric tailored to their program or exhibition as well as the skills to continue using rubrics in the future. Participants should bring to the workshop a description of a program or exhibition, including the goals and objectives.

Barbara H. Butler, Retired ISE Program Director, National Science Foundation,
Marcella Wells, Consultant, Wells Resources, Inc.

During this workshop, Wells and Butler will offer an opportunity for collaboration among educators, planners, evaluators, and administrators to integrate educational/interpretive planning and evaluation. This workshop will examine the interpretive planning process and demonstrate the integration of visitor’s studies using the Visitor Centered Evaluation Hierarchy. Participants are encouraged to bring an interpretive or informal learning project that is in the planning stage for the afternoon case studies exercise. The workshop will also explore the potential of the Visitor Centered Evaluation Hierarchy as a diagnostic tool, a management tool, a teaching tool, a research tool, and a communication tool for informal learning institutions.