Program: RIP™ Based
Inquiry and Statistics for Grades 612 Teachers at the University
of Hawaii, Graduate Professional
Development Series Seminar for Science Teachers 2002, ANOVA Science Education
Corporation, Honolulu, HI.
Program evaluation
submitted: 9182002
The overall purposes of this graduate seminar were to review
and strengthen the participants’ understanding of and ability and confidence to
use the research investigation process (RIP) used in scientific inquiry, and to
introduce 612 grade teachers to research design, data handling and summary,
and data analyses techniques and procedures as they are used in the RIP. Specifically, this threeday seminar was
designed for teachers to explore the research investigation process; to use the
inquiry process to learn how to design and conduct scientific research studies;
to become familiar with techniques to assist in guiding students through the
scientific inquiry process; understand and be able to obtain random samples;
develop research investigations using independent and dependent designs; to
learn how to represent and present data obtained from research investigations;
to examine, practice, understand, and become competent in the ability to apply
data analysis techniques, including the quantification of error and statistical
tests for significance, to decisionmaking in science; and to increase
confidence in using scientific research in their approach to instructing
students in science and in addressing the scientific inquiry benchmarks and
science inquiry content standards. The
research investigation process (RIP) was reviewed and teachers were provided
the opportunity to further develop their understanding of each of the elements
of the RIP through their participation in and development of actual research
investigations. Techniques in data
summary, analysis and presentation were explored in the context of hypothesis
testing and decisionmaking in science.
Finally, numerous strategies for the teaching of science through true
scientific inquiry were also emphasized.
All aspects of this seminar were aligned with the State of Hawaii Science Content
and Performance Standards.
The data for this graduate seminar evaluation were obtained
from assessments of the 14 teacherparticipants at the beginning of
(PreAssessment) and again at the end (PostAssessment) of the 3day
seminar. Items on the assessments
required demonstration of knowledge and understanding about the scientific
inquiry process, data analyses procedures, and decisionmaking in science. A number of these items required participants
to demonstrate their knowledge and understanding through application. Selfreport items measured teacher confidence
levels in understanding and using data summary, representation, and analysis
techniques in scientific inquiry conducted in the classroom, and in comprehending
and applying the scientific inquiry content standards to their
instruction. A concept inventory
determined teachers’ familiarity with and ability to teach elements of
scientific inquiry, research design, and data summary and statistical analysis
techniques. The preseminar and
postseminar assessment items were the same except for three additional
selfreport items included on the postassessment. These additional items assessed the teachers’
perceptions of how much their understanding of the research investigation
process and scientific inquiry process improved, and understanding of
application of statistics to data increased, as a result of participation in
the seminar. The data were statistically
analyzed using dependent ttests to determine significant differences
(indicating change) between pre and postassessment mean values. For comparisons in which the assumption of
normality was violated, the Wilcoxon Signed Rank Test (nonparametric ttest)
was used to compare central tendencies.
The Pearson Product Moment Correlation Coefficient was used to test for
linear relationships. The criterion for
statistical significance (a) was set at
0.05.
Scientific Inquiry, Scientific Research, and the Research
Investigation Process (RIP)
This seminar focused on review and strengthening of teacher
understanding, abilities, and confidence related to instruction of science
through scientific inquiry, conducting scientific research, and engaging their
students in scientific research.
Although it was generally expected that teacherparticipants would be
familiar with these aspects of scientific inquiry, a preseminar phone survey
of the participants’ perceived knowledge and abilities indicated that it would
be important to address the RIP before examining research design and analysis
strategies and procedures. Specific
emphases were placed on teacher demonstration of understanding the logic of
organization of the RIP elements, as well as the components and concepts
involved in each element.
Although participant selfreported familiarity
with/understanding of the RIP appeared to slightly increase from its
preseminar level by the end of the threeday seminar (Figure 1, below), the
difference was not statistically significant.
However, the power of the ttest was low and so the negative
statistical finding should be interpreted cautiously (see Figure 1 caption for
more details).
Mean (+SEM) RIP Concept Inventory Score


Figure 1. Familiarity with and
understanding of concepts related to methodology in the RIP.
Mean postassessment value
did not differ from the mean preassessment value [t (13) =
1.99, p=0.068]. Note: Because the power of the performed
statistical test (0.345) was below the desired power of 0.800, the results
should be interpreted cautiously.
At first glance, the lack of a significant change in
selfreported familiarity/understanding of the RIP concept appears surprising
because participantdemonstrated knowledge and understanding, and selfreported
confidence (discussed below) increased consistently for all of the RIP
assessment items. However, both the pre
and postassessment concept inventory mean values were relatively high and
indicated that the participants felt that they were already very familiar with
and had a good understanding of (could teach the concept to others) the RIP
before participating in the seminar.
Postseminar teacherparticipant demonstrated knowledge and
understanding of the RIP components and their contents (Figure 2, below) and of
the organization of the RIP elements (Figure 3, below) significantly increased
compared to preseminar levels.
Mean (+SEM) Level of Demonstrated Knowledge and
Understanding
of the RIP


Figure 2. Knowledge and
understanding of the research investigation process (RIP) components and
content. (Highest possible value for
knowledge and understanding was six.)
*Mean postassessment value differs from the mean
preassessment value [t (13) =
3.29, p=0.006].
Mean (+SEM) Level of Demonstrated Knowledge of
Organization of the RIP Elements


Figure 3. Knowledge and
understanding of the organization of the elements of the research investigation
process (RIP). (Highest possible level
of knowledge was seven.)
*Mean postassessment value differs from the mean
preassessment value [t (13) = 2.49, p=0.027].
By the end of the threeday seminar compared to
preseminar levels, participants’ selfreported confidence levels for both
ability to use scientific inquiry in their instruction and their ability to
teach and engage students in scientific research activities increased from
“confident” to midway between “confident” and “very confident.” Although these increases were only slight to
moderate, they were both statistically significant (Figures 4 and 5, below).
Mean (+SEM) Confidence
Score


Figure 4. Selfreported
confidence levels for ability to use scientific inquiry in instruction.
Mean postassessment value is significantly greater
than mean preassessment value [t (13) = 2.60, p=0.022].
Figure 5. Selfreported
confidence levels for ability to teach and engage students in scientific
research activities.
* Mean postassessment value
is significantly greater than mean preassessment value [W (N=14)
= 54.0, p=0.034].
Design and Analysis
The seminar participants exhibited increases in their
knowledge and understanding as well as in their confidence related to the
elements involved in designing research investigations so that they can be
analyzed using the appropriate statistical tests. By the end of the 3day seminar, participants
demonstrated a statistically significant increase in their knowledge and
understanding of research designs, statistical concepts including, but not
limited to, random sampling, error (variability), and statistical tests (Figure
6, below).
Mean (+SEM) Research Design and Statistics Score


Figure 6. Demonstration of knowledge and understanding
of research design and statistical concepts.
(Scores were based on the number of items correct
out of 10.)
* Mean postassessment value
is significantly greater than mean pre
assessment value [t (13) = 4.70, p<0.001].
The postseminar increase in teacherparticipant knowledge
and understanding of the research process was accompanied by a significant increase in teacher’ selfreported
familiarity and understanding of concepts related to research design and
methodology (Figure 7, below) and data analysis including descriptive and
inferential statistics (Figure 8, below).
Mean (+SEM) Research Design/Methodology Concept
Inventory Score


Figure 7. Familiarity with and
understanding of concepts related to research design and methodology.
* Mean postassessment value is significantly greater
than mean preassessment value [t (13) = 6.38, p<0.001].
Mean (+SEM) Data Analysis Concept Inventory Score


Figure 8. Familiarity with and
understanding of concepts related to data analysis, including descriptive and
inferential statistics.
* Mean postassessment value is significantly greater
than mean preassessment value [t (13) = 5.57, p<0.001].
By the end of the seminar, the average participant’ response
to the concept of research design and methodology rose from between “somewhat
familiar, but do not really understanding what it means” and “familiarity with
fair understanding of the concept” to “very familiar with concept and could
teach it to others.” Likewise, the
average participant’ response to the concept of data analysis rose from
“somewhat familiar, but do not really understanding what it means” to between
“familiarity with fair understanding of the concept” and “very familiar with
concept and could teach it to others.”
These findings indicated that teachers recognized their increased
knowledge and understanding related to research design and statistical
concepts.
With the increase in knowledge and understanding, the
teacherparticipants also demonstrated an increase in selfreported confidence
in their understanding of the application of basic statistics to analyze research
data (Figure 9, below).
Mean (+SEM) Confidence
Score


Figure 9. Selfreported
confidence levels for understanding of the application of basic statistics to
analyze research data.
* Mean postassessment value
is significantly greater than mean preassessment value [t (13) =
6.07, p<0.001].
By the end of the threeday seminar, the mean participant
confidence score more than doubled, significantly rising from between “not at
all confident” and “somewhat confident” to near “confident.”
Teacher Understanding of and Ability to Apply Data
Summary, Presentation, and Analysis techniques to DecisionMaking in Science
By the end of the threeday workshop, seminar participants
demonstrated a significant 33% increase in their ability to determine the
appropriate statistic for describing the center of a group of data (Figure 10,
below).





Mean (+SEM) Score for Ability to Use Appropriate
Measure of Central Tendency





Figure 10.
Demonstration of the ability to determine the appropriate measure of
central tendency for a group of data.
(Highest possible ability score was four.)
* Mean postassessment value
is significantly greater than mean pre
assessment value [t (13) = 2.79, p=0.015].
In addition to gains in understanding of the application of
descriptive statistics, participants also demonstrated a postseminar increase
in knowledge of the procedures used to calculate the three measures of central
tendency (Figure 11, below).
Mean (+SEM) Score for Ability to Calculate the
Three Measures of Central Tendency


Figure 11.
Demonstration of the ability to calculate the three measures of central
tendency. (Highest possible ability
score was six.)
* Mean postassessment value
is significantly greater than mean preassessment value [t (13) =
3.16, p=0.008].
The seminar participants’ selfreported familiarity with,
and understanding of the concepts of, the three measures of central tendency
also significantly increased by the end of the threeday seminar (Figure 12,
below). This increase coincided with the
demonstrated increases in understanding and ability to calculate the measures
of central tendency (both discussed above), indicating that the participants
were able to accurately perceive and report their relative knowledge of and
ability to summarizing groups of data.
Mean (+SEM) Measures of Central Tendency Concept
Inventory Score


Figure 12. Familiarity with and
understanding of concepts related to the three measures of central tendency.
* Mean postassessment value is significantly greater
than mean preassessment value [t (13) = 3.60, p=0.003].
Figure 13 (below) presents a scatterplot showing the pre to
postseminar change in participants’ familiarity/understanding of the concept
of measures of central tendency plotted as a function of change in demonstrated
participant knowledge of descriptive statistics. Pearson Product Moment Correlation
Coefficient revealed a significant moderate positive relationship between these
two variables (Figure 13).










Amount of
Change in SelfReported Familiarity/ Understanding






Amount of Change in Demonstrated Knowledge


Figure
13. Scatterplot of pre to postseminar
change in familiarity/understanding as a function of change in demonstrated
knowledge of measures of central tendency.
By the end of the threeday seminar, participantreported
familiarity/understanding to the concept of error variance (standard deviation)
steeply increased compared to the preseminar value (Figure 14, below). Before taking the seminar, the
teacherparticipants reported that the
concept “standard deviation” was between “somewhat familiar” and “familiar,” but have less than “a fair understanding of
what it means.” After participating in
the seminar, teachers reported that they were “very familiar with this concept,
but would have some difficulty teaching it to others.”
Mean (+SEM) Standard Deviation Concept Inventory
Score


Figure 14. Familiarity with and
understanding of the concept of standard deviation.
* Mean postassessment value is significantly greater
than mean preassessment value [t (13) = 7.78, p<0.001].
The participants confidence in their understanding of how to
use statistics to describe the center and dispersion (variability) of a group
of data also substantially increased from less than “somewhat confident” to
slightly greater than “confident” by the end of the seminar (Figure 15,
below).
Mean (+ SEM) Confidence
Score


Figure 15. Selfreported confidence
levels for ability to use statistics to describe the center of and variability
for a group of data.
* Mean postassessment value
is significantly greater than mean preassessment value [t (13) =
4.52, p<0.001].
Seminar participants exhibited a statistically significant
increase in their ability to correctly identify the purpose of, and accurately
interpret data presented in graphs displaying central tendencies for two groups
(Figure 16, below) and scatterplots (Figure 17, below).
Mean (+SEM) Score for Ability to Recognize and
Interpret Bar Graphs


Figure 16.
Demonstration of the ability to correctly recognize the purpose and
accurately interpret data presented in graphs displaying the central tendencies
of two groups. (Highest possible ability
score was five.)
·
Mean postassessment value is significantly greater than mean preassessment
value [t (13) = 2.81, p=0.015].








Mean (+SEM) Score for Ability to Recognize and
Interpret Scatterplots



Figure 17.
Demonstration of the ability to correctly recognize the purpose and
accurately interpret data presented in scatterplots. (Highest possible ability score was five.)
* Mean postassessment value
is significantly greater than mean preassessment value [t (13) =
3.83, p=0.002].
Increases in the ability of participants to construct graphs
for displaying of data and comparison of central tendencies also significantly
increased by the end of the seminar (Figure 18, below).
Mean (+SEM) Score for Ability to Construct Graphs


Figure 18.
Demonstration of the ability to construct graphs for display of data and
the comparison of central tendencies.
(Highest possible ability score was five.)
* Mean postassessment value
is significantly greater than mean preassessment value [t (13) =
3.52, p=0.004].
Consistent with demonstrated increases in abilities to
summarize data, identify, accurately interpret, and correctly construct tables
and graphs, significant increases in participant selfreports of familiarity
and understanding for concepts related to representations of data, including
tables and graphs, were exhibited at the end of the threeday seminar (Figure
19). At the start of the seminar,
teachers reported being “familiar with these concepts, with a fair
understanding of what they mean.” By the
end of the seminar, the teachers reported being “very familiar with these
concepts, but would have some difficulty teaching them to others.”
Mean (+SEM) Data Representation Concept Inventory
Score


Figure 19. Familiarity with and
understanding of concepts related to representing data in tables and graphs.
* Mean postassessment value is significantly greater
than mean preassessment value [t (13) = 4.97, p<0.001].
Hawaii
State Science Content Standards
Teacher confidence in
ability to address science content standards relating to collecting, organizing,
displaying and analyzing data in their classroom was also affected by the end
of the seminar. Selfreported confidence
in ability to address content standards in the classroom rose significantly
from between “confident” and “somewhat confident” to above “confident” by the
end of the seminar (Figure 20, below).
Mean (+SEM) Confidence
Score


Figure 20. Selfreported
confidence levels for ability to address data summary and analysis science
content standards in the classroom.
* Mean
postassessment score is significantly greater than mean preassessment score [t
(13) = 3.45, p=0.004].
Teacher
Perceptions of Impact from their Participation in this Graduate Seminar
The postassessment contained three additional
selfreport items designed to assess how much teacherparticipants believed
their understanding about the scientific inquiry process, data analysis, and
the science inquiry standards improved as a direct consequence of their
participation in the seminar. The
results from these items are presented in Figures 2123 below.
An overwhelming majority of the seminarparticipants
(10 of 14) claimed that their understanding of the research investigation
process improved a large amount to completely as a result of their
participation in this seminar, while three participants claimed it changed a
moderate to a large amount (Figure 21, below).
One teacher reported that his understanding improved slightly less than
a moderate amount.
Figure
21. Pie chart representing
teacherparticipants’ responses to “what extent, if any, did your understanding
of the research investigation process improve as a result of your participation
in this graduate seminar?” The scale for
responses included none, a small amount, a moderate amount, a large amount, and
completely.
A majority of the seminarparticipants (9 of 14)
claimed that their understanding of the application of statistics to analyze
data obtained in a research investigation improved a large amount to completely
as a result of their participation in the 3day seminar (Figure 22,
below). Four of the participants claimed
it improved a moderate to a large amount as a result of their participation. One teacher reported only a small to
moderate improvement on this item.
Figure
22. Pie chart representing
teacherparticipants’ responses to “what extent, if any, did your understanding
of the application of statistics for understanding the data obtained from a
scientific research investigation increase as a result of your participation in
this graduate seminar?” The scale for
responses included none, a small amount, a moderate amount, a large amount, and
completely.
Fifty percent of the seminarparticipants (7 of 14)
claimed that their understanding of the science inquiry standards improved a
large amount to completely as a result of their participation in this seminar,
five (about 36%) claimed it improved a moderate to a large amount, and two
reported that it improved a small to moderate amount (Figure 23, below).
Figure
23. Pie chart representing
teacherparticipants’ responses to “what extent, if any, did your understanding
of the scientific inquiry standards improve as a result of your participation
in this graduate seminar?” The scale for
responses included none, a small amount, a moderate amount, a large amount, and
completely.