Evaluating the Effects of Intervention on Teacher Learning in an Online Heat and Temperature Course.

Authors: Roxane Johnson, Manasa Chakravarthi, Judah Schwartz

I. Analysis of online discussion forums: Radiation Session (session 5)-Cohort One

After the second Fulcrum course had ended for Cohort One in December of 2006, some of the archived discussions and teacher work from a science session on radiation (F2, session 5) were analyzed. The session had as its main goals for teachers to understand that:

• Every object with a temperature above absolute zero (-273° C) emits radiation.
• Thermal radiation is energy transferred by electromagnetic waves.
• There are several differences between heat transfer due to radiation, convection, and conduction.

In the radiation session teachers were asked to consider the following scenario:

You're going to heat up some beans for a pot-luck dinner. You wonder if you should put them in your white casserole or your black casserole. Will one cool down faster than the other? You don't like the idea of carrying a burning hot pot. The color of the pots wouldn't affect heat transfer by conduction or convection. But you may have had an experience like this before when color did affect heat transfer: You're walking with a friend. It's a spring day, about 20° C and cloudy. You are wearing identical shirts, only yours is black and your friend's is white. Both of you left your sweaters at home. The sun comes out. Your friend doesn't believe you when you remark after 10 minutes or so how warm you are. How do you explain what's going on?

The Fulcrum participants were asked to make predictions about how two cans, one black, one white, would heat up and cool off over time. Temperature was automatically recorded by heat probes inserted into each can and linked to microcomputer based laboratory software. For the "heating" prediction (and experiment) they were to use a lamp bulb focused on the cans to heat up the cans and once the cans had reached equilibrium, they were to turn off the light. They were asked, "How do you think the temperature in the black and white cans will change during heating?" For cooling condition #1 they were asked: "What will happen once the light is turned off?" They were asked to draw a prediction graph in their science journal of temperature vs. time for each can. They were instructed:

Be sure to consider the starting temperature, the final temperature, and how the shape of the curve reflects the rate of temperature increase and of temperature decrease. Annotate your curves, giving your rationale for how energy is transferred.

They were also asked to consider the shape of the cooling curves if the two cans started at the same temperature by letting them heat in the same water bath and then allowing them to cool on the counter (cooling condition no. 2.) They were asked to think about how conduction, convection, and radiation were affecting the heating and cooling of the cans.

The teachers were then asked to perform the experiments, collect and post their data. Regarding their reports, the following instructions were given:

In your report, include data from your investigation as evidence to support your conclusions. Compare your results with your prediction. Can you find a reasonable explanation for any discrepancies between your predictions and your results? Explain your results in terms of radiation, convection and conduction.

After Cohort One completed the radiation session of Fulcrum course two the individual predictions, reports and graphs that were posted to the online forum during the radiation session were analyzed to find if first, the teachers were making the required predictions, testing their predictions, and reporting their findings and secondly, to find if the overall understanding of the teachers met the session understanding goals and for the purpose of informing course revisions. For Part A of this analysis, several questions were asked of each participant's post:

1. Did the teacher make a heating prediction and was the heating prediction (for the lamp heating the cans) consistent with scientific norms? (Black can heats up at a faster rate and heats to a higher temperature than the white can. The white can heats up but at a slower rate and to a lower temperature.)
2. Did the teacher make a cooling prediction (after the cans had been heated by the lamp) and was the cooling prediction consistent with scientific norms? (The hotter black can will cool faster than the cooler white can.)
3. Did the teacher make a cooling prediction of the cans starting at the same temperature and was the cooling prediction consistent with scientific norms? (The black can will cool faster than the white can. Both cans decrease in temperature at a decreasing rate with the black decreasing faster.)
4. Did the teacher report results for each test and are the reported results for each test consistent with the scientific norm?
5. Is there at least one Logger Lite graph posted to the report?
6. Is the graph labeled and consistent with the claims reported in the written report?

For Part B of this analysis, the discussions of two groups were analyzed. Group profiles (case studies) were created by reading the posted predictions, reports, and graphs and then studying the threaded discussions that occurred from Tuesday-Thursday of that session. Each member's report thread and peer responses were evaluated for science content, experimental data, and final overall understanding of radiation by the end of the session. A summary of each student's report and peer responses was written for each of the group members.

II. Analysis of Pre and Post Responses to a Thought Experiment (Cohort One and Cohort Two)

The teachers conducted a thought experiment at the beginning and again at the end of F2 and posted their explanations of the physical phenomena pertaining to heat and temperature as if the experiment were to be conducted in reality.

Thought Experiment:

It's a hot, still summer day. You're having a party and soon the guests will arrive. Your friend brings over 2 cases of soda, but they're warm and now you have a real dilemma. What's the fastest way to cool them?

1. Put them in the refrigerator
2. Put them in a cooler with ice
3. Put them in a cooler with water at 0C
4. Put them in a cooler with ice and pour room temperature water over the ice

Spend 30 minutes taking notes about the temperature change and heat transfer that occurs in each instance.

• Sketch graphs showing temperature change over thirty minutes at two locations: in a can of soda, and in the environment (i.e. the air inside the refrigerator in case 1, the ice in case 2, the water in cases 3 and 4).
• Draw a diagram, showing where and by what process(es) heat transfer is occurring in the system and explaining all the ways that heat is transferred.
• Imagine you have a pair of magic eyeglasses that allow you to observe the system on a microscopic scale. In each case, as heat is transferred from the warm soda to its surroundings, what changes occur?
• Use your notes to write a report explaining the heat flow and the temperature change occurring. Please give as much detail as possible, and explain the process using both a macroscopic and a microscopic model.

The written responses of the seventeen teachers who completed both the pre and post tests to this thought experiment constitute the data for this study. In order to determine whether there was any improvement in teachers' conceptual understanding of the mechanisms of heat transfer, a list of essential ideas was made based on the 'ideal responses' obtained from a Tufts University physicist and Fulcrum staff member, Roger Tobin. The key ideas were related to the following properties:

1. Rate of Heat Transfer
2. Temperature differences between substances
3. Heat flow direction
4. Use of words conduction or convection without further explanation
5. Equilibrium
6. Phase change
7. Volume of source and sink relative to each other
8. Speed of molecules of the substances in the system
9. Molecular interaction between molecules of (a) substance(s) at different temperatures
10. Molecular interaction between molecules of two different substances
11. Contact between the surfaces of source and sink
12. Kinetic energy of molecules
13. System in which the heat transfer takes place

These responses posted and counted only once if they occurred. The ideas themselves are neither wrong nor right; for example one of the key ideas was 'conduction'. This by itself is not correct or incorrect but when looking at a particular response in the context it was used, and comparing it with the 'ideal response' it can be called Normative (confirms to scientific norm) or not. The frequency of each key idea was counted for the whole group, both pre and post test.

The analysis includes a random sampling of the total number of key concepts invoked by seven teacher participants during pre-test and the post-test (Cohort One). Each key idea used by a teacher was given a nominal value '1' if normative (scientific) and '0' if not. The ratio of normative to non-normative ideas was computed for these seven teachers.

A similar analysis will be undertaken with Cohort Two, F2 pre and post tests, once Cohort Two has completed the course (December 21, 2007). The results of Cohort One and Cohort Two will then be compared.

III. Analysis of online discussion forums: Radiation Session (Session 8)-Cohort Two

The work done by Cohort Two during the radiation session (Session 8) will be analyzed in much the same way as Cohort One reports posted to the online discussion forums were analyzed. By necessity, the precise questions that will be asked will change due to the "new" radiation session that was created in response to the findings of Cohort One's response to the radiation session in 2006.

In addition, the cohort analysis will include assessments of the effectiveness of the discussion leader. In hopes of improving online discussion, one of the changes that

the Fulcrum Institute made between cohort one and cohort two was to require that each teacher take a turn being a discussion leader during one online science session and one online pedagogy session. The discussion leader will be identified for each group. A transcript of the discussion leader for each group will be analyzed for the number of posts made, the immediacy of replies to peers, the types of questions asked, the response of these questions, and the types of comments made.