defining the landscape

Examining national science education uses a macro-level view. Studying what occurs in classrooms involves the micro-level. Our interest in schools — their organization and their leadership — operates at the meso-level. But before we could go much further, we had to identify the components of a meso-level framework. Fortuitously, school effectiveness researchers have been working at this challenge for several years. Even though our focus is on science achievement, we can draw on the intellectual foundations of others.

James Coleman suggested that social capital consists of three overlapping elements: communication networks, interpersonal trust, and organizational norms. But schools are complex organizations: there are administrators and teachers, there is “inside” the school and the necessary obligations to “outside” groups, there are opportunities to collaborate as well as incentives for competition. Essentially, we combine all the survey items that others had designed that were relevant to our project — and asked experts to judge these materials.

We asked practicing teachers, school effectiveness researchers, and science leaders to rate the items based on their alignment to one another and the accuracy with which they measured what we were after. That process cut our pool of items down by about half. We then gave these items to classroom teachers whose feedback indicated how the items could be further improved. Finally, the items were sent to a nationally representative pool of K-8 teachers and their responses were statistically analyzed. What emerged from all of this work were NINE categories of responses that are distinct from one another. These will provide the framework to guide us as we move forward on this research project.

how we began

A common dream for every educator is to start one’s own school. However, once you are invited to design a science magnet school, the tolerance for crazy ideas suddenly disappears and is replaced by a need to know exactly what the right thing is to do. One tactic is to look at science performance across a variety of schools in an effort to uncover features that might be essential for success.

This graph shows fifth grader performance on the annual statewide science test. Each symbol represents a school within a single urban district with the percent of “minority” students (i.e., those who are not White) along the horizontal axis and science scores on the vertical axis. The statistical trend is that the scores  go down as the proportion of minority students increases. What is surprising is how much variation there is in science performance. At the 80% mark we see substantial differences in scores — and the causes for this are intriguing.

Those who work with statistics have an aversion for unusual data. Rather than throw outliers away, schools with performance far above what the equations predict are cause for investigation. That is how this project began. We wondered why schools with similar populations of students had such uneven performance in science. Even though the science test is given during the fifth grade, we felt the test measured more than what a child learned with one teacher during one school year. And this made us wonder whether the school as an organization could help explain the outliers in this data.