Multi-campus Project: Promoting and Assessing Science Literacy in General Education Science Courses
- Edward Nuhfer, Faculty Development & Geology, Channel Islands
- Jerry Clifford, Physics, Channel Islands
Christopher Cogan, Environmental Sciences & Resource Management, Channel Islands
- Anya Goodman, Biochemistry, San Luis Obispo
- Carl Kloock, Biology, Bakersfield
- Beth Stoeckly, Physics, Channel Islands
- Christopher Wheeler, Geology, Channel Islands
- Gregory Wood, Physics, Channel Islands
- Natalie Zayas, Science Education & Environmental Sciences, Monterey Bay
Students in Natalie Zayas' Nature of Science course at CSU Monterey Bay hike
to Moro Cojo Slough to conduct field tests of water quality.
A world understood is a world less threatening, and understanding the world is what, in the end, science is about (James Trefil, 2008, Why Science? p. 1).
The primary learning goal of science courses that fulfill universities' general education requirements is the achievement of science literacy—a general term that communicates that one knows the conceptual processes employed in understanding the physical world. Misconceptions serve as particularly formidable barriers to this understanding.
Popular media provide one source of misconceptions. A person educated through popular media may believe that science supports the interests of a particular political party; that science is an appropriate means to validate or refute particular religions; and that science includes investigation of the paranormal. A wide range of valid disciplines that have nothing to do with understanding the physical world nevertheless adopt the term science to describe their own frameworks of investigation, which adds further confusion to students about the nature of science.
Finally, neglect of teaching the frameworks of reasoning and process of science in the very courses that have the responsibility to produce science literacy means that the misconceptions about science continue past the conferring of college degrees. Conveying the content and computational procedures of specialty disciplines simply does not produce conceptual understanding of science as a process.
A number of Cal State faculty recognized the problem, and some had expended considerable individual efforts to remedy it. These scientists would not have met except for an online database called the Colleague Connector, designed by the CSU's Jeff Gold and his team at the Center for Distributed Learning at Sonoma State, which enabled faculty on different CSU campuses with similar scholarly interests to connect and participate in the ITL's faculty Disciplinary Research Project. We came together as a result of a successful proposal to ITL: Instilling Conceptual Thinking in Introductory Science Courses (see ITL Connections, v1n2).
At our initial two-day meeting in April 2008, we formulated some challenges based upon our pooled experiences.
- How can we best integrate multiple teaching approaches into classes to better convey both content and general long-term goals like critical thinking and knowledge of science?
- Does the incorporation of active learning techniques improve performance in science literacy compared to a traditional lab/lecture format?
- Why should we better convey what science is and how can we do that?
- Can we create ways to track learning produced by our courses through pre-post measures?
- As a group, can we come to consensus on what constitutes a good understanding of science literacy in a general education introductory-level course?
- Does changing our teaching approach(es) make a difference in student learning?
- What are effective techniques to develop scientific literacy, and can science literacy offer an effective pathway to opportunities for developing critical thinking skills?
- How can students who are non-majors recognize a scientific question, study, or investigation when they encounter one in class or in the media?
- How can updating our individual teaching philosophies inform us about how to better design our courses and achieve what we desire?
The meeting offered resources on DVDs, books, and training sessions by several faculty developers that addressed items 1, 2, 6 and 9.
Participants returned to their campuses and incorporated a variety of tools such as knowledge surveys, course redesign based on Dee Fink’s model, and a number of interactive engagement techniques. During the following year, we pooled our knowledge to pursue a literature research to better address items 3, 4, 5, 7, and 8. The group also procured and read Lewis Wolpert’s The Unnatural Nature of Science, Robert Hazen and James Trefil’s Science Matters: Achieving Scientific Literacy, and James Trefil’s Why Science?
Students in Natalie Zayas' Nature of Science course at CSU Monterey Bay measure the water pH at a sampling site along Moro Cojo Slough.
We remained unsatisfied that the literature offered what we sought in operationally defining science literacy for purposes of general education. We sought to define it in terms of measurable learning outcomes that involved conceptual reasoning, and we eventually deduced twelve, the first seven of which we consider the most essential.
A graduate who is science literate should be able to:
- Articulate in her/his own words a reasonable definition for what constitutes science.
- Describe, using at least two specific examples, how science literacy is important in everyday life to an educated person.
- Explain why the attribute of doubt has value in science.
- Explain how scientists select which among several competing working hypotheses best explains a physical phenomenon.
- Explain how theory as used and understood in science differs from theory as commonly used and understood by the general public.
- Explain why peer review generally improves our quality of knowing within science.
- Explain how science uses the method of reproducible experiments to understand and explain the physical world.
- Name one assumption that underlies all science.
- Provide two examples of science and two of technology and use these to explain a central concept by which one can distinguish between science and technology.
- Cite a single major theory from one of the science disciplines and explain its historical development.
- Explain and provide an example of modeling as used in science.
- Explain why awareness of ethics becomes increasingly important as a society becomes increasingly advanced in science.
If these outcomes truly represent conceptual thinking, then each should correspond to a clearly articulated concept of science. The correlative concepts we deduced follow, and each has correlative misconceptions that impede understanding.
- Science explains physical phenomena based upon testable information about the physical world.
- In modern life, science literacy is important to both personal and collective decisions that involve science content and reasoning.
- Doubt plays necessary roles in advancing science.
- The strongest scientific hypotheses remain as accumulating evidence supports them and fails to support weaker hypotheses.
- A theory in science rests on considerable evidence and offers a unifying explanation for observations that result from testing several hypotheses.
- Peer review generally leads to better understanding of physical phenomena than can the unquestioned conclusions of involved investigators.
- Science can test certain kinds of hypotheses through controlled experiments.
- All science rests on fundamental assumptions about the physical world.
- Science differs from technology in that science is concerned with explaining the underlying phenomena behind a technological application.
- Scientific knowledge is discovered, and scientific discovery relies upon the history of previous scientific discoveries.
- Science employs modeling as a method for understanding the physical world.
- Scientific knowledge imparts power that must be used ethically.
At this point, we had addressed our main problem of aligning teaching with a focused awareness on what we needed to address to develop science literacy along with content mastery. However, we also wanted to assess whether our efforts to develop science literacy produced any measurable changes in students.
When we reviewed instruments that purported to measure science literacy as conceptual understanding, we found none that met our needs. Concept inventories such as the Force Concept Inventory of physics or the Geoscience Concept Inventory of geology focused on disciplinary understanding but failed to address conceptual understanding of the process of science. Other tests tended to define science literacy as possessing certain factual information or by responses to true/false questions that were not clearly related to measurable learning outcomes or conceptual understanding. When we realized that the instrument we sought did not exist, we turned to the literature (particularly that of Julie Libarkin) on preparing concept inventory tests and created our own Science Literacy Concept Inventory, which, like other concept inventories, is in multiple-choice format.
We queried whether each item drafted passed the following screening criteria:
- Item reflects clearly the science concept being tested?
[Note: Multiple-choice tests consist of discrete items. Each item consists of a stem that asks the question to set up the situation for a choice from multiple options. The choices consist of one correct item and several incorrect options, also called distracters.]
- Item can likely be answered with science background of a college freshman?
- Stem is constructed as a question?
- Stem is simply worded and unambiguous?
- Item avoids technical jargon?
- The response options are plausible?
- Item employs 3-5 response options?
- Item avoids requiring responses as a combination of statement choices?
- Item avoids absolutes and complexities?
- Response options are of similar length?
We realized during the vigorous debate that occurred during review and revisions of these items why this particular inventory could never have been created by a single investigator or by a team of investigators from a single science discipline. In addition to the many other instructional benefits, we learned to write better test items in the course of creating the inventory. This project proved to be a monumental development opportunity.
We are now ready to begin pilot testing nearly eighty items we created to address the twelve concepts based upon two subtests of forty items each. We had just time at the end of spring term, 2010, to run one subtest of the items in one upper-division science class and one lower- division class. The number of students involved is far too small to make meaningful conclusions, but the results illustrate some of the kinds of information that we can begin to obtain.
Figure 1. Results of post-course pilot test on one upper and one lower division non-majors science class.
Overall, the upper division class scored 72% on the inventory, whereas the lower division class scored 59%. Juniors and seniors better understood the value of peer review (Concept 6), modeling (Concept 11), and the relationship between ethics and the practice of science (Concept 12). Little difference existed between lower division and upper division students in understanding that science does rest on some basic assumptions (Concept 8), in ability to distinguish statements as testable hypotheses or the strongest among multiple working hypotheses (Concept 4), or in ability to distinguish science from technology (Concept 9). Students seemed weakest in distinguishing why science literacy is relevant to their lives (Concept 2) and in distinguishing science from technology (Concept 9).
To do good item analyses and to screen out bias from factors such as gender or ethnicity requires several thousand students. A number of our items will need to be revised, replaced, or discarded. Creating a good concept inventory is a major effort, but once completed, it will fill a major need in assessing any general education science course and help convey a clearer vision of what students who achieve science literacy should be able to do to demonstrate understanding.
Thus, we invite other faculty who teach general education science courses to participate in this important project.
Contact Dr. Edward Nuhfer firstname.lastname@example.org for access to the inventory to run as an online survey at the start and end of your courses.
||Teaching & Learning
Teaching a Course outside Your Expertise: Ten Suggestions
Dr. Cynthia Desrochers
Faculty Director, ITL, CSU Office of the Chancellor
Have you ever taught a course on a topic you hadn’t mastered? Did you view this as unusual and believe that you were alone in being asked to teach a course you didn’t feel fully prepared to teach? Therese Huston calls this practice one of higher education’s dirty little secrets, bringing it to light for frank discussion in her book, Teaching What You Don’t Know (Harvard, 2009).
Does teaching what you don’t know occur in the CSU? At a CSU meeting I attended last month, this very issue came up: faculty in a School of Education are experiencing reduced enrollment due to limited public school teaching positions; consequently, their faculty who were hired to teach language arts methods to future teachers are now preparing to teach freshman composition courses in the English department. This illustrates a typical example of teaching outside one’s area of expertise, but within one’s related area of knowledge and skill. Thankfully, no one is asking the math faculty to teach freshman composition, at least not yet!
What follows are ten suggestions extracted from Huston’s book to assist faculty who find themselves teaching a course outside their area of expertise.
- Read all student-assigned readings before the term begins.
This may sound obvious, but faculty credibility can take an immediate nose dive when we misquote materials which the students have read in their assigned readings. Think of it this way, given the choice between reading extensively in an area of research that is not part of your students’ assigned readings and reading a student-assigned chapter, the student-assigned chapter comes first. Moreover, it is almost essential that you read all the assigned materials before the term begins (assuming you didn’t get the course assignment the weekend before the start date) because teaching any new course requires additional preparation time throughout the term. Dee Fink, expert in course design, estimates that teaching a familiar course represents 20% of faculty workload, but teaching a new course represents 40% -- and that’s within one’s area of expertise.
- Communicate that as a professional it is appropriate for you to be learning, too.
Huston explains that faculty struggle with whether or not they should let students know that teaching a course will be a learning experience for them, too. She suggests that rather than make yourself miserable trying to keep up the appearance that you know more than you actually do, remove some of the pressure by telling students that “The life of a scholar is always to be learning, so we’ll all be learning together in this course.”
- Demonstrate your confidence and credibility as a teacher.
Faculty should recognize that many of us feel like impostors when addressing various concepts, even those within our area of expertise. While clearly we should know more than the students know about the concepts we are teaching, we should have the confidence to recognize that it isn’t necessary to be a credentialed expert in an area to be an effective teacher. Moreover, be aware that students view credibility as more than subject-matter expertise. Research shows that instructors often lose credibility with their students when they: show up late for class; lack familiarity with the text; cannot explain difficult concepts; rarely ask students if they understand their explanations; don’t make any attempt to answer students’ questions; provide unclear expectations about course policies, tests, and graded assignments; fail to follow the course policies outlined in the syllabus; and fail to remind students of upcoming deadlines.
- Capitalize on your content-novice perspective towards the topic.
I have seen it happen that a student can explain a topic to another student better than I can. This likely occurs because students are closer to one another on the learning curve, and the student doing the explaining is using more simplified examples, vocabulary, and strategies than I would use. Witness the stereotypical expert professor who knows more about her or his topic than anyone on the planet, but who is completely unintelligible to others when attempting to explain it. When we are teaching out of our area of expertise, we have that content-novice perspective that allows us to be closer to our students as novice learners. Recognize this and capitalize on it; you may better be able to explain the concept at hand than the expert professor down the hall because of your novice perspective—KISS (keep it simple sweetie).
- Focus on teaching concepts that are familiar to you.
Faculty likely do this when teaching most courses, but consider doing it intentionally when teaching outside of your expertise. Obviously, we aren’t able to do this exclusively, but I suggest that we begin the course as well as begin each class with familiar-to-us concepts, if possible. This advice applies the primacy effect, which is defined as a student’s tendency to learn the first things presented in a sequence, and is one of the oldest findings in educational psychology. Translated into how it can operate in our classrooms, if we teach familiar-to-us concepts first, they have a greater probability of being learned by our students than the concepts we present in the middle of the term or the middle of each class; moreover, our credibility in teaching these familiar concepts will likely also be remembered by students, leaving those concepts taught in the middle, where we struggled, likely items on our students’ forgetting curve.
- Avoid over-preparing and lecturing too much on a topic you are still learning.
Huston cites these as common mistakes faculty make when teaching outside their area of expertise, even faculty who normally incorporate discussion and active learning into their courses. The reason this occurs is that we have more control under lecture conditions as compared to the decreased predictability with active learning and group work—considered by some faculty as high-wire teaching without a safety net. Moreover, in discussions, students may ask questions we are not prepared to answer regarding this relatively-new-to-us topic. Teaching outside one’s area of expertise using only the lecture format takes hours of preparation to prepare the lecture notes, plus the time-intensive crafting of PowerPoint slides for every concept, if you add those to your lectures. But perhaps more importantly, research has shown time and again that students’ learning from lecture is shallow if not coupled with a deep learning activity where they can actively use the concepts and relate them to what they already know.
Student from CSU Northridge participates in a class poster session, an active learning experience.
In Chapter 5, Huston shares ten active learning strategies we might use. Two are explained below. To learn about the others, suggest that your teaching and learning center director facilitate a faculty book club using Huston’s book.
Research cautions the lecturer that the average learner attention span wanes during standard lecture every 15-20 minutes. In order to restart students’ attention clock, stop lecturing for two minutes, and let students pair up to compare their notes with a neighbor, filling in each other’s gaps and misunderstandings.
This activity is designed to help students prepare and build confidence to participate in class discussion by giving them time to write, think about, and perhaps even rehearse their answers. Simply tell the students your question and ask them to write the answer in their notes. Select a question that requires some thoughtful analysis versus simple recall from memory. For example in a California history class, the question, "Name three major cities in California?" requires less thoughtful analysis than the question, “Which city, Los Angeles or San Francisco, has played the more significant role in the history of California? Please explain." After a few minutes of writing, a class discussion of the answers ensues.
For more lecture breaks, please see Multi-purpose Lecture Breaks (The Best of the Teaching Professor, 2005, Magna Publications).
- Make time for student questions during class.
It is likely that students will ask questions you cannot answer when you’re teaching outside your area. Some advice for responding to students includes: clarifying the question; acknowledging the student for asking it; asking if the class can answer it; offering an educated guess and labeling it as such; or admitting that you don’t know the answer and offer to find it for the next class meeting. Huston suggests that students may respect your candor and appreciate that their questions are insightful enough to drive your learning as well as theirs. And she ends with a warning: Never Fake It.
- Invite guest speakers for your least familiar topic.
For that critical topic where you feel least adequate, invite a guest speaker to class and arrange an exchange with a course she or he is teaching where you are the guest.
- Prepare an Emergency Assessment Kit to use if you finish early.
Perhaps one reason we tend to overplan these out-of-expertise courses to a fault is in order to make sure that we have enough material to fill the entire class period. It’s scary to finish 15 minutes too early and try to improvise when you are only a few chapters ahead of the students. In preparation for the possibility of extra time, put together an Emergency Assessment Kit as a sponge activity (i.e., an activity that productively sops up time). Introduce the assessment activity with a positive comment, such as, "I was hoping that we would have time to do this today,” versus “Oh, my land, I ran too short, but here’s a filler you can do."
Only one assessment activity is explained below, but you can check Classroom Assessment Techniques (CATs) (Angelo and Cross, 1993) for 50 more.
Announce a concept from your course; ask students to write that concept at the top of a sheet of paper; and then ask them to label three columns as follows:
- What makes sense to me about this concept.
- Why it makes sense to me.
- What I’m still working to understand. When the grid is individually completed in writing by your students, each column will reveal important elements about their understanding of that concept.
- Pack your humor before coming to class.
And, finally, when you DO make a mistake, rather than panic or become upset, use humor to make light of your mix up: you were catching up on your Netflix and didn’t get enough sleep; you stepped on and broke your eye glasses; or you turned on the lecture record without listening to what was playing – whatever suits your personality to keep the classroom climate pleasant. This may be a challenge, because as I once heard a colleague comment about a classroom problem, "When you feel like you’ve been pushed from an airplane at 30,000 feet, it’s difficult to enjoy the view."
Bottom line: Huston’s book is filled with suggestions that are absolute gems. Consider reading it cover to cover!
Things turn out best for the people who
make the best of the way things turn out.
John Wooden, teacher and coach,
October 6-8, 2010, 8:30 a.m. to 5:00 p.m. 16th Annual Conference on Learning and Teaching
Registration opens: August 30, 2010
Hosted by Dr. Lee Altier, Director
Highlights include a keynote presentation and workshop by Dr. Laura Rendón on teaching to the whole student and tracks on course transformation, internationalization, technology in the classroom, and civic engagement.
Center for Excellence in Learning and Teaching
Questions: (530) 898-3094 or email LAltier@csuchico.edu