My Goal in Blogging

I started this blog in May of 2008, shortly after my election to the School Committee, because I believed it was very important to both provide the community with an opportunity to share their thoughts with me about our schools and to provide me with an opportunity for me to ask questions and share my thoughts and reasoning. I have found the conversation generated on my blog to be extremely helpful to me in learning community views on many issues. I appreciate the many people who have taken the time to share their views. I believe it is critical to the quality of our public schools to have a public discussion of our community priorities, concerns and aspirations.

Thursday, July 1, 2010

More Thoughts on IMP

My June Oped in the Bulletin regarding math in ARMS and ARHS has led to a number of responses, including two letters in today's Bulletin (not yet online, but will be soon) and an official response on the ARHS website (http://www.arps.org/hs/Parents/IMPMathIntro.php). The high school response makes a number of claims about the benefits of IMP, including that this program is associated with students taking more math classes and achieving higher standardized test scores. I would encourage parents and community members to read the research posted in this link carefully: what is very clear is that IMP was developed in the early 1990s, and that ALL of the research cited in this link is from the early to mid-1990s, when students taking IMP math were still in high school. Some evidence now suggests that this approach didn't really stand the test of time -- meaning that when students trained in IMP entered college-level math classes, they struggled considerably. This might be why of the 23 districts that, like Amherst, are part of the Minority Student Achievement Network (MSAN), 16 do NOT offer an IMP math track. It might also be why there is considerable controversy in the mathematics community about IMP (and other forms of reform math), as described in an editorial by David Klein, a math professor at California State Northridge, in the American Journal of Physics entitled "School math books, nonsense, and the National Science Foundation (http://www.csun.edu/~vcmth00m/).

Most importantly, what the response from the high school does NOT say is that "we have tested the effectiveness of the IMP program in ARHS and have found the following results" -- because absolutely no analysis of the effectiveness of this program has been conducted in our high school. And that is the entire point of my oped, which in fact ended with the following sentence: As part of the upcoming review of the mathematics curriculum in Amherst, we need to objectively evaluate the effectiveness of both the IMP and extensions programs so that we can adequately advise students about both the benefits and costs of their choices, and ideally help all students make choices that expand rather than limit their possibilities. My piece did NOT say "let's immediately end the IMP math program" -- it said we need to evaluate the effectiveness of this program in our school, since some national data is suggesting it has some real problems for college-bound students (particularly those who want to study math or science).

As a member of the Regional School Committee, I pushed hard last fall for an evaluation of K to 12 math, including an examination of the effectiveness of the IMP program. This would be quite an easy evaluation to conduct: since we currently have kids in both the IMP and traditional tracks, one could track their MCAS scores over time and directly compare rate of change (improvement) as a function of which math track was chosen. One could also compare whether one approach was better as a function of gender, or level of math proficiency, etc. Again, this would be a very easy analysis to do, and yet it hasn't ever been done so we really don't know how effective this alternative math program is in general or how effective it is for particular students.

I have been a member of the School Committee for over two years, and a member of the Math Curriculum Council for a year prior to that, and at no time has any high school math teacher (or the HS principal) come to a SC or math curriculum council meeting and requested this type of rigorous evaluation so that we would definitively understand the benefits and/or costs for all or some students of choosing the IMP math track. Yet this seems like an evaluation that all SC members, math teachers, parents, and principals, should agree is long overdue, and highly important for our students.

35 comments:

Anonymous said...

Couldn't agree with you more! It is so important that we evaluate the effectiveness of our math programs. And btw, why is it that no schools in the Valley teach Singapore math? It's a widely heralded model and requires a good deal of critical thinking. It's used in several high-performing public school districts, including the district of Scarsdale, New York.

TomG said...

If you are looking for a consensus on whether we should evaluate, then you have my support.

Go further!

1) Ask SC members, math teachers, parents, and principals, to agree to conduct the study.

2) Ask how important they think it is for our students.

Anonymous said...

Get students feedback on these classes! I thought the letter from the student in today's Bulletin was well-developed and made a strong argument for academic choices at the high school, including math. And you, Catherine, make a good argument for evaluating traditional and IMP math so that the costs/benefits of the various academic choices can be clear for students and their families.

Like Darius Peyton's oped, these students continue to impress me with their maturity and their thoughtfulness. We ought to make sure that even if we don't share their opinions, that we support their speaking up and speaking out on behalf of their education in Amherst. Their letters also tell us that overall, the high school is doing a very good job.

Anonymous said...

There's been more constructive public dialogue in Amherst about what's being taught in our schools in the past 2+ years than there was in the previous ten years.

Gee, I wonder why that is.

The thoughtful Mr. Peyton is a graduate of Crocker Farm School, where many fine students have gone over the years.

Rich Morse

Anonymous said...

In the early 1970's, I took the traditional math program in Northampton public schools, including calculus senior year. I took the SATs and did well in the 700s on three math tests, including the Achievements. I then took two math courses at Catherine's employer, Amherst College, and bombed miserably. (I did pass.)

So I was skeptical about traditional math. My child is taking IMP, and, from what I can tell, her math scores have NOT suffered for it.

I agree that an evaluation would be beneficial about a number of aspects of math (including how parents are presented with the choice), but agreeing on what the standards of evaluation are going to be may be tricky. Relative performance on testing may not capture all of the benefits of IMP.

Rich Morse

Anonymous said...

I think we should stay away from making statements such as, "...overall, the high school is doing a very good job." In fairness, we should also stay away from generalizing that the HS is doing less than a very good job.

In reality, many students have a great experience at the HS, some students just so-so, some students have a bad experience and some drop out. Some of these scenarios are within our control and others are not.

Let's continue to reflect on and evaluate our individual programs rather than making sweeping generalizations about the overall value of the HS.

Looking at IMP so that students and parents can make more informed choices makes sense. Especially because IMP could be a great choice for certain students and a bad choice for other students. Just as whole language instruction is a good choice for some and a bad choice for others.

Anonymous said...

on evaluation of IMP, math, and anything else:
Rich Morse says, "agreeing on what the standards of evaluation are going to be may be tricky. Relative performance on testing may not capture all of the benefits of IMP."

So true. And applies to more than just math.

Anonymous said...

I wish that rather than attack Catherine, people would read and comprehend what she is writing. She wants quantitative data on IMP, not anecdotal stories from students or parents. The data shows that kids who took IMP in high school did not do well in college; don't shoot the messenger. Thank goodness we have a school committee member who digs into this stuff. This is a first! Thank you Catherine! akab

Tom Porter, Amherst said...

Yes, Mr. Payton's letter indicates a strong writing style, clear devotion to the school system, and a commitment to the best academic outcomes for those who struggle. I commend him for stating his case. And as a fellow Crocker Farm grad ('70), I salute him and wish him great rewards for his courage and hard work in his coming senior year, and beyond.

This said, the letter does suffer from a few small flaws, including a number of instances of a priori reasoning and some ungrounded assertions. In particular, as one who attended the event I would disagree with the statement "I barely saw any people of color walk across the stage at the 2009-2010 Underclassmen Awards for Science, English and Math at Amherst Regional High School." This was a tremendous night of public recognition for high achievement, with apparent broad, strong representation by students of all backgrounds.

However, in the case of this as with other possibly inflammatory assertions - such as the notion that there need to be more teachers of color (an aim highlighted by SC member Rick Hood during the election campaign, if I remember correctly), I wonder if ARPS would help itself by actually counting and posting the figures.

I have no idea what such an analysis would show, but to do so would lend a factual texture to the ongoing discussion of Dr. Rodriguez' challenge that Amherst improve itself.

Ed said...

For what it is worth, UMass has something like 20-30 sections of remedial math each semester. That comes to about a fifth of the Freshman class needing to learn high school math before they can do their college work -- and these are all kids whose high school grades (in everything including math) were good enough to get into UMass....

todd h said...

from such a community as we hope to be, come first year college, all our children should be testing out of freshman math, not retaking high school.

couldn't be simplier, to be prepared.

Anonymous said...

Instead of just testing the students, the teachers and administrators should be regularly testing the programs and teaching methods they use with students. It should be a near constant cycle of analysis, reflection and correction, if needed. We also should look to other schools for ideas and to see what they are doing effectively to avoid getting stagnent.

Nina Koch said...

Catherine,

Can you provide a link to a source for this statement?

"Some evidence now suggests that this approach didn't really stand the test of time -- meaning that when students trained in IMP entered college-level math classes, they struggled considerably."

What kind of evidence are you referring to?

Anonymous said...

Surprise surprise, Nina doesn't like the evidence presented about IMP not standing the test of time.

Nina Koch said...

I don't believe any evidence was presented. That is why I asked my question.

Anonymous said...

I'm sure you don't Nina. That's the problem.

Nina Koch said...

Maybe I missed something. Can you point me to the place where you see the evidence presented?

Anonymous said...

I can't speak for Catherine's evidence. I did find a review done by a U.C. Berkeley professor. math.berkeley.edu/~wu/IMP2.pdf

I have to apologize to you Nina for my tone. You ended up being the target of my years of total frustration with the schools here. I sincerely apologize.

At the end of the analysis of IMP, the professor basically says that sometimes you have to memorize formulas. To be exact, and to be able to be scientific, you have to know formulas. They aren't fun. Ask any high school football player how much they like memorizing football plays? They don't. Same goes for math. It's not fun, but if you're going to be scientific, not anecdotal, you have to know formulas.

Nina Koch said...

Thanks, 8:08, for the link to Professor Wu's critique of IMP. It certainly is true that there are college professors who don't like reform math, either at the k12 level or at the college level. And then there are also professors who really want to see math taught differently, such as Deborah Hughes Hallett of Harvard who wrote the most widely used textbook for reform calculus. I'm sure Wu has a critique of that text also. Wu lays out some arguments based on a certain set of values that may or may not be shared by others.

Professor Wu does not, however, offer any of the evidence that Catherine appeared to be referring to. Commenter akab said above "The data shows that kids who took IMP in high school did not do well in college; don't shoot the messenger." But where is this data? No one has provided a reference to it. Yet akab is convinced it exists, based on Catherine's original post.

Catherine, I feel if you don't have a source for this evidence, you shouldn't be mentioning it.

Catherine A. Sanderson said...

Nina - as I've said before, I have been sent papers by colleagues on IMP, which is what led to my concerns. One was by Professor Wu (Berkeley); another was by Professor Milgram (Stanford). I imagine you (or others) could google IMP and Milgram or IMP and Wu and read the whole papers yourself (and I believe I've given both of these cites on my blog before).

But here's the key -- surely you want to know whether IMP works, right? I have suggested, based on some concerns by math professors in California, that it might not work as well at preparing kids for college math/science. So, I have suggested an EVALUATION of it. I am having trouble imagining that you aren't supporting a rigorous evaluation of the IMP program so that we learn whether this program does work at ARHA -- that's the whole thing. I would imagine you, and other math teachers at ARHS, would be pushing urgently for an evaluation of IMP ... which would show whether it does or does NOT work as well as a traditional math approach. I have yet to hear from you, or any math teacher at ARHS, that you are in favor of such an evaluation ... which I find concerning.

Nina Koch said...

but you don't have any data about the performance of IMP students in college, right? Yet you suggested that you did. I'm just trying to get you to stop making misleading statements.

Anonymous said...

Nina, the problem is, some kids DO want to get into Ivy League colleges. Maybe that's not what the rest of the town wants, but some kids do want it. The school should offer AP classes or any accelerated classes those kids want. We shouldn't have to turn to private schools for that, which is what we have to do at the present time.

Anonymous said...

Whether they want to or not, anyone in medicine is a teacher. That's just the way it is, from doctors to nurses. They all teach. They only practice and teach Evidence Based Practice. Evidence Based Practice is the product of studies, peer reviews and finally publishing these results. Why shouldn't curriculum go thru the same tough standards?

Anonymous said...

My sense is that the argument about IMP is going to be a lot more complicated and involved than a lot of other debates we could have. It may be worth doing, but I see LOTS of sticking points.

I have a bunch of initial questions:

1) Are we actually implementing the whole IMP curriculum?
2) Just what's so wonderful about the traditional math curriculum?
3) How do we know that what is being evaluated in standardized testing either in Massachusetts or nationally has anything to do with THE broadest and deepest understanding of mathematics and how it works in our world?
4) Isn't there a way to eliminate the fork in the road and combine the best aspects of both math programs?

My sense is that deciding "whether IMP works" or not, however that is defined, is probably a very fair fight. And it involves not just looking at how IMP students "perform", but it also involves looking at what we think is being taught, whether it's actually being taught, and deciding simply whether we in this community like it or not. In short, I don't think that this argument is just about data.

My impression is that IMP teaches a more holistic, more problem-solving approach to math than the traditional program. I don't understand why so many parents have voted against it with their decisions for their children in Amherst. After conversations with a number of people teaching math outside of the Amherst system, we voted YES and, although the question is now out there, I'm not seeing YET that we made a mistake.

Rich Morse

Catherine A. Sanderson said...

Two quick responses:

Nina - Professor Wu, math department at Berkeley, has written a paper describing his frustration teaching kids trained in IMP when they enter college (an earlier anonymous poster has given the precise link - THANKS!). Other math professors have reported similar problems (including professors at U Mass - ask around!). That concerns me. I would hope it would concern you.

But again, there is an easy solution - let's examine how kids at ARHS do in math/science as a function of whether they are in traditional or IMP math. Then when parents or students ask about differences in preparation, we can answer them honestly -- with data. I am still waiting to hear your support for an evaluation.

Rich - good questions. And they could be answered -- both with kids at ARHS now, and at kids who attended ARHS and have now gone on to college. We could test, for example, how kids in both math sequences do in AP calculus (grades and AP scores), how they score on SAT-1s and IIs, how they do in science (e.g., AP physics, which requires a fair amount of math background, etc.).

I continue to wonder why asking for information on how something is working is restricted so strongly. Don't we ALL want to know how our students are doing?!?

Anonymous said...

I'm not fond of some of the Big Casino aspects of picking courses in the Amherst Regional schools.

Rich Morse

Anonymous said...

it is too bad professor wu is having a hard time doing his job. but anecdotal stories of being frustrated, especially from one professor, are not evidence. it sounds like professor wu is struggling, and that doesn't say anything about his students.

and "asking around" isn't going to provide me with any convincing data either.

i hear you, nina.

Nina Koch said...

Catherine,

The paper by Professor Wu is not about his "frustration teaching kids trained in IMP when they enter college". It's a review of the content of the curriculum when Berkeley High School was first testing it out. He just looked at the materials; it doesn't mention data from student performance at college at all. Wu's paper was originally written in 1992, with some updates a few years later. By the way, Berkeley High School is still offering an IMP pathway today.

The fact remains that you had absolutely no basis for this statement:

"Some evidence now suggests that this approach didn't really stand the test of time -- meaning that when students trained in IMP entered college-level math classes, they struggled considerably."

You should not have made it. For one thing, it tips your hand. It shows that when you are calling for an evaluation of IMP, you already have a negative impression of the program before that evaluation even starts. You are not proceeding in inquiry mode.

I do support evaluation of IMP, just as I support evaluation of the traditional approach to mathematics instruction. I think Rich Morse has made some very good points and asked some excellent questions. The most important thing is to establish our criteria when we are measuring the success of a mathematics program. A rigorous evaluation, in my view, entails multiple factors, not just student performance on a standardized test. We have to decide what we want our kids to be able to do and what we value. For example, we might decide that it's important for kids to be able to explain their thinking and their approach to solving a problem (both orally and in writing). Then we would have to decide how to measure that, and it won't be easy. The MCAS open-response questions would give us some measure of that and that might be a place to start.

On a national scale, I don't believe that the traditional math curriculum has performed very well. Do college professors complain about their students' preparation? Yes, but it's students from everywhere, not just one particular program. They were complaining when I was at UMass in the 1980s before reform curricula had even been written. One half of all the students entering UMass at that time did not pass the placement test to enter Calculus. Ten percent of the students were placed in a remedial, non-credit course to teach basic arithmetic. The US government was writing about A Nation at Risk, Sheila Tobias was writing about Math Anxiety, and John Paulos was writing about Innumeracy before reform curricula were a factor at all. In fact, it was due to these lackings that reform curricula emerged -- to respond to the observed problems in math education!

Nina Koch said...

Catherine,

The paper by Professor Wu is not about his "frustration teaching kids trained in IMP when they enter college". It's a review of the content of the curriculum when Berkeley High School was first testing it out. He just looked at the materials; it doesn't mention data from student performance at college at all. Wu's paper was originally written in 1992, with some updates a few years later. By the way, Berkeley High School is still offering an IMP pathway today.

The fact remains that you had absolutely no basis for this statement:

"Some evidence now suggests that this approach didn't really stand the test of time -- meaning that when students trained in IMP entered college-level math classes, they struggled considerably."

You should not have made it. For one thing, it tips your hand. It shows that when you are calling for an evaluation of IMP, you already have a negative impression of the program before that evaluation even starts. You are not proceeding in inquiry mode.

I do support evaluation of IMP, just as I support evaluation of the traditional approach to mathematics instruction. I think Rich Morse has made some very good points and asked some excellent questions. The most important thing is to establish our criteria when we are measuring the success of a mathematics program. A rigorous evaluation, in my view, entails multiple factors, not just student performance on a standardized test. We have to decide what we want our kids to be able to do and what we value. For example, we might decide that it's important for kids to be able to explain their thinking and their approach to solving a problem (both orally and in writing). Then we would have to decide how to measure that, and it won't be easy. The MCAS open-response questions would give us some measure of that and that might be a place to start.

On a national scale, I don't believe that the traditional math curriculum has performed very well. Do college professors complain about their students' preparation? Yes, but it's students from everywhere, not just one particular program. They were complaining when I was at UMass in the 1980s before reform curricula had even been written. One half of all the students entering UMass at that time did not pass the placement test to enter Calculus. Ten percent of the students were placed in a remedial, non-credit course to teach basic arithmetic. The US government was writing about A Nation at Risk, Sheila Tobias was writing about Math Anxiety, and John Paulos was writing about Innumeracy before reform curricula were a factor at all. In fact, it was due to these lackings that reform curricula emerged -- to respond to the observed problems in math education!

Anonymous said...

To July 12, 10:51 pm: Professor Wu also wrote his critique of IMP in conjunction with these other professionals, quoted below. Do you think all these people also had difficulty teaching college math to their students?

"I am very much indebted to my friends whose help in various forms back in 1992 made
the writing of this review possible: Professors P. Chernoff, T.Y. Lam, B. Parlett, M.H.
Protter, J. Sethian, and P. Vojta of UC Berkeley, Dr. Ian Brown of Lawrence Berkeley
Laboratory, and Professors S.Y. Cheng and R.E. Greene of UCLA. I should also record
the list of the people whom I have consulted in the writing of this review and thank them
all for their courtesy, without implying in any way that they are responsible for the factual
accuracy of what follows: Ms. Lynne Alper, Professor Dan Fendel, Ms. Sherry Fraser, Ms.
Heidi Boley, and Mr. Harvey Garn. Finally, special thanks are also due Mr. Todd Boley
and Mr. Fred Dunn-Ruiz for their critical comments which led to significant improvements
in the exposition. 1"

Anonymous said...

Catherine,

I was getting an error message on submission, so mine went through twice. Can you please remove the second one and also not publish this?

Thanks,
Nina

Janet McGowan said...

Since Catherine and Nina and Rich, and probably many others, agree that the IMP program at the high school should be evaluated, I am wondering what is the next step. Asking the math department head for a study? Or Mark Jackson? Or designing the study? Going to the School Committee?

Also, is there a School Committee policy about regular evaluation of programs--or should there be one? If there is one, how is that implemented and how does a specific program get in line?

Anonymous said...

Janet,

There is a K-12 math review going on now, with a report due in October.

Anonymous said...

Is this the same thing as a study or program evaluation?

Anonymous said...

your link to the northridge report/commentary isn't working.