Improving Our Program through Evaluation

“Our primary goal for the Postdoctoral Fellowship Awards is to draw new researchers into the breast cancer field. Our survey indicates we are succeeding.”

California taxpayers deserve to have the funds they provide for breast cancer research spent wisely. At the CBCRP, we're committed to evaluating our program to make sure we're doing the best job we can. We're planning a major evaluation of our entire program over the coming years. During 2001, we got started in two ways. First, we evaluated an important component of our work by taking a close look at one type of grant we make, our Postdoctoral Fellowship Awards. Second, we completed work on a survey of outof- state experts who review our research proposals.

Evaluating CBCRP Postdoctoral Awards

The CBCRP's Postdoctoral Fellowships go to scientists in training who work under a mentor on breast cancer research. The grants-up to $80,000 for as long as two years-are designed to prepare researchers for a long-term professional research career. The CBCRP has invested almost $6.4 million in 94 postdoctoral awards since 1995. They make up 20-25% of the grants we award.

To evaluate the Postdoctoral Fellowship Awards, we surveyed researchers who were CBCRP postdocs during the first four years of the program.

Our primary goal for the Postdoctoral Fellowship Awards is to draw new researchers into the breast cancer field. Our survey indicates we are succeeding. Forty-four percent of our former postdocs used the CBCRP Award to gain first-time experience in breast cancer research. Of these, 70% have stayed in the field, and 27% say they would not be involved in breast cancer research today without the CBCRP's support. Moreover, 62% of the CBCRP's former postdocs say the grant helped them stay in the breast cancer field.

Another goal for our Postdoctoral Fellowship Awards is encouraging high-quality researchers from a variety of disciplines. We are meeting the first part of this goal well. Our survey turned up a number of indications that we funded high-quality researchers. As a result of research at least partly funded by the CBCRP fellowship, 39 former postdocs published a total of 108 articles in scientific journals, gave 78 presentations at scientific conferences, and received four patents. Seven former postdocs received a total of nine awards for this work. However, the CBCRP fell short on another part of this goal-encouraging research from a variety of disciplines. The majority of Postdoctoral Fellowships went to researchers in the field of pathogenesis. The CBCRP is taking steps to increase postdoctoral fellowship training in other fields.

When we began making Postdoctoral Fellowship Awards, we hoped the awards would help develop the careers of new researchers. We seem to be succeeding here. More than half of our former postdocs told us the fellowship helped them gain a higher-level job, recognition, self-confidence, and skills to become a better researcher. We also hoped the Postdoctoral Fellowships would help launch promising new breast cancer research, and they appear to be doing so. Our first four years of Postdoctoral Fellowships have led to various funding agencies providing over $36 million in grants to continue avenues of research begun with our funding.

Along with letting us know that Postdoctoral Fellowships are generally succeeding, our former postdocs also made a number of suggestions for improving the program. These included making grants to researchers who want to work part-time, offering training in grant writing to postdocs, and getting the word out about these awards so more researchers will apply. The CBCRP is investigating ways to put these and other suggestions from former postdocs into practice. We plan to evaluate the awards again in four or five years.

Surveying Our Reviewers

Every grant the CBCRP makes must pass a review by out-of-state research scientists and breast cancer advocates. Together, these reviewers make up a pool with a high level of expertise and knowledge about breast cancer. During 2001, we tapped this talent pool with a survey that asked these out-of-state reviewers their opinions on what the CBCRP has achieved and on how we should move forward.

In general, the people who review our proposals agreed with the CBCRP's approach to funding breast cancer research. There was no agreement on the need for any major changes.

This survey's most striking finding is strong support for including advocates-generally women who have or have had breast cancer-in decisions about which research to fund. The advocates told us they learn a tremendous amount. The scientists said advocates keep them focused on the human side of the disease. As one scientist commented, “Most advocates have important and significant information to convey. They may not know the science, but they remind us all of what our end point should be.”

More Evaluation Next Year

Over the coming year, we plan to evaluate more of our program, as part of our continuing effort to target our research dollars in the way that will do the most to make breast cancer a disease of the past.

“This survey's most striking finding is strong support for including women who have or have had breast cancer in decisions about which research to fund.”