July 13, 2009
Charity Navigator's New Course
As president of the nonprofit watchdog group Charity Navigator, Ken Berger is accustomed to hostile audiences.
And he was indeed met with some skepticism and a flurry of questions from international nonprofit officials at the annual conference of InterAction last week, where he spoke about his efforts to change the way Charity Navigator rates nonprofit groups.
The watchdog organization has long been criticized for focusing too much attention on how much charities spend on administration. One nonprofit official who spoke at the InterAction conference, for example, said his group was so concerned about high administrative costs that it was reluctant to spend money keeping staff members safe overseas.
Mr. Berger said he understands such concerns and wants to find ways to resolve them. He said that Charity Navigator intends to modify the system it uses for rating charities’ financial health, and also include accountability and openness, as well as “outcomes,” in how it assesses nonprofit groups.
Mr. Berger, who joined the nonprofit watchdog last year, said he’s working with a group of charity leaders, grant makers, and philanthropy experts known as the Alliance for Effective Social Investing to determine ways to assess charities’ effectiveness and their impact on the communities they serve. (See The Chronicle article on the alliance).
He stressed that the process was still in its early stages, and he welcomed feedback from charity officials in how to shape a revised ratings system.
Many questions remain. For example, he says he’s just beginning to wrap his head around how to evaluate advocacy organizations.
Maliha Khan, director of program impact at CARE, said she was concerned that her charity would be less open to self-analysis and self-evaluation if it knew its mistakes would be aired on a site such as Charity Navigator.
Mr. Berger told Ms. Khan not to worry: Any system Charity Navigator developed, he said, would be so “macro” that it wouldn’t get into the sorts of evaluations that Ms. Khan is undertaking at her charity. He said the fact that CARE has hired Ms. Khan to oversee efforts to assess the effectiveness of its programs, for example, means that it’s already far ahead of most charities and would perform well on the evaluation system Charity Navigator developed.
Mr. Berger said Charity Navigator would rely on charities to establish their own measurements and standards, rather than imposing standards on them.
But many in the audience wondered if trying to produce easily digestible, four-star-style ratings based on effectiveness was an impossible task. Some thought charities’ missions were too diverse to compare.
“I feel sorry for you,” said one woman in the audience. “You are comparing apples and oranges.”
“And also vegetables and steak, too,” said Mr. Berger.
He said donors want to know whether they are giving to strong charities and that if Charity Navigator doesn’t provide this sort of information, “somebody else will.”
What do you think of Charity Navigator’s efforts?
Comments
Commenting is closed for this article.
Previous: New York Grant Maker to Close Due to Decline in Assets
Next: In Defense of Gates Foundation's Work in India
I applaud Ken’s efforts and would like to comment about comparing apples and oranges. There is a way to compare orgs with different missions, goals, and intended outcomes. Regardless of what you intend to achieve there are certain organizational characteristics that can inform a social investor as to the likelihood that the org has any chance of achieving its mission, goals, and intended outcomes. For instance, if an organization doesn’t have a theory of change, doesn’t diligently collect data that informs them of the affect of their efforts, and doesn’t use that data to continuously improve then it’s unlikely that the org will succeed. If we can identify qualities like these then we can more effectively compare and invest in the right orgs. From numerous discussions with Ken I know that this is what he is now exploring.
— Jeff Mason Jul 13, 10:26 AM #
Using the analogy quoted by Caroline, there is a common quality to “apples, oranges, steak and vegetables” – they are all food.
So too in the nonprofit sector, as Jeff Mason notes above, there ARE certain qualities that all share that can be compared. The three broad categores we plan to look at going forward are financial health, accountability (including transparency) and the potential for outcomes. The Social Value Assessment Tool on the Resources page of the Alliance for Effective Social Investing web site (www.alleffective.org) is a perfect example of how this might work in the case of potential for outcomes.
Ken Berger
Charity Navigator
— Ken Berger Jul 13, 02:28 PM #
My problem with Charity Navigator is that it will change the numbers on an organization’s Form 990 if it doesn’t agree with the overhead allocation. Even if the underlying financial statements are audited. No where on the Charity Navigator site does it disclose that they do this. Furthermore, admin costs are a necessary part of a nonprofit’s mission – how can a charity be effective without infrastructure?
— kn Jul 13, 03:35 PM #
It is indeed a fruit salad of comparisons, but I can’t think of a better approach than that of Charity Navigator. I view CN as a service to investors vs. a ‘gotcha’ type of watchdog. Its good and needed to have a light shone on the sector and even better that CN works hard to evolve on behalf of the sector at the same time. Strong non-profits strive to make themselves fully accessible to and conversant with their supporters, and should view CN ratings not as punitive but a good reason to show they are worthwhile stewards or seeking to be better.
— Tim Deuitch Jul 13, 03:37 PM #
KN is correct about the numbers game that Charity Navigator plays behind the scenes. They take it upon themselves to determine whether the 990 is accurate and will not hesitate to change numbers and percentages based on their adjustments to the audited numbers. Unfortunately, anyone reviewing the statistics will not be informed that any changes were made to the actual numbers. CN doesn’t mention this in their evaluations and they present them as ‘objective.’
— BB Jul 13, 04:15 PM #
The problem with Charity Navigator and other organizations like it is – at some point a line has to be drawn. No matter where that line is, or how it is defined, there will always be people on the other side thinking it’s unfair.
That having been said:
1 Apples and Oranges might look different. But, I am told by my more science oriented children, they are virtually the same chemically. We need another metaphor.
2. As a career fundraiser, I often have difficulty with people who just use ratios. Depending on the cause and the geographic location of it (as in city or community) the costs involved of mounting a truly successful fundraising campaign can vary greatly. Also, if you are at the beginning of building a fundraising program, your cost ratios will generally be higher than when managing a more mature fundraising program.
Charity Navigator has established a yardstick and measures all of us in a somewhat equal fashion. That is merely one measure of effectiveness. I am certain, however, that Mr. Berger and his team take their job of observing and reporting stewardship of donated funds as seriously as we who raise those funds take our jobs. So, let’s continue to work through the issues as we clarify and perfect systems of analysis. Because, in the final analysis, if we don’t continue to improve performance and measuring performance, we will all end up in the same barrel with the scam artists and thieves of which we have so recently become acutely aware.
— Sam Prince Jul 13, 04:20 PM #
You know, for decades people formed community-based organizations because they saw a need and had the desire to address it. They brought together people who shared their sense of mission. Some succeeded and continued to this day. Some succeeded and therefore put themselves out of business—a great result. Some failed and kept on going out of inertia. Some just failed. But until recently, none of them. whether they succeeded or failed, spelled out a “theory of change,” and few of them could collect as much data as we now expect to do. It’s a bit presumptuous to think that one trendy bit of conceptual apparatus can be the be-all and end-all of evaluation—no matter how useful it may be for organizations doing our own planning!
Some foundations ask grant seekers to state what we, the organizations on a mission, consider success. That’s a much more appropriate and valid approach. If Charity Navigator and other organizations would find it impossible to do that for all nonprofit organizations, then that’s a problem with their approach, not with the organizations.
— Dennis Fischman Jul 14, 09:22 AM #
I believe that the nonprofit sector at all levels has to intentionally work with an organization like Charity Navigator. Charity Navigator needs to recognize the dynamics involved in nonprofits, especially the faith-based nonprofits, in their efforts of doing good. No organization is perfect! The faith-based nonprofits are often getting a ‘bum rap’ by the watchdogs which they often don’t deserve. If everyone of the people of faith put their donations where their claim of faith is, the faith-based would dominate the industry, but the faith-based trails far below the non-faith based nonprofits in donations. I am waiting very patiently for the watchdog organizations to come up with a reasonable explanation for this. Another issue I have with the watchdogs: Why are they buying in so boldly to the prejudice against faith-based nonprofits in the media? Why is there such a prejudice in the non-faith-based nonprofits against those who are or have worked in the faith-based? The truth is, faith-based do a whole lot of good, with a whole lot less! But, why should they? Why is it that the media and the market assumes that all faith-based ministers and administration have taken a vow of poverty? We haven’t for the most part. Often times we in the faith-based nonprofit consider ourselves in ministry roles, but this does not mean we do everything for free. For instance, I have taken over 10 years of full-time college and many more years of part-time college credits. I have spent close to $100,000 on my education. Many of my faith-based colleagues are just like me. We have educated ourselves into excellence, but we are very often treated as though our education is not up to the standards because it is faith-based and therefore tainted! We are far too often passed over for quality development jobs because people somehow believe ministers and/or people who have worked for churches and Para-church ministries have no real marketable skills. Or, they think that we are out to proselytize even their company, to somehow turn it into a church. Like we don’t understand the role required of us as we work amongst the non-faith based organizations. Why is this true? Perhaps they really have faced idiots in faith-based clothing before. I am not an idiot, and I have met a few of those myself. This should not be an automatic response when looking at a resume from someone like me.
— Pastor Chuck Cunningham Jul 14, 01:42 PM #
I applaud what Charity Navigator is hoping to do down the road. Ken, I have a friendly but earnest challenge – while you are waiting to find other ways to evaluate nonprofits, stop using the administration:program ratio now. It is unjust, discriminatory, inaccurate, broken and dangerous. The argument I hear for keeping it is “it’s the only measure we have.” That’s like taking your baby’s temperature with a broken thermometer because it’s the only thermometer you have in the house. It’s insane, literally. Others say, “it’s the only thing we have to weed out fraud.” The Supreme Court has ruled – twice – that there is no nexus between fraud and high overhead. Besides that, what perpetrator of fraud is going to report it in line item detail on their Form 990? Charity Navigator could transform the conversation overnight. Transform it. It could send a roaring wake-up call to the media, challenge all of the other watchdogs and advance the conversation by years if it were to issue a major press release tomorrow saying that it is discontinuing its use of the ratio. This would take enormous courage, but believe me Ken, it would pay off in spades, not only to CN, but to the entire sector and all of the causes we care about. I challenge Charity Navigator to lead – boldly, and audaciously. Surprise the world. That’s the only way it ever changes.
People might call you crazy. What’s crazy is to continue using a measure that we all agree does not work simply because it’s the only measure we have. We’d be far better off without one than to have one that does all of this damage.
— Dan Pallotta Jul 14, 04:09 PM #
While I applaud a focus on effectiveness, I simply don’t see how this can be reduced to a rating system that fits nicely into the Charity Navigator format.
Outcome evaluation is extremely complex or we’d have solved this problem of measurement long ago.
Rather than starting with the relatively easy measures, I dare Charity Navigator to start with the hardest: Arts, cultural and humanities organizations; advocacy organizations; prevention organizations.
Much social change is the result of complex interactions of systems of nonprofits, from the national or international stage down to the neighborhood level. The smaller the organization, the less likely they will have clearly defined instruments that measure their “effectiveness” and thus they will continue to be penalized as they are now in the administrative cost rating system.
Or how about highly complex organizations such as UNICEF, which Givewell decided was “not recommended.” If this is what we can expect, we are in deep trouble.
I’ve written about some of my concerns in articles like “Measure the Right Thing” on my blog The Butterfly Effect http://tinyurl.com/dlewj2
Yes, we can and should hold many individual organizations accountable for where they spend their dollars and the quality of their investments. But there really isn’t an easy way to “rate” effectiveness to channel donors. Donors are going to have to do more work investigating organizations. Sorry. Also, if we get distracted and only focus our attention on what nonprofits donors should invest in, as a community we distort the massive, interlocking systems changes needed to dramatically move the needle for all.
Funding the best of charter schools won’t change public education as long as charters enroll miniscule numbers of kids. Funding a few individual organizations that adopt child survival isn’t enough to eliminate communicable disease when 100% of kids need to be immunized.
We must hold accountable whole communities, states and countries for the investments they make in the outcomes we profess to desire. Our society must invest in practices at the scale needed to reach every kid or every adult and not just the fortunate few who may win the nonprofit assistance lottery.
While I believe in the power of nonprofits to change lives, I also know that our institutions are a small part of the picture. The easily measured usually serve as band aids or incubators. It’s a lot harder to measure the efforts of the advocates or catalysts for widescale change.
— Gayle Gifford, ACFRE Jul 14, 05:07 PM #
It’s unfortunate that so many in the nonprofit world seem to resist any meaningful move toward greater accountability and transparency. If you’re going to ask people to just GIVE YOU MONEY then you better be able to prove to them that you are 1) HONEST and 2) EFFECTIVE. Darn few nonprofits can do both. Yet there seems to be a tendency in this sector to fight tooth and nail against all efforts toward this outcome. There’s endless whining about why “we can’t do that” or “we can’t do this” – whining very reminiscent of the commercial sector every time they’re caught spewing filth in a river or conducting another financial scam and there’s a call for greater regulation. The nonprofit world better wake up and realize that it’s on trail – on trial in the court of public opinion – and the verdict is in doubt.
Asking a nonprofit to define the guidelines by which it will be evaluated (intimated at in comment #7) is like asking a convicted criminal to set his/her own sentence. It’s nonsense. Do employees get to set their own salaries? Do students set their own grades?
I agree that the administrative/program services ratio is a faulty mechanism, but it is not entirely without value. Interesting that Mr. Pallotta is so adamant about throwing it out entirely instead of working to refine it and improve its effectiveness. (One wonders just exactly what he would do with the bath water and baby of lore.) Part of the difficulty in the overhead ratio is defining the constituent elements that comprise program services vs. administrative costs. I’ve analyzed many 990s where I thought costs included in admin overhead could have been legitimately booked to program services.
Ken Berger has impressed me by his willingness to buck the traditional paradigms of the third sector and suggest it might be possible to develop better models. Nonprofit leaders can decide who they’d rather partner with to improve nonprofit models of accountability and governance – Ken Berger and groups like Charity Navigator, or federal and state regulatory agencies.
There needs to be a major push to educate the public about how the nonprofit sector actually functions. Educated donors and supporters are the best safeguard against nonprofit fraud and mediocre results. Even people employed in the sector occasionally display an amazing lack of knowledge of its history and underpinnings. For instance, with all the emphasis on donations, how many of you are aware that more than 50% of all the revenue entering the sector in the US is a result of fees charged by nonprofits?
And I don’t know where Pastor Chuck (comment #8) obtains his data, but as far as I know, the overwhelming majority of private philanthropy in this nation goes to churches and church-related organizations.
— Benton Carson Jul 15, 02:40 PM #
As a foundation grantor, I applaud CN in their effort to publish a charity’s measures of their effectiveness in meeting their goals. I recently read the Mulago Foundation’s impact measures and think this is what we are looking for also. If a charity says it’s reducing poverty, it’s up to that charity to use measures that measure that, not measure their methods or activities. CN can just publish what they say, not evaluate it. We grantors will evaluate it. If a charity offers something less than a valid measure of impact, we’ll just support one who does.
— Jim Gennrich Jul 15, 08:21 PM #