• January 25, 2015

Charities and Watchdog Groups Clash Over Monitoring Systems

By Raymund Flandez

New York

Battles between charities and the watchdog groups that help donors decide where to give escalated last week when a major trade association released drafts of two reports by scholars who say the watchdogs may do more harm than good.

The studies, paid for and released by the Direct Marketing Association’s Nonprofit Federation at a meeting here, charge that the watchdog groups use evaluation systems that are confusing and simplistic.

What’s more, they said, because the watchdog groups are influential, charities often take steps that are not smart in the long-term simply to garner high ratings.

The watchdog organizations they studied are the American Institute of Philanthropy, Better Business Bureau’s Wise Giving Alliance, and Charity Navigator.

Leaders of those groups say the studies were biased.

Ken Berger, chief executive of Charity Navigator, took issue with the idea that “our rating system can cause harm.” He charged that the Direct Marketing Association’s studies were designed to “foster the myth that all charities are good, that all charities are fundamentally equal. That is a lie.”

'Severe Shortcomings’

The studies of the watchdog groups took different approaches.

One, conducted by George E. Mitchell, a research assistant at the Transnational NGO Initiative at the Moynihan Institute of Global Affairs at Syracuse University, was based on in-depth interviews with top leaders from 152 nonprofit organizations.

  • Watchdog ratings may be biased against smaller organizations since economies of scale make larger charities seem more financially efficient.
  • The watchdogs “fail to provide adequate guidance about what their ratings measure.” The American Institute of Philanthropy, for instance, gives letter grades but “provides little guidance about what the scores actually mean” and does not “disclose how the raw scores are actually converted into their final designations.”
  • Benchmarks are arbitrary. Charity Navigator scores organizations that work on different missions in different ways. The BBB Wise Giving Alliance adjusts several of its standards if an organization protests or argues for special accommodations, the report says.

The watchdogs aren’t entirely at fault, Mr. Mitchell’s report says.

Lack of good data hinders their ability to put forth information that can be easily consumed and understood by donors. Most of the watchdogs rely on the IRS Form 990, which has financial data, and information from charity Web sites.

Overhead Measures

Ms. Sowa’s study criticizes the watchdogs for placing too little emphasis on programs’ operations and outcomes. For instance, their requirement of “low overhead as a measure of effectiveness” means that many nonprofit organizations decide it’s wise to offer low salaries for managers when they need highly trained managers “more than ever.”

“Financial health is important. However, it’s only one measurement of a broad spectrum of how organizations work,” says Ms. Sowa.

“It’s a delicate balance between holding nonprofits accountable on how they spend but not overemphasizing thrift for the sake of performance.”

She suggested that the watchdogs could use an accreditation process to evaluate and validate nonprofit organizations, require charities to release more information, and add more detailed information on their Web sites to counter the shortcomings of the rating system.

Focus on Results

Charity Navigator and the Wise Giving Alliance agree with the scholars that more needs to be done to focus on a charity’s accomplishments in meeting its mission.

But that is no easy task, Mr. Berger says.

“The sad state of reality is that there are no standardized agreed-upon measures of effectiveness that most charities gather,” he says. “The data isn’t quite there yet.”

His group announced two years ago that it would revamp its ratings system to look beyond nonprofits’ financial health and is now working to grade charities’ accountability, ethical practices, good governance. and transparency. Donors will see the results of those changes early next year, he says.

H. Art Taylor, chief executive of BBB Wise Giving Alliance. says the Wise Giving Alliance is working with GuideStar, a site that collects 990 data and other information on charities, and Independent Sector, a coalition of charities and foundations, to develop a format for organizations to report their effectiveness. It expects to release a new reporting format next year.

Mr. Taylor says the characterization of his group as making arbitrary exceptions is flawed. But, he says, some charities do ask to be excused from one or more of the 19 standards the alliance uses to rate charities. “We’ll certainly hear what the charities have to say,” he says. “If they’re legitimate reasons, we’ll certainly make some adjustments, or not.”

Lack of Disclosure

Daniel Borochoff, president of the American Institute of Philanthropy, says the biggest issue for charities and the watchdogs that review them is accountability.

“There’s a lot of bad reporting and poor disclosures going on,” Mr. Borochoff says. “Certainly, we’re open to realistic ways in which to improve,” but, he says, “for them to somehow say that financial ratios are not something of interest or something that donors don’t need in order to consider making a donation, it’s just stupidity. It’s very misguided for nonprofits to belittle the importance of the finances and how they’re spending their money.”

The Direct Marketing Association was not surprised by the reaction from the watchdogs. Christopher Quinn, executive director of the Nonprofit Federation, said in an e-mail: “These studies speak for themselves, and it’s only fair that so-called 'watchdogs’ are held to the same scrutiny as they expect of others,” he wrote. “There has to be a better way to evaluate charities than the current flawed and varied systems.”


1. charitywatch - September 01, 2010 at 02:23 pm

The George E Mitchell study is seriously flawed and it was NOT based on in-depth interviews with top leaders of nonprofits but rather on off the cuff remarks from a few who were being interviewed for other studies. According to Mitchell the leaders were not specifically asked about the watchdogs.

The American Institute of Philanthropy clearly discloses in detail its criteria for ratings here: http://charitywatch.org/criteria.html and in each edition of the the Charity Rating Guide and Watchdog Report.

DMA members, who promote inefficient and wasteful fundraising practices will not fool the donating public by hiring, in the author's own words, "a starving student" to write a highly biased report. AIP was not even contacted about the study before it was released and the author admitted to me that he had only "poked around" our web site.

Daniel Borochoff
American Institute of Philanthropy

2. vincegabor007 - September 02, 2010 at 04:28 am

<Comment removed by moderator>

3. mwolcott - September 02, 2010 at 09:47 am

Good for the DMA for taking this head-on. Not sure about the level of expert research but the writing is clearly on the wall. Almost makes me want to re-up my membership to help support such research. I'm sure Palotta is smiling somewhere out there too.

4. cragusa1 - September 02, 2010 at 10:45 am

The study points out the limitations of the watchdogs, and in that way it is in no way flawed. If one reads the entire study (which I have) you will see it is not biased at all.

The consumer is mislead in thinking they can simply look for stars or seals and know that an organization is doing what they've promised, and that is simply not the case.

I hope the media take hold of this study and start the discussion about what consumers REALLY need to know before they donate.

5. gbf2102 - September 08, 2010 at 03:26 pm

Ratings agencies, for securities and nonprofits alike, survive because private individuals erroneously believe that they somehow have more information, better information, or are more analytically qualified to design metrics and make judgments. This debate, like the recent financial crisis serve as reminders that there is no substitute for doing your homework, and no satisfaction greater than knowing that you took the time to form your own opinion.

Add Your Comment

Commenting is closed.

  • 1255 Twenty-Third St., N.W.
  • Washington, D.C. 20037
subscribe today

Raise more money and increase awareness with trusted insight.