Rootcause
Peer Performance Exchange

Independent Program Analysis Methodology

Overview


Root Cause's methodology for analyzing nonprofits is based on the theory that organizations which use evidence-based best practices for their programs and organizational systems are most likely to achieve the highest possible level of outcomes. The building blocks for healthy organizations are: financial sustainability, strong management and governance, and effective performance measurement systems. The building blocks for high-performing programs are dependent on program quality metrics specific to the social issue being addressed. Therefore, our standard framework for analysis includes metrics in two key categories: organizational health and program quality. Each category is comprised of subcategories, indicators, and best practices, which are described below in the assessment details section. Across the framework, analysts review almost 500 best practices to score organizations using a one-to-five scale for each subcategory. Based on the total number of points earned across all the subcategories, organizations earn one of the following levels of recognition.

image description   High-Performing: These programs display an extremely high level of competence in all the program quality and organizational health areas assessed. This level is very difficult to attain.
 
image description   Meets Standards: These programs have effective program models and management practices, meeting the minimum standards for all or most of the program quality and organizational health areas that Root Cause assesses. This level is difficult to attain because meeting the minimum standards requires a firm commitment to excellence and often many years of practical experience.
image description   Trending Up: These programs are committed to improving their program models and management practices. Therefore, they are likely meeting the minimum standards in some areas and are working to meet the standards in others. With some targeted improvements, these programs can be expected to meet all the minimum standards in the future.

Organizations receive two reports that describe the results of the analysis.

  • Independent Program Analysis Report: This confidential and comprehensive report details the results of the assessment and identifies strengths, gaps in service, and opportunities for stronger performance, while also providing actionable information for short- and long-term change. It contains data on all the best practices assessed and provides benchmarking information to help the organization understand where it fits within the context of similar organizations that have also been assessed.
  • Transparency Report: This summary report distills the most pertinent findings from the analysis into a compelling narrative that can be used to attract support in a variety of ways, including on several donor platforms. Members control when and where this report is posted; Root Cause will not share the results of the assessment without express permission from the organization.

Organizations are encouraged to share the results of the assessment with key audiences by displaying their seals on their websites and marketing materials. Organizations also have the opportunity to post their Transparency Report on donor platforms such as GuideStar and Charity Navigator as well as sharing it on their website and directly with funders.

Analysis Process

The analysis process is designed to gather as much information as possible while placing minimal burden on the organizations being assessed. The standardized metrics allow the following streamlined process:

  • Online questionnaire through which members submit outcome data, documentation, and information about the program and organization. Analysts review the information submitted through the questionnaire to prepare for the site visit.
  • Site visit during which analysts conduct a series of meetings with key leadership and staff to gather in depth information about the organization and program, while also verifying information submitted through the online questionnaire. The site visit takes approximately one day and typically includes meetings with:
    • Executive Director and leadership team
    • Program leadership and key staff
    • Individual responsible for outcome data tracking
    • Program Participants
    • Board member(s)

After gathering the data, at least two analysts review the information and come to a consensus about the health of the organization and quality of the program. The analysts then draft the Independent Program Analysis Report. Based on the results of the assessment and the organization’s capacity, Root Cause tailors its engagement for the rest of the membership term (two years) to help members improve program quality and organizational health to meet the needs of the member organization.

Members always have the opportunity to renew their membership and be re-analyzed prior to the expiration of their Independent Program Analysis Reports.

Assessment Details

The graphic below shows the subcategories for organizational health and the three program quality areas assessed by Root Cause. The framework was developed through a rigorous research methodology, during which analysts conducted a literature review and interviewed experts from academia and government, as well funders and practitioners, to determine organizational health and programmatic best practices. The metrics are updated annually.

Organizational Health

Within organizational health Root Cause assesses the following elements. These areas are the same regardless of social issue the organization is addressing.

Financial Sustainability: The organization’s ability to work in a fiscally responsible manner and to plan for future growth with reliable, recurring revenue streams.

Management and Governance: The organizational leadership team’s experience and ability to develop and manage organizational infrastructure, the program leadership’s ability to ensure that the program is effective, and the board’s ability to oversee the organization’s operations.

Performance Measurement: The organization’s ability to collect and use data to make decisions.

Program Quality

Root Cause currently assesses the program quality of programs addressing three different social issues: college access, college persistence, and youth career development. Below are the specific program elements that Root Cause assess for each social issue.

Root Cause has program quality metrics for additional social issues. Please contact Anne Radday to inquire about other social issues that we can analyze. She can be reached at aradday@rootcause.org or 617-649-1536.

College Access Program Quality

College Aspirations and Knowledge: The organization’s ability to instill college aspirations, assist in college application and selection processes, and help participants successfully transition to college.

Academic Preparation and Enrichment: The organization’s ability to assist participants to develop academic and non-cognitive skills required to succeed in college, including academic advising, standardized test preparation, and study skills development.

College Affordability: The organization’s ability to help participants understand and overcome the financial barriers to attending college.

College Persistence Program Quality

Degree Attainment Support: The organization’s ability to work with participants to ensure that they fulfill the college requirements and maintain the financial support needed to complete a bachelor’s degree.

Advising: The organization’s ability to provide guidance and support to participants on personal and academic matters, including on-campus life, creating goals, and developing support systems.

Employment Support: The organization’s ability to assist participants in developing job-skills, gaining work experience, exploring career options, and getting a job after graduation.

College Integration Support: The organization’s ability to help participants adjust to the college environment and find needed resources on or near campus.

Youth Career Development Program Quality

Applied Skills Development: The organization’s ability to assess and provide academic, job, and 21st century skills development to improve participants’ potential for employment and educational opportunities.

Career Exploration: The organization’s ability to assist participants to learn about a variety of career opportunities and engage in a meaningful work experience.

Youth Development Services: The organization’s ability to connect participants with comprehensive social services to address and overcome individual challenges to educational, job, and career success.

Scoring Details

Root Cause analysts score organizations based on the percentage of critical and advanced best practices that they systematically implement using the following scale.. Critical best practices are those that all organizations (regardless of size or age) should implement to ensure strong organizational health and solid program quality. Advanced best practices indicate that an organization has complex organizational systems and infrastructure and delivers high-quality programs through more in-depth and intensive program elements. These organizations are performing at the highest level.