Efficiency of Irish Charities: Insights Through Data Envelopment Analysis

1. Introduction

Ireland has a sizable number of nonprofit organisations (NPOs)^[1], with charities making a significant and growing contribution to this segment of economic activity. In Ireland, there are approximately 11,600 charities (Charities Regulator, 2023) included within the wider Irish nonprofit sector, which has an annual income of approximately €14 billion (Benefacts, 2022). Studies of charity activity by jurisdiction have identified Ireland as a generous nation and one in which charitable activity is pervasive (for example, World Giving Index (Charities Aid Foundation, 2016)). In Ireland, the sector employs more than 100,000 people and has over 560,000 volunteers, with the proportion of charities using volunteers (and relying on such inputs) increasing over time (Irish Nonprofit Knowledge Exchange, 2012).

The growth in the size and influence of charities, combined with several highly publicised governance and fundraising scandals, has led to increased charity sector visibility and public scrutiny (Burke-Kennedy, 2013; Hind, 2017; O’Brien, 2013). Moreover, the aftermath of the 2008 financial crash and accompanying austerity measures, COVID-19 and the cost-of-living crisis since 2022 have intensified pressures on funding and demand for services, creating a ‘perfect storm’ for charities whereby they must do more with less (The Wheel, 2014; Visram, 2022). Given this challenging operating environment and enhanced profile, charity accountability, performance and transparency in performance reporting have received increased attention (Coupet, 2018; Coupet & Berrett, 2019), with performance often analysed as a production process that includes inputs, activities and outputs (American Accounting Association, 1989; Brace et al., 1980; Breckell et al., 2011; Carter et al., 1992; W. K. Kellogg Foundation, 2004). This is discussed further in Section 2.1.

Based upon this production-process concept, two key criteria for judging charity performance are efficiency and effectiveness (Boateng et al., 2016; Connolly et al., 2013, 2018), with charity efficiency being the focus of this paper. The disclosure of performance information by charities has increased over time, leading to complexities in developing meaningful measures. The limited reporting of such metrics – typically of a financial nature and provided through the annual report – makes assessing performance difficult. However, notwithstanding the difficulties associated with performance measurement, with a results-based culture becoming more pervasive, the demand for improved transparency and the provision of efficiency measures as a means of assessing charity performance is growing (Eckerd, 2015; O’Dwyer & Boomsma, 2015).

Data envelopment analysis (DEA) is a linear programming technique that measures the efficiency of a decision-making unit (DMU), which has multiple inputs and outputs. Those DMUs with the optimal outputs from given inputs represent the ‘efficient frontier’ (that is, the most efficient charities in our research). While DEA has been subject to some critique, many organisational types, including charities, have been subject to DEA to assess their efficiency (e.g. the arts (Golden et al., 2012), supply chains and agriculture (Emrouznejad & Yang, 2018)) (see also Section 2.2). Despite the substantial contribution, profile and size of the sector, few studies have explored charities using DEA, with notable exceptions including Coupet (2018) and Coupet and Berrett (2019). As Coupet and Berrett (2019) focus only on Habitat for Humanity-affiliated organisations, a key contribution of our paper is to apply DEA to a larger number of organisations engaged in a variety of charitable activities, with the calculated DEA scores being used to determine efficient charities (see Section 3). We also offer an approach to measuring beneficiaries (number of direct human beneficiaries) and introduce a variable that captures the accessibility of financial reporting information (that is, a measure of transparency based on the clicks to reach the annual accounts) on the basis that the availability of such information to stakeholders should encourage an organisation to use its inputs efficiently.

Using DEA, the objective of this paper is to consider the relative efficiency of charities, specifically whether this is influenced by transparency in reporting. The remainder of this paper is structured as follows. The next section provides an overview of the measurement of charity efficiency to provide context for this study (Section 2.1), together with the application of DEA in the study of organisational efficiency (Section 2.2). Then, the sample, DEA model, regression model and hypotheses are presented (Section 3). Subsequently, the results are reported (Section 4), and some discussion points and conclusions offered (Sections 5 and 6).

2. Charity Efficiency and Data Envelopment Analysis

2.1. Charity Efficiency

Based upon concepts of accountability, stakeholder theory and principal-agent theory (Ebrahim, 2003; Freeman, 1984; Lawry, 1995; Stewart, 1984), and the need for organisations to provide “adequate information to allow stakeholders to assess the overall performance of the charity” (Charity Commission, 2004, p. 2) (that is, transparency), measuring and disclosing good quality performance information is important (Anthony & Young, 2002; Charity Commission, 2004; Connolly et al., 2013; Hyndman & McConville, 2016). For example, it may provide information that enables informed discussion (and decision-making) and improves organisational planning and control systems. Furthermore, by externally reporting performance information, charities can discharge accountability and establish connections with key stakeholders, which may help build external legitimacy (Connolly & Hyndman, 2017; Ebrahim, 2003; Hyndman & McConville, 2018).

As mentioned briefly in Section 1, charity performance and accountability are considered important, with it being possible to consider charity performance in terms of a production process (Connolly et al., 2017, 2018). Various elements of some charity production-process models are illustrated in Table 1 (see also Connolly et al., 2017; Connolly & Dhanani, 2009).

While accepting that there are no standardised definitions for each of the terms shown in Table 1 – inputs, processes (activities), outputs and results/outcomes/impact – the following explanations are broadly accepted (Connolly et al., 2017, 2018). Inputs are the resources used in providing a service (for example, expenditure incurred or number of staff). Processes represent the activities undertaken by a charity (for example, number of visits or projects funded), with outputs being the actual goods or services provided (for example, number of children fed, or projects completed). Results (or outcomes/impact) are the effects of a charity’s activities on beneficiaries and society (for example, change in the level of education or overall level of satisfaction with the services provided). While these elements are presented as distinct, this may not be the case in practice and there may be some blurring at the edges. Difficulties, for example, may arise in distinguishing between an output and a result (outcomes/impact) in certain circumstances.

Drawing on the concept of a production process, charity efficiency (the ratio of outputs to inputs or the amount of input per unit of output), which is the focus of this paper, is an important measure of charity performance that may differ depending upon the nature of the organisation’s activities. For example, it could be the cost (an input) per child fed (an output) in an overseas aid organisation, or the number of cases handled (an output) per employee (an input) in a social welfare organisation. Often such measures are used in a comparative, rather than an absolute, sense. For example, it is not normally said that an organisation is 90% efficient, but rather that it is more (or less) efficient than a comparable organisation, than a prior year or than budgeted for (Anthony & Young, 2002).

Stakeholders (particularly donors) are especially interested in fundraising and administrative efficiency information because they do not want to be associated with inefficient charities (Callen & Falk, 1993; Qu & Daniel, 2021). Moreover, charities spending more on direct charitable activities (and less on other activities such as fundraising, administration or governance) are perceived as more efficient and thus more attractive to potential donors (Parsons, 2003; Qu & Daniel, 2021; Tinkelman & Mankaney, 2007). Consequently, increased attention has been paid to developing and reporting performance metrics, particularly indicators of charity efficiency, many of which have a financial focus (Eckerd, 2015; Epstein & McFarlan, 2011; Hyndman & McConville, 2016).

For example, conversion ratios are often used as proxies for efficiency on the basis that a charity’s primary purpose is to translate funds received into direct beneficiary support, with common examples including program ratios (costs of charitable activity as a percentage of total costs), administration cost ratios (administration costs as a percentage of total costs) and fundraising ratios (fundraising costs as a percentage of total costs, or as a percentage of funds generated) (Connolly et al., 2013). With such ratios, the argument is that charities exist to convert funds received (inputs) into direct benefits for beneficiaries (outputs). Admittedly, while it is not necessarily the case that a decrease in administrative or fundraising cost ratios is the result of a more efficient management, it does suggest that more resources have been used in pursuing the charitable objectives of the organisation. Thus, on this basis, increasing charitable activity ratios and decreasing administrative or fundraising cost ratios signal improved efficiency (Tinkelman & Mankaney, 2007; Van der Heijden, 2013).

However, while such measures have obtained legitimacy due to their apparent precision, there can be complications for organisations attempting to develop meaningful measures of efficiency and their use can be problematic. For example, broadly drafted organisational objectives, often done to enable varied and changing activities to be undertaken, can make it difficult to develop meaningful measures, while aligning high- and low-level performance measures can be challenging because of multiple (frequently competing) organisational goals and influences, unclear input-output relationships, and the long-term nature of many costs and benefits (Hedley et al., 2010). In addition, ratios may not take account of different organisational structures and activities, and there is not necessarily a relationship between the amount spent on charitable activities and the quality of services provided (Qu & Daniel, 2021; Sargeant et al., 2009). This can lead to inappropriate comparisons and decisions due to charities’ dissimilar organisational structures, size and activities (Coupet & Berrett, 2019). A charity that makes extensive use of volunteers (that is, at no cost to the charity) may have a different cost structure to one that operates largely with paid staff, while the cost structure of a charity that directly delivers charitable services is unlikely to be the same as a grant-making charity. In addition, ratios by themselves only reflect part of the picture and may encourage donors and charities to pay insufficient attention to beneficiaries and clients who should be front and centre when considering charity performance (Tinkelman, 2006, 2009; Tinkelman & Donabedian, 2007, 2009).

Furthermore, there are considerable difficulties in measuring organisational impact, and over-reliance on such metrics may encourage charities to engage in behaviour that results in favourable ratios (Hong, 2014). For example, some organisations allocate joint costs (such as human resources, legal, information technology and finance department costs) subjectively as there is no standardised method for such allocations (Sargeant et al., 2009), which may signal misleadingly high charitable activity levels (Jones & Roberts, 2006; Keating et al., 2008; Trussel, 2003). Moreover, such notions of efficiency might encourage reduced spending to appease external expectations of administrative leanness even if doing so is contrary to the organisation’s aims (for example, the provision of free services) or lead to decisions that do not further the organisation’s aims (for example, reducing expenditure on fundraising might ‘improve’ the fundraising ratio but decrease the overall amount of money raised) (Calabrese, 2012).

Indeed, reporting such ratios could reinforce the notion that some costs are ‘good’ (charitable spend) and other costs are ‘bad’ (administration), and that optimal or benchmark levels can and should be set. For example, the Better Business Bureau (BBB) suggests that charitable organisations should achieve a charitable activity ratio of at least 65% and that charities should not have fundraising costs of more than 35% of fundraising income (BBB, 2023). Such an approach, however, perhaps fails to provide sufficient discrimination between different types of charities (for example, fundraising and grant-making), or between charities with varying pools of volunteers. Relatedly, Coupet and Berrett (2019) argue that overhead-related ratios do not measure efficiency, and that there should be a move away from concepts and measures of efficiency based on financial ratios, and toward ones that capture what charities do and whom they seek to serve.

In the UK and Ireland, for example, while charities are required to report on their achievements during the year, including measures or indicators used by the charity to measure those achievements, they are not required to (but may) report specific ratios as part of their trustees’ annual report (TAR) and financial statements. Prior research has examined the extent of disclosure of performance information by UK and Irish charities (primarily in the TAR and financial statements) (Breen, 2013; Connolly et al., 2013; Connolly & Dhanani, 2009; Dhanani & Connolly, 2015; Hyndman & McConville, 2016). Overall, the findings indicate that while performance reporting has increased over time, it remains poor and there are significant shortcomings in terms of making such reporting transparent (that is, understandable and candid). It is argued that these inadequacies make assessing performance difficult for most users of charity reports.

2.2. Data Envelopment Analysis: An Overview of Prior Research

As an alternative to conversion or overhead ratios, regression-based methods (for example, stochastic frontier analysis) can be used to evaluate the relative performance of charities. However, these methods use only one dependent variable (output) to measure efficiency (Hong, 2014), and charities typically generate multiple outputs or have multiple goals (including stakeholder satisfaction). In contrast, DEA was developed on the basis that organisations produce outputs by transforming inputs, with this methodology being particularly suitable when it is difficult to analyse the performance of entities because it is necessary to include numerous variables to do so. Using multiple inputs and outputs, rather than for example just one dependent variable, it measures the relative efficiency of DMUs to assess whether inputs are being translated into outputs efficiently (Emrouznejad & Yang, 2018; Gattoufi et al., 2004).

Thus, DEA is appropriate for analysing the relative performance of charities as such organisations have many types of input and output, and rarely produce a homogeneously beneficial amount of output, often without price (Golden et al., 2012; Kim & Lee, 2018). It is particularly useful for analysing the performance of entities that have difficulty valuing their outputs (for example, charities) (Bowlin, 1998), as it is necessary to incorporate multiple variables to assess them and has advantages over the other methods mentioned above (Callen & Falk, 1993). For instance, DEA identifies the best practice DMUs and constructs an ‘efficient frontier’ based on these units (that is, charities in the context of our research), thus allowing the relative efficiency of the remaining charities to be measured based on proximity to the efficient frontier. In addition, DEA can simultaneously consider multiple inputs and outputs, identifying whether inefficiency is a result of scale or technical factors. Furthermore, as a non-parametric technique, DEA does not make assumptions on the production function, estimating the relationship between inputs and outputs with given data only, and thus avoiding errors in the function setting of the analysis.

DEA has been applied in, for example, research on banking (Emrouznejad et al., 2008), supply chains and agriculture (Emrouznejad & Yang, 2018), and to assess the efficiency of nonprofit and public sector organisations involved in the arts (Golden et al., 2012; Hong, 2014), education (Colbert et al., 2000; Coupet, 2018; Reichmann & Sommersguter-Reichmann, 2006), health services (Hollingsworth, 2008; Van der Wielen & Ozcan, 2015) and humanitarian assistance (Coupet & Berrett, 2019; Kim & Lee, 2018) on the basis they have many inputs and outputs (often without price).

Overall, compared to other organisational types, there is relatively less research on charities, with much of this research focusing on Spanish charities, whose findings provide mixed evidence of the factors that drive efficiency. For example, Marcuello (1999) concluded that the efficiency of Spanish charities was affected positively by size and negatively by legal form, while García and Marcuello (2007) determined only size was significant. Several studies have also applied DEA to Spanish foundations^[2]. For example, Romero (2007) established that size, donation concentration and board diversity positively influenced efficiency, with the age of the organisation being a negative influence. Martínez Franco and Guzmán Raja (2014) found that long-term debt had a positive influence on efficiency while size and age had a negative effect, while Martínez Franco and Guzmán Raja (2017) reported that the size of the paid workforce negatively affected performance and the number of volunteers had a positive impact. Solana et al. (2017) similarly noted those foundations with volunteers complementing paid employees were more efficient. These findings are summarised in Table 2.

With respect to charitable organisations specifically, Joo et al. (2007) analysed the operating efficiency of nine charity resale shops (not all of the charity’s operations), concluding that, as the relatively efficient shops were located in affluent communities, location was a major contributor to operating efficiency. Tofallis and Sargeant (2000) examined charity spend on administration and fundraising in relation to the amount of voluntary income generated, finding that the performance of many charities fell short of the efficient frontier for no immediately apparent reason, with the efficient charities being a diverse group. They suggested a more detailed analysis taking account of a wider number of factors might be instructive for not only improving the operational stewardship of the sector, but also in providing guidance to donors on organisations likely to have the greatest impact on their chosen cause.

3. Methods

3.1. Sample

While data on Irish charities is available from the Public Register of Charities^[3], this only includes registered charities, and it does not contain detailed financial information (for example, only gross incomes and expenditures are provided). Thus, the primary data source for this study was the more comprehensive Benefacts Database^[4]. The database was queried for organisations registered with the Charities Regulator (Ireland) that acknowledged adopting the charities Statement of Recommended Practice (SORP)^[5], which stipulates the information to be reported by charities. The results showed that 294 of 4,535 Irish charities were identified as preparing their accruals-based financial statements in accordance with the charities SORP. While this limits the sample size, it ensures standard data can be obtained and that the organisations are registered charities. Of the 294 charities preparing accruals-based accounts, data was available for all of the variables for the years 2013–2015 for 117 charities.

3.2. Data Envelopment Analysis Model and Regression

The constant returns to scale (CRS) DEA model developed by Charnes et al. (1978) utilises a linear combination of outputs (for example, charity deliverables) over a linear combination of inputs (for example, the items a charity needs to accomplish its goals) to determine relative DMU efficiency. Banker et al. (1984) modified this model to incorporate variable returns to scale (VRS), thus enabling relative efficiency and technical efficiency (that is, maximum outputs relative to inputs) to be calculated so that the scale of operations of DMUs can be compared. In our research, given the varying size (and scale) of charities, we have adopted a VRS approach whereby DMUs are compared with those of their scale size, a peer group comparison in effect. Both the CRS and VRS models allow an input-oriented approach (reducing inputs whilst maintaining proportions of outputs) and an output-oriented approach (obtaining maximum expansion of outputs from a given level of inputs). As a charity’s output is driven by its often voluntary and uncontrollable income (Carroll & Stater, 2009; Hager, 2001), an input orientation is considered appropriate as a charity can control expenditure (Tofallis & Sargeant, 2000).

As the trend of efficiency indices does not incorporate movements in the efficient frontier between periods, a Malmquist Productivity Index (M_i) can be calculated assuming CRS (Färe et al., 1994). In brief, M_i captures changes in productivity over time. Values of M_i higher than unity indicate an increase in productivity from period t to t + 1; lower values depict decreased productivity. M_i can be separated into technical efficiency change and technological change. The change in technical efficiency refers to the movement of DMUs in relation to the efficiency frontier (which represents fully efficient DMUs) and whether the DMU is approaching it or moving in the opposite direction. Technological change refers to movements of the efficiency frontier itself. If it increases, for example, this is considered evidence of innovation of the DMUs that comprise the efficiency frontier.

With the dependent variable for our regression being the efficiency score derived from the VRS model, we adopted a two-stage approach to assess the main determinants of charity efficiency and calculate M_i (Golden et al., 2012; Hong, 2014). The singularity of the DEA efficiency scores (that is, 1 being efficient and between 0–0.99 inefficient) complicates the choice of a statistical analysis method. Prior DEA research has used Tobit regression to examine exogenous variables affecting efficiency levels (Golden et al., 2012; González & Rúa, 2007). However, it cannot be assumed that the distribution of the dependent variable (the efficiency score) fits with a censored regression method, as the accumulation of values at the highest level of efficiency is not the result of a defect in the data but a consequence of the definition of the problem. Also, occasionally, it is not possible to assume variable normality as required by Tobit regression (Chilingerian, 1995), which is the case with our research (Shapiro & Wilk, 1965)^[6]. Indeed, the use of Tobit regression in DEA analysis has been criticised for this reason (Andrés et al., 2010; Drake & Simper, 2003; Gálvez, 2012; García & Marcuello, 2007; Marcuello, 1999; Romero, 2007). Therefore, we employed a Firth logistic regression model, which applies Firth’s penalised maximum likelihood/the standard maximum likelihood method for the logistic regression, and which is considered an ideal approach for the analysis of binary outcomes with small samples (Firth, 1993).

As we utilise charity data over several years, and Firth logistic regression assumes homoscedasticity, a further consideration is the use of panel data (for example, the same charities every n years) or a pooled sample (for example, the data of n charities for x years pooled together). This was addressed by running a Breusch-Pagan Test (Breusch & Pagan, 1980) with a logit regression model of a pooled sample as the null hypothesis. The Breusch-Pagan Test results (chi²: 48.46, p-value: 0.000) imply rejection of the null hypothesis (p-value < 0.05) (that is, heteroscedasticity is present). Thus, the use of panel data in the regression model is supported.

The DEA model was executed using the DEAP v2.1 software^[7], with the output of the DEA model (the efficiency score, as 0 or 1) being the dependent variable (DEA_it) for the Firth logistic regression below.

\[ \begin{aligned} \mathrm{DEA}_{\mathrm{it}}&=\rho_0+\rho_1 \mathrm{AS}_{\mathrm{it}}+\rho_2 \mathrm{AIW}_{\mathrm{it}}+\rho_3 \mathrm{TS}_{\mathrm{it}} \\&\quad +\rho_4 \mathrm{CE}_{\mathrm{it}}+\rho_5 \mathrm{GS}_{\mathrm{it}}+\rho_6 \mathrm{AGE}_{\mathrm{it}}\\ &\quad+\rho_7 \mathrm{PF}_{\mathrm{it}}+\rho_8 \mathrm{LTC}_{\mathrm{it}}+ \mathrm{u}_{\mathrm{it}} \end{aligned} \]

Where:

DEA_it: Dichotomous variable for charity i for period t, with value of 1 if charity efficient per VRS model at 0.99 or greater, 0 if not;

AS_it: Sector of charity i for the period t (see H1 below);

AIW_it: Availability of accounting information on charity i’s website for the period t, with value of 1 if available within three mouse clicks, 0 if not (see H2 below);

TS_it: Total number of employees for charity i for the period t (see H3 below);

CE_it: Cost per employee for charity i for the period t, calculated as total staff cost/total number of staff (see H4 below);

GS_it: Charity i’s governance structure size for the period t, calculated as the number of directors reported (see H5 below);

AGE_it: Years since the establishment of charity i for the period t (see H6 below);

PF_it: Ratio of public funds to total funds of charity i for the period t, measured as public funds/total funds, range 0–1 (see H7 below);

LTC_it: Ratio of long-term credit to total credit for charity i for the period t, measured as long-term credit/total credit (see H8 below); and

u_it: Residual for charity i for the period t.

3.3. Data Envelopment Analysis Model Variables and Hypotheses

Data, except for the number of direct human beneficiaries (see below), was obtained from the Benefacts Database. A DEA model’s discrimination power is related to the number of model variables and the sum of inputs and outputs should exceed the number of DMUs (charities in our research) threefold (El-Mahgary & Ladhelma, 1995). As the number of charities (117, as indicated in Section 3.1) exceeds the number of DEA model variables (five, as indicated above), our DEA model is discriminatory and should provide some useful insights on charity efficiency.

Given the nature of charities’ activities and funding, and in contrast to for-profit organisations, charities typically receive funds (an input) and then use these funds in conjunction with input from their staff and volunteers to provide charitable activities (an output) to assist beneficiaries. Thus, our DEA model input variables – each of which has been included in prior nonprofit research (see below) – are staff costs (SC), total funds (TF) (for example, revenue grants, donations and bequests) and total assets (TA). Our output variables are activity expenditure (AE) and number of direct human beneficiaries (NB).

With respect to our input variables, as staff fulfil organisational aims, staff costs (SC) are important and have been included in prior nonprofit efficiency models (Drake & Simper, 2003; Martínez Franco & Guzmán Raja, 2017; Romero, 2007). Total funds (TF), which include revenue grants, donations and bequests, are a key determinant of charity activities, and this variable has also been included in nonprofit efficiency models (Marcuello, 1999; Martínez Franco & Guzmán Raja, 2017; Romero, 2007). Total assets (TA), as reflected in the balance sheet, support charitable activities (Martínez Franco & Guzmán Raja, 2017; Romero, 2007; Solana et al., 2017).

Moving to our two output variables, unlike traditional for-profit organisations who spend money to generate income, charities receive income and spend that income delivering activities to assist those they seek to help (Ibáñez & Benito, 2013). Therefore, in our model, total expenditure on activities (AE) represents an output derived from the funds received. As discussed in Section 2.1, whilst acknowledging the difficulties of measuring impact, we use this as an indicator of a charity’s impact on those it seeks to help (Gámez et al., 2012; Martínez Franco & Guzmán Raja, 2014, 2017; Solana et al., 2017). Carman (2009) contends that two key variables when measuring charity actions are: the expenditure incurred assisting beneficiaries; the impact of said expenditure or activities, which is measured in our model by the number of beneficiaries assisted. Although disclosure of the number of direct beneficiaries (NB) is mandatory in some countries (for example, Spain) (Galindo et al., 2012), this is not the case for the charities in our Irish dataset (although some did). Thus, where not provided explicitly, this was estimated using other information contained in the charity annual reports and/or websites. For example, with respect to the Dogs Trust, an animal welfare charity, we equated a dog rehomed to one direct human beneficiary; although, it is accepted there are likely more (indirect) beneficiaries.

As per Coupet and Berrett (2019), each input should positively and significantly increase output. Thus, we regressed each output (AE and NB) on the three inputs (SC, TF, and TA) and found all inputs statistically significant at 99% confidence. While the r-squared value for the AE regression was 0.9983, the corresponding value for NB was 0.6581, which is likely reflective of the difficulties of measuring beneficiary numbers in different charities; however, it indicates that the model has reasonable predictive value. As DEA is sensitive to outliers, we applied the super-efficiency model (Andersen & Petersen, 1993), whereby organisations presenting efficiency indexes greater than 1.5 are eliminated. No outliers were identified in this study.

Our hypotheses are now presented. While the Benefacts Database classifies charities into different sub-sectors based upon the ‘type’ of charitable activity and, acknowledging that certain ‘types’ of charities may be more or less efficient than others (Callen & Falk, 1993), the number of charities in some groupings was insufficient for analysis purposes. Also, as suggested by Callen and Falk (1993, p. 54), more homogenous groupings of charities imply similar “production technology”. Therefore, drawing on Martínez Franco and Guzmán Raja (2017) – who split their sample of charitable foundations into “care” and “education/cultural” sectors and noted sectoral efficiency differences – we similarly categorise charities into two activity sub-sectors (AS): (i) health, social services or the environment (n = 69); and (ii) arts, education, religion or philanthropy (n = 48). Thus, H1 is:

H1: There is a positive relationship between the charity sub-sector (AS) and efficiency.

Disclosure in charity financial reports is a dimension of transparency (Hyndman & McConville, 2016), and increased accessibility to such data should enhance stakeholder awareness of charity management, motivating a charity to use its inputs efficiently (Valencia et al., 2015). A positive relationship has been shown between the availability of financial reporting data and various variables (for example, Styles & Tennyson, 2007). In our research, we measured transparency based upon the accessibility of financial reporting information on a charity’s website, with such accounting information being deemed available if it was accessible within three mouse clicks of the charity home page (Styles & Tennyson, 2007). Therefore, H2 states:

H2: There is a positive relationship between ease of access to accounting information available on a charity’s website (AIW) and efficiency.

Staff numbers have been used in prior efficiency studies (Andrés et al., 2006; González & Rúa, 2007; Martínez Franco & Guzmán Raja, 2017), although results are mixed. Here, we expect that as a charity’s activities are often staff intensive, there will be a positive relationship between staff numbers and charity efficiency. Thus H3, like prior studies (e.g. Martínez Franco & Guzmán Raja, 2017), contends:

H3: There is a positive relationship between total staff (TS) and efficiency.

Higher wage levels (which are reflected in higher staff costs (SC)) attract a more skilled workforce, reduce staff turnover and improve productivity (Ingene, 1982; Lusch & Moon, 1984). Due to the sector’s increased professionalisation (Hwang & Powell, 2009; Tucker, 2010; Valeau, 2015), and acknowledging that volunteers may deliver elements of a charity’s services, H4 proposes:

H4: There is a positive relationship between cost per employee (CE) and efficiency.

Charities are typically governed by a board (of trustees) who is responsible for ensuring the organisation operates within its charitable purposes. Larger governance structures can have a detrimental impact on efficiency by making communication, coordination and decision-making more difficult (Andrés et al., 2006; Marcuello, 1999; Martínez Franco & Guzmán Raja, 2017), and thus H5 proposes:

H5: There is a negative relationship between governing structure size (GS) and efficiency.

While an older charity may have greater prestige, position and experience compared with one formed more recently, with longevity perhaps indicating good management, older organisations may struggle to adapt to change (for example, to new technologies and higher professional standards) (Andrés et al., 2006; González & Rúa, 2007; Martínez Franco & Guzmán Raja, 2014, 2017; Romero, 2007; Solana et al., 2017). In this study, H6 proposes:

H6: There is a negative relationship between age (AGE) and efficiency.

Many charities are publicly funded (that is, by government/related bodies) to a greater or lesser extent, rather than purely from direct public donations. Prior literature indicates that government-based funding can hinder efficiency; although, drawing on data from private nonprofit teaching-oriented colleges, Coupet (2018) concluded that government funding may not have a quantifiable negative effect on efficiency. As receiving such money often imposes increased (bureaucratic) demands that may reduce efficiency (Solana et al., 2017), H7 proposes:

H7: There is a negative relationship between public funds (PF) and efficiency.

Finally, non-current assets can be funded from surpluses and/or through debt, which if managed appropriately, can improve productive capacity and efficiency (Martínez Franco & Guzmán Raja, 2014). Thus, H8 is:

H8: There is a positive relationship between long-term credit (LTC) and efficiency.

4. Results

The results of the DEA and Firth logistic regression models are now presented. The sample data average, median and standard deviation figures for each of the five efficiency variables are presented in Table 3. The DEA model scores shown in Table 4 indicate average technical efficiency^[8] of 70.1% (0.701) over the three-year period. This suggests that, on average, charities could deliver the same level of activities to the same number of direct human beneficiaries using 29.9% less inputs. Furthermore, there was a decrease in this efficiency index from 2014 to 2015, from 78.4% (0.784) to 59.6% (0.596) (see Table 4). In the interests of brevity, while the individual efficiency scores are not presented, the lowest score in 2013 was 0.39 (a hospital foundation), 0.20 in 2014 (a literary arts charity) and 0.28 in 2015 (a visual arts charity). Table 4 also indicates an average scale efficiency of 83% (0.830) over the three-year period. As 1.0 represents full-scale efficiency, this implies that charities (on average) are near their optimal scale of operations but could improve.

Details of fully efficient charities (that is, a DEA model score of 1) are presented in Table 5. Of the 117 charities included in our sample for the three-year period, 18, 17 and 10 were fully efficient in 2013, 2014 and 2015 respectively. While not illustrated separately in Table 5, only three charities, each from sub-sector (i) (health, social services or the environment), attained a score of 1 in each of the three years included in our study. Although six charities from sub-sector (ii) (arts, education, religion or philanthropy) were found to be fully efficient in 2013, none was classified as such in 2014 and 2015 (see Table 5).

Table 6, which complements Table 4, indicates that the percentage of the total sample of 117 charities which were efficient was 15% in 2013, 18% in 2014, and 9% in 2015. In each of these years, these efficient charities contributed more than 50% of the total output of our sample (Table 6: 2013 – 51%; 2014 – 70%; 2015 – 62%). In input-based DEA models, slack variables measure the inefficiency of particular inputs. Here, Table 6 reveals the input with the least inefficiency is total funds (TF) in each of the three years, with total assets (TA) showing increasing inefficiency from 9% in 2013 to 54% in 2015, suggesting more attention should be given to managing this input. Staff costs (SC) present an added inefficiency of 13% and 12% in 2013 and 2015 respectively, but only 0.3% in 2014 (see Table 6).

By eliminating the inputs one-by-one, their weight in the DEA model can be determined by re-calculating the efficiency score (Quindós et al., 2003). The weight of each input refers to how the efficiency index varies if inputs are removed from the model. If the efficiency index varies substantially, an input’s weight has a greater impact on the model results (Quindós et al., 2003). Here, removing staff costs (SC) and total assets (TA) results in similar changes in the level of efficiency, while excluding total funds (TF) has a much greater effect (see Table 6). This would be expected for such a critical resource.

As the trend of efficiency scores does not address changes in productivity, since it does not assess movements of the efficient frontier between periods, M_i was calculated (Table 7). Our results show a decrease in average total productivity of 4.9% over the period 2013–2015 (M_i = 0.951 (i.e. 1–0.951)). Separating the Mi into technological change (CTC) and technical efficiency change (CEF) indicates that this decrease can be attributed to declining technical efficiency (−16.8%) (CEF = 0.832 (i.e. 1–0.832)). Technological progress increased by 10.9% (CTC = 1.109 (i.e. 1–1.109)). This suggests that charities, in general, were less capable of changing organisational processes and/or service technologies over the three-year period relative to those on the efficient frontier.

As outlined in Section 3.2, we employed a two-stage approach. Therefore, after calculating the efficiency scores, we applied the Firth logistic regression model. This was justified by the Breusch-Pagan Test (Chi²: 48.46, p-value: 0.000), which also warrants using a nested regression model (that is, a panel sample), with the Hausman test results (Chi²: 2.09, p-value: 0.5544) supporting using random rather than fixed effects.

The sample data average, median and standard deviation figures for each of the nine Firth logistic regression variables are presented in Table 8 (see also below and Table 10). The Firth logistic regression results in Table 9 indicate that, of the eight hypotheses presented earlier, four variables are significant: activity sector (AS); accounting information available on website (AIW); total staff (TS); and cost per employee (CE). The activity sector (AS) has a statistically significant negative coefficient (ρ1 = −2.530, p ≤ 0.01, odds ratio 0.08). Thus H1 is rejected. The availability of accounting information on a charity’s website (AIW) presents a statistically significant negative coefficient (ρ2 = −0.949, p ≤ 0.01, odds ratio 0.387). Thus H2 is also rejected. This finding is somewhat counterintuitive as it might be expected that a more efficient charity would wish to create a positive impression by making its results readily available (Merkl-Davies & Brennan, 2007). Given the statistically significant positive coefficient for total staff (TS), (ρ3 = 0.001, p ≤ 0.01, odds ratio 1.001), the results offer support for H3. Similarly, cost per employee (CE) presents a statistically significant positive coefficient (ρ4 = 0.000, p ≤ 0.10, odds ratio 1.000), and the results offer support for H4, suggesting higher wage levels do not contribute to improved efficiency. There is no statistically significant correlation between charity efficiency governance structure size (GS), the age of the charity (AGE), the proportion of public funds (PF), and the proportion of long-term credit (LTC). Therefore, H5–H8 are rejected. The regression results per Table 9 were derived using a similar method to prior studies of charities using DEA as a dependent variable. Some of the variables used in this study are not normally distributed, namely total staff (TS), cost per employee (CE) and age of the charity (AGE). Table 10 shows the results of a Firth regression where the log of these three variables is used. The results show that AS and AIW remain statistically significant (respectively: ρ1 = −2.726, p ≤ 0.01; ρ2 = −0.761, p ≤ 0.05), but TS and CE no longer report as statistically significant. However, the degree of public funding (PF) is positively statistically significant under this model (ρ7 = 0.946, p ≤ 0.05, odds ratio 2.575), offering support for H7.

4.1. Robustness Tests

For the dependent variable (DEA_it) in the Firth logistic regression model (see Section 3.2), we assigned a value of 1 (efficient) if the charity had a score greater than or equal to 0.99 or 0 (inefficient) if less than 0.99. A sensitivity analysis equating scores of 0.95 and 0.97 or above as fully efficient confirmed the results shown in Tables 9 and 10. Using the actual DEA scores in an ordinary least squares linear regression produces a similar pattern of results. While not reported in the tables, running a truncated (Tobit) regression (using the same DEA_it values) reveals activity sector (AS) as statistically negatively significant (ρ1 = −0.354, p ≤ 0.10) and total staff (TS) as statistically positively significant (ρ3 = 0.000, p ≤ 0.01). It also reports the coefficients of AIW and GS as negative but not statistically significant. However, as noted earlier, use of a Tobit regression in DEA research has been criticised. In sum, various regression modelling methods provide broadly similar results, lending some robustness to the analysis.

5. Discussion

Charity efficiency is the focus of considerable debate and evaluation (Eckerd, 2015). A key contribution of this paper is specifically to extend the work of Coupet and Berrett (2019) by applying DEA to a broader set of charities and to add to existing NPO efficiency-related research more generally. Some variables included in this research are found to be statistically significant determinants of charity efficiency as shown in Tables 9 and 10. Across the various regression models used, both activity sector (AS) and the online availability of accounting information (AIW) are statistically significant, but negatively so. As can be seen in Table 10, the proportion of public funding (PF) is positively statistically significant in that regression model (with logged variables for TS, CE and AGE), whereas Table 9 shows the total number of staff (TS) and cost per employee (CE) are positively statistically significant in that model (without logged variables).

This study reveals three significant results. First, the efficiency scores are higher than similar previous research, with an average efficiency score of 70.1% (Table 4) compared with scores of less than 50% from studies of international development organisations (Gámez et al., 2012; García & Marcuello, 2007) and Spanish foundations (Martínez Franco & Guzmán Raja, 2014, 2017; Romero, 2007). Second, Table 5 reveals that, while those charities obtaining a modelled perfect efficiency score are in both charity sub-sectors in 2013, they are involved in health, social services or the environment (sub-sector (i)) in 2014 and 2015. These activities arguably have more defined operations compared with those in sub-sector (ii) (arts, education, religion or philanthropy). Third, our study uncovers that employee numbers have a positive effect on efficiency. This contrasts with research by Martínez Franco and Guzmán Raja (2017) and seems counterintuitive as more staff might create a greater administrative burden. Further research is required to better understand the impact of differences within charity sub-sectors, including the role of volunteers.

Whilst the robustness tests indicate that our DEA model provides reasonable results, at least in an Irish context, it is problematic that charities are not obliged to provide quantified beneficiary data, as this requires such information to be estimated. Whilst acknowledging the difficulties in doing so, charity regulators, working with practitioners and academic researchers, could likely develop a suitable approach to measuring the number of ‘direct beneficiaries assisted’ for public disclosure. Such data would be of interest to a variety of stakeholders and also facilitate more meaningful DEA modelling. In our research, all the inputs and outputs in the DEA model, except for the number of direct beneficiaries assisted, are available in a relatively standard format through the financial statements. Adding beneficiaries as a required disclosure, while not without problems, would facilitate and improve modelling such as that reported here. Moreover, at the individual charity level, this could provide a better understanding of the extent to which output could be increased given the inputs available, together with identifying areas that need to be addressed – a key advantage over the frequently used, but often criticised, overhead ratios (Coupet & Berrett, 2019). Furthermore, DEA could assist organisations in identifying potential strategic alliances (for example, with respect to service delivery and/or marketing) and facilitate benchmarking, which could assist donors in their decision-making (for example, in deciding which organisations might best allocate charitable resources).

We acknowledge that to obtain data prepared in a consistent manner (that is, in accordance with the charities SORP) limited the number of charitable organisations included in our sample, together with the sub-sectors into which we were able to classify them. Whilst there are similarities between the different organisations in each of our sub-sectors, we accept that there will be operational differences. Thus, this study is limited by the fact that we do not consider more detailed charity sectors. Therefore, care must be taken in applying the results beyond our sample to different types of charities and/or other jurisdictions (i.e. the results here may not be generalisable). It should also be noted that our smaller sample size may imply the efficiency scores are biased upwards. While a limitation, the results remain meaningful (Alirezaee et al., 1998). Indeed, the various tests on the data suggest our methodology can be replicated, indicating there is scope to develop our research on larger sample sizes. We also recognise that performance is a relative measure in that it is defined by other DMUs, with the efficient frontier being very sensitive to outliers. Moreover, different or additional inputs may change the results (for example, including volunteer hours or location differences in cost per employee).

Although some prior studies have applied DEA to the NPO sector, with its appropriateness for this study being discussed in Section 2.2, DEA as a technique should be applied with care. For example, Gong and Sickles (1992) suggest DEA may be more suitable over other similar techniques when measurement error is less likely, and production is assumed to be non-parametric. In our study, while elements of accounting standards (i.e. the charities SORP) are subjective, all organizations in the sample measure their accounting data on a similar basis. Additionally, while we group the charities into two sectors for analysis, the activities of each are varied and thus can be considered non-parametric. Simar and Wilson (2007) note that DEA does not consider noise (i.e. any deviation from the efficiency frontier is due to inefficiency). We acknowledge this limitation, which implies that the (in)efficiency reported in this paper may be due to the limits of the DEA method and not reflective of actual performance (Bryce et al., 2000).

However, despite the noted limitations, given the size of the NPO sector globally, and the increasing emphasis on its accountability and performance – with measures of efficiency being considered an important element of this – it is suggested that this is an area appropriate for further DEA research. We hope our study encourages this. Data limitations, such as we experienced with the number of direct beneficiaries, or potentially useful inputs like volunteer hours, could be resolved over time voluntarily or through a regulatory requirement to disclose such information.

6. Conclusion

While this paper adopts a DEA approach to assess charity efficiency, it is accepted that the use of any financial ratios or rankings per se can be problematic. For instance, they may not provide sufficient discrimination between different types of charities (for example, fundraising and grant-making) or between charities with naturally different cost structures (for example, due to the availability of volunteers) (Sargeant et al., 2009). In addition, focusing excessively on ratios may encourage charities to pay insufficient attention to broader issues of impact and performance (Tinkelman, 2006, 2009; Tinkelman & Donabedian, 2007, 2009), and make operational decisions to the detriment of their charitable aims to report more acceptable ratios (Tinkelman, 2009). Charity financial statements alone cannot easily portray what a charity has done (its outputs) or achieved (its results). This is because financial statements have inherent limitations in terms of their ability to reflect the full impact of activities, not least because it is not possible to measure everything in either monetary or numerical terms.

The Trustees’ Annual Report, which is an important element of a charity’s annual reporting, provides an opportunity for charity trustees to explain the areas that the financial statements do not explain. Given the evidence that many charities report little or no performance information (Connolly & Dhanani, 2009), and acknowledging the difficulties associated with calculating and interpreting ratios, encouraging the reporting of such metrics is important, especially if it is supplemented with a contextual narrative. Without this, charities risk alienating potential donors who, if they are unable to find such information, may infer that the lack of transparency is an indication of poor efficiency and trustworthiness. Therefore, charities should be motivated to report more fully on their efficiency and social impact, augmenting quantitative information with a supporting qualitative commentary.