Income Forecasting Model - Report - 27 November 2017

CORPORATE SERVICES SCRUTINY PANEL

INCOME FORECASTING MODEL

Presented to the States on 27th November 2017

SR.11/2017

Introduction

In preparation for its review of the Draft Budget 2018, the Corporate Services Scrutiny Panel identified a need to undertake some background research and data gathering in relation to three areas of tax policy. The three areas were:

• The link between Impôts duties on alcohol and tobacco and health considerations

• The income forecasting model used by the Department for Treasury and Resources

• Changes in the taxation burden on individual taxpayers and the link between tax revenue income for the States and increases in population

The reason for selecting these particular areas was in order to follow up on issues identified in the Panel's report on the Draft Budget 2017 and also in the context of a new population policy which the Chief Minister is expected to publish before the end of this year.

This report covers the second of the three areas identified above. The Income Forecasting Model is used by the Treasury Department to forecast States income from taxation. A review of two key elements of the model (forecasting employment income and pensions income) was undertaken by external consultants Oxera and the outcome was published by the Treasury Department in June 2017.

The Panel engaged MJO Consulting to provide it with expert technical advice on the changes to the model recommended by Oxera. The Panel also received a briefing from Treasury officials on how the Income Forecasting Model works in practice. The Panel is now pleased to present its advisors report.

AN ASSESSMENT OF OXERA'S REVIEW OF THE INCOME FORECASTING MODEL1

MJO Consultancy

1 We would like to acknowledge all the help received from the Economics Unit and the States Statistician.

Contents

Executive Summary

1. Introduction
2. An assessment of the review of employment income forecasting methods discussed by Oxera
3. An assessment of the review of pension income forecasting methods discussed by Oxera
4. Conclusions

Executive Summary

In the draft Budget Statement for 2017 it was reported that the Economics Unit intended to work with the Income Forecasting Group to review the personal income tax forecasting model. This review was deemed necessary as it followed concerns about the performance of the model in recent years; namely the difference between forecast employment income and actual (outturn) employment income. Oxera were asked to review the approach to forecasting both pension and employment income and whether improvements could be made to both. In their review, Oxera identified a number of weaknesses with the employment-forecasting model and proposed several ways to improve this. They also suggested an alternative approach to forecasting short-term pension income.

In this assessment, we concur with Oxera's criticisms of what they term the current approach' to forecasting employment income and agree that the relationship between the percentage change in total wages and salaries (Compensation of Employees) and employment income has become weaker in recent years.[2] However, we raise several concerns about the models discussed. First, the model selection strategy needs to be clarified and theoretically grounded with more detailed specification and diagnostic testing. This would help the assessment of the different models presented in the review. Secondly, we highlight a key concern about the overall forecasting power' of the models discussed. The proposed new model seems better in replicating and modelling the past, but the accuracy of the forecast rests on the observation relative to one single period. There is a need for augmenting the number of observations on which the estimations are carried out. Thirdly, there is little consideration of the economic theory, which accompanies both the model and the variables selected in specifying the model, and this is something which needs to be addressed.[3]

In their review of the pension income forecasting methods, Oxera suggested a move away from the approach which takes the compound annual growth rate in total pension income over the previous five years and uses the average growth rate to predict next year's growth rate. We share the doubts about the stability of the proposed new forecasting system of the pension model put forward by Oxera itself. The model presented seems effective in replicating past series, but if the need is for a model that is stable enough to be consistently used for decades to come, the approach needs to go through major revisions.

1. INTRODUCTION

The two reviews by Oxera (2017a; 2017b) are intended to evaluate Jersey's approach to forecasting employment and pension income. Their review about the forecasting methods are well articulated and we share the doubts raised in particular about the documented positive impact of the variables and effects that have been omitted from the estimations. However, in this assessment we argue that the updated methods proposed by Oxera for both employment income and pension income forecasts suffers from both theoretical and methodological inconsistencies.

Section 2 of this report assesses the employment income forecasting methods discussed by Oxera. Section 3 assesses the approaches to forecasting short-term pension income discussed by Oxera. Section 4 provides some conclusions.

2. AN ASSESSMENT OF THE EMPLOYMENT INCOME FORECASTING METHODS DISCUSSED BY OXERA

Section 2 is divided into four sections. Section 2.1 examines the income-forecasting model prior to Oxera's report. In the short-term, Oxera suggested two different proposals. The first method to improve the forecasting accuracy of the model decomposes Compensation of Employees (CoE) into full-time-equivalent employees (FTEs) and average earnings and is discussed in Section 2.2. A further disaggregation of the model is discussed in Section 2.3. Finally, Section 2.4 considers some issues with the approach and results discussed by Oxera.

1. The income-forecasting model prior to Oxera's report

As Oxera (2017a) note in Section 2 of their report, the method of forecasting income in Jersey has long been based on a very simple regression, namely:

=  1 +  2

where:

• is the annual percentage change in total employment income;
• is the annual percentage change in Compensation of Employees (CoE);
• 1is a constant term; and
• 2 is a coefficient representing the effect that CoE has on employment income.

In short, what this means is that the percentage change in the variable employment income' is ultimately explained by the percentage change in total wages and salaries as registered in the national accounts (i.e. the CoE variable). However, the assessment of the historical forecast performance of this method is based on the estimation which includes a statistically non-significant term, i.e. the intercept 1(or constant) term (see Oxera 2017a, pp. 66-67). The estimation of any econometric effect (e.g.  1 and  2) is an exercise to test the likelihood that a relationship between two (or more) variables (in this specific case, between and ) is explained by a relation that is not a random chance. In our case, given that the intercept term ( 1) is statistically insignificant, we cannot exclude that the estimated relationship is a by chance' result. Although the intercept is usually a relatively less interesting term with respect to the estimated strength of the relationship (i.e.  2), the computation from the estimated equation would be biased.

It is well known that the inclusion/exclusion of a variable from a multivariate regression analysis has an effect on the magnitude, the statistical significance of the other variables, and ultimately on the interpretation of the whole estimated model. In brief, since we cannot gather consistent information from a statistically non-significant variable, this should be excluded from the estimated model. Notwithstanding this, an eventual exclusion of the intercept from the regression would require a discussion and justification about the relevant assumptions. In fact, by simplifying we can say that the intercept (or constant term) is collecting some of the information omitted by the predictor(s) and is a common feature of any econometric models.[4]

Thus, this means that the blue dotted line in Figure 2.1 (Oxera 2017a, p. 67) cannot be understood as a fair representation of the forecasting power of the model (as acknowledged, giving alternative reasons, by Oxera themselves in a note just below Figure 2.1).[5] However, notwithstanding several shortcomings, Oxera's report seems to use the evidence presented in Figure 2.1 (generated through a biased estimation) as the base for the proposal of a new, alternative approach for the estimation of the change in employment income.

Oxera commented on the good performance of the forecasting method so far but highlighted that in 2015 the performance weakened, overestimating the forecasted figure. The discrepancies between the average of differences from 2011 to 2014 (1.05 percentage points) and the last difference in 2015 are substantial. Oxera therefore proposed a revision of the forecasting method, and in particular the introduction of additional (or decomposed) explanatory variables within the specification of the regression model.

2. Proposal of an alternative approach

In Section 3 of their report, Oxera (2017a) proposed an alternative approach which is encapsulated in the following equation:

=  1 + 2 + 3 + 4 09

where:  

• is the annual percentage change in total employment income;
• is the annual percentage change in FTE employment;
• is the annual percentage change in nominal GVA;
• 09 is equal to 0 up to 2008 and equal to 1 from 2009 onwardsit is a dummy term used to control for a structural break in the data;
• 1 is a constant term;
• 2 is a coefficient which represents the effect that changes to FTE employment have on employment income;
• 3 is a coefficient which represents the effect that changes to nominal GVA have on employment income;
• 4 is a coefficient which represents the effect of the structural break in the data on annual changes in employment income.

In short, this is an extension of the specification of the original model and what this means is that the key drivers are the number of people working (captured by FTE employment) and changes to wages/bonuses (captured by GVA).

There are three theoretical and technical issues in the alternative approach outlined by Oxera in Section 3 of their report. First, the estimation based on the decomposition in two potentially highly correlated variables would bias ordinary-least square (OLS) estimation, simply violating one of the core assumptions of this estimation method. In other words, it is possible that the values of the variable FTE and the ones of the variable GVA are characterized by correlation during the period analysed, given what they are respectively measuring. In fact, FTE and GVA could share a common time trend component, and they capture similar phenomena. These are two possible causes of a high level of correlation between the two explanatory variables. This is a feature of the chosen estimator that needs to be carefully addressed, especially if the model aims to explain the same relationship on the basis of alternative samples (e.g. dataset from a different time period). However, we do not have information about correlations between explanatory variable to properly assess this point. Secondly, Oxera (2017a, p. 68) acknowledge that in recent years, FTE employment and average earnings have not been good predictors of CoE, which is an additional reason for testing an equation based on FTE employment and average earnings directly'. Finally, wages and bonuses are measured through a proxy variable, i.e. GVA.

With respect to the model specification, we agree with the introduction of a dummy for the financial crisis (dummy 09) and its impact on nominal GVA. It is surprising that this was omitted in the previous forecasting method. Oxera note that they tested a regression that included changes in average earnings; however, as the value of the coefficient was negative and close to zero, it was not included in the final formula' (Oxera 2017a, p. 69). It is not clear why a significant variable with an opposite sign to what expected should be excluded. The rationale for model selection needs to be clarified further.

The inclusion of non-significant coefficients in the final estimated model shown in Table 3.1 (i.e. change in GVA in the first column and, again, the intercept in the second column) raises some doubts about the consistency and robustness of the other estimated coefficients. There is neither a discussion about the insignificant estimated effect of GVA, nor an evaluation of possible alternative specifications. The evaluation of forecast performance in Figure 3.1 can be biased since the estimated effect of FTE might be sensitive to the exclusion of insignificant effects of GVA. Moreover, it is not clear from the discussion in the report how the differences in the tax contributions can be considered for the many employment contracts by definition collected under the FTE category.[6] Another serious concern is the potential effect of reverse causality between, for example, changes in employment income and number of people working' (FTE). In fact, as it is specified and estimated (OLS), the model is not taking into account this effect. It is also plausible that the higher the change in employment earnings, the higher the number of FTEs.

In Figure 3.1, which compares the actual and estimated employment income, the underlying assumption is that the future will be like the past. The proposed model seems better in replicating and modelling the past, but the accuracy of the forecast rests on the observation relative to one single period. This is a fairly strong assumption to insert in economic forecasting, and should be addressed by introducing estimation techniques more advanced than the standard OLS.

3. Refinement of the alternative approach

In discussion with the Economics Unit, Oxera have discussed the possibility of refining the alternative approach. In Section 4 of their report (Oxera 2017a), the refinement approach' is outlined, which essentially introduces a further disaggregation of the explanatory variables used in the estimations discussed above.

The main solution in the short-term is to decompose the GVA variable (which was non- significant in the previous estimations) into CoE and gross operating surplus (GOS), a measure of firms' profits. The two variables are further split into financial and non- financial profits, providing a richer model specification (Oxera 2017a, p 72). However, it is unclear from the report whether the GOS for the non-financial sector is included in the estimation since it resulted in a negative effect. This would mean that an increase in non-financial operating surplus (profits) would see a reduction in compensation of employees. This is actually plausible given that, at the enterprise

level, a negative relationship between profits and wages can be observed. It would have been useful if the report could have explored further the theoretical relationship between gross operating surplus and employment earnings and reported the alternative estimations.

With respect to GVA, the estimated coefficient of the change in financial CoE is not significant (according to Table 4.1 the coefficient is 0.069). If a general-to-specific' model selection is the one to be applied going forward, this variable should be removed/substituted, and the model re-estimated to check the robustness of the other variables.[7]

We endorse the bottom-up approach suggested by Oxera in Section 4.2 of their report, which is listed as a longer-term solution. We hope that the long-term proves to be the medium term as it is our understanding that a more joined up approach for data sharing between the Social Security Department and Tax Office will be possible by 2021.

4. Issues with the approach and results discussed by Oxera

We have drawn attention to several issues with the approach and results in Oxera's (2017a) report and below we provide a brief discussion of the possible alternatives to overcome these.

First, there is a need for augmenting the number of observations on which the estimations are carried out. Making statistical inference on the basis of less than 20 observations is risky, since the OLS estimator rests on the assumption of normality of the distribution (and thus the central limit theory) of the included variables. Using quarterly data or even monthly ones would provide more efficient and unbiased forecasting.[8] We recognize and understand the difficulties surrounding data collection in Jersey and this is absolutely not a criticism of the excellent work conducted by the Statistics Unit. It is hoped that the new Census and Statistics Law will provide more granular data for more frequent analysis.

Second, a consideration of economic theory should accompany both the model and the variable selection in specifying the variables to be used at the outset of the econometric model. This would help and better justify the discussion about the decomposition of some of the variable discussed in Oxera's report.

Finally, a more developed evaluation criteria of the model design should be considered. In fact, there is no mention of having addressed time-series properties like integration, cointegration, or error-correction. A note in the main report would have been sufficient if this had been done and would not have compromised readability of the report.

Thus, we suggest the use of estimation and specification methodologies as alternatives to the one presented, specifically Auto-regression models (even Vector Auto-regression ones) or Error Correction models.

3. AN ASSESSMENT OF THE REVIEW OF PENSION INCOME FORECASTING METHODS DISCUSSED BY OXERA

Oxera (2017b) then turn to discuss approaches to forecasting short-term pension income. They note that the approach of the States of Jersey has been to take the compound annual growth rate in total pension income over the previous five years and to use the average growth rate to predict next year's growth rate. As they remark, while this might provide a reasonable prediction in some cases, it will be unable to account for future changes to key determinants of pension income that diverge from previous trends because the prediction is purely backward-looking' (Oxera 2017b, p. 75).

To derive an alternative forecasting approach, Oxera first determine which factors are likely to affect pension income. These are discussed in more detail on page 76 of the report (Oxera 2017b) but in summary the main drivers are: (1) earnings; (2) inflation; (3) population (pensioners); (4) lump-sum payments; (5) contribution history and (6) long-term investment performance.

There are two important points to note. First, Oxera excludes components (5) and (6) from the analysis since they are unlikely to have a material effect on the year-on-year change in total pension income' (Oxera 2017b, p. 76). We think that a more clear justification for this decision should be provided. Moreover, item (4) has not been included because of data availability. We would also like to see a greater discussion about the inclusion/exclusion of private pension contributions as estimating changes in pension income without consistently taking into account the  role of a private integrative system could lead to biased results and reduce forecasting stability.

Second, the model presented is a result of various tests for which it is not possible to provide an evaluation since the results are not reported. The final selected model is given below:

=   +   + 65 + [9]

1 2 3 4

where:

• is the annual percentage change in total pension income;
• is the annual percentage change in average (nominal) earnings;
• 2 is the annual percentage change in average lagged (nominal) earnings (by one year)e.g. the value in 2014 will be the percentage

difference in earnings between 2013 and 2012;

• 65  is the annual percentage change in the number of people aged 65 or over (based on 2001 and 2011 Census data and the Jersey population

projections in the 2016 release report);

• 1 is a coefficient which represents the effect that annual changes in average earnings has on annual changes in total pension income;
• 2 is a coefficient which represents the effect that annual changes in average lagged earnings has on annual changes in total pension income;
• 3 is a coefficient which represents the effect that changes in the number of people over 65 has on changes in pension income;
• 4 is a constant term.

This explains change in pension income with change in nominal earnings, the lagged level of the change in nominal earnings, and the annual percentage change in the number of people aged 65 or over. This specification seems better with respect to the existing one, since it takes into account the potential dynamic effect present in the series. However, as in the employment income model discussed earlier, the constant term (the intercept) in the pension model is statistically insignificant, and the reasons for this should be discussed (see Table 2.1, page 78).

After reviewing the results, Oxera highlight the need for a bottom-up' approach in which pension income is projected at an individual level. As they recognise, this would be needed because of the physiological discrepancies between new pensioners' and old pensioners' earnings. However, as they admit, this new variable would cause multicollinearity in the equation to be tested. This is not just causing the forecast to be vulnerable to shocks in earnings growth' but more importantly is violating one of the assumptions on which the OLS estimation is built (Oxera 2017b, p. 79).

As discussed in Section 2 of this report, the major issues with these estimations is not merely technical, but have to do with the number of observations on which all the evaluations are provided. In this case, the estimation rests on 14 observations, far below the theoretical threshold of 20. The 0.4 difference in the forecasting performance of the previous and the alternative model proposed seems not big enough to somehow justify the proposed evolution towards projections on individual pension income.

Oxera are also cautious about the use of the proposed alternative, since it is likely to be less resilient than the current approach to movements in earnings growth year-on- year away from the long-term trend, which may reduce the ability of the alternative model to forecast changes in total pension income accurately in future' (Oxera 2017b, p. 81). Again, it seems that the approach suffers from a lack of theoretical grounding for the selection of variables and model specification, which is referred to in their conclusions:

the ability of any formula to predict future outcomes accurately is dependent on the ability to forecast the explanatory variables used in the formula (in this case, the size of the retired population and earnings growth)

(Oxera 2017b, p. 81)

The alternatives to the previous method of forecasting pension income would be very similar to the one proposed above about the forecast of employment income. In particular, the number of available observations should be increased, the theoretical foundations of model selection taken into account, and the time-series properties of the model carefully addressed.

4. CONCLUSIONS

1. Overall, the evidence provided by Oxera (2017a; 2017b) shows a better performance with the alternative models; however, we have raised some issues about these models in this report.
2. The model selection strategy should be clarified and theoretically grounded. In addition, the  p-values (or  t-values) and standard errors, as well as other diagnostic tests (e.g. F-test) should be presented along with the estimated coefficients. At least a plot analysis of the regression residuals should also be presented to check for the presence of heteroskedasticity. All these would help the assessment of the different models presented.
3. Another major concern has to do with the overall forecasting power' of the models. In fact, in all of them the total observations are below the threshold of 20, under which it is very ambitious to argue in terms of inference for future values. For many economists and econometricians, this would be the first problem to be addressed, moving from a qualitative decomposition of the variables to a quantitative composition of the time dimension of the series, thus sensibly increasing the number of observations.
4. We would be really cautious in basing the expectation about employment earnings, i.e. one of the components of taxable income, on the proposed methodology, as also recommended by Oxera itself.
5. A specific issue with the pension income model is the lack of discussion about the effect of private pension schemes, a component that we think should be taken into account.
6. We share the doubts about the stability of the proposed forecasting system of the pension model put forward by Oxera itself. The model presented seems effective in replicating past series, but if the need is for a model that is stable enough to be consistently used for decades to come, the approach needs to go through major revisions.

7. Given the importance of employment income as the main source of revenue for expenditure in Jersey, we would draw attention to the revised income forecasts in March 2017 (Sates of Jersey 2017). These showed that Oxera's review coupled to a more prudent approach to forecasting the yield resulted in offsetting reductions of up to £5m in the personal tax forecast by 2020.[10] In previous reports we have expressed our view that the structure of Jersey's economy was changing even before the Global Financial Crisis, and we remain concerned whether this is fully reflected in the assumptions behind some of the explanatory variables. We agree with Oxera's (2017a, p. 74) cautionary note in their conclusion: prior to adopting any additional forecasting approach, it is therefore important to understand whether the explanatory variables can be forecast with a reasonable degree of accuracy in the short to medium term'.

References

Oxera, (2017a), Income tax forecasting phase one: review of employment income forecast', Note prepared for Government of Jersey, 25 April 2017, reproduced in Revised forecast of States income for March 2017', R.66 (reissue), Jersey: States of Jersey.

Oxera, (2017b), Income tax forecasting phase two: review of pension income forecast', Note prepared for Government of Jersey, 25 April 2017, reproduced in Revised forecast of States income for March 2017', R.66 (reissue), Jersey: States of Jersey.

States of Jersey (2017), Revised forecast of States income for March 2017', R.66 (reissue), Jersey: States of Jersey.

Panel Membership and Terms of Reference

Panel Membership:

Deputy John Le Fondré (Chairman) Deputy Simon Brée (Vice Chairman) Deputy Kevin Lewis

Senator Sarah Ferguson

N.B. The  Connétable of St John was a member of the Panel at the commencement of the review, but resigned on 10th October 2017 following his appointment as Chairman of the Public Accounts Committee.

Budget 2018, Phase 1 – Terms of Reference

1. To examine the available evidence for linking increases in Impôts duties to the reduction in consumption of alcohol and tobacco.
2. To assess the outcomes of the review of the Income Forecasting Model undertaken by the Department for Treasury and Resources and consider any proposed follow up work.
3. To evaluate the change in the total tax burden (taxation and all significant charges) on individual tax payers.
4. To examine the correlation between changes in tax revenue income for the States and increases in population.
5. To review the level of supplementation required by the Department for Social Security.

[2] Since Oxera's report was published the improved regressions provided by Oxera have now been used in the latest personal income tax forecasts. The phrase current approach' therefore refers to the previous approach, i.e. before Oxera's suggestions were incorporated by the Economics Unit.

[3] The theoretical rationale for variable inclusion is something which is non-technical and could have been disclosed in the report.

[4] In more technical terms, the inclusion of the intercept assures that the mean of the residuals  from  the  estimation  is  zero,  which  is  a  condition  for  consistency  and unbiasedness of the OLS estimation.

[5] The same applies to data for the approach presented in Figures 3.1 and 4.1 (Oxera 2017a). In fact, assuming for example that the estimated value for  1 is 0.4, the one for  0.5  and  that  the  value  of   is  0.3,  the  resulting  value  for

2 would be 0.55 ( 1 + 2 0.4 + 0.5*0.3 =0.55). It is easy

to see that the exclusion of the constant effect (the intercept) from this computation would give us a different estimated change in employment income, also because it is likely that the value of the estimated coefficient  2 will vary.

[6] Technical note: In an OLS regression setting, the most common reasons explaining the  insignificance  of  an  estimated  coefficient  are  first,  the  absence  of  a  linear relationship between the variable and, secondly, issues with the measurement of the explanatory variable. This could be explored either by plotting the dependent variable against the explanatory one, or by estimating a non-linear specification (i.e. elevating the explanatory variable to the power of 2).

[7] Technical Note: the variable change in CoE (FS) is significant at the 5% level, whilst the dummy for the crisis (structural break dummy) is significant at the 10% level. Although the actual values of p or t-statistics are not provided and thus a careful examination is not possible, it is legitimate to expect that these two variables will be very sensitive to the model specification (e.g. inclusion/exclusion of variables).

[8] Given  the  period  of  14  years  considered  (2001-2015),  the  total  number  of observations will be 56 with quarterly data (14*4) and 168 with monthly data (14*12) against the 13 to 15 observations currently used.

[9] 4, which is again the intercept, is not written within the specification of Oxera's

model. However, it is listed in Table 2.1 (Oxera 2017b, p. 78) and it is not significant (as per the earlier discussions).

[10] The economic assumptions were revised by the FPP in August 2017 and have been used by the Income Forecasting Group for new income forecasts for Budget 2018.