National Evaluation of the Neighbourhood Nurseries Initiative - March 07

March 2007

National Evaluation of the Neighbourhood Nurseries Initiative:

The Relationship between Quality and

Children's Behavioural Development

Research Report SSU/2007/FR/022

National Evaluation of the Neighbourhood Nurseries Initiative:

The Relationship between Quality and Children's Behavioural Development

Sandra Mathers and Kathy Sylva

Department of Educational Studies University of Oxford

The views expressed in this report are the authors' and do not necessarily reflect those of the Department for Education and Skills.

ACKNOWLEDGEMENTS

We are grateful to the Regional Research Officers, Faye Linskey, Margaret Kehoe and Denise Jennings, who visited the 103 Neighbourhood Nurseries to carry out observations and collect data; Ana Maria Aricescu and Jenna Jones who entered and cleaned the data; James Hall who contributed to the analysis; and our research colleagues in the NNI Research Team, particularly Teresa Smith, Kate Coxon and Maria Sigala who have worked closely with us as their Implementation Study Report and our Childcare Quality and Children's Behaviour Report have drawn from the same nurseries. We are also grateful to our DfES sponsors who commissioned the research – particularly Sam Mason, Richard White, Jane Simmonds, Ganka Mueller, Teresa Downing, Emma Marshall, Laura Sukhnandan, and Rajkiran Sidhu; and the members of our Advisory Committee. Finally, we thank all the Neighbourhood Nurseries, their managers and staff, who agreed to take part in the study, and the regional advisors and local authority advisors who agreed to be interviewed – without their participation there would be no study and no report.

THE NNI RESEARCH TEAM ( * denotes principal investigator)

Implementation

Dept of Social Policy and Social Work, University of Oxford Teresa Smith*, Kate Coxon, Dr Maria Sigala

Childcare quality and children's behaviour Dept of Educational Studies, University of Oxford Professor Kathy Sylva*, Sandra Mathers

Neighbourhood change

Dept of Social Policy and Social Work, University of Oxford George Smith*, Dr Maria Sigala

Impact and economic analysis

Institute for Fiscal Studies

Professor Lorraine Dearden*, Alissa Goodman, Jonathan Shaw, Luke Sibieta National Centre for Social Research

Ivana La Valle*, Dr Susan Purdon, Ruth Smith, Alice Bell

FOR FURTHER INFORMATION ABOUT THIS REPORT CONTACT:

Sandra Mathers

Department of Educational Studies, University of Oxford 15 Norham Gardens, Oxford OX2 6PY

(01865) 274043

Sandra.mathers@edstud.ox.ac.uk

CONTENTS

Page EXECUTIVE SUMMARY 5

1. CONTEXT AND AIMS

1. Introduction 14
2. The Childcare Quality and Children's Behaviour Study 14
3. The research context 15
4. The structureof this report 16

2. METHODOLOGY

1. Sampling 17
2. Data collection 18
3. Design andanalysis 24

3. QUALITY OF PROVISION

1. Overall quality of provision(as measured by the ITERS-R) 28
2. Dimensions of quality 31

4. FACTORS RELATED TO QUALITY OF PROVISION

1. Introduction 34
2. Relationshipof quality to sector 35
3. Relationshipof quality to community partnerships andlinks 40 withother programmes
4. Relationshipof quality to the physical environment 43
5. Relationshipof quality to staff qualifications 44
6. Relationshipof quality to centresize 47
7. Relationshipof quality to group characteristics 48
8. Relationshipof quality to the populationof children and families served 51
9. Summary of findings: which centre characteristics relate to quality? 52

5. EFFECTS ON CHILDREN'S BEHAVIOURAL DEVELOPMENT

1. Introduction 56
2. Characteristics of thesample 58
3. Effects on children's social behavioural development 59

6. CONCLUSIONS

1. Quality of provision 69
2. The effects of quality on children's behaviour 73
3. The effects of other centre characteristics on children's behaviour 74
4. The effects of time spentincentre-based childcareonchildren's 76 behaviour
5. Childand family influences on children's behaviour 78
6. Final conclusions and recommendations 80

REFERENCES 82 APPENDIX 1: CHARACTERISTICS OF THE SAMPLE 85 APPENDIX 2: UNI/ MULTIVARIATE ANALYSES 88 APPENDIX 3: INSTRUMENTS 100

EXECUTIVE SUMMARY

Introduction

The aim of the Neighbourhood Nurseries Initiative (NNI) was to reduce unemployment - and thus tackle child poverty - by offering high quality, affordable childcare in the most disadvantaged areas of the country. By August 2004, 45,000 new places for 0-4 year olds had been created. This research was part of the NNI National Evaluation, and had two main strands.

The Childcare Quality strand described the quality of provision offered by a random sample of 103 Neighbourhood Nurseries. The aim was to establish whether the new places and nurseries created by the NNI were of sufficient quality to foster the development of the children attending them. It also considered a number of other centre characteristics (for example, sector, qualifications of centre staff, centre size), with the aim of establishing which of these characteristics were related to, and predicted, quality of provision.

The Quality and Children's Behaviour strand used the information gathered by the quality strand to explore the effects of early centre-based childcare on 810 children[1] attending the sample Neighbourhood Nurseries. It aimed to establish (after taking into account child and family background):

• The effect of provision quality on children's social and behavioural development;

• Which centre and childcare characteristics (in addition to quality) were related to children's social and behavioural development.

This study was intended to fill two major gaps in the UK literature. Firstly, the majority of studies exploring the relationship between quality and child outcomes have focused on provision for 3 and 4 year old children. The current study focused on children under the age of 3years. Secondly, although there is a comprehensive body of research which considers the impacts of childcare on children's intellectual development (for example language and reasoning abilities), much less is known about impacts on behavioural outcomes. Previous research has drawn mixed conclusions, and there have been a number of worrying findings relating to the effects of childcare on anti-social behaviour. For this reason, the current study focused on the relationships between quality of care and young children's social behaviour.

This report can be read as a stand-alone document, or in conjunction with the NNI Implementation Study Report[2].

Methodology

Visits to the sample Neighbourhood Nurseries took place between February 2004 and July 2005. Two observational instruments were used to assess quality in rooms providing for infants and toddlers:

The Infant Toddler Environment Rating Scale (Harms, Cryer & Clifford, 2003) This revised version of the ITERS scale (the ITERS-R) is designed to assess provision for children from birth to 2 years, and covers a comprehensive range of quality features:

• Space and furnishings (e.g. layout of the room, resources, display);

• Personal care routines (e.g. health and safety, hygiene, meal times);

• Listening and talking (e.g. supporting children's language development);

• Activities (e.g. dramatic play, sand and water, fine motor play);

• Interaction (e.g. supervision, discipline, staff-child and peer interactions);

• Program structure (e.g. opportunities for free play, group activities);

• Parents and staff (e.g. information for parents, staff training).

The Caregiver Interaction Scale (Arnett, 1989)

The positive relationships' subscale of the CIS (indicating warmth and enthusiasm in interaction with children) was used in this study.

Where the term overall quality of provision' is used in this report, it refers to the mean total score achieved on the ITERS-R. Where individual dimensions of quality' are referred to, this relates to one or more of the seven individual subscales of the ITERS-R and the CIS positive relationships subscale.

Information on general centre characteristics (for example, sector, qualifications of centre staff, centre size) was provided by the NNI Implementation Study, with the aim of establishing which of these characteristics were related to children's behaviour. Finally, information on children (and their families) was collected using:

The Adaptive Social Behaviour Inventory (Hogan et al, 1992)

The ASBI records information on the social and behavioural development of pre-school children across five dimensions:

• Co-operation and conformity;

• Peer sociability;

• Confidence;

• Anti-social behaviour;

• Worried and upset behaviour.

Family Profile

As previous research had shown that home background has a large impact on children's social behaviour, it was necessary to take account of these influences when considering the effect of the NNI provision. This questionnaire, devised for the study, recorded information on child characteristics (e.g. gender), family characteristics (e.g. family structure and work status) and current childcare and childcare history (e.g. hours per week in group care).

Key Findings: Overall quality of provision in the NNI sample

Overall quality of provision varied widely across the sample: while some centres were offering a good-to-excellent standard of provision, others were of poor quality. The vast majority of Neighbourhood Nurseries were offering at least adequate quality of provision for children under the age of 3 . Most (70%) of the rooms observed were rated as adequate (above minimal but below good). Around one quarter (23%) of the rooms observed offered a good standard of provision. These centres provided children with a nurturing, educationally stimulating and healthy environment. A small proportion (7%) offered less than minimal quality. These centres were missing basic elements of quality provision such as hygiene, safety, educational stimulation and warm staff-child interactions. On the whole, providers in the maintained sector offered the highest quality of provision. The private sector had the lowest mean quality rating, but also displayed the broadest variation in quality, with some centres operating at a very high standard.

Neighbourhood Nurseries were the most successful at providing children with pleasant and appropriate staff-child interaction – they offered good quality provision in this regard. Interactions were warm and respectful, staff helped children to develop appropriate behaviour with their peers, and employed appropriate levels of supervision and positive discipline strategies. The sample centres did less well at providing hygienic and appropriate care routines such as meal times, toileting and naps. The provision of stimulating educational activities was also limited. For example, many centres did not provide opportunities for children to explore natural materials, or use everyday events such as the weather to help children develop their understanding of nature and their environment. This finding is of particular concern, as the provision of educational opportunities during the early years is related to later school success.

The most important predictors of provision quality (as measured by the ITERS-R) were:

• Sector: fully maintained Local Authority (LA) provision offered the most stimulating environment for children's developing language and cognitive skills, as well as the highest quality physical environment. This effect can partly be explained by variation in staff qualifications between the sectors – the maintained centres in the sample had the most qualified workforce. Maintained pre-school centres also have access to the educational infrastructure' and support systems, for example curricular and pedagogical input to planning and access to specialist staff (e.g. speech and language therapists, educational psychologists).

• Children's Centre status: Neighbourhood Nurseries that were also Children's Centres offered higher quality provision than centres with no involvement in the Children's Centre Programme. Again, one factor behind this positive effect may be the qualifications of centre staff – Children's Centres had a more qualified workforce than non-Children's Centres. The higher quality offered by Children's Centres related to social interactions and daily program structure, rather than to educational provision.

• Centre size: the larger Neighbourhood Nurseries (number of children registered) offered better overall quality of provision and, in particular, higher quality in relation to personal care routines, language (listening/talking), program structure and provision for parents and staff. Economies of scale mean that larger centres are able to offer a greater range of resources and facilities for children, staff and parents. In addition, the larger centres are likely to have a larger staff base, with a richer and more diverse adult social environment and a broader range of experiences and interests to draw upon when specialist knowledge is required. However, centre size had a complex relationship with children's behaviour, with larger centres offering some advantages, but also some disadvantages (see p.10).

• Age of children: many of the rooms observed catered for children over the age of 3 as well as for younger children. Quality scores were significantly higher in these mixed age groups than in rooms which provided only for young children. This supports previous research, for example the NICHD study (1996), which found that children in classrooms providing only for infants and/or toddlers received less positive care-giving than infants in mixed age rooms. Younger children experienced better quality provision in rooms with clear and stretching' aims for children's development: the dimensions which improved with the presence of older children were those related to educational provision. In a mixed age room, young children were able to experience higher level language, communication and educational activities developed to meet the needs of the older children. They also had the opportunity to interact with, and model the behaviour of, more mature peers. Thus, they had access to a richer and more stimulating environment than they would have experienced in a room which catered only for children under the age of 3 . However, the presence of older children was not always beneficial for the younger ones in terms of behavioural outcomes (see p.10).

• Staff childcare qualifications had a significant positive relationship with overall quality of provision. The better qualified staff teams provided a more stimulating environment for children's developing communication, and a more appropriate environment in terms of the daily schedule: opportunities for free play, group activities and provision for children with special needs. The qualifications of the centre managers were also important, and were related to overall quality of provision in the infant and toddler rooms (although not to any individual dimensions of quality).

No relationship was found between the population of children and families served and quality of provision. This is an important finding and suggests that families from very different backgrounds, and with different needs, were being offered comparable quality of provision.

Key Findings: Effect of quality on children's behaviour

The identified effects of quality on children's behaviour were significant but modest. Overall quality of provision in the sample Neighbourhood Nurseries (mean total ITERS-R scores) was not significantly linked to children's behaviour, and there was no significant effect of staff-child interaction as measured by the Caregiver Interaction Scale. However, a number of effects were identified for the individual dimensions of quality measured by the ITERS-R subscales.

• The quality of the physical environment was important. Children displayed significantly fewer worried and upset behaviours in centres which offered a spacious, well maintained and pleasant physical environment, with appropriate furniture for care routines and educational activities, and comfortable areas for

children to relax and spend quiet time. This study confirms the findings of the EPPE project, which concluded that high quality provision can reduce some of the negative behaviours associated with attending centre-based provision.

• The structure of the day was related to older children's levels of sociability. Children aged between 33 and 42 months were more sociable in centres which scored highly on the program structure' subscale of the ITERS-R. These centres offered a predictable yet flexible daily schedule, many opportunities for free play and high quality group play activities. The children attending them were more likely, for example, to say nice or friendly things to others (ASBI item 12) or play games and talk with other children (ASBI item 19) than children in centres which offered lower quality program structure.

• When the sample was split into age groups, the older group displayed a negative relationship between the personal care routines' subscale of the ITERS-R and children's co-operative behaviour, social skills and confidence. Children in centres which scored highly on this subscale were rated as less co-operative, less sociable and less confident. It could be that, in centres where hygiene and care routines are paramount, less time and attention is paid to developing children's interactions and social behaviour.

Key Findings: Effect of centre characteristics on children's behaviour

Staff qualifications

The qualifications of centre staff were related to children's social and behavioural development. Children with access to a qualified teacher (either working in their room or as the nursery manager) were significantly more co-operative and sociable than children without access to a trained teacher. These children were more likely to share their toys or possessions (ASBI item 20), say please' and thank you' when reminded (item 16) or be sympathetic towards other children's distress (item 7). Children in rooms with high mean staff qualification levels were also more co-operative, and displayed fewer worried and upset behaviours, than children cared for by less well-qualified staff teams. Finally, centres with better qualified managers had younger children who were less anti-social. The findings echo previous research in identifying the importance of high quality staffing for both the provision of high quality caregiving and for child outcomes (Melhuish et al, 2000; NICHD, 1999b/ 2000; Peisner-Feinberg and Burchinal, 1997).

Sector

No specific effect of sector was identified in relation to children's social and behavioural development, despite maintained status being identified as an important predictor of centre quality. This apparent contradiction is not surprising when we consider the specific quality subscales on which the maintained sector excelled. Maintained centres provided significantly higher quality in those domains related to educational provision - thus, we would expect to see an impact on children's cognitive outcomes but not necessarily on their social behaviour.

Children's Centre Status

Involvement in the Children's Centre Programme had a positive relationship with children's co-operative behaviour, particularly for the younger age group. Children under 2 years 9 months attending Children's Centres were more co-operative than their counterparts in centres not involved in the Children's Centre Programme.

Centre size

The Childcare Quality strand had identified centre size as a predictor of provision quality, with the larger Neighbourhood Nurseries offering higher quality provision. Children in larger centres (number of FTE places and number of children registered) were also less anti-social and displayed fewer worried and upset behaviours than children in smaller centres. For example, they were less likely to tease other children or call them names (ASBI item 21), or get upset when not paid enough attention (item 6). However, the picture was not the same for positive behaviours. Centre size had a negative relationship with children's co-operation and sociability: children in larger centres displayed fewer of these behaviours than children in smaller centres. The effect of centre size on children's behaviour is clearly a complex one. It is possible that larger centres show lower rates of anti-social behaviour because they have more explicit procedures for dealing with children's negative behaviours, in comparison to smaller centres which may operate more ad hoc and informal approaches to discipline. However, large centres may also be rather overwhelming for young children, who are just beginning to develop their social skills.

Age range

The age range of the rooms had a weak but significant effect on children's worried and upset behaviour. Children under the age of 3 years displayed more worried and upset behaviours when they attended a mixed age room with children aged 4 years and over. In mixed groups, they were more likely to frown, shrug, pout or stamp their feet when given an idea for playing (ASBI item 4), or to be worried about not getting enough attention, or access to toys, food or drink (item 28). This is particularly interesting, since mixed age rooms were rated as being of higher quality. In-depth analysis of quality showed that the elements of provision which improved with the presence of older children related to educational provision. Thus, mixed age rooms may be better for children in terms of cognitive outcomes, but not in terms of behavioural outcomes.

Workless households

Attending a centre with a high proportion of working families had a positive relationship with children's behaviour. Children in centres with high proportions of workless households were less co-operative and more anti-social than children from centres with high proportions of working households (families with at least one employed adult). In fact, attending a centre with a high proportion of children from working households had more of an effect on anti-social behaviour than the child's own family employment status. Only one employment effect was found for the child's own family - children living in houses with at least one working adult were more sociable with their peers than those living in workless families (and this effect was stronger for the older age group than for the younger). This evidence provides strong support for the aims of the Neighbourhood Nurseries Initiative, and suggests that encouraging parents to return to work will have positive benefits for children.

Key Findings: Effect of time spent in centre-based childcare on children's behaviour

The findings confirm previous research (e.g. Melhuish et al, 2001) in suggesting that attending centre-based childcare provision has both beneficial and detrimental effects on children's social and behavioural development. The more time (hours and days) children spent each week at a childcare centre, the more confident they were, and the more sociable they were with their peers. Staff in the sample centres rated them as more likely to say nice or friendly things to others (ASBI item 12) or to join a group of other children playing (item 13), and as more confident with other people (item 22). Children who spend more time each week at a pre-school centre have greater opportunities to mix with other children, and to become confident in their social skills, than those who attend for less time. Additional analysis was carried out to establish how much time children need to spend in centres to see significant benefits in terms of their social behaviour and confidence. The results suggest that the tipping point' is around 35 hours and/or five days attendance i.e. almost full-time.

Attending centre-based provision also had a number of less positive relationships with children's behaviour. Children who attended for at least 30 hours and/or 3 days every week were rated as more anti-social, for example more likely to tease other children and call them names (ASBI item 21), prevent other children from carrying out routines (item 23) or be bossy and need their own way (item 29). In addition, children who attended for at least 35 hours and/or 5 days each week displayed more worried and upset behaviours. These tipping points' identified in relation to time spent in centre-based provision are similar to those identified by the NICHD study (2005).

The effect of time spent in centre-based provision was more important for the younger children in the sample than for the older children. When the sample was split by age, the effects of hours and days attended each week on sociability, confidence and worried/ upset behaviour were significant only for the younger half of the sample i.e. children under 2 years 9 months. However the effect on anti-social behaviour was significant for both age groups, and this suggests that intensity of child care (measured in hours/days per week) is relevant for children up to the age of 3 years.

Length of day (the number of hours children attended their Neighbourhood Nursery each day) did not appear to be detrimental: there were no significant differences between children who attended for long periods each day and those who attended for shorter days in terms of co-operative behaviour, peer sociability, anti-social behaviour or worried/ upset behaviour. A significant effect was seen on children's levels of confidence, but only at very high levels of daily attendance (in comparison to children who attended less than 5 hours per day on average, children who attended for 9 hours or more were significantly more confident). However, the fact that very few of the children in the sample attended for short days (only 16% attended for fewer than 5 hours) means that these results should be interpreted with caution: there was not enough variability in the sample to accurately assess the effects of length of day on children's behaviour.

Duration of childcare during the early years also had a statistically significant effect – the longer children had been attending their Neighbourhood Nursery, the more likely they were to display anti-social behaviours. Interestingly, the age at which children started attending their Neighbourhood Nursery did not have an impact on their behaviour (either positive or negative). Thus, it is not the age at which children start at their centres which is important, but the cumulative number of months they attend, and the amount of time they spend in centre-based provision each week.

Key Findings: Effect of child and family characteristics on children's behaviour

In line with previous research, children's positive behaviours were most strongly predicted by child and family characteristics (Melhuish et al, 2001; NICHD, 1998b) – although negative behaviours were more strongly related to childcare experiences and centre characteristics. In general, girls displayed more positive behaviours than boys, and older children were rated as being more confident, sociable and co-operative than younger children. Perhaps not surprisingly, children with special needs were rated by their caregivers as being less sociable with their peers and less confident than children without special needs. Children for whom English was not the first language spoken at home were also rated as less sociable with their peers.

Final conclusions

• There was wide variation in the quality of provision for children in infant and toddler rooms.

• Higher quality was seen in the Local Authority maintained sector, in Children's Centres and in larger centres.

• Observers found higher quality provision, particularly educational provision, in mixed age rooms which included older children as well as under threes. However, the presence of older children was not always beneficial for the younger ones, who displayed more worried and upset behaviours in mixed age rooms.

• No relationships were found between the population of children and families served and quality of provision. This suggests that families from very different backgrounds, and with different needs, were being offered the same quality of provision.

• The findings highlighted the importance of a well qualified workforce for the provision of high quality caregiving and for child outcomes. Children with access to a trained teacher were more co-operative and sociable, and children in rooms with a better qualified workforce were more co-operative and displayed fewer worried and upset behaviours than children cared for by less well-qualified staff

teams.

• The quality of the physical environment was identified as important. Children displayed fewer worried and upset behaviours in centres which offered a spacious, well maintained and pleasant physical environment, with appropriate furniture for care routines and educational activities, and comfortable areas for children to relax and spend quiet time.

• Older children (those aged between 33 and 42 months) showed more peer sociability in centres which provided a high quality daily schedule, for example an appropriate daily routine, opportunities for free play and high quality group play activities.

• The effects of quality on children's behaviour were significant, but moderate in size compared with other (stronger) influences, such as gender, age, special needs and time spent in centre-based childcare.

• Time spent in centre-based childcare (hours/days per week) had some beneficial effects on children, such as greater confidence and sociability. This effect was stronger for the younger children in the sample (those under 2 years 9 months), and for children attending 35 hours per week or more. However, time spent in centre-based childcare was also related to negative behaviours. Children who

attended 30 hours or more each week were rated as more anti-social, while children who attended 35 hours or more displayed more worried and upset behaviours.

• Although the age at which children started attending their Neighbourhood Nursery did not have an impact (either positive or negative) on their behaviour, duration of childcare during the early years was important: the longer children had been attending their Neighbourhood Nursery, the more likely they were to display anti-social behaviours.

• Although larger centres were generally of higher quality, the effects of centre size on children's behaviour was mixed. Children in larger centres were less anti- social and displayed fewer worried and upset behaviours, but were also rated as less co-operative and less sociable than children in smaller centres.

• Attending a centre with a high proportion of working families had a positive effect on children's co-operative behaviour, and also reduced anti-social behaviour. This supports the aims of the Neighbourhood Nurseries Initiative, and suggests that encouraging parents to return to work may have positive benefits for the development of their children.

Recommendations

1. The development of a well-qualified childcare workforce is vital for improving quality and positive child development. In particular, employing qualified teachers to work with children under the age of 3 will have a significant impact on children's developing co-operation and other peer skills.
2. The development of Children's Centres should be supported. NNI settings with Children's Centre status were of higher quality and had better child outcomes. Future support (and evaluation) of the programme should focus on the educational aspects of provision to ensure that the learning' aspects of the curriculum are given equal weight to the more social' aspects.
3. This research supports the development of larger centres: these offered higher quality (measured on the ITERS-R scale) and their children showed reduced levels of anti- social and worried/upset behaviour. However, larger centres need to be supported in finding ways to ensure that their children are not overwhelmed by size, and are provided with the nurturing environments they need to develop their confidence and sociability.
4. Further research into the impact of mixed age rooms is recommended. They may enhance cognitive development at the price of emotional security.
5. More research is also required to explore the effects of length of day on children's behaviour. In particular, the effects of attending for a small number of long days over a week, as compared to a greater number of short days, need to be explored.
6. A broad social mix is recommended for early childhood settings – higher proportions of working families were related to decreased anti-social behaviour. Initiatives such as the NNI which address unemployment should be encouraged and supported.
7. Maintained centres should continue to be supported and developed, as these were particularly effective at offering high quality educational provision. Nurseries in other sectors need further support to raise the quality of the provision they offer.

1. CONTEXT AND AIMS

1. Introduction

The Neighbourhood Nurseries Initiative (NNI), launched in 2000, was one of a number of programmes aiming to expand early years services following the government's announcement of the National Childcare Strategy in 1997. Its remit was to increase the supply of childcare for working parents in disadvantaged neighbourhoods. The NNI programme aimed to tackle child poverty through parent employment, whilst also offering quality childcare and early learning in suitable buildings and environments.

The NNI evaluation was carried out by the Department of Social Policy and Social Work and the Department of Educational Studies in the University of Oxford, the Institute for Fiscal Studies (IFS) and the National Centre for Social Research (NatCen), on behalf of the Department for Education and Skills (DfES).

2. The Childcare Quality and Children's Behaviour Study

The Childcare Quality and Children's Behaviour Study was part of the NNI Implementation Study (Smith et al, in press)[3]. It had two main strands.

The Childcare Quality strand described the quality of provision offered by a random sample of 103 Neighbourhood Nurseries. The aim was to establish whether the new places and nurseries created by the NNI were of sufficient quality' to foster the development of the children attending them. It also considered a number of other centre characteristics (for example, sector, qualifications of centre staff, centre size), with the aim of establishing which of these characteristics were related to, and predicted, quality of provision.

The Quality and Children's Behaviour strand used the information gathered on the quality of Neighbourhood Nursery provision to explore the effects of quality on children's social and behavioural development. This was intended to fill a major gap in the UK literature – that is, the relationships between the quality of early centre-based childcare and young children's social and behavioural development. The EPPE findings (Sylva et al, 2004) show that some children whose parents retrospectively reported using group care before age 3 were more anti-social than children from similar backgrounds who did not have early care. High quality care between the ages of 3 and 5 ameliorated (although did not eradicate) the anti-social effects of care in the very early years. However, because the quality of settings for children under 3 was not measured in EPPE, we do not know whether the early group care which was related to increased anti-social behaviour was of low quality. The Quality and Children's Behaviour strand aimed to answer this question by investigating the effects of quality of provision on the behaviour of children under the age of 3 years from disadvantaged areas, after controlling for family background. The study also explored which other centre and childcare characteristics were related to children's social and behavioural development.

3. The research context

There is strong evidence from studies in many countries that attending pre-school can be beneficial for children (Melhuish, 2004a& b). The US NICHD study (2002) found that the benefits of attending pre-school - compared to not - were greater than the effects of social disadvantage. In the UK, the Effective Provision of Pre-School Education (EPPE) project has shown conclusively that children benefit intellectually and socially from attending pre-school (Sylva et al, 2004), and that the effects continue into primary school. Research also highlights the role of early years provision in combating disadvantage – the EPPE project found that children from disadvantaged families benefited most from attending childcare provision.

Effects of childcare on children's social behaviour

There is a comprehensive body of research which considers the impacts of childcare on children's intellectual development, for example language and reasoning skills. However, this study focuses attention on children's social behaviour. This area is of particular interest, as previous research has drawn mixed conclusions. On the one hand, it suggests that attending pre-school provision can have a positive impact on desirable' behaviours such as social competence, confidence and compliance. The NICHD study (1998a) found that attending group care was linked to more co-operative behaviour at age 2. Likewise, EPPE found that children who begin attending centre-based provision early in life (before the age of 3) are more sociable with other children. However, some of the research findings are more worrying. Some of the EPPE early starters' – children who began attending centres before the age of 3 – were found to have higher levels of anti-social behaviour. Similarly, the NICHD study (2003) found that the more time children spent in non-maternal care across the infant, toddler and pre-school years, the more problem behaviours they showed at 54 months of age and in kindergarten.

So, does all pre-school provision have the same effect on children? Are the centres which benefit children, making them more confident and sociable, the same ones that increase levels of anti-social behaviour? The evidence suggests not – and highlights the importance of quality.

Quality matters

Research suggests that high quality pre-school centres not only have a positive impact on desirable' behaviours, but can also reduce problem behaviours (e.g. Howes, 1988; Melhuish 2004a & b; NICHD1998a; Peisner-Feinberg and Burchinal,1997; Peisner- Feinberg et al, 2001; Whitebook, Howes and Phillips, 1989). In fact, EPPE found that high quality education and care between the ages of 3 and 5 reduced the early effects of childcare on anti-social behaviour. Perhaps more importantly, it seems that entering low quality provision at an early age can have a negative effect on children's behaviour (e.g. Howes, 1990; Volling and Feagans, 1995).

It is clear that the quality of pre-school provision needs to be taken into account when considering the role childcare plays in shaping children's behaviour. However, some studies have found the effects of early years quality to be modest in comparison to the influence of children's home background (e.g. Deater-Deckard et al, 1996). EPPE argues strongly that family factors need to be taken into account when considering the impacts of educational provision on children's development, stating that children's outcomes are

the joint product of home and pre-school, and any research on the effects of early education will have to take into account influences from home' (Sylva et al, 2004). It is possible that the relative influences of home and pre-school vary according to the child's experience of each. Howes (1990) found that, for children enrolled as infants, childcare was a better predictor of outcomes than family socialisation – while the opposite was true for those enrolled as toddlers or pre-schoolers.

This study aimed to explore some of these issues and establish, after child and family characteristics have been accounted for, the effects of early centre-based childcare on young children's social behaviour. In particular, it explored the role of childcare quality in encouraging the development of positive behaviours and/or reducing the likelihood of problem behaviours.

The majority of studies to date have focused on provision for 3 and 4 year old children. This study looked specifically at provision for children under the age of 3 , in areas of disadvantage targeted by the Neighbourhood Nurseries Initiative.

What constitutes quality of provision'?

Melhuish (2004b) identifies two dimensions of provision quality:

• Process quality is dynamic and interactional. It relates to children's experiences, for example their interactions with adults and peers and educational experiences.

• Structural quality relates to fixed features of the environment such as staff:child

ratios, group sizes and caregiver qualifications and training.

This study considered the role of both process and structural features of provision quality. In addition, a number of other centre characteristics identified by previous research[4] as being potentially relevant to quality of caregiving and/or to children's development were considered. For example:

• Sector/ type of centre (e.g. maintained, private, voluntary);
• Community partnerships and links with other programmes;
• The physical environment (e.g. safety);
• The age ranges of children catered for.

4. The structure of this report

This report presents the findings of the Childcare Quality and Children's Behaviour Study. The methodology of the study is outlined in Chapter 2. Chapters 3 and 4 consider the quality of provision offered by a sample of 103 Neighbourhood Nurseries. In Chapter 5, the effects of quality on children's social and behavioural development are considered. Finally, Chapter 6 draws conclusions and discusses implications.

2. METHODOLOGY

1. Sampling

1. Centres

The sampling procedure began with 102 Neighbourhood Nurseries randomly selected for the NNI Implementation Study (Smith et al, in press). This sample was stratified to be representative of the overall population of Neighbourhood Nurseries on several criteria – their geographies (London, East West, etc), sectors (private, voluntary, etc) types (new build, refurbishment, etc), and links to other related programmes (Sure Start Local Programmes, Children's Centres, etc). Nurseries were sampled in three stages, to ensure early openers were represented, as well as later groups that took more time to set up. The stratified sampling procedure was undertaken for all nurseries that were declared to be open, had more than ten NNI places and (for stages 2 and 3) had not been included in the sampling process at an earlier stage.

A number of nurseries in the original sample of 102 were not yet offering provision for infants and toddlers. Since this was the main focus of the Childcare Quality and Children's Behaviour Study, these centres were not suitable for inclusion in this element of the evaluation. In addition, a number of Neighbourhood Nurseries in the implementation sample had two NNI funded sites. While the Implementation Study considered both sites, only one was selected and visited as part of the Childcare Quality and Children's Behaviour Study (usually the site with the largest number of children under the age of 3 ). As a result, only 96 of the centres in the implementation sample were included in the quality sample.

In order to meet the target sample of 100 nurseries for the Childcare Quality and Children's Behaviour Study, a number of additional centres were selected using the same randomised method described above. This resulted in a sample of 103 Neighbourhood Nurseries for the Childcare Quality strand, 100 of which also had children eligible for the Quality and Children's Behaviour strand (see following section).

2. Children

Information was collected on 810 children attending 100 Neighbourhood Nurseries. Children were selected using the following criteria:

• Age: between 20 months and 42 months;

• Length of attendance at Neighbourhood Nursery: at least 6 months;

• Hours of attendance: at least 10 hours per week[5].

The mean age of children in the sample was 33 months (2 years 9 months). A maximum of 20 children were selected from each centre. In many cases, the number of eligible children in each centre was less than 20. The upper limit was imposed to ensure that the larger centres (i.e. those with more than 20 eligible children) did not have an undue influence on the sample.

An additional method was employed to ensure that, where possible, the quality of provision measured for each centre was that experienced by the children in the sample. Where there were more than 20 eligible children in any one centre, children were selected according to the amount of time they had spent in the room observed.

2. Data collection

1. Information collected on quality of provision

Visits to the sample Neighbourhood Nurseries took place between February 2004 and July 2005. Observations of up to a day were conducted in one of the rooms providing for children under the age of 3 . Two observational instruments were used to assess quality:

The Infant Toddler Environment Rating Scale (Harms, Cryer & Clifford, 2003)

The revised version of the ITERS scale (the ITERS-R) consists of 39 items divided into seven subscales, covering a comprehensive range of quality features:

• Space and furnishings (e.g. layout of the room, resources, display);

• Personal care routines (e.g. health and safety, hygiene, meal times);

• Listening and talking (e.g. supporting children's language development);

• Activities (e.g. dramatic play, sand and water, fine motor play);

• Interaction (e.g. supervision, discipline, staff-child interactions, peer interactions);

• Program structure (e.g. opportunities for free play, group activities, transitions);

• Parents and staff (e.g. information for parents, staff training opportunities).

Each item is rated on a 7 point scale from 1 (inadequate), through to 3 (minimal), 5 (good) and 7 (excellent). For the purposes of this study, scores between 1 and 3 were labeled below minimal', scores between 3 and 5 were labeled adequate quality' and scores between 5 and 7 were labeled good quality'.

The average of item scores in a subscale gives the mean score for that subscale. An overall quality rating for each centre is calculated by taking the mean of all items across all subscales. An overview of the items and subscales which make up the ITERS-R is shown in Appendix 3.

The ITERS-R is designed to assess provision for children from birth to 2 years. A second scale in the same family', the Early Childhood Environment Rating Scale or ECERS-R, assesses provision for children from 2 to 5 years. Since this study focused on provision for infants and toddlers, the ITERS-R was selected as the quality assessment instrument. In practice, many rooms observed also catered for older children: 36% catered for children over the age of 3, and 19% for children aged 4 years or over (see Section 4.7). However, since 64% provided only for children aged 3 or under, the ITERS- R was considered to be a more appropriate instrument than the ECERS-R.

The Caregiver Interaction Scale (Arnett, 1989)

The CIS consists of 26 items forming four subscales, each of which measures a different aspect of caregiver-child interaction. Each item is rated on a four point scale, ranging from 1 (not at all) to 4 (very much), according to how often caregivers display a particular behaviour. For example:

• Seems to enjoy the children 1 2 3 4

The positive relationships' subscale (indicating warmth and enthusiasm in interaction with children) was used for this study. This subscale has been shown to predict children's later academic success, for example in pre-reading (Sylva et al, 2004). The 10 items which make up the positive relationships subscale of the CIS are shown in Appendix 3.

Quality terminology used in this report:

• Where the term overall quality of provision' is used in this report, it refers to the mean total score achieved on the ITERS-R.

• Where individual dimensions of quality' are referred to, this relates to one or more of the seven individual subscales of the ITERS-R and/or the CIS positive relationships subscale.

Inter-rater reliability

In any study of this nature, it is important to check inter-rater reliability, i.e. how consistently members of the fieldwork team are using the observation instruments. This provides evidence that any differences in observed quality are real, rather than arising from differences between raters. Sixteen paired visits were conducted, where two observers scored independently of each other and then compared ratings.

Inter-rater reliability on the ITERS-R was assessed using Cohen's Kappa. This measures the level of concordance between two raters, allowing for the level of chance agreement. A Kappa value of 0.8 or above indicates an excellent level of agreement between two raters. A value of between 0.6 and 0.8 is reasonable. The average Kappa value for ITERS-R scores across all 16 paired visits was 0.8, with scores ranging from 0.6 to 0.9. This indicates that the reliability for this instrument was very good.

2. Information collected on centre characteristics

The NNI Implementation Study (Smith et al, in press) provided information on the general characteristics of centres in the sample, including[6]:

• Sector (Local Authority maintained; voluntary; private; joint[7]);

• Community partnerships and links with other programmes (involvement in Children's Centre Programme; links with Sure Start Local Programmes; school site location);

• Physical environment (new build; extension; refurbishment);

• Centre size (number of FTE places; number of children registered);

• Populations of children and families served (proportion of: lone parent families; families living in disadvantaged postcodes; workless households; children from ethnic minorities; and children with special educational needs).

When the sample centres were visited for the quality observations, information was also gathered on the general characteristics of the rooms observed, and on the qualifications of staff:

• Group size (number of children registered; maximum capacity of room);

• Staff:child ratios;

• Age ranges of children (age-segregated; mixed rooms);

• Childcare qualifications (nursery manager; all paid staff working at least 10 hours per week in the room observed).

3. Information collected on children and families

Information on the children in the sample (and their families) was collected using the Adaptive Social Behaviour Inventory (ASBI) and a family profile devised for this study.

The Adaptive Social Behaviour Inventory (Hogan et al, 1992)

The ASBI is made up of a series of 30 behaviour statements. The person completing the profile rates each statement from 1 to 3, according to how often the child displays that particular behaviour. For example:

Rarely Some- Almost or never times always

The items of the ASBI are grouped into five categories, each measuring a different dimension of children's social and behavioural development:

The co-operation and conformity' subscale contains items such as:

• Is helpful to other children'

• Is calm and easygoing'

The peer sociability' subscale contains items such as:

• Understands others' feelings, like when they are happy, sad or mad'

• Will join a group of children playing'

The confidence' subscale contains items such as:

• Is open and direct about what s/he wants'

• Is confident with other people'

The anti-social' subscale contains items such as:

• Teases other children and calls them names'

• Is bossy/ needs to have his/her own way'

The worried and upset' subscale contains items such as:

• When you give him/her an idea for playing, s/he frowns, shrugs shoulders, pouts or stamps foot'

• Gets upset when you don't pay enough attention'

The five subscales of the ASBI (and the items which relate to each) are shown in Appendix 3.

The original sample on which the ASBI was developed and tested comprised children aged 36 months (Hogan et al, 1992). However, the children in the Hogan study were all born prematurely. Hogan suggests that the ASBI scale should in fact be sensitive to a broader developmental span' and have the greatest utility for children who range in age from younger to slightly older than 36 months i.e. 24 – 40 months'. Since this is very close to the age range seen in the current sample (20 – 42 months), the ASBI was considered to be the most appropriate instrument for this study.

In the original Hogan study, conducted on pre-term children, the 30 ASBI items were grouped into three subscales – social competence', comply' and disrupt'. More recently, the EPPE study (Melhuish et al, 2001) developed the five subscale resolution used in the current study. These five dimensions of behaviour were thought to be more appropriate for the current sample, since the EPPE analysis was conducted more recently than the Hogan study, comprised a larger sample of typically developing children and, most importantly, was carried out in the UK.

Staff at the Neighbourhood Nurseries were asked to complete the ASBI. This was seen as an important strand of the evaluation methodology, and one which ensured accurate ratings of child behaviour. In each case, the ASBI was completed by the member of staff who knew the child best, usually their key worker. The nurseries were paid £20 for each completed ASBI (and accompanying family profile and consent form) to reflect the serious nature of the data collection.

Family Profile

A Family Profile was devised for this study to collect information on the children's home backgrounds. The families of children attending the sample Neighbourhood Nurseries were not the main focus of this study. However, as previous research had shown that home background has a large impact on children's social behaviour and development, it was necessary to take account of these influences when considering the effects of the NNI provision. The full family profile is shown in Appendix 3. Information was collected on:

• Child characteristics (e.g. gender, age, birthweight, special needs, ethnic group);

• Family characteristics (e.g. family structure and work status, number of siblings, age of mother/ father/ partner, age mother/ father/ partner left full time education, language spoken at home);

• Current childcare and childcare history (e.g. starting age at Neighbourhood Nursery, duration of time at Neighbourhood Nursery, hours/ days per week in centre-based/ family/ childminder care, childcare history aged 1 to 2 years, childcare history under the age of 1).

4. Ethics

The study reported here was carried out under the ethical guidelines of the Department of Educational Studies at the University of Oxford. It meets the standards laid down by the British Educational Research Association as regards to consent, confidentiality and data protection.

5. Summary of data collected

Tables 2.1 and 2.2 summarise the data collected on both outcomes and predictors.

Table 2.1 Outcome data collected

Table 2.2 Predictor data collected

Child

• Gender

• Age

• Birthweight

• Special needs

• Medical problems during first three months of life

• Ethnic group

Family

• Family structure (living with both parents/ parent plus partner; living with lone parent)

• Number of siblings

• Place in birth order

• Language spoken at home (English; other language)

• Age of mother

• Age of father/ partner

• Age mother left education

• Age father/ partner left education

• Mother work status (full time; part time; not working)

• Father/ partner work status (working; not working; father or partner not present)

• Household work status (anyone in household working; workless household)

Current childcare and childcare history

• Starting age (age at which child first attended Neighbourhood Nursery)

• Duration of childcare (months spent at Neighbourhood Nursery)

• Weekly attendance at centre-based provision:

• Number of hours per week (at Neighbourhood Nursery; at any centre)
• Number of days per week (at Neighbourhood Nursery)

• Average hours per day at Neighbourhood Nursery (derived variable: hours pw/ days pw)

• Hours per week spent in family care

• Hours per week spent in childminder care

• Childcare between the ages of 1 and 2 years (centre-based; family care; childminder)

• Childcare under the age of 1 (centre-based; family care; childminder)

General centre characteristics

• Sector (maintained; private; voluntary; joint project)

• Level of involvement in Children's Centre Programme

• Links with Sure Start Local Programmes

• School-site location

• Physical environment/ project type (new build; extension; refurbishment)

• Centre size (total FTE places; number of children registered)

• Group size (room observed)

• Staff:child ratio (room observed)

• Age range of children catered for (room observed)

• Staff childcare qualifications:

o Nursery manager qualifications

o Mean qualification level of staff (working 10 hours or more) in room observed

o Qualified teacher present (either working in room or as manager)

• Characteristics of children and families served (by the whole centre):

o Proportion of lone parents

• Proportion of families living in deprived postcode areas
• Proportion of families with no working parent/s
• Proportion of families from ethnic minorities
• Proportion of children with special education needs

Quality of provision*

• Quality of provision (as measured by the ITERS-R)

• Individual subscales:

Space and furnishings

Personal care routines

Listening and talking

Activities

Interactions

Program structure

Provision for parents and staff

• Overall quality of provision (mean of all items)

• Quality of staff-child interactions (as measured by the CIS)

* For the childcare quality' strand, quality was an outcome measure. For the quality and children's behaviour' strand, quality was a predictor variable.

3. Design and analysis

1. The Childcare Quality strand

The first aim of the Childcare Quality strand was to describe the quality of provision offered by a random sample of 103 Neighbourhood Nurseries, in order to establish whether the new places and nurseries created by the NNI were of sufficient quality to foster the development of the children attending them. Two observational instruments were used to assess quality of provision: the Infant Toddler Environment Rating Scale (ITERS-R) and the Caregiver Interaction Scale (CIS) (see section 2.2 for details).

The second aim of the quality strand was to establish which centre characteristics were related to, and predicted, quality of provision. The majority of characteristics considered were those identified by previous research as being relevant to quality of provision (e.g. sector, staff qualifications). In addition, a number of characteristics particularly relevant to the sample were collected – for example, level of involvement in the Children's Centre Programme and links with Sure Start Local Programmes. Information on the populations served by the sample centres was also gathered, with the aim of establishing whether centres providing for high proportions of disadvantaged families offered comparable quality to those catering for a less disadvantaged clientele.

Figure 2.1 shows the analysis carried out to explore the relationships between the characteristics of the sample centres and quality of provision as measured by the ITERS-R and the CIS. Relationships between each individual characteristic (e.g. sector) and quality were first established using univariate techniques (analysis of variance, t-tests and correlation). Once initial relationships had been identified, multiple regression analyses were used to explore which centre characteristics were most predictive of quality of provision. This technique makes it possible to establish the individual contribution made by each characteristic, after controlling for the effects of other relevant variables.

Figure 2.1 Identifying contributors to quality

Predictors Quality Measures

Sector

(LA maintained, private, voluntary, joint)

Programme links

(Children's Centre status, Sure Start links, school- site location)

Physical environment/ project type

(New building, refurbishment, extension)

Quality of provision (n = 103): Centre size

(FTE places, number of children registered) Overall quality (mean total ITERS-R score)

• Individual dimensions of quality:

Populations of children/ families served: o ITERS-R subscale scores

• Proportion of lone parents o CIS positive relationships subscale

• Prop. of workless families

• Prop. of families living in deprived

postcode areas

• Prop. of families from ethnic minorities