There used to be the Halifax house price index and one from Nationwide. That was it. They were definitive; house prices went up or down, and you knew exactly by how much.

Now, there’s a much wider range of residential property indices. IPD, LSL, Hometrack, Rightmove, Primelocation, it can be a confusing landscape, with some telling you prices are rising, other that they’re falling.

They don’t even agree on the average house price. Academetrics, which produces the LSL index, points out that the average house price in 2010 might have been £220,000 or £170,000, depending on which survey you chose to believe.

If you’re confused, you’re in good company, back in 1998 Mervyn King, Governor of the Bank of England, complained that the Nationwide and Halifax indices had diverged widely and neither he nor his economists could really understand what was going on with house prices. Nationwide was reporting a 12 per cent rise in prices, Halifax said it was 5.6 per cent, and the Bank, after careful research, thought it was probably about nine per cent. So why do these indices give such different answers? And do we
need them all?

To understand why the indices have different results, you have to look at the methodologies they use, what data they are based on, what is excluded, and how the raw data is adjusted. They’re also of different ages; Halifax goes back to 1983, the government’s Department for Communities and Local Government (DCLG) to 1968, and Nationwide to 1952 (though it was only annual until 1974), while some of the other indices don’t yet cover a full market cycle. So if you’re looking to understand the very long term movements of the market, only the older indices are going to be much help.

Base data

First of all, the indices all use different base data. The Halifax and Nationwide data use the mortgages arranged by each lender as the basis for compiling the index, so, automatically, that excludes houses sold for cash. But it also means that the index will reflect the lending criteria being used by the lender, of course.

Land Registry and the LSL index are based on completions, using the whole market as the base for the index, but Land Registry excludes repossessions and houses bought at auction, which, it says, don’t represent the true market value. Meanwhile, Rightmove tracks prices for properties added to the website during the month, but appears not to capture changes in the price of properties that are already on the site. (Maybe a Property Snake index would be a good corrective!) Halifax excludes buy-to-let and other non-owner occupation purchases, while DCLG includes buy to let, though not purchases by sitting tenants.

Other indices exclude properties above and below a certain value. For most, this cuts out only very low value transactions (beach huts, garages, wrecks) or very high value. Home.co.uk publishes an asking price index using over 750,000 prices, but excludes those over £1m and under £20,000; the difficulty, of course, is that in the London market in particular, the £1m-plus exclusion cuts out many reasonably ordinary properties.

On the other hand, Primelocation excludes the vast majority of the market in order to report only prime properties (top quartile) and, within this, prime platinum (top decile). If you’re selling prime property, this index might be more worth tracking than the better known indices, from which it diverged in 2011. Despite three months of fall-back, the Primelocation index still shows prime property prices increasing 3.65 per cent year on year to February 2012, against national prices 0.9 per cent down (Nationwide).

Some idea of the variety of datasets used can be gained by looking at how many properties each index tracks. At the top end, Rightmove tracks 200,000 a month, and Land Registry, 100,000 a month; at the lower end, Halifax tracks 15,000, and the DCLG index takes a sample of 25,000 completions from lenders.

Timing is everything

The stage at which the data is taken also varies, and this affects both the prices reported, and the timeliness of the index. For instance Rightmove reports asking prices at the time the property is put on the market; this will quickly reflect changes in market sentiment so it can be seen as a lead indicator. Halifax and Nationwide report mortgages on approval of the loan, a little later in the chain, while the Land Registry and LSL indices report completions. While that’s high quality data, it is not particularly timely, full figures might not be available for three months after the date of completion. Academetrics notes that “speed to market comes at a price”, the samples in the earliest reporting indices may not be as reliable as the full Land Registry data.

LSL/Academetrics gets round the problem of tardy Land Registry data by issuing forecasts which use the other indices to adjust the previous month’s Land Registry figures. The forecast is then updated twice in the month as the figures become available.

Techniques

The LSL figures also report the level of transactions, as well as price. That makes LSL a particularly useful index for estate agents who can easily gauge how the market is developing and check their own inventory development against the market.

Different indices also use different techniques for smoothing out seasonal fluctuations. Some don’t adjust, though this exposes the index to high volatility. Some adopt seasonal adjustment techniques that allow for the fact that the spring always sees more house-hunting and purchase activity than is the case later in the year. Halifax and Nationwide report both seasonally adjusted and non-adjusted data.

Seasonal adjustments are often reported in the press. But what’s not widely reported is how the various indices adjust for the mix of properties represented. For instance, many of the indices are ‘mix adjusted’, but not all of them are adjusted in the same way. This is what explains the differences in average house price. The Land Registry takes a simple average of all the prices of property transactions in the month, so if there are more London properties, or more larger detached houses, the average price will increase simply because of the mix.

Nationwide, on the other hand, says “we track representative house prices over time rather than the simple average price.” It uses a hedonic regression model, in plain English, this accounts for the characteristics of the properties in the index, so for instance it tries to analyse the price of ‘terracedness’ or ‘detachedness’, the number of bedrooms, and so on. Halifax uses a similar approach.

Other indices are mix-adjusted. Instead of trying to assess the characteristics of a property and ‘price’ each one, they divide the market into ‘cells’ (one bedroom flats, terraced houses, large detached houses) and allocate each transaction price to the relevant ‘cell’. The average prices for each type of property are then used to produce the overall average. This can work at a high level of detail, the DCLG index for instance has a specific cell just for “second-hand, semi-detached houses with six rooms bought by first-time buyers in the North-East.”

Fascinating facts

While the exact mathematics is only of interest to the more nerdy of us, the adjustment chosen does tend to skew indices one way or the other. For instance the mix-adjusted index rose a third faster than simple average prices in the 1980s as cheaper houses predominated.

Hometrack produces a survey, rather than an index, of 5000 agents.’

There are other statistical sources, of course, as well as indices. For instance Hometrack produces a survey, rather than an index, asking 5,000 agents and surveyors about pricing, rental values,  and so on, for various types and sizes of property. Obviously, this is based on individuals’ subjective assessments of the market, rather than achieved prices, but its big advantage for professionals (as opposed to homeowners) is that it gives other data too, the average time property spends on the market, changes in the volume of property listings, and what percentage of the asking price is being achieved. (The latter is quite interesting; it was 96 per cent at the top of the market, reached its nadir in February 2009 at 88 per cent, and has now rebounded to 92.9 per cent.)

Similarly, RICS carries out a survey, giving the percentage of surveyors who think that prices are rising or falling. That is obviously subjective, but it has proven to be a timely forward indicator, and quite good at predicting changes in the market’s direction. It’s also a good check on inventory.

One name that is rarely mentioned in relation to house prices is IPD, better known for its commercial property indices. But IPD does produce a residential property index; its big difference is that it looks at homes from the point of view of the portfolio investor, analysing returns rather than prices. IPD’s Mark Weedon says the index showed strong returns continuing in 2011, with total returns at 11.3 per cent, ahead of all other classes of property.

With residential property increasingly viewed as an investment, and institutional investors for the first time getting interested in the residential market, we will probably be hearing a lot more about the IPD index over the next few years. In February this year, IPD included residential in its ‘all property’ index for the first time, sending a very clear message that rental yields are as important as house prices. For agents involved in selling newbuild, there’s another specialised index that might be relevant – Knight Frank’s development land index. Unsurprisingly perhaps, the latest release showed London land prices rising by 20.3 per cent in 2011, while across the UK as a whole land prices were only just better than flat.

Adding it all up

You may not have been counting, but the total so far is 12 indices and surveys, tracking house prices could, if you’re not careful, become a full time job. That’s where the Chesterton Humberts Poll of Polls comes in. It aggregates all the major national house price and asking price indicators. Where it’s particularly clever is in the way it weights them according to how accurate they have been in the past, as well as a number of other criteria. It then adjusts the data it has to reflect the housing stock as a whole, rather than just those properties sold in the month. With this index, we come full circle, because it summarises the differences between the different indices included in the Poll of Polls. The January release showed that the other indices couldn’t even make up their minds whether the market was growing or slowing. Hometrack and the Land Registry said the market was flat; Nationwide said it was down 0.2 per cent, Home.co.uk thought it was down 0.3 per cent, and DCLG said it was down 0.1 per cent. But then the Halifax not only thought it was up, but very substantially up, with a 0.6 per cent rise. (One clue to the difference is that not all the latest releases referred to the same calendar month, but both Halifax and Nationwide were stating figures for January, so that’s nearly a whole per cent difference between the two of them.)

If you don’t keep up to date, you’ll be putting yourself at a disadvantage.’

There is no single figure that ‘works’. But if I had to pick one or two most important polls to follow, the Poll of Polls would certainly be one, because it gives a good feel for the other major polls, and the commentary is well written and insightful. I’d also want to follow RICS, because it is a good lead indicator, and because of the useful information on the balance of buyers and sellers and the level of sales and inventory per branch.

And I think I would keep a very close eye on the IPD figures. If the big investing institutions come into the housing market, yields and returns will become increasingly important, and if you don’t keep up to date, you’ll be putting yourself at a disadvantage.