Rationale, aims and objectives  Studies have challenged the validity and underlying measurement model of the physical and mental component summary scores of the 36-item Short-Form Health Survey in, for example the elderly and people with neurological disorders. However, it is unclear to what extent these observations translate to physical and mental component summary scores derived from the 12-item short form (SF-12) of the 36-item Short-Form Health Survey. This study evaluated the construct validity of the SF-12 in elderly people and people with Parkinson's disease (PD) and stroke.

Methods  SF-12 data from a general elderly (aged 75+) population (n = 4278), people with PD (n = 159) and stroke survivors (n = 89) were analysed regarding data quality, reliability (coefficient alpha) and internal construct validity. The latter was assessed through item-total correlations, exploratory and confirmatory factor analyses.

Results  Completeness of data was high (93–98.8%) and reliability was acceptable (0.78–0.85). Item-total correlations argued against the suggested items-to-summary scores structure in all three samples. Exploratory factor analyses failed to support a two-dimensional item structure among elderly and stroke survivors, and cross-loadings of items were seen in all three samples. Confirmatory factor analyses showed lack of fit between empirical data and the proposed items-to-summary measures structure in all samples.

Conclusions  These observations challenge the validity and interpretability of SF-12 scores among the elderly, people with PD and stroke survivors. The standard orthogonally weighted SF-12 scoring algorithm is cautioned against. Instead, when the assumed two-dimensional structure is supported in the data, oblique scoring algorithms appear preferable. Failure to consider basic scoring assumptions may yield misleading results.