Analysis of duration data from longitudinal surveys subject to loss to follow-up

Abstract

Data from longitudinal surveys give rise to many statistical challenges. They often come from a vast, heterogeneous population and from a complex sampling design. Further, they are usually collected retrospectively at intermittent interviews spaced over a long period of time, which gives rise to missing information and loss to follow-up. As a result, duration data from this kind of surveys are subject to dependent censoring, which needs to be taken into account to prevent biased analysis. Methods for point and variance estimation are developed using Inverse Probability of Censoring (IPC) weights. These methods account for the random nature of the IPC weights and can be applied in the analysis of duration data in survey and non-survey settings. The IPC estimation techniques are based on parametric estimating function theory and involve the estimation of dropout models. Survival distributions without covariates are estimated via a weighted Kaplan-Meier method and regression modeling through the Cox Proportional Hazards model and other models is based on weighted estimating functions. The observational frameworks from Statistics Canada's Survey of Labour and Income Dynamics (SLID) and the UK Millenium Cohort Study are used as motivation, and durations of jobless spells from SLID are analyzed as an illustration of the methodology. Issues regarding missing information from longitudinal surveys are also discussed.