Comparison of self-reported and register data on sickness absence among Norwegian patients participating in an occupational rehabilitation program

Background: Sick leave and return to work are common outcome variables in studies where the aim is to measure the effect of targeted interventions for individuals that are on sick leave benefits or other allowances. Use of official register data is often restricted, and research on sick leave and return to work are often based on the participants self-reports. However, there is insufficient documentation that there is agreement between self-reports and register data on sick leave benefits and allowances. Aims: The aim of this study was to analyse the individuals’ knowledge about states of sick leave benefits or allowances compared with register data from The Labour and Welfare Administration (NAV) in Norway. Method: 153 individuals, sick-listed or on allowances, participated in a 4-week inpatient occupational rehabilitation program. 132 (86%) answered a questionnaire on assessments of work, sick leave, and allowances three months after completed rehabilitation. Self-reported data were compared with register data from NAV according to four categories: working, sick-listed, on medical/vocational rehabilitation allowance or disability pension. Agreement between self-reported and register data was evaluated in cross-tabulations and reported with kappa values. Stratified analyses were done for gender, age, education, medical diagnosis and length of sick leave/allowances at baseline. Results: Good agreement was found for medical/vocational rehabilitation allowance (kappa=.70) and disability pension (kappa=.65). Moderate agreement was found for working (kappa=.49) and fair agreement for sick-listed (kappa=.36). Stratified analyses showed significant better kappa values for individuals that had been sick-listed less than 12 months before entering the rehabilitation program. Conclusions: Agreements from good to fair were found between self-reported and official register data on sick leave. However, official register data is preferred in research because this will ensure complete data sets. Data on sick leave and other benefits are not absorbing states, and there are often multiple and recurrent episodes. These data may be hard to obtain from self-reports.


INTRODUCTION
Return to work (RTW) is often used as an outcome after occupational rehabilitation and in research on sick leave and disability (Wasiak et al. 2007;Pransky et al. 2005).Register data are often lacking, or access is restricted due to ethical limitations concerning patient integrity (Fredriksson et al. 1998;Ferrie et al. 2005) or due to old or incomplete data sets and lack of validity (Fredriksson et al. 1998;Burdorf et al. 1996).Therefore, research often has to rely on self-reported data on sick leave (Ferrie et al. 2005;Voss et al. 2008).Surprisingly few studies have examined the level of comparability between self-reported data from questionnaires and data from registers.We have identified a total of seven studies comparing self-reported and register data on sick leave benefits (Agius et al. 1994;Burdorf et al. 1996;Severens et al. 2000;van Poppel et al. 2002;Ferrie et al. 2005;Voss et al. 2008).The only study examining the validity and reliability of self-reported data compared with official register data was a Swedish study (Fredriksson et al. 1998) where sufficient validity of retrospectively collected self-reported sick leave data was shown.Good agreement was also found between number of self-reported sick leave days and number of sick leave days recorded in employers' registers, during the last year (Ferrie et al. 2005;Voss et al. 2008).However, adequate agreement was not found between self-reported sick leave and employers' registers in the Netherlands (van Poppel et al. 2002).
Self-reported sick leave is less reliable when the recall periods (Severens et al. 2000;Fredriksson et al. 1998) or the length of episodes of sick leave are long (van Poppel et al. 2002;Ferrie et al. 2005).Differences in agreement have also been reported between women and men and between different levels of education (Ferrie et al. 2005;Voss et al. 2008), as well as between different occupations (van Poppel et al. 2002).
Within the national insurance system in Norway it is possible to be part-time working and at the same time receive graded sickness benefit, medical/vocational rehabilitation allowance or disability pension.Combinations between work and various types of part time benefits complicate the collection and analyses of sick leave data.Due to variations in data collection and presentation on data on RTW it is difficult to compare the results from different studies and there is a need for clear instructions about how to measure and report data, also on part-time benefits (Voss et al. 2008).There are large intra-national variations in how sick leave is registered in official registers and employers' registers (Hensing 2004).Furthermore both terminology and chosen measurements vary between different studies (Hensing et al. 1998).Measurements of sick leave should be chosen with care, because the results may be influenced by type of measure being used (Borg et al. 2006).
The aim of this study was to examine if selfreported data on sick leave benefits and allowances were comparable to register data from the Labour and Welfare Administration (NAV).A secondary aim was to examine if gender, age, education, medical diagnosis, and length of last sick leave spell or period on allowance before entry in an occupational rehabilitation program explained any variance in agreement.The third aim was to examine if number of sick leave spells, mean number of sick leave days per sick leave spell, and total number of sick leave days during the ten last years before entry in the occupational rehabilitation program explained any variances in agreement between self-reports and register data.

Participants and procedure
153 individuals participated in a 4-week inpatient occupational rehabilitation program during the autumn 2002.A questionnaire with self-reported assessments of current work and sick leave situation was distributed to all participants 3 months after completed rehabilitation.132 individuals (86%) answered the questionnaire.
Baseline data was retrieved from patient journals at entry in the rehabilitation program.Mean age was 46 years (SD=8.34)with a range from 24 to 61 years, and 67% (n=88) were women.The mean education length was 13.7 years (SD=3.3)with a range from 8 to 22 years.All participants were on sick leave or on medical/vocational allowances, with a mean duration out of work of 10.5 months (SD=2.8)before inclusion in the study.About one-third (33%) (n=43) of the partici-pants had been out of work for more than one year.104 participants (77%) had only one sick leave spell the last 12 months, whereas 28 participants (23%) had several sick leave spells.23 participants (17%) were part-time sick-listed at inclusion and 6 of the participants (4%) received graded disability benefit.All participants' medical diagnoses had been supplied by their GPs, and coded according to the International Classification of Primary Care (ICPC).55 participants (42%) were sick-listed for a diagnosis related to the musculoskeletal system, 51 (39%) for psychiatric diagnosis and 26 (20%) had other or unspecified diagnosis.The 132 participants answering the questionnaire at 3 months follow up did not differ from the total sample (N=153) on any baseline characteristics.
During the stay at the rehabilitation centre, the participants gave permission that register data from NAV would be collected after they had completed their participation in the rehabilitation program.Selfreported data were linked to the register data and prepared for statistical analysis.

Measures
Measures of self-reports: Self-reported data on sick leave benefits were measured with 4 questions about the work and sick leave situation, sickness-/unemployment benefits and disability pension: 1. What is your current work and sick leave situation (with 7 response alternatives)?: Returned to ordinary work, returned with adjusted work tasks, different work tasks/same employer, new employer, "workrelated re-employment"/active sick leave (at work paid by the social security benefit system), vocational rehabilitation and not working at the moment (due to sickness or unemployment).2. Do you receive sickness and/or unemployment benefits (with 4 response alternatives)?: Sickness benefit; 100%, 50% or other percent rate, medical/vocational rehabilitation allowance; 100%, 50% or other percent rate, unemployment benefit or other benefits (for instance private pensions or social security benefits).

Have you applied for disability pension? 4. Do you receive disability pension? (If so write percent rate).
The responses were not mutually exclusive.

Measures from the Labour and Welfare Administration (NAV):
Data from NAV on each individual included all states of benefits with start and end date on each allowance.The states were given as sick-listed, medical rehabilitation, vocational rehabilitation and disability pension.Available data from NAV did not contain any information if a person was working.In this study, all time periods where a person was not registered as receiving any allowances, the person was considered working at the current time.Data from NAV included continuous data during a time period going back 10 years before the participant entered into the rehabilitation program and ending 5 months after completed rehabilitation.

Comparison of self-report and NAV data
Self-reported and register data were not fully comparable.Therefore the measures were merged into 4 new comparable categories; working, sick-listed, medical/ vocational rehabilitation and disability pension (see table 1).
All participants received the questionnaire on work and sick leave situation 3 months after departure from the rehabilitation centre.First we conducted an analysis comparing self-report with data from NAV on the exact date 3 months after departure.
However, there is some uncertainty on the exact dates where the participants actually filled out their responses.We adjusted for this by inserting an interval of 10 days in front of and 60 days after the exact dates, before the main analysis comparing these data with the register data from NAV.Thus, if the individual e.g.reports to be on sick leave and the individual was reported to be on sick leave in the register data from NAV within the minus 10 to plus 60 days interval, we consider the information given from the individual to be correct.
Individuals may be part-time working and at the same time receive part-time benefit.Both the selfreported and register data from NAV contained graded states of benefits.We intended to examine agreement between self-reported and register data 3 months after the rehabilitation program including if the participants had been on full-or part time benefits.The same per-son may therefore be "counted" several times.For instance an individual registered with 50% sickness benefit and 50% returned to work, will be included in both categories.Consequently, 49 participants (from a total of 132) were counted in several categories at the same time.

Statistics
SPSS 14.0 for Windows was used for the statistical analysis.Agreement between self-reported and register data from NAV was evaluated in cross-tabulations.Agreement was reported with Kappa values.The Kappa coefficient measures statistical agreement between two observations adjusted for agreement occurring by chance.Kappa values between .80 -1.00 are explained as very good agreement, .60 -.79 good agreement, .40-.59 moderate agreement, .20 -.39 fair agreement, whereas Kappa values less than .19are slight agreement (Altman 1991).
Stratified analyses were done for gender, age, education, medical diagnosis and length of sickness absence/allowances at baseline.Discrepancy in Kappa values indicates differences amid the subgroups.In the stratified analyses we judged differences between 0 and .19 to be minor, differences between .20 and .39 to be moderate and differences from .40 to 1 to be considerable.
Earlier sick leave during the ten years before rehabilitation was described with three variables; number of sick leave spells, mean number of sick leave days per sick leave spell and total number of sick leave days.Each of the three variables was dichotomized by the median into two new categories before agreement was evaluated in cross-tabulations.* The grade has been calculated when it was not reported in the questionnaire.E.g. when the participant has ticked off both returned to work and graded benefit.# The total is greater than 132 due to graded benefits.

Agreements between self-reported and register data
Agreements from good to fair were found between register data and self-reports 3 months after rehabilitation.Best agreement was found for medical/vocational rehabilitation (kappa=.70)and for disability pension (kappa=.65).Moderate agreement was found for working (kappa=.49)and fair agreement for sick-listed (kappa=.36)(see table 4).All reported kappa values are adjusted inserting an interval of 10 days in front of, and 60 days after the exact dates.Unadjusted numbers are reported in table 3 showing a minor discrepancy after the adjustment.
Self-reports of the participants current work and sick leave situation is reported in table 2 and concurrent registered data from NAV is reported in table 3. Graded benefits are included in the calculation of the Kappa values, but due to unnecessary complexity, the numbers of individuals on different graded benefits are not reported.

Do baseline characteristics influence agreement?
The agreements between self-reports and register data were influenced by duration of the last sick leave spell before entry into the occupational rehabilitation program.Individuals sick-listed less than 12 months at baseline showed better agreements on all four categories; working, sick-listed, medical/vocational rehabilitation and disability pension (see table 4).Moderate differences in agreements were found between women and men, for age and education.Men had better agreements on the categories sick-listed and disability pension compared with women.Individuals above 45 years had better agreements than individuals below 45 years on the category medical/vocational rehabilitation.It was also better agreements on medical/vocational rehabilitation for individuals with more than 12 years education compared with individuals with 12 years or less at school.Moderate but no systematic differences were found in agreements between different medical diagnoses.
Earlier sickness absence episodes during the last ten years before rehabilitation did not explain any agreements between self-reported and register data (see table 5).* Adjusted with an interval of -10/+60 days in front and after the exact day respectively.# Available data from NAV did not contain any information about a person actually working.Participants without any benefits were considered working.The working category was adjusted with regards to the benefits.

DISCUSSION
This study compared self-reported assessments of work, sick leave benefits and allowances with register data from The Labour and Welfare Administration (NAV) in Norway.Agreements were assessed for four categories; working, sick-listed, medical/vocational rehabilitation and disability pension and agreements varied between good and fair.Best agreements were found for medical/vocational rehabilitation and for disability pension.Acceptable agreement between self-reported and register data has been found in similar studies (Burdorf et al 1996;Fredriksson et al. 1998;Severens et al. 2000;Ferrie et al. 2005;Voss et al. 2008).Only the study from Fredriksson et al. (1998) had access to official register data from regional social insurance registers whilst the other studies utilized employers' registers.All these studies examined retrospective data, where the participants had to recall sick leave spells and number of sick leave days.The data were therefore subject to recall bias.In Burdorf's study the recall period was 6 months, in the studies of Voss, Ferrie and Severens the recall period was 12 months and in Fredriksson's study it was 4 years.Our study was not subject to this type of recall bias.We simply measured the individuals knowledge about correct states of benefits or allowances checked with the current official data registered at NAV.Thus this study is mainly related to Norwegian conditions and we do not know of any comparable Norwegian study exami-  Kappa values adjusted with an interval of -10/+60 days in front and after the exact day.
# No individuals with sick leave periods shorter than 100 days or with total number of sick leave days less than 365 days was registered with disability pension.
ning agreement between self-reported and registered data on sick leave and other allowances.
Retrospective self-reported assessments of sick leave are frequently performed in epidemiologic studies and are often associated with self-reported general health (Voss et al. 2008).A typical question in such a survey would be: "Have you during the last year been sick-listed -if that is the case due to which health problem, how many times, for how long time?etc".Sick leave data is also applied in research on return to work (RTW).However, the research question would then often be phrased differently: "Have you returned to work or are you still sick-listed?Do you receive any allowances or graded benefits?etc".Consistent, reliable and standardized RTW measurements both in self-reports and registered data are requested (Wasiak et al. 2007;Pransky et al. 2005).There is also need for outcome measurements visualizing the complexity and endurance of the RTW process in some of the longterm cases of sick leave, taking the developmental nature of the RTW process into account (Young et al. 2005;Lie et al. 2008).The findings from our study may contribute to improve data sampling and analysis both research on RTW but also in the processes leading to exit from work and entry into disability benefit.
Stratified analysis showed minor to considerable differences within the subgroups gender, age, education, medical diagnosis and sick leave length at baseline.Greatest and considerable differences in agreements were found for sick leave length at baseline, where self-reports from participants sick-listed less than 12 months agreed best with the registered data on all four categories.The findings give support to earlier findings (van Poppel et al. 2002;Ferrie et al. 2005) and may be explained by a more complex and complicated benefit/allowance situation for those out of work more than 12 months.However, these findings were not confirmed examining differences amid subgroups with long and complex history of sickness absence the last 10 years before the rehabilitation program.Moderate gender differences in agreements were found with better agreements for men on the categories sick-listed and disability pension.This is in accordance with findings from other studies (Ferrie et al. 2005;Voss et al. 2008).
No systematic differences in agreements were found within groups of lower or higher education.Better agreement was found for those with more than 12 years of education, but for one category only.
Moderate but no systematic differences in agreements were found between the three subgroups of medical diagnoses, musculoskeletal, psychiatric and other/unspecified.Reported diagnosis (ICPC) was based on the GPs diagnosis, entered in the medical journal at the rehabilitation centre.In another study based on the same set of data, we have found a high degree of comorbidity, with more than half of the participants having two or more medical diagnoses (Øyeflaten et al. 2008).The most common was having both a musculoskeletal diagnosis and a psychiatric diagnosis.This indicates a complex pattern of comorbidity in long term sick-listed individuals and may explain inconsistency in the findings.We are not aware of studies examining differences in diagnosis, except one study where it was reported better agreements among individuals with back pain diagnosis compared with respiratory diagnosis (Burdorf et al. 1996).
Limitations of this study include a small sample.After stratification the examined subgroups became even smaller, but division into maximum 2-3 subgroups still makes acceptable analysis possible.Data was not primarily collected for the purpose of doing these analyses.But since agreements between selfreported and register data had not earlier been assessed in Norwegian studies, we wanted to make use of the previously gathered data in this study.Knowledge from this report may be useful in the design of a later and more comprehensive study on a Norwegian sample.
There was some uncertainty in the exact dates for when the self-reported data ought to be compared with the registered data, since the questionnaire on self-reports did not enclose exact dates for when they were filled out.However we believe that the adjustment inserting an interval of -10/+60 days on the register data compensates satisfactorily for this uncertainty.It has also been claimed that there may be a delay in the update of the official data registers, but since data in this study was gathered some time after the time of registration this should not be of any importance.
Another weakness in the self-reports was that some items in the questionnaire did not correspond exactly with the categories in the NAV registers.The NAV data separated medical rehabilitation and vocational rehabilitation and this was not the case in the selfreported data.These two categories were therefore merged into one.Since rather few participants were registered on vocational rehabilitation, we do not believe this is of significant importance.
Furthermore it is a weakness that the register data did not contain any information about a person actually working or not.Due to this we had to assume that participants who didn't receive any full or graded benefits at the current time had returned to work.Additionally it might be a weakness that we had no access to other benefits such as social benefits and unemployment benefits.
Despite these limitations, register data should be preferred to self-reports because response rates always will be lower when the researcher has to relay on the patients' willingness to reply (van Poppel et al. 2002).The response rate will also decrease by length of the follow up period.Low response rate may bias the data since the responders may differ from the nonresponders.In research on sick leave or RTW these arguments are of essential importance since high quality studies with longitudinal design are required.In longitudinal studies there will also be a risk for participants moving to a new address or other conditions may complicate access to data without using official registers.
Return to work after rehabilitation can be a lingering process (Pransky et al. 2005), therefore measures of sickness benefits and allowances at a single point of time may not give accurate information of this complex process (Young et al. 2005;Lie et al. 2008).Data on a single point of time cannot give accurate information about the sick-listed individuals' movements in and out of different benefits before and after this point of time.
In conclusion, we believe that access to register data is crucial in research on return to work or in research examining the process towards disability benefits.This study found acceptable agreements between self-reported and register data giving a good cross sectional picture of the current situation, but when more accurate information about the complex return to work process is requested, official register data and sophisticated statistical analyses (Lie et al 2008) will be preferred.

Table 1 .
Categories of work and sick-leave situation based on self-reported measures from questionnaires and register data from NAV.

on self-reports Register data 4 shared categories: Self-reports on current work and sick leave situation Self-reports on current benefits Information from NAV
# Participants without any benefits from NAV at a certain date were considered working.

Table 2 .
Number of individuals reporting different work and sick leave situation (n=132).

Table 3 .
Number of individuals registered within NAV as receiving different benefits (n=132).

Table 4 .
Agreements reported with kappa values, between self-reports and register data from NAV stratified for gender, age, education, medical diagnosis and length of sick leave/allowances at baseline.
* Moderate differences; > .20 or considerable differences; > .40 in agreements between subgroups for the four categories.# No individuals below 45 years registered with disability pension.

Table 5 .
Agreements reported with kappa values, between self-reports and register data on number of sick leave spells, mean number of sick leave days per sick leave spell and total number of sick leave days the last ten years before rehabilitation (n=132).