Validation of Hospital Anxiety and Depression Scale in an Indonesian population: a scale adaptation study

Study design, sample size and sampling procedure

A cross-sectional study was conducted between June and November 2018 to investigate the validity and reliability of the HADS-Indonesia in measuring anxiety and depression in the Indonesian population. This study was designed to measure construct validity of the scale using factor analysis, convergent validity, face validity, test–retest reliability using intraclass correlation coefficient (ICC) and internal reliability. A minimum factor analysis sample size is required to yield a stable factor model. The minimum ratio of participants to items has been reported to be 10:1 (10 participants for one item), although more participants might provide a more robust model.14 Thus, we needed 140 or more participants to run a factor analysis and produce a stable model. A minimum of 28 responses has been deemed sufficient for test–retest analysis reliability using ICC in a review article and only a small number of responses is usually needed for the study.15

The study’s inclusion criteria were adults (18 years old and older) with basic Indonesian language literacy, including reading and writing. There were no specific exclusion criteria in this study. Using a convenience sampling method, we invited 200 participants from three villages in Jatinangor subdistrict, Sumedang regency, West Java province Indonesia, to take part. The villages were chosen based on their profiles (ie, their urban and suburban characteristics).

We provided an incentive to each participant in the form of daily food and other necessities, valued at approximately IDR50 000 (approximately US$3.50). All data were collected and stored by the researchers and was only used during data analysis without disclosing participants’ personal information.

Hospital Anxiety and Depression Scale

The HADS is copyrighted under R. P. Snaith and A. S. Zigmond, 1983, 1992 and 1994. HADS was initially published in Acta Psychiatrica Scandinavica 67, 361-70, Munksgaard International Publishers, Copenhagen, 1983.6 Permission has been obtained for its translation and use.

Given its simplicity, HADS has been widely used and validated for measuring anxiety and depression cases.12 HADS is a self-completion questionnaire consisting of 14 items comprising depression and anxiety subscales, which can be completed in 2–5 min. In this study, respondents’ answers were based on their feelings within a week of the data collection. Each respondent’s score was the sum of the ratings for all items (each score ranged from 0 to 3 points) in each subscale. The maximum score for each subscale is 21, with 0–7 points meaning no depression/anxiety cases (ie, non-cases), 8–10 points representing borderline or doubtful cases, and ≥11 points suggesting anxiety or depressive cases.6

Translation and back translation of HADS

The adaptation process followed the guidelines furnished by Beaton et al.16 The first stage in the adaption process was the translation of the original English HADS text by two native Indonesian translators with different profiles and backgrounds. Two independent Indonesian language versions of HADS were produced along with the translators’ reports on challenging phrases. The reports and translations were followed up with a discussion between the translators and experts to synthesise the translations into one unified version. Next, the unified version was discussed and evaluated by a committee consisting of the researchers, a psychiatrist, a study methodology consultant and the two translators. The experts agreed on a prefinal version of the instrument at this stage.

A native English translator conducted a back-translation process to ensure the prefinal Indonesian version of HADS reflected the original version. The back-translation version suggested that the Indonesian version of the questionnaire reflected the original version.

Face validity evaluation

The prefinal version of the Indonesian version of HADS was tested on 35 subjects selected from the target population. Each subject completed the questionnaire and underwent an interview with one of the researchers. The interviews aimed to probe each subject’s understanding of each item and the subject’s responses. This process provided several insights which supported the development of the final version of the Indonesian HADS.

Test–retest reliability and convergent validity evaluation

After face validity evaluation was completed, 41 participants were invited to complete the questionnaire twice. At the first data collection stage, they were asked to complete three questionnaires, the final version of the HADS-Indonesia, Zung’s Self-rating Anxiety Scale (SAS)7 and Zung’s Self-rating Depression Scale (SDS).11 SAS has been adapted and validated in the Indonesian population and was found to have a Cronbach’s alpha score of 0.65 with additional evidence of convergent and construct validity.17 SDS has also been validated in the Indonesian population with a Cronbach’s alpha score of 0.88.18 The second data collection stage was conducted 3 months later, and the participants were reinvited to complete the final version of HADS-Indonesia, evaluating the scale’s test–retest reliability. A 3-month interval was chosen for test–retest evaluation to avoid the possibility that the participants might have memorised their previous answers. Also, a longer interval was chosen since the diagnosis of depression and anxiety needs a longer duration (>2 weeks).19

Data collection method

After the second data collection and the analysis of the convergent validity and test–retest reliability data, 30 people from Jatinangor were trained to prepare the questionnaire for data collection, identify potential participants, assess participants’ eligibility and collect the data. They visited the potential participants in their target area as trained and collected the data directly.

Data analysis

Patient and public involvement

Patients or the public were not involved in the design, or conduct, or reporting, or dissemination plans of our research.

Participants and demographic profiles

A total of 200 responses were collected and the response rate was 100% as all participants invited consented to participate. The mean age of the participants was 42.41 (14.25) years old. Most of the participants were female (n=109, 54.5%) and graduated from middle school (n=120; 60%). Most of the women in our study were homemakers (n=85, 78% of the total women participants). Table 1 shows the demographic profile of the study’s population.

Face validity

A to the original version of HADS, the HADS-Indonesia consists of 14 items (online supplemental appendix 1). The questionnaire was completed by all participants (n=35) within 2–5 min. Several amendments were made based on 35 participants’ reviews of the items. Changes were made to item 4 (ie, I can laugh and see the funny side of things), item 5 (ie, worrying thoughts go through my mind), item 8 (ie, I feel as if I am slowed down), item 9 (ie, I get a sort of frightened feeling like ‘butterflies’ in the stomach), item 10 (ie, I have lost interest in my appearance), item 11 (ie, I feel restless as if I have to be on the move), item 13 (ie, I get sudden feelings of panic) and item 14 (ie, I can enjoy a good book or radio or TV programme). The changes to each item were applied to the item statement, to the response options, or both. This evaluation resulted in the final version of HADS-Indonesia used for the validation process which followed.

Convergent validity

A total of 41 participants completed the HADS-Indonesia, SAS and SDS scales. The Spearman correlation result indicated a significant correlation between the Indonesian version of HADS-Anxiety (HADS-A) and SAS (r_s=0.45, p=0.030, N=41). As was true for the Indonesian version of HADS-A, there was a statistically significant correlation between HADS-Depression (HADS-D) and SDS (r_s=0.58, p<0.001, N=41).

Test–retest reliability

After a 3-month interval, 41 participants completed the final version of HADS-Indonesia for the second time to measure the instrument’s intrarater reliability. ICC estimates and 95% CIs were based on a mean-rating, absolute-agreement, two-way-mixed effects model, which showed an ICC score of 0.98 (excellent reliability) for the whole scale. Each item’s ICC value ranged between 0.72 (moderate reliability) (item 1; I feel tense or ‘wound up’) and 0.96 (excellent reliability) (item 3; ‘I get a sort of frightened feeling as if something awful is about to happen’). Table 2 presents the ICC value of each item and scale and the total scale indicating the test–retest reliability results in this study.

Structural validity

The KMO (KMO=0.89) and Bartlett’s test of sphericity (χ²(91, N=200)=1052.38, p<0.001) analysis indicated sufficient data for EFA. All items had a communality of >0.40, and inter-item correlation showed an AIC of 0.36. A putative factor extraction using PCA yielded three factors with an eigenvalue greater than 1. However, the scree plot supported the thesis of two significant factors, meaning the third factor would explain an insignificant amount of variance and, therefore, was not retained (figure 1).

Request permission

An orthogonal (ie, varimax) rotation based on the two-factor model showed that all items load to a factor (factor loading >0.40). However, item A7 (ie, I can sit at ease and feel relaxed) and D8 (ie, I feel as if I am slowed down) were cross-loaded into both factors (D8: 0.47 to factor 1 and 0.46 to factor 2; A7: 0.47 to factor 1 and 0.47 to factor 2). Therefore, as the protocol of this study was to put the cross-loaded items into the theoretically relevant factor, item A7 was put into factor 1 and item D8 into factor 2. After completing factor analysis, we found that the HADS-Indonesia consisted of 14 items like the original version. All the anxiety items converged into factor 1 and all the depression items gathered into factor 2; therefore, we labelled factor 1 as ‘Anxiety Subscales/HADS-A’ and factor 2 as ‘Depression Subscales/HADS-D’. Table 3 shows the result of the factor analysis in this study, including each item’s factor loading, commonalities and descriptive statistics. The factor model explained 50.80% (40.40%+10.40%) of the variance in the data.

Internal consistency

The HADS-Indonesia consisted of 14 items with a 2-factor model. The HADS-A subscale consisted of 7 items (alpha=0.85), and the HADS-D subscale consisted of seven items (alpha=0.80). Table 4 presents each factor’s internal consistency and descriptive statistics.

Summary of findings

This study aims to adapt the HADS scale for use in the Indonesian language and to provide evidence of its validity and reliability. It is a validation study of one of the most widely used instruments for measuring anxiety and depression in different settings. A face validity evaluation showed that HADS-Indonesia could be understood by, and was acceptable to, the target population. Next, this study showed an excellent convergent validity for each subscale when they were compared with SAS and SDS, as well as excellent test–retest reliability. PCA analysis conducted on 14 items yielded a 2-factor model for HADS-Indonesia while also maintaining the original HADS structure overall. Also, each factor showed good internal consistency.

Validity of HADS-Indonesia

This study evaluated three pieces of validity evidence of HADS-Indonesia; face validity, convergent validity and structural validity. The face validity evaluation of HADS-Indonesia showed that this scale was acceptable to the target population. Several phrases in the original version could not be directly translated into the Indonesian language. For example, item 9 (ie, I get a sort of frightened feeling like ‘butterflies’ in the stomach). The idiomatic phrase ‘butterflies in my stomach’’ has no direct equivalent in the Indonesian language and was translated into Indonesian using the word for ‘nauseous’. Problems with translating HADS have been reported in the literature28 and could have result in several problems impacting the validity of the adapted version. In our study, this problem seemed to be solved, however, by a proper face validity evaluation involving the target population, resulting in changes in item statements and answer options.

The convergent validity test of the HADS-Indonesia was conducted by comparing each scale to SAS and SDS using Spearman correlation analysis. Spearman’s analysis showed a good correlation between the anxiety subscale of HADS-Indonesia and SAS, as well as the depression subscale of HADS-Indonesia and SDS, thus providing HADS-Indonesia with evidence of its convergent validity. Different studies have compared HADS to SAS and SDS. To the best of our knowledge, all studies reported a positive correlation between the anxiety subscale of the HADS and SAS as well as between the depression subscale of the HADS to SDS.29–31 Our study, therefore, adds to the current body of literature, strengthening the evidence of HADS convergent validity.

An EFA using PCA analysis provided construct validity evidence for HADS-Indonesia with a two-factor solution, which explained 50.80% of the total variance. Conceptualisation of the items loaded into each factor led us to label the first factor as the anxiety subscale (ie, reflecting the anxiety sub-scale of the original version/HADS-A) and the depression subscale (ie, reflecting the depression subscale of the original version). In this study, two items were cross-loaded into both factors (ie, item A7, ‘I can sit at ease and feel relaxed’) and D8 (ie, ‘I feel as if I am slowed down’). Following the original version, both items were loaded into the anxiety and depression subscale and put into the relevant factor. A similar situation has been reported in several adaptations of the HADS, such as in Cantonese,32 Japanese33 and Canadian French.34 In terms of factor structure, our study’s EFA retained the original subscales developed by Zigmond and Snaith.6 This result adds to the body of literature showing results resulting from retaining the original subscales, such as the validation conducted in the case of the Italian population,35 the Norwegian and Swedish populations,36 and the Spanish population.37

Reliability of HADS-Indonesia

Two pieces of reliability evidence were evaluated (ie, internal consistency and test–retest reliability). Both factors showed an excellent internal consistency, 0.85 for HADS-A and 0.80 for HADS-D. The AIC also yielded a good inter-item correlation, which indicated a good correlation between items. To the best of our knowledge, the Cronbach’s alpha score of HADS-A in different populations was between 0.76 (Portuguese translation)38 to 0.86 (Spanish translation).39 HADS-D had an excellent internal consistency, although it was slightly lower than HADS-A, between 0.63 in the Hong Kong translation40 to 0.86 in the Iranian41 and Arab translations.42 In addition, this result was also comparable to the findings of studies conducted in the general population or primary healthcare by el-Rufaie and Absood (Cronbach’s alpha of 0.78 for HADS-A and 0.86 for HADS-D),42 Roberge et al (Cronbach’s alpha of 0.82 for HADS-A and 0.83 for HADS-D),34 and higher than one conducted in Columbia (Cronbach’s alpha of 0.77 for HADS-A and 0.75 for HADS-D).43

The ICC showed excellent test–retest reliability (ICC=0.98). The ICC score showed that HADS-Indonesia was still reliable despite the 3-month evaluation period. In addition, all items showed a good ICC score (ICC=0.72 (item 1) to 0.96 (item 3)). One study by Quintana et al examined the reliability of HADS in the Spanish population.39 They reported a similar test–retest reliability score between 0.85 and 0.91.

Study limitations, implications and future research

To the best of our knowledge, this study is the first to provide multiple validity and reliability evidence of HADS in the Indonesian population. Although the best measurements possible have been conducted, this study has several limitations. First, HADS is a self-reported questionnaire which is therefore prone to social desirability bias. The study participants can answer the questionnaire according to the applicable value within their society, which prevented us from obtaining objective information. Second, this study was a preliminary study of HADS-Indonesia. Although HADS itself is a sound scale, which have been adapted to different populations and settings, this study is still the first research to evaluate its validity in the Indonesian population. Therefore, the validity and reliability evidence of HADS-Indonesia needs to be studied further.

Our results may have valuable implications for the body of literature on HADS. First, as this study provides the first validity and reliability evidence of HADS in the Indonesian population, it may open the possibility that HADS can also be adapted into other languages and used in relatively similar cultures, such as that of the Malay language and other cultures in the Southeast Asian region. Second, our study provides healthcare workers and researchers working in mental health and other field a valid and reliable instrument to measure anxiety and depression, which may assist the early detection of mental ill health conditions and help researchers in the study area. Finally, these results open the possibility for further research. New studies using confirmatory factor analysis (CFA) are required to provide more robust validity evidence. Other validity studies, such as those examining discriminant validity, should be undertaken. Finally, as this study was conducted in a predetermined population, further replication studies are required, especially in other areas of Indonesia and in different settings, to evaluate HADS-Indonesia’s reproducibility and validity in these settings.