Table 1 Field testing for reliability and convergent validity, including original wording of questions at first test, changes made, and revised wording at second test

Q.1 How likely do you think it is that pandemic influenza will occur in Australia?

(Not likely, somewhat likely, very likely, extremely likely, don't know)

Weighted kappa = 0.43 (0.33–0.54)

Agreement = 61%

Don't know = 2.1%

Refused = 0%

None to question.

Response altered to a Likert scale: increased from 4 to 5 options.

Q.1 How likely do you think it is that pandemic influenza will occur in Australia?

(Not at all likely, a little likely, moderately likely, very likely, extremely likely, don't know)

Weighted kappa = 0.38 (0.28–0.47)

Indicator kappa = 0.44 (0.30–0.58)

Agreement = 45.5%

Don't know = 3.9%

Refused = 0%

Q.2 How likely do you think it is that you or your family would be directly affected by an influenza pandemic in Australia?

(Not likely, somewhat likely, very likely, extremely likely, don't know)

Weighted kappa = 0.48 (0.38–0.58)

Agreement = 62%

Don't know = 2.6%

Refused = 0%

Additional context added before question to provide better context.

Likelihood of being affected was changed to concern about being affected, to tap a sense of vulnerability rather than probability.

Responses altered to reflect concern and increase to 5 options.

Q.2 If an influenza pandemic were to occur in Australia, how concerned would you be that you or your family would be directly affected by it?

(Not at all concerned, a little concerned, moderately concerned, very concerned, extremely concerned, don't know)

Weighted kappa = 0.28 (0.18–0.38)

Indicator kappa = 0.25 (0.12–0.38)

Agreement = 39.1%

Don't know = 0%

Refused = 0%

Q.3 Do you feel you have changed the way you live your life because of the possibility of an influenza pandemic in Australia?

(Yes/No)

Kappa = 0.51 (0.34–0.68)

Agreement = 87%

Don't know = 0%

Refused = 0%

Changed from a binary Yes/No question to a Likert scale response assessing the degree to which respondents had made any change.

Question wording was altered to reflect that change.

Q.3 How much have you changed the way you live your life because of the possibility of an influenza pandemic?

(Not at all, a little, moderately, very much, extremely, don't know)

Weighted kappa = 0.34 (0.24–0.44)

Indicator kappa = 0.43 (0.30–0.56)

Agreement = 57.5%

Don't know = 1.5%

Refused = 0%

Q.4 In case of an emergency situation, government authorities might request cooperation from the public in a number of ways. Please indicate how willing would you be to do the following: Receive vaccination?

(not willing, somewhat willing, very willing, extremely willing don't know, refused)

Weighted kappa = 0.39 (0.28–0.50)

Agreement = 53%

Don't know = 2.1%

Refused = 0%

Additional context added to the introduction to specify pandemic influenza and make questions more relevant.

Response altered to a Likert scale: increased from 4 to 5 options.

Q.4 In case of an emergency situation such as an influenza pandemic, government authorities might request cooperation from the public in a number of ways Please indicate how willing would you be to receive vaccination?

(not at all willing, a little willing, moderately willing, very willing, extremely willing don't know, refused)

Weighted kappa = 0.45 (0.35–0.56)

Indicator kappa = 0.51 (0.34–0.67)

Agreement = 57.1%

Don't know = 0%

Refused = 0%

Q.5 Isolate yourself from others?

(not willing, somewhat willing, very willing, extremely willing don't know, refused)

Weighted kappa = 0.51 (0.42–0.60)

Agreement = 56%

Don't know = 1.0%

Refused = 0%

Question stem repeated.

Response altered to a Likert scale: increased from 4 to 5 options.

Q.5 How willing would you be to isolate yourself from others if needed?

(not at all willing, a little willing, moderately willing, very willing, extremely willing don't know, refused)

Weighted kappa = 0.4 (0.31–0.5)

Indicator kappa = 0.48 (0.33–0.62)

Agreement = 52.7%

Don't know = 1.0%

Refused = 0%

Q.6 ...Wear a face mask?

(not willing, somewhat willing, very willing, extremely willing don't know, refused)

Weighted kappa = 0.50 (0.40–0.60)

Agreement = 56%

Don't know = 1.0%

Refused = 0%

Question stem repeated.

Response altered to a Likert scale: increased from 4 to 5 options.

Q.6 How willing would you be to wear a face mask?

(not at all willing, a little willing, moderately willing, very willing, extremely willing don't know, refused)

Weighted kappa = 0.48 (0.39–0.56)

Indicator kappa = 0.51 (0.39–0.63)

Agreement = 45.8%

Don't know = 0.5%

Refused = 0%