Background and purpose

We previously found considerable variation in information provision on preoperative radiotherapy (PRT) in rectal cancer. Our aims were to reach consensus among patients and oncologists on which benefits/harms of PRT should be addressed during the consultation, and to assess congruence with daily clinical practice.

Materials and methods

A four-round Delphi-study was conducted with two expert panels: (1) 31 treated rectal cancer patients and (2) 35 radiation oncologists. Thirty-seven possible benefits/harms were shown. Participants indicated whether addressing the benefit/harm was (1) essential, (2) desired, (3) not necessary, or (4) to be avoided. Consensus was assumed when ⩾80% of the panel agreed. Results were compared to 81 audio-taped consultations.


The panels reached consensus that six topics should be addressed in all patients (local control, survival, long term altered defecation pattern and faecal incontinence, perineal wound healing problems, advice to avoid pregnancy), three in male patients (erectile dysfunction, ejaculation disorder, infertility), and four in female patients (vaginal dryness, pain during intercourse, menopause, infertility). On average, less than half of these topics were addressed in daily clinical practice.


This study showed substantial overlap between benefits/harms that patients and oncologists consider important to address during the consultation, and at the same time poor congruence with daily clinical practice.

Preoperative radiotherapy (PRT) improves local control of rectal cancer. Although not demonstrated in randomised controlled trials, there might be a small survival benefit at the population level [[1], [2]]. Due to the good local control with surgery alone, there is a high number needed to treat to prevent one local recurrence [[2], [3]]. In addition, PRT is associated with adverse outcomes, such as higher chances of bowel and sexual dysfunction than with surgery alone [[3],[4]]. When deciding about treatment, the possible benefit in terms of local control should therefore be balanced against the possible harms, taking into account patient preferences. Patients need to be informed about the most relevant benefits and harms of treatment in order to develop a preference. Informing patients also prevents them from overestimating the impact of treatment on cure [5]. Moreover, patients who are well-informed experience better health-related quality of life and may cope better with treatment side effects [[6], [7]].

In earlier research, we found considerable variation in information provision regarding benefits and harms of PRT during the decision consultation between rectal cancer patients and their radiation oncologist [8]. This variation indicates a lack of clarity on which benefits and harms of PRT should be discussed with newly-diagnosed patients. In general, treatment guidelines provide little or no recommendation on which benefits and harms to communicate to patients. The Dutch guidelines for the treatment of rectal cancer for example state that clinicians need to ‘discuss the possible benefits and harms of radiotherapy with the patient’, without further specification [9].

The aims of this study were to (1) reach consensus among rectal cancer patients and radiation oncologists and compose a core list of benefits and harms of PRT that should minimally be addressed during the decision consultation, and (2) assess congruence with daily clinical practice.

Materials and methods


A Delphi study was performed in two panels: treated rectal cancer patients and radiation oncologists. One of the most critical requirements in the Delphi method is the selection of experts, rich in information and experience [10]. Eligible patients had received radiotherapy and had finished their oncologic treatment at least 4 months ago. Patients treated at the Leiden University Medical Center who participated in an earlier study were approached via mail. Furthermore, members of the Dutch colorectal cancer patient organization were approached through the monthly newsletter of their association. Members of the Gastrointestinal-subsection of the Dutch Society for Radiation Oncology were approached for participation. All 45 radiation oncologists who were members of this platform were considered to be clinical experts.

We aimed to include at least half of the radiation oncologists from the platform, and an equal number of rectal cancer patients.


In order to reach consensus, we used the Delphi technique. This is a structured process that uses a series of questionnaires or ‘rounds’ to gather information until consensus in the panels is reached [11]. As we expected differences in opinions between patients and radiation oncologists, we aimed to reach consensus in each panel separately [12]. Based on previous Delphi studies, we intended a maximum of three online rounds in which participants could indicate which benefits and harms should always be addressed during the decision consultation [11]. Since there was only consensus on a limited number of benefits/harms after three rounds, we organized additional and separate consensus meetings with a fourth and final voting round. Between January and September 2013, the participants completed an iterative series of four questionnaires with feedback reports. In the first online questionnaire, socio-demographic and treatment- (patients) or work- (radiation oncologists) related details were obtained.

To assess congruence between the results of this Delphi-study and daily clinical practice, we compared the core list that was obtained to results of a previous study on information provision regarding benefits and harms of PRT [8]. In that study, we audiotaped and analysed 81 decision consultations between radiation oncologists and rectal cancer patients.

Questionnaire rounds

The first questionnaire consisted of 37 benefits and harms, ordered by subject matter (see Table 1). These were obtained from all benefits/harms that had been discussed in any of the first 45 of 81 previously audio taped decision consultations between radiation oncologists and rectal cancer patients [8]. Benefits/harms related to inconvenience or costs were excluded. We complemented the list with outcomes described in the literature [[2], [3], [13], [14], [15], [16], [17], [18]]. This led to a total of 30 outcomes on which PRT could have an effect for all patients, three for male patients only, and four for female patients only. In both panels, the same brief description of the items was given to help minimize interpretation differences. Information on probable prevalence was given in words and ranges (rare: 0–5%; sometimes: 5–25%; often: 25–75%; (almost) always: 75–100%).

Table 1Benefits and harms of preoperative radiotherapy presented in the first Delphi-round.
1. Local control
2. Overall survival
3. Secondary tumours
4. Altered defecation pattern (short term)
5. Altered defecation pattern (long term)
6. Faecal incontinence (short term)
7. Faecal incontinence (long term)
8. Soiling
9. Increased rectal blood loss
10. Decreased rectal blood loss
11. Small bowel adhesions
12. Bladder dysfunction
13. Urinary incontinence
14. Infertility (women)
15. Infertility (men)
16. Avoidance of pregnancy
17. Erectile dysfunction (men)
18. Ejaculation disorder (men)
19. Vaginal dryness (women)
20. Pain during intercourse (women)
21. Menopause (women)
22. Anastomotic leakage
23. Increased blood loss during surgery
24. Abdominal wound healing problems
25. Perineal wound healing problems
26. Increased readmission rate
27. Nerve damage (short term)
28. Nerve damage (long term)
29. Muscle weakness
30. Skin irritation
31. Hair loss (local)
32. Fatigue
33. Longer recovery
34. Feeling unwell
35. Less appetite
36. Cardiovascular problems
37. Fistula

Participants were asked to indicate whether they thought that addressing the outcome during the first consultation was (1) essential, (2) desired, (3) not necessary, or (4) to be avoided. Participants were asked to respond to all outcomes. For example, all participants (including female patients) were asked to indicate the importance of addressing ‘erectile dysfunction’ during consultations with male patients. After each subject matter, participants could comment on the item descriptions or suggest additional outcomes. The first questionnaire was pilot-tested in eight radiation oncologists and eight lay people. The final version of the first questionnaire was adjusted according to their feedback.

Based on the literature, we defined consensus as at least 80% of the participants in one panel ticking the same answer category (e.g., 1 ‘essential’) and no more than 15% an answer category two or three categories away (e.g., 3 ‘not necessary’ or 4 ‘avoid’) [11]. Outcomes on which consensus was reached were removed from the subsequent questionnaire(s). The other items were included in the subsequent questionnaire, together with feedback on the responses of the panel and the participant’s own responses. Radiation oncologists also received feedback on patients’ responses. Feedback on participants’ responses in each of the categories was shown as a percentage and a column bar. In the second and third questionnaire, participants were asked to reconsider their previously given responses in light of the opinion of other panel members.

Consensus meetings

After the three online questionnaires, we organized a separate in-person consensus meeting for each panel, with the aim to discuss the importance of addressing benefits/harms for which no consensus had been reached in the online rounds. All participants who had completed the third round were invited. The meetings started with a brief presentation on the background of the study, followed by the results up to then. After a group discussion on the importance of addressing the benefits and harms, participants’ final opinions were assessed anonymously.

At the consensus meeting, several participants indicated that the response categories 1 (essential) and 2 (desired) were only marginally different. We therefore decided to merge these categories in the analysis of the responses in this final round.

Statistical analyses

Responses of patients and radiation oncologists were analysed separately. Descriptive statistics were used to report patients’ and radiation oncologists’ characteristics, their views on which benefits and harms should be addressed and congruence between the results of this study and daily clinical practice. Using Chi-square tests and Mann–Whitney U tests, characteristics and responses of participants who did versus did not complete the study were compared. A two-sided p-value of ⩽0.05 was considered statistically significant.


Of the 38 eligible patients approached, 23 (61%) completed the first questionnaire. An additional eight were included through the patient organization. Of these 31 patients, 28 patients completed the second and third questionnaires (90% of those who started). Ten patients attended the consensus meeting and completed the final voting round (36% of those who completed the third questionnaire).

Of the 45 radiation oncologists who are members of the platform, 35 (78%) completed the first questionnaire. The second and third questionnaires were completed by 32 and 29 oncologists, respectively (91% and 83% of started, 71% and 64% of total). All 29 oncologists who completed the third round also completed the final voting round.

In Table 2 participant demographic and treatment- (patients) or work- (oncologists) related characteristics are listed. Radiation oncologists working at a non-teaching centre compared to an academic or non-academic teaching centre significantly more often declined further participation in the study (N = 4, 44% vs. N = 1, 8% vs. N = 1, 7%; χ2 = 6.36, p < 0.05). Otherwise, no significant differences were found between characteristics of participants who did versus did not complete the study. To assess bias caused by the 36% response rate of the consensus meeting, we compared the scores in round 3 of attenders and non-attenders. We found that patients who attended the consensus meeting rated ‘bladder dysfunction’ as significantly more important than patients who did not attend the consensus meeting (χ2 = 10.04, p < 0.01). After receiving feedback on the answers given in round 3 and a discussion during the consensus meeting, patients eventually reached consensus that this outcome need not necessarily be discussed. Otherwise, no significant differences were found between the answers of those who did versus those who did not attend the consensus meeting.

Table 2Participant characteristics in round 1.
N (%)
Patients (N = 31)
 Member of patient association 10 (32)
 Mean age, years ± s.d. (range) 64 ± 10.7 (32–85)
 Mean time since diagnosis, years ± s.d. (range) 2 ± 2.2 (0.3–9)
 Male 18 (58)
 Educational levela
  Low 2 (7)
  Intermediate 16 (55)
  High 11 (38)
 Neo-adjuvant treatment
  PRT (5×5 Gy) 19 (61)
  Chemoradiation 12 (39)
  No stoma 12 (34)
  Temporary 11 (36)
  Permanent 8 (26)
Radiation oncologists (N = 35)
 Mean age, years ± s.d. (range) 47 ± 8.1 (35–66)
 Mean time since specialization, years ± s.d. (range) 12 ± 9.0 (2–39)
 Mean number of new rectal cancer patients per month ± s.d. (range) 5 ± 2.3 (1–10)
 Male 12 (34)
 Current institution
  Academic teaching centre 14 (40)
  Non-academic teaching centre 12 (34)
  Non-teaching centre 9 (26)
aEducational levels included low = completed no/primary school, intermediate = completed lower general secondary education/vocational training; or high = completed pre-university education/high vocational training/university. Two patients did not respond to this question.

Patients and radiation oncologists reached consensus on, respectively, 29 and 30 of the 37 benefits/harms. Both panels agreed that six benefits/harms should be addressed with all newly-diagnosed rectal cancer patients, together with three benefits/harms for male patients only and four for female patients only. They also agreed that 11 benefits/harms need not necessarily be addressed. According to the panels, none of the benefits/harms should be avoided during the first consultation. The final core list of benefits/harms that should be addressed and items that need not necessarily be addressed can be found in Table 3.

Table 3Benefits and harms of PRT that should be or need not be addressed with newly-diagnosed rectal cancer patients, and number of rounds needed by expert panel before reaching consensus.
Inline Img fx1

a Men and premenopausal women only.

b Premenopausal women only.

Note: Light grey shades refer to benefits/side effects concerning male patients only, dark grey shades refer to benefits/side effects concerning female patients only.

In total, there were 11 topics on which one panel reached consensus, and the other panel did not. On two topics, patients’ and oncologists’ opinions were almost contrary. Patients agreed that ‘increased readmission rate’ should be addressed, while oncologists approached consensus that this is not necessary (72% agreement). Also, oncologists agreed that ‘short-term altered defecation pattern’ should be addressed, while patients tended to rate this as ‘not necessary’ (60% agreement).

To assess congruence between the results from the Delphi-study and daily clinical practice, we compared the core list to results on information provision regarding benefits and harms of PRT, based on 81 audiotaped decision consultations [8]. We found that in daily clinical practice, male patients received information on 3.3 (37%) of the nine topics from the core list (range, 1–6) on average. Female patients on average received information on 3.2 (32–46%) of the seven topics from the core list for postmenopausal women or 10 topics for premenopausal women (range, 1–6). In none of the 81 audiotaped consultations, all benefits/harms as defined in the core list were addressed. As can be seen in Fig. 1, only the effect of PRT on local control was addressed in all consultations. There were seven (9%) consultations in which local control was the only topic from the core list that was addressed.

Thumbnail image of Fig. 1. Opens large image

Fig. 1

Topics which should be addressed and the percentage of consultations in which the benefit/harm was addressed. Note:* as a percentage of consultations with patients from relevant patient group (male/female patients). ‡ Only relevant for patients undergoing abdominoperineal resection, all patients in our sample underwent a low anterior resection. ST = on the short term; LT = on the long term. Black bars = consensus in both panels, Grey bars = consensus in radiation oncologist panel only.

Several topics which are not on the core list are frequently addressed in consultations. Both panels reached consensus that ‘skin problems’ and ‘feeling unwell’ are not necessary to address in the first consultation. In daily clinical practice, these topics were addressed in 27% and 31% of the consultations, respectively. In addition, the patient panel agreed that ‘fatigue’ and ‘bladder dysfunction’ need not necessarily be addressed. In respectively 53 and 70% of consultations, patients received information on these harms of treatment.


The first aim of this study was to reach consensus among rectal cancer patients and radiation oncologists on which benefits and harms of PRT should minimally be addressed in the decision consultation. The patient and oncologist panels agreed that six benefits/harms should be addressed with all newly-diagnosed rectal cancer patients, together with three benefits/harms for male patients only and four for female patients only. It is noteworthy that all topics in the final core list are long-term benefits/harms. Indeed, during the consensus meeting, patients indicated to be less interested in temporary short-term effects. The long-term benefits/harms include the effect of PRT on local control, survival, defecation and sexual functioning, and these effects are well-established and described in the literature [[1], [2], [3]].

Of particular interest are the topics on which panels had different opinions. Firstly, oncologists reached consensus that ‘short-term altered defecation pattern’ should be addressed during the consultation, but patients’ opinion differed. This might be due to the fact that most patients already experience an altered defecation pattern at the time of the consultation. Secondly, oncologists agreed that ‘short-term nerve damage’ should be addressed, while patients’ opinions were divided. Because of the very low prevalence of short term nerve damage and patients’ bias towards their own experiences, it might be difficult for patients to understand the consequences of this harm, despite the description we provided. This lack of consensus highlights that oncologists should be aware that patients’ information needs might differ from what they themselves consider important to address.

The second aim of the study was to assess congruence between the core list and daily clinical practice. We found that patients received information on a limited number of topics from the core list. On average, less than half of the topics from the list were addressed during the consultation. Even more importantly, almost one in ten patients received no information on any of the adverse effects that should have been addressed, according to both patients and radiation oncologists. The need for implementing this list in daily clinical practice is therefore clearly demonstrated.

Panels agreed that several topics not necessarily need to be addressed with newly-diagnosed patients. Some of these topics are nevertheless discussed in a large part of the audiotaped consultations. Possible reasons to address these topics may have to do with patient characteristics or patient’s question asking behavior. In previous research we found that four out of five patients did not initiate discussion on any benefits/harms by asking questions or raising new topics. Furthermore, no clear association between benefits/harms mentioned and patient’s characteristics such as gender, age or educational level was seen. However, factors like co-morbidity and medical history of individual patients might give a reason to discuss certain additional topics.

Our study has some limitations. Firstly, although we pilot-tested our questionnaire and panellists were given the opportunity to comment on the questionnaire in each round, participants only indicated during the consensus meeting that the meaning of the answer categories ‘essential’ and ‘desired’ was only marginally different. We decided to merge these categories in the analysis of the responses on the final round. Had we started the Delphi study with three categories, consensus on some topics might have been reached earlier. Secondly, of the 28 patients who completed the third questionnaire, only 10 patients attended the consensus meeting and completed the final questionnaire. However, the characteristics of patients who attended compared to those who declined further participation did not differ significantly. Although there was a significant difference in how attending and non-attending patients valued the discussion of ‘Bladder irritation’, the attenders converged to the opinion of the larger group of non-attenders on this topic. Finally, we have no information on which adverse effects of PRT the members of our patient panel experienced during or after their treatment. Therefore, we cannot make any statements on the influence of patients’ own experience on their views about whether or not to address the benefits/harms offered in our questionnaire.

Delphi-methods have been used before in order to develop core lists. However, so far no studies have been published on core lists of benefits and harms of treatment to be communicated to patients during the consultation. Even though previous research has shown that cancer patients have a strong need for information about side effects of treatment [[19],[20]], our study showed that treated patients are capable of prioritizing those benefits/harms they think are necessary to address during the consultation. Therefore, patients’ perspectives are valuable when creating core lists. The method we used thus seems feasible for creating core lists for other treatments and other cancer types. As can be seen from a number of recent publications, the interest in the sequelae of rectal cancer treatment, and other cancer treatments as well, is rising [[21], [22]]. Our study is thus timely in showing a feasible method to determine which such sequelae should be communicated with patients during the consultation.

In conclusion, our results showed substantial overlap between which benefits and side effects of PRT patients and radiation oncologists consider important to address in the first consultation. These topics were poorly addressed in daily clinical practice. Our core list can be supplemented with outcomes of relevance to the individual patient. Addressing information on these major outcomes of PRT will better enable individual rectal cancer patients to balance possible side effects against the possible benefit in local control when deciding about PRT.


This work was supported by the Dutch Cancer Society (Grant number UL2009-4431 ). The study sponsor had no involvement on the study design, in the collection, analysis and interpretation of data, in the writing of the manuscript, and in the decision to submit the manuscript for publication.

Conflict of interest

The authors have declared no conflicts of interest.


The authors would like to thank Hans Pieterse for designing and building the online questionnaires. Furthermore, we thank the Dutch colorectal cancer patient organization ‘Darmkanker Nederland’ and all participating patients and radiation oncologists for their efforts.


[1]Glimelius, B., Gronberg, H., Jarhult, J., Wallgren, A., and Cavallin-Stahl, E. A systematic overview of radiation therapy effects in rectal cancer. Acta Oncol. 2003; 42: 476–492

[2]van Gijn, W., Marijnen, C.A., Nagtegaal, I.D. et al. Preoperative radiotherapy combined with total mesorectal excision for resectable rectal cancer: 12-year follow-up of the multicentre, randomised controlled TME trial.Lancet Oncol. 2011; 12: 575–582

[3]Birgisson, H., Pahlman, L., Gunnarsson, U., and Glimelius, B. Late adverse effects of radiation therapy for rectal cancer – a systematic overview. Acta Oncol. 2007; 46: 504–516

[4]Stephens, R.J., Thompson, L.C., Quirke, P. et al. Impact of short-course preoperative radiotherapy for rectal cancer on patients’ quality of life: data from the Medical Research Council CR07/National Cancer Institute of Canada Clinical Trials Group C016 randomized clinical trial. J Clin Oncol. 2010; 28: 4233–4239

[5]Hack, T.F., Degner, L.F., and Parker, P.A. The communication goals and needs of cancer patients: a review.Psychooncology. 2005; 14: 831–845

[6]Dunn, J., Steginga, S.K., Rose, P., Scott, J., and Allison, R. Evaluating patient education materials about radiation therapy. Patient Educ Couns. 2004; 52: 325–332

[7]Guleser, G.N., Tasci, S., and Kaplan, B. The experience of symptoms and information needs of cancer patients undergoing radiotherapy. J Cancer Educ. 2012; 27: 46–53

[8]Kunneman, M., Marijnen, C.A.M., Rozema, T. et al. Decision consultations on preoperative radiotherapy for rectal cancer: large variation in benefits and harms that are addressed. Br J Cancer. 2014; ([Epub ahead of print])

[9]Comprehensive Cancer Centre The Netherlands. Guidelines for the management of colorectal cancer and colorectal liver metastases; 2014.

[10]Okoli, C. and Pawlowski, S.D. The Delphi method as a research tool: an example, design considerations and applications. Inf Manage. 2004; 42: 15–29

[11]Keeney, S., Hasson, F., and McKenna, H. Consulting the oracle: ten lessons from using the Delphi technique in nursing research. J Adv Nurs. 2006; 53: 205–212

[12]Mease, P.J., Arnold, L.M., Crofford, L.J. et al. Identifying the clinical domains of fibromyalgia: contributions from clinician and patient Delphi exercises. Arthritis Rheum. 2008; 59: 952–960

[13]Holm, T., Singnomklao, T., Rutqvist, L.E., and Cedermark, B. Adjuvant preoperative radiotherapy in patients with rectal carcinoma. Adverse effects during long term follow-up of two randomized trials. Cancer. 1996; 78:968–976

[14]Marijnen, C.A., Kapiteijn, E., van de Velde, C.J. et al. Acute side effects and complications after short-term preoperative radiotherapy combined with total mesorectal excision in primary rectal cancer: report of a multicenter randomized trial. J Clin Oncol. 2002; 20: 817–825

[15]Pollack, J., Holm, T., Cedermark, B. et al. Late adverse effects of short-course preoperative radiotherapy in rectal cancer. Br J Surg. 2006; 93: 1519–1525

[16]Birgisson, H., Pahlman, L., Gunnarsson, U., and Glimelius, B. Occurrence of second cancers in patients treated with radiotherapy for rectal cancer. J Clin Oncol. 2005; 23: 6126–6131

[17]Lange, M.M., Marijnen, C.A., Maas, C.P. et al. Risk factors for sexual dysfunction after rectal cancer treatment. Eur J Cancer. 2009; 45: 1578–1588

[18]Lange, M.M., Den, D.M., Bossema, E.R. et al. Risk factors for faecal incontinence after rectal cancer treatment. Br J Surg. 2007; 94: 1278–1284

[19]Meredith, C., Symonds, P., Webster, L. et al. Information needs of cancer patients in West Scotland: cross sectional survey of patients’ views. BMJ. 1996; 313: 724–726

[20]Jenkins, V., Fallowfield, L., and Saul, J. Information needs of patients with cancer: results from a large study in UK cancer centres. Br J Cancer. 2001; 84: 48–51

[21]Engels, B., Platteaux, N., Van den Begin, R. et al. Preoperative intensity-modulated and image-guided radiotherapy with a simultaneous integrated boost in locally advanced rectal cancer: report on late toxicity and outcome. Radiother Oncol. 2014; 110: 155–159

[22]Burbach, J.P., den Harder, A.M., Intven, M. et al. Impact of radiotherapy boost on pathological complete response in patients with locally advanced rectal cancer: a systematic review and meta-analysis. Radiother Oncol. 2014;