If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
The aim of our work was to test the usability of fecal calprotectin (FC) home-based test in inflammatory bowel disease (IBD) patients.
Methods
IBD patients were prospectively recruited. They had to measure FC with a dedicated tool and smartphone application, 5 times at two weeks intervals over an 8 weeks period. They had to fill in a usability questionnaire at the first and the last FC measurement. A System Usability Scale (SUS: 0–100) and the Global Score of Usability (GSU: 0–85) were calculated. FC was also centrally measured by ELISA.
Results
Fifty-eight patients were recruited. Forty-two performed at least one FC measurement and 27 performed all the FC requested measurements. The median (IQR) SUS (0–100) at the first and last use were 85 (78–90) and 81 (70–88), respectively; the median (IQR) GSU (0–85) at the first and last use were 74 (69–80) and 77 (68–83), respectively. Adherence to the planned measurements and usability of the tool were higher in females and in less severe disease. The intra-class correlation coefficient between home-based and centrally measured FC was 0.88.
Conclusion
The adherence to home-based measurement of FC was fair. Usability scores for the home-based test were high. There was a good correlation with the centrally measured FC by ELISA.
Inflammatory bowel diseases (IBD)—Crohn’s disease (CD) and Ulcerative Colitis (UC)—consist in chronic inflammatory conditions of the gastrointestinal tract that most of the time evolve through remitting and relapsing periods. Recently, the importance of achieving mucosal healing in these pathologies has been highlighted [
]. This has led to the concept of treat to target, the target being a composite outcome including patients reported outcome as well as intestinal healing [
]. However, endoscopy presents some limitations: it is relatively expensive and its repetition is not widely accepted by patients due to its invasiveness and a low risk of complications.
A non-invasive and easily available marker of bowel inflammation is fecal calprotectin (FC). It is protein produced by neutrophils. This biomarker is well correlated to endoscopic inflammation and can even detect histological inflammation despite endoscopic remission [
Fecal calprotectin more accurately reflects endoscopic activity of ulcerative colitis than the lichtiger index, C-reactive protein, platelets, hemoglobin, and blood leukocytes.
Current tests for FC need to be performed in laboratories, which means to collect and bring the stool samples to the hospital for analysis. That is particularly uncomfortable for patients, which may lead to relatively low adherence to the monitoring plan. In a recent large nationwide survey in France, main reasons for low acceptability of stool collection were embarrassment to collect the stools, the dirtiness feeling and the problem of transporting the sample [
Patients’ point of view regarding acceptability and usefulness of inflammatory bowel disease monitoring tools: results from a nationwide multicentre study (the ACCEPT study).
Recently, thanks to technological developments, it has become possible to measure directly the stool concentration of calprotectin in individual samples through a rapid immunochromatographic assay captured by a smartphone’s camera, potentially transforming the measurement of FC into a home-based test allowing the patient to measure FC him/herself [
] and the practicability of such measurements in routine practice has still to be confirmed.
Hence the primary aim of our study was to evaluate the usability of such a home-based test for the measurement of FC in IBD patients and the adherence to the planned measurements.
2. Patients and methods
2.1 Patients
We prospectively recruited patients with CD or UC, in clinical remission or with mild activity of their disease, from three IBD centers in Barcelona, Oslo and Liège. To be recruited, the patients had to be diagnosed with CD or UC for at least 6 months, be >18 years of age, have a Harvey Bradshaw Index (HBI) [
] <3 for UC, a stable treatment with no change and no surgery planned in a near future. They also had to own a smartphone compatible with the IBDoc® tool (BÜHLMANN) and to declare themselves motivated to perform measurements of FC and to answer questionnaires.
Data collected for each patients were: age, sex, clinical centre (Lovisenberg Diaconal Hospital Oslo, Hospital Universitari de Bellvitge of Barcelona or CHU of Liège), type of IBD (CD or UC), treatment(s) (5ASA, immunosuppressants (IS), corticoids and/or biologics) and HBI (for CD patients) or partial clinical Mayo score (for UC patients).
The protocol was accepted by the Ethical Review Board of each participating centre and all patients gave their written informed consent before inclusion.
2.2 Standardized training
Patients received a standardized information on the IBDoc® device, given by trained IBD nurses or IBD doctors and got access to information documents and demonstration procedures. They also received materials and accessories in an IBDoc® test kit, composed by a CALEX Valve device filled with extraction solution, a test cassette, 2 stool collection papers and one camera test card, to get trained. They had to download an app called “CalApp” on their smartphone, available on iTunes App store or on Google Play store. They had to enter their IBDoc® account email address and password, and to read and accept the End User Licence Agreement (EULA).
2.3 FC measurements and questionnaires
Patients were asked to collect stool and measure FC with the CalApp (IBDoc®), as demonstrated during the training, five times, two weeks apart, over an 8 weeks period. At the second, third or fourth measurements a subset of patients (patients included at CHU of Liège hospital) was asked to perform 3 extraction procedures to test for reproducibility of the extraction process. For these extracts, as well as for all the measurements performed with the CalApp, a stool extract was sent to the central laboratory at BÜHLMANN for FC measurement by ELISA. This extract was sent by the patient in specific envelopes on the day of the collection to BÜHLMANN laboratory where it was received within 2–3 days and immediately frozen at −80 °C. It was defrozen only once for the fecal calprotectin measurement, within one year after having been collected and frozen. The measurement at central laboratory was blinded from the home-based test results. The patients also had to answer a structured questionnaire twice: once before the first stool collection (day 0), and once at the end of the study period, at week 8. The questionnaire contained Likert scales (0–5) and simple questions with yes/no answer (17 overall) on the understanding of the methods, the ability to use the extraction device, the ability to use the reading cassette and the smartphone camera, and the use of the portal where results were stored. From there, we calculated a Global Score of Usability (GSU; calculated by summing up the values of the Likert scales and attributing 5 points for yes and 0 for no in the yes/no questions; ranging 0–85). The questionnaire also contained 4 other simple questions with yes/no answer, about the trust in the results, the added value of the test for disease management, the appropriateness of the use of a smartphone for a medical test and the foreseen use of this tool in the future in the routine setting. Finally a set of 10 Likert scale questions led to a second global usability score: the System Usability Scale (SUS; ranging 0–100). The System Usability Scale (SUS) is a simple, widely used survey developed in the 1980s as a subjective measure of system usability. SUS is technology independent and has since been tested on hardware, consumer software, websites, cell-phones, IVRs and other devices [
We also analyzed the adherence to the monitoring plan by calculating the proportion of FC measurements done. We divided the patients in three groups: group 1: no measurement performed, group 2: 1–4 measurements and group 3: all 5 measurements.
2.4 Statistics
The results are presented as medians and quartiles (Q1–Q3) for continuous variables, and as frequency tables for categorized variables. To compare the variables between measures at day 0 and week 8, the Wilcoxon sign rank test was used for continuous variables and the McNemar test for categorized variables. The ordinal logistic regression was used to study the adherence to planned measures (according to the 3 pre-specified categories: no measurement performed, 1–4 measurements performed, all 5 measurements performed) with respect to clinical characteristics, demographics, usability scores and 4 yes/no questions about the trust in the results, the added value for disease management, the appropriateness of the use of a smartphone for a medical test and the foreseen use of this tool in the future in the routine setting. A linear regression was used to study the scores GSU and SUS with respect to the same factors. The adherence to planned measures, GSU and SUS were also evaluated in a multivariate analysis with a stepwise selection of the variables. Other comparisons were done between factors (Spearman correlation coefficient between continuous variables, chi-square test between categorical variables and Kruskal–Wallis between continuous and categorical variables).
Reproducibility of the stool extraction process by the patient was assessed by the calculation of the variance components for the three successive stool extracts performed on the same stool by the subset of patients who accepted to perform these successive stool extracts. Agreement between FC measured by the home-based test and the central ELISA was assessed by the intra-class correlation coefficient (and its 95% lower confidence limit). The difference between the two measures was assessed by the Wilcoxon sign rank test. The Bland and Altman method and the Spearman correlation coefficient were used to assess the correlation between the magnitude of the measure and the difference between the two techniques. FC measurements were also categorized in three categories: normal (≤100 μg/g), borderline (101–299 μg/g) and elevated (≥300 μg/g). The agreement between the two methods to classify the semi-quantitative FC value was also estimated by the Kappa coefficient of Cohen.
Variability of the fecal calprotectin measure over time was assessed by measuring the coefficient of variation between successive measures and by analyzing the impact of time (representing the successive measures) in a generalized linear mixed model.
The performance of the home-based test to predict a fecal calprotectin >300 μg/g was assessed, calculating sensitivity, specificity, negative predictive value and positive predictive value.
Results were considered significant at the level of uncertainty of 5% (p < 0.05). The calculations were performed using SAS version 9.4 and figures were done in R version 3.2.2.
3. Results
3.1 Patients’ characteristics, global adherence to the measurement plan and usability
In total, 58 patients were included into the study. Characteristics of the patients are shown in Table 1. Among the 58 patients, 42 performed at least 1 FC measurement (72.4%), and 27 (46.6%) performed the 5 requested FC measurements (Fig. 1).
Table 1Patients’ demographic and clinical characteristics (N = 58).
Fig. 1Adherence to the planned fecal calprotectin (FC) measurements with the home-based test (N = 58). Patients were split into three categories depending on the number of measurements they performed, over a total of 5 which was initially planned. 27.6% of the patient did not perform any measurement, 25.9% performed 1–4 measurements and 46.6% performed all 5 measurements.
The questionnaires were filled in at day 0 by 39 patients and at week 8 by 40 patients. Those patients correspond to a high proportion of the patients who measured FC at least once with the home-based test.
The median GSU was 74 (IQR: 69.1–80) at day 0, and 77 (IQR: 68–83) at week 8. There was no statistically significant difference between GSU at day 0 and at week 8 (p = 0.48) (Fig. 2).
Fig. 2Global Score of Usability (GSU) and System Usability Scale (SUS) at day 0 and week 8 (N = 58). Between day 0 and week 8, there was no significant change in the GSU (74 (IQR: 69.1–80) vs. 77 (IQR: 68–83)), while there was a slight but significant decrease in the SUS (85 (IQR: 77.5–90) vs. 81.3 (IQR: 70–87.5) (p = 0.0025)).
The median SUS significantly decreased between day 0 (85; IQR: 77.5–90) and week 8 (81.3; IQR: 70–87.5) (p = 0.0025) (Fig. 2).
There was no statistically significant difference between day 0 and week 8 concerning the trust in the tool (87.1% and 90.3%; p = 0.32), the perception of a better management of the disease thanks to the tool (86.1% and 83.3%; p = 0.56), the confidence in using a smartphone for a medical purpose (100% and 94.4%; p = 0.16), and the probability of future use of the tool (100% and 97.2%; p = 0.32).
3.2 Factors associated with adherence and usability of the test
The adherence to the planned FC measurements was significantly higher in females (p = 0.034), in 5ASA treated patients (p = 0.027). It significantly increased with lower disease activity in CD (p = 0.016) and higher GSU at day 0 (p = 0.010). It differed between centers (p = 0.025); it was higher in Barcelona compared to Liège. It was also higher in patients declaring that this tool would help them to better manage their disease at day 0 (p = 0.016) (Table 2). In multivariate analysis, it was higher in females (p = 0.0077) and in 5ASA treated patients (p = 0.042).
Table 2Ordinal logistic regression of the adherence to the planned measurements of FC with the home-based test with respect to factors (N = 58).
Number of FC measurements
0
1–4
5
P
Sex (n = 58)
Female
23.3%
13.3%
63.3%
Male
32.1%
39.3%
28.6%
0.034
Centre (n = 58)
Oslo
22.2%
27.8%
50.0
Liège*
40.0%
32.0%
28.0%
Barcelona*
13.3%
13.3%
73.3%
0.025*
IBD type (n = 58)
CD
25.0%
32.5%
42.5%
UC
33.3%
1.1%
55.6%
0.73
Treatment (n = 53)
5-ASA
17.6%
5.9%
76.5%
No 5-ASA
27.8%
33.3%
38.9%
0.027
Steroids
25%
0%
75%
No steroids
24.5%
26.5%
49%
0.42
IS
23.8%
28.6%
47.6%
No IS
25%
21.9%
53.1%
0.81
Biologics
28%
32%
40%
No Biologics
21.4%
17.9%
60.7%
0.20
Age (n = 56) (years; median (IQR)
32 (22–44)
28.5 (24–36)
37 (31–46)
0.11
HBI (n = 39) (CD; median (IQR)
4 (2–6)
0 (0–3)
0 (0–2)
0.016
Partial clinical Mayo (n = 17) (UC; median (IQR)
0 (0–0)
3.5 (1–6)
0 (0–2)
0.86
GUS at day 0 (n = 43; median (IQR)
57.1 (32.3–79.5)
72.1 (70–80.3)
75 (71.2–81.8)
0.010
GUS at week 8 (n = 41; median (IQR)
66.4 (61.2–84)
77 (68–80)
78.3 (72.5–84)
0.14
SUS at day 0 (n = 39; median (IQR)
88.8 (77.5–93.8)
83.8 (75–86.3)
87.5 (77.5–90)
0.90
SUS at week 8 (n = 40; median (IQR)
82.5 (65–85)
75 (67.5–82.5)
83.8 (72.5–87.5)
0.10
Trust in the device (at day 0; n = 39)
No
0%
12.5%
12%
Yes
100%
87.5
88%
0.67
Comfortable with smartphone use for a medical purpose (at day 0; n = 38)
No
25%
0%
0%
Yes
75%
100%
100%
0.98
Future use of the device (at day 0; n = 40)
No
0%
0%
0%
Yes
100%
100%
100%
0.99
Better disease management (at day 0; n = 39)
No
60%
12.5%
7.7%
Yes
40%
87.5%
92.3%
0.016
The * is indicating that the difference is significant between Barcelona and Liège.
The GSU at day 0 differed between centers (p = 0.013); it was significantly higher for Barcelona patients as compared to Liège. It also inversely correlated with disease activity assessed by HBI in CD (r = −0.47, p = 0.015). No factor was associated with GSU in multivariate analysis at day 0.
The SUS was only significantly higher in females at day 0 in univariate (p = 0.040) and multivariate analysis (p = 0.042).
Patients who were not treated with biologics expressed a greater trust (100% vs. 72.2%, p = 0.022) in the tool and thought it would help them to better manage their disease (100% vs. 65%, p = 0.007). No factor was significantly associated with the confidence in using a smartphone as a medical device. At week 8, the patients who stated they would like to use the device in the future for the management of the disease were significantly younger (36 vs. 55 years, p = 0.012).
3.3 Reproducibility of the extraction and reading procedures
Eleven patients recruited at the CHU of Liège Hospital performed three successive stool extraction processes on their second, third or fourth measurement. The mean of the individual coefficients of variation was 17.5% (range: 0–42%). The analytical variation coefficient was 25.1% while the subject variation coefficient was 96.7%.
3.4 Correlations between FC measured with the home-based test and the central ELISA
A total of 169 individual FC measures were available, coming from 42 out of the 58 patients. A median of 3 matched Central ELISA/home-based tests was performed per patient. The correlation between the two measures is shown in Fig. 3. The intra-class correlation coefficient between measures was 0.88 with an inferior limit of the coefficient at 0.82. The values obtained with the central ELISA were significantly higher than with the home-based test (158 (63–581) vs. 120 (49–419); p < 0.0001). Bland and Altman method showed that the greater the FC value, the greater the difference between the central ELISA and the home-based test (r = 0.90, p < 0.0001). When categorized in normal (≤100 μg/g), borderline (101–299 μg/g) and elevated values (≥ 300 μg/g), 140/169 measures (82.8%) were similarly classified with the two methods and the Cohen Kappa coefficient was 0.73 (0.65–0.82). Among these 140 measures, 65 were normal, 22 borderline and 53 elevated. Among the 29 misclassified, 14 were normal with the home-based test and borderline with the ELISA, 4 borderline for the home-based test but normal with the ELISA, 6 borderline for the home-based test but elevated with the ELISA, 5 elevated with the home-based test but borderline with the ELISA.
Fig. 3Correlations between fecal calprotectin (FC) measured with the home-based test and the central ELISA (N = 169). The intra-class correlation coefficient between measures was 0.88 with an inferior limit of the coefficient at 0.82. The values obtained with the central ELISA were significantly higher than with the home-based test (158 (63–581) vs. 120 (49–419); p < 0.0001).
The coefficient of variation of fecal calprotectin over time was 59% for ELISA and 69.4% for the home-based test. There was no significant impact of time in a generalized linear mixed model on the level of fecal calprotectin measured by either central ELISA (p = 0.16) or home-based test (p = 0.07).
The sensitivity, specificity, negative predictive value and positive predictive value of the home-based test to predict a fecal calprotectin >300 μg/g by ELISA were, 89.8%, 95.5%, 91.4% and 94.6%.
4. Discussion
Several studies concerning the evaluation of new tools for measuring FC by patients themselves and with the use of smartphones have been recently published. These were intended to compare the measurement of FC with these tools with the gold standard ELISA. These studies mainly concerned healthy subjects and rarely patients [
]. Our study confirms the good correlation between home-based and central ELISA measurement of FC on a relatively large number of pathological samples from IBD patients. It also shows a good usability of the home-based tool and a fair adherence to the planned FC measurements. Finally, it highlights patients’ profiles associated with lower usability scores and at risk of lower adherence to the planned measures using this tool.
Globally, the correlation between the FC measured by the home-based tool and the central ELISA was good. Particularly when FC values were categorized in normal, borderline or high, only a minority were misclassified. This is in agreement with previously reported results with healthy subjects and highlights the fact that this good correlation persists for higher FC values [
]. The variation between FC coming from three different extracts from the same stool sample was also relatively low when considering the range of possible variations of FC in pathological situations and its clinical relevance. This is reassuring on the validity of the extraction process made by the patients. This confirms previous studies on the feasibility of stool extraction by the patient him/herself and reproducibility between several extractions from the same bowel movement [
Clinical importance of faecal calprotectin variability in inflammatory bowel disease: intra-individual variability and standardisation of sampling procedure.
Scandinavian Journal of Gastroenterology.2016; 51: 548-555
The global adherence to the planned measurements was fair with three quarters of the patients at least performing one measurement over the 8 weeks study period. Nevertheless, the majority of patients did not strictly adhere to the planned measures and a small number even did not perform any test. This reflects the difficulty to obtain full adherence to this kind of monitoring plan. To our knowledge, this is one of the first time this kind of adherence was formally measured in IBD. In a previous landmark and pioneering study, a higher adherence rate, around 85%, was reported from a prospective cohort in which self-measurement of calprotectin was used in the setting of a more global management plan leading to tailored dosage of infliximab in CD [
]. The higher adherence in that study may have been linked to the more extensive education and training given to the patient as well as to the formal empowerment which was linked to it, the patients having to contact the health care providers for the planning of the next infliximab infusion [
]. Also, in the present study, we requested a measurement of fecal calprotectin every other week, which is higher than usually done or recommended in routine practice. This high frequency of measurement mat have lowered the adherence. Interestingly, in the present study, we could identify profiles of patients at high risk of non-adherence: non 5ASA users, and males. Non 5ASA users may be considered as having more aggressive disease requiring immunosuppressants or biologics to be controlled. The ability to be empowered in chronic disease may be influenced by disease severity and activity, and for example, hospitalized patients with disease flare may be less keen to be empowered while outpatients in longstanding remission may be willing to take greater part in their disease management. Likewise, patients with biologics, usually more severe, had less trust in the tool and significantly less of them stated this tool would help them to manage their disease in the future. However, this correlation between empowerment ability and disease severity has not been uniformly confirmed in other chronic diseases [
]. Lower adherence also correlated to lower global usability score. This highlights that by improving usability, particularly in these specific categories of patients we might improve the adherence. Self-management can be beneficial for the patient, but it requires, on the one hand, as suggested in the Pedersen et al. study [
], a formal patients empowerment, and on the other hand a sufficient level of self-efficacy. Techniques have been developed to increase self-efficacy by specific problem-solving skills training [
]. In other chronic diseases where biological or clinical measures are also done by the patients, like diabetes, the adherence to self-management plan is associated with several dimensions: cost, participants' emotion, and the Self-Monitoring Blood Glucose process [
]. More particularly, the barriers identified included: frustration related to high blood glucose reading, stigma, fear of needles and pain, cost of test strips and needles, lack of motivation, knowledge, and self-efficacy. Facilitators were also identified: experiencing hypoglycemic symptoms, desire to see the effects of dietary changes, desire to please the physician, and family motivation. Although some of these factors may be similar in fecal calprotectin self-monitoring in IBD, specific barriers and facilitators should also be searched in this setting.
We found that the usability, as measured by two different usability scores, can be considered as excellent [
]. There was no improvement over time but the scores were already very high for the first use. The absence of improvement over time could suggest that there is a subset of patients having difficulty to understand and use this kind of device. Those patients should be identified from the start and a complementary training should be given to them. We didn’t ask patients about the highest level of education they had completed, so we don’t know if those for whom the test was the easiest were those who had the highest level of education. Another recent study found a significantly higher correlation between home-based and central ELISA measured FC for academic patients, but they couldn’t conclude if they simply better understood the instructions, were more precisely oriented when performing the test, or if they asked more help if needed [
]. Only a few factors were associated with the tool usability. Particularly, the female sex was associated with a higher SUS score at day 0, both in univariate and multivariate analysis. This may have also contributed to better adherence. On the opposite, on the top of having lower adherence, CD patients with more active disease, also had lower scores of usability.
Finally, we found that patients who thought to use the tool in the future were on average younger. This probably has to do with the modern character of the tool.
In conclusion, this study confirms that the use of a home-based tool for measuring FC by the patient him/herself is feasible and reached high usability scores in the patients who decided to use it. We also confirmed a good correlation between the measurements made by this home-based test and the central ELISA. Although a small half of the patients actually performed all the planned tests, a quarter did not perform all the planned test and the last quarter did not even perform any test. The adherence to the planned tests was associated with the Global Usability Score suggesting the fact that more appropriate explanations for some patients may improve the adherence by increasing self-efficacy. We also disclosed specific profiles of patients associated with low adherence and low usability, for whom the tool or the training should be adapted.
Conflict of interest
None declared.
Funding
This work was supported by a grant of the Bühlmann Company, Switzerland.
Appendix A. Supplementary data
The following is Supplementary data to this article:
Fecal calprotectin more accurately reflects endoscopic activity of ulcerative colitis than the lichtiger index, C-reactive protein, platelets, hemoglobin, and blood leukocytes.
Patients’ point of view regarding acceptability and usefulness of inflammatory bowel disease monitoring tools: results from a nationwide multicentre study (the ACCEPT study).
Clinical importance of faecal calprotectin variability in inflammatory bowel disease: intra-individual variability and standardisation of sampling procedure.
Scandinavian Journal of Gastroenterology.2016; 51: 548-555
30886-1&id=gr1.jpg){kind=link}
30886-1&id=gr2.jpg){kind=link}
30886-1&id=gr3.jpg){kind=link}