Would Artificial Neural Networks Implemented in Clinical Wards Help Nephrologists in Predicting Epoetin Responsiveness?

Luca Gabutti; Nathalie Lötscher; Josephine Bianda; Claudio Marone; Giorgio Mombelli; Michel Burnier

Disclosures

BMC Nephrology 

In This Article

Methods

Patient Characteristics

We did a secondary analysis of the survey on Anaemia Management in dialysis patients in Switzerland (AIMS); a prospective, open label, non-randomized observational study designed to assess anaemia management in the dialysis centres in Switzerland [Lötscher N, et al. Swiss Med Forum 2004; 4: S7; Abstract]. In this study (inclusion and exclusion criteria were: current dialysis treatment, age > 18 y, renal anaemia requiring epoetin therapy, ferritin > 200 μg/L and respectively absence of vitamin B12 or folic acid deficiency, unstable angina pectoris, untreated hypertension, haemoglobinopathy, haemolysis, epilepsy), data about sex, weight, age, presence or absence of a diabetes mellitus and/or a cardiomyopathy, haemoglobin, epoetin beta dose and its administration route (subcutaneous vs. intravenous), ferritin and creatinine were collected in 340 hemodialysis patients. In order to increase the statistical power of the study and to gain more information about factors influencing erythropoietin responsiveness, a further 92 haemodialysis patients (with complete historical data and selected with the same criteria applied for the AIMS survey) were included in the analysis. These additional patients were selected based on the same inclusion criteria from the Renal Services of the Ente Ospedaliero Cantonale (EOC), Bellinzona, Switzerland after having been followed for at least 12 months (the 2 databases pooled together being called AIMSEOC data). For each patient the sex, weight and age, the haemoglobin, epoetin beta dose and its administration route (subcutaneous vs. intravenous), ferritin, creatinine, urea, Kt/V, pH, phosphate, ionized calcium, albumin, parathyroid hormone (PTH), C reactive protein (CRP) and intravenous iron dose over the 12 treatment months were collected. Comorbidity information about the concomitance of diabetes mellitus, cardiomyopathy with impaired left ventricular ejection fraction (<50%) and ACE-I or ARB medication were also registered.

Kt/V Calculation

The Kt/V, a parameter of dialysis adequacy defined as the dialyzer clearance of urea multiplied by the duration of the dialysis treatment and divided by the volume of distribution of urea in the body, was estimated with a second generation single-pool Daugirdas formula[39]: Kt/V = -ln (R -0.03) + [(4-3.5 × R) × (UF/W)] where: R = post-dialysis urea/pre-dialysis urea, UF = net ultrafiltration and W = weight.

Estimation of the Erythropoietin Dose Required to Obtain a Haemoglobin of 11.5 g/dL

To calculate the epoetin dose that should have been prescribed in order to obtain a haemoglobin of 11.5 g/dL a linear regression plotting epoetin dose against haemoglobin was built for each patient. The choice of using suboptimal tools like linear regressions to estimate the ideal epoetin dose was made considering that (i) the small number of observations reduced the statistical options at our disposal and (ii) an epoetin dose approximating the ideal one was necessary to build the models.

Artificial Neural Networks

In order to build the non-linear continuous functions expressing the interdependency between the collected data and the epoetin dose a series of artificial neural networks (ANNs) were built, trained, cross-validated and tested using the NeuroSolution for Excel 4.32 software, NeuroDimension Inc.

ANNs are composed of one input layer (collecting input variables expected to be predictive), one output layer (collecting the predictions, known in training and unknown in testing and validation cases) and one or more hidden layers (performing a weighted sum of the inputs and passing the resulting value through a non-linear function to the output layer). Individual weights are progressively adapted, using for instance a back-propagation algorithm, to minimize the difference between calculated and expected outputs; the weights assuring the best results then being used to test and compare the performance of the ANNs (see Figure 1 for a schematic representation).

Figure 1.

Schematic representation of an artificial neural network. A typical ANN consisting of one input layer, two hidden layers and one output layer is represented. The basic structure, fed forwards and trained by back-propagation is called Multilayer Perceptron (MLP) while models designed with connections jumping over hidden layers (---) are called Generalized Feedforward Networks (GFN).

The databases selected to be analyzed were randomized using the same software package. After selecting 25% of the data for the validation phase, the remaining pool was divided into the training, cross validation and testing subgroups assigning respectively to each one 50, 10 and 40% of the data. As a network structure only Multilayer Perceptrons (MLP) (layered feed-forward networks trained with static backpropagation) and Generalized Feedforward Networks (GFN) (generalization of the MLP with connections jumping over hidden layers) were used (see Figure 1). Seeing that the critical difference between the two network structures is the amount of training data requested to optimize the performance (higher in the MLP and lower in the GFN) and because of the database with a limited extension used, each set of variables was used to train both models. A hyperbolic tangent transfer function (maximum and minimum output 1 and -1 respectively), as recommended by the producer (default setting), was chosen for each neuron. All initial connection weights were randomized before beginning a training phase. As a learning rule a gradient and weight change one (momentum) was chosen. To avoid overtraining, the training phase was stopped when the minimum squared error (MSE), displayed as a function of the training epochs, between the predictions and the desired output in the cross validation subgroup (indirect indicator of the level of generalisation) began to increase. All the networks were built using only one hidden layer while the ideal number of hidden neurons was determined by training, cross validating and testing a MLP, increasing unitarily the number of neurons beginning with 4 and stopping when the MSE between the predictions and the desired output in the testing subgroup began to increase. Once a definitive MLP was obtained, its performance was compared with a GFN with an identical number of processing elements (being both trained, cross validated and tested 5 times) and the best performing model (best compromise in the testing subgroup between linear correlation and normalized mean squared error r/NMSE) (see "Statistical and data analysis" paragraph for details) was selected to be tested in the validation subgroup. All studied variables were first analyzed separately and then combined in order to achieve the best prediction performance. The study was designed according to the prescriptions of Cross et al.[33]

Prediction by the Nephrologists in Charge of the Patients of a Follow-Up Haemoglobin Level Below the Target of 11.0 g/dL

The sensitivity, specificity and the positive and negative predictive values in detecting a follow-up haemoglobin < 11.0 g/dL were calculated by interpreting the decision of the nephrologists to increase the epoetin dose as if it had been a prediction of a follow-up haemoglobin below the target level. Respecting statistical and modeling exigencies the haemoglobin measured one month after the adaptation in the dose has been considered as the follow-up haemoglobin.

Selection of the Data for the Prediction of the Monthly Epoetin Dose Adjustments

The data necessary to build and test the ANN predicting the monthly epoetin dose adjustments were selected from the AIMSEOC pool. Complete data of 4 consecutive months from months 3 to 6 and/or 7 to 10 were requested. The data of the months 3-5 and 7-9 were used as input while the data of months 6 and 10 were used as desired output.

Statistical and Data Analysis

Statistical and data analysis was performed using a statistical software package (SPSS 12.0; SPSS Inc.). SPSS was also used to build linear regressions (forward stepwise method based on the F probability including collinearity diagnostics) and to infer receiver-operating-characteristic curves (ROC) plotting the sensitivity against 1 minus the specificity for each prediction from the ANN, the linear regressions or the nephrologists (the areas under the curves were calculated by the trapezoidal nonparametric method and are expressed with the 95% confidence interval). Accuracy was expressed by the Combined Root Mean Square Error (CRMSE) calculated as the square root of [(mean difference in estimate-observed)2 + (standard deviation of the difference)2]. Agreement between the predictions and the basis data was expressed by "limits of agreement", "95% confidence interval for the bias" and "95% confidence interval for the lower and upper limits of agreement" according to Bland and Altman.[40] The mean difference in estimate - observed, also called "bias", and the standard deviation of the difference of the same subtraction, also called "precision" are included concepts in both CRMSE and "limits of agreement". Histograms comparing the performance of the ANNs and the best performing linear regression were built using Excel SP3, Microsoft Inc. With the intention of facilitating the graphical representation of the differences in the predictive performances the ratio r/NMSE was calculated dividing the Pearson linear correlation coefficient r by the normalized mean squared error (obtained dividing the mean squared error by the variance of the reference population); the higher the value the better the performance. Percentages were compared using a Fisher exact test. The values are in mean ± standard deviation (SD). A P-value < 0.05 was considered statistically significant.

Comments

3090D553-9492-4563-8681-AD288FA52ACE
Comments on Medscape are moderated and should be professional in tone and on topic. You must declare any conflicts of interest related to your comments and responses. Please see our Commenting Guide for further information. We reserve the right to remove posts at our sole discretion.

processing....