introduction

Routine use ofthe “ADE Scorecards”,

an application forautomated ADE detection,

in a general hospitalEmmanuel Chazarda, Michel Luyckxb, Jean-Baptiste Beuscartc, Laurie Ferreta, Régis Beuscarta

a Public Health Department, Lille University Hospital; UDSL EA 2694; Univ Lille Nord de France; F-59000 Lille, France.

b Hospital Pharmacy, Lille University Hospital; UDSL EA 4481; Univ Lille Nord de France; F-59000 Lille, France.

c Geriatrics Department, Lille University Hospital; UDSL EA 2694; Univ Lille Nord de France; F-59000 Lille, France.

2Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

Introduction

I. Adverse Drug EventsII. The PSIP European ProjectIII. The “ADE Scorecards”

2013-08-22

2013-08-22

Adverse drug events ADEs = Adverse Drug Events Several definitions. Institute of Medicine (2007):

“An injury resulting from the use of a drug” “An injury due to medication management

rather than the underlying condition of the patient” Epidemiological data:

98,000 deaths per year in the US An ADE would occur in 5-9% of inpatient stays

Two fields of research: Retrospective ADE detection Prospective prevention of ADEs by CDSS (clinical decision support

systems)

3Routine use of the “ADE Scorecards”, an application for

automated ADE detection, in a general hospital.

2013-08-22

Funded by the European Research Council, 7th framework program(agreement N°216130)

The PSIP European ProjectPatient Safety Through Intelligent Procedures in Medication



Hospitals:• Lille University Hospital (F)• Rouen University Hospital (F)• Denain general hospital (F)• Hospitals from the Region Hovedstaden

(Dk)Industrial partners:

• Oracle (Europe)• IBM Denmark (Dk)• Medasys (F)• Vidal SA (F)• KITE solutions (I)• Ideea Advertising (Ro)

Academic partners:• Aristotle Thessaloniki University (Gr)• Aalborg University (Dk)• UMIT – Innsbruck University (Au)

2013-08-22

The “ADE Scorecards”, a tool for automated ADE detection in EHR



EHR DatabaseADE de-

tection rules

Computation step

List of potential ADE cases

Contextualized statistics

Web-based display tool

- Statistics on ADEs- Description of

rules- Cases review

Routinely-collected data

(diags, lab, drugs, etc.).

/!\ ADEs are not flagged.

236 validated complex rules

Statistics and automated filters

are site-dependant

Real past inpatient

stays

Multilingual (En, Fr, Dk,

Bu)

2013-08-22 6Routine use of the “ADE Scorecards”, an application for




Number of cases per month Histogram of appearance delay



83


automated ADE detection, in a general hospital.2013-08-22

2013-08-22

The ADE Scorecards, a tool for automated ADE detection in EHR

Usage: Installed in 5 hospitals (2 Danish, 2 French and 1

Bulgarian) Routinely used by the physicians and pharmacists of a

French general hospital during three years Objective: show the results of its use in real-life

situation




Material and methods

I. Quantitative analysisII. Qualitative analysis

2013-08-22

Quantitative & qualitative analysis

Analyzed data: January 2007 to August 2012 Description: Patients, medical background (95% confidence intervals) Comparisons:

Between medical departments (displayed here: Gynecology-Obstetrics vs. Cardiology)

Chi-2 tests & Student’s t-tests (with =5%) Focus on 2 outcomes:

Hyperkalemia: defined as K+>5.5mmol/lmay induce lethal cardiac rhythm troubles

INR increase: defined as INR≥5 (INR=international normalized ratio) due to VKA overdose or interaction (VKA=vitamin K antagonist)may induce a severe hemorrhage

Observation of the daily use by a human-factors specialist and a pharmacist




Results

I. Characteristics of the patientsII. Potential ADEs with INR increaseIII. Potential ADEs with hyperkalemia

2013-08-22

27.7%

24.4%18.4%

15.8%

10.0%

3.8%

Main medical department

Surgery

Cardiology

Internal medicine

Pneumology

Gynecology Obstetrics

Geriatrics

Characteristics of the patients(n=73,836 inpatient stays)



0

2000

4000

6000

8000

10000

12000

14000

16000

2007 2008 2009 2010 2011 2012(Jan-Aug)

Nb of stays

Nb of analyzed stays

Parameter Overall Cardiology Gyn. Obs. p value (all)Age (years) 60.2 67.6 28 p<0.001

Length of stay (days) 8.01 8.19 11.6 p<0.001

Proportion of men 40.80% 42.80% 0.00% p<0.001

Potential ADE cases with INR increase (INR≥5)

Estimated proportion:

0.99% [0.89%;1.09%]

Interesting data:



Parameter Overall Cardiology Gyn. Obs. p val (all)VKA 8.34% 15.50% 0.00% p<0.001Chr. hepatic insuf. 4.90% 13.70% 0.04% p<0.001INR increase (all) 2.46% Potential ADE 0.99% 1.36% 0.00% p<0.001

0.00%

0.50%

1.00%

1.50%

2.00%

2.50%

3.00%

2007 2008 2009 2010 2011 2012*

Potential ADE cases with hyperkalemia (K+>5.5mmol/l)

Estimated proportion:

2.03% [1.89%;2.18%]

Interesting data:



0.00%

0.50%

1.00%

1.50%

2.00%

2.50%

3.00%

2007 2008 2009 2010 2011 2012*

Parameter Overall Cardiology Gyn. Obs. p val (all)Diuretics 23.30% 41.10% 0.00% p<0.001

Chr. renal insuf. 2.02% 3.04% 0.04% p<0.001Hyperkalemia (all) 5.43% Potential ADE 2.03% 2.65% 0.00% p<0.001

Results of qualitative evaluation and Discussion

Initial design:



ADE Scorecards

Physicians

Pharmacist

Physicians in charge of quality

of care

Regular spontaneous

use of the soft

Improvement of the drug

management

Decrease of morbidity

Good accuracy (precision 50%,

recall 95%)User-centered design,

good evaluation (questionnaire)

Prescription analyses

Patient records

screening

Weekly meetings

Morbidity and mortality

reviews

Results of qualitative evaluation and Discussion

support to a global quality improvement approach:



Physicians

Pharmacist

Physicians in charge of quality

of care

Improvement of the drug

management?

Decrease of morbidity?

Prescription analyses

Patient records

screening

Weekly meetings

Morbidity and mortality

reviews

• Faster preparation• Real local cases (not

perceived as theoretical)

Faster preparation for drug-related morbidity


Thank you for your attention! Free demonstration:

http://psip.univ-lille2.fr/prototypes/public/

The research leading to these results has received funding from the European Community’s Seventh Framework Program (FP7/2007-2013) under Grant Agreement n°216130 - the PSIP project.

Contact: [email protected]

2013-08-22

http://psip.univ-lille2.fr/prototypes/public/

mailto:[email protected]

2013-08-22

The “ADE Scorecards”General procedure

Databases of inpatient stays

Statistics computation

Tool for comprehensive

visualization: the ADE Scorecards

Data Mining &expert

validation

Identification of potential ADE cases

ADE detection rules



Available data: ~155,000 inpatient stays from 6 hospitals (F, Dk, Bu)

2013-08-22

DiagnosesE119 DiabetesI251 AthérosclérosisI10 Arterial hypertensionN300 Cystitis

Lab resultsNPU03230 Potassium

Administered drugsProceduresZZBQ002Thorax radiography



Demo. & AdminAge 80Man? 0Dead? 0Length of Stay 9 (…)

Data aggregation,Mappings,data management

Simplified database with potential outcomes and context

Data mining(decision trees, etc.)

ADE rulesADEcases


Which methods forretrospective ADE detection ?

Reporting systems: Based on spontaneous case reports Mandatory, but underreporting bias:

less than 5% cases are declared!

2013-08-22

Objective: using data reuse & data mining to: Automatically identify past ADE cases Generate ADE detection rules Computing probabilities of occurrence

Expert-operated chart reviews Reference method, expert validation Time consuming: 30 min per case, and some

ADEs are very rare…



Administrative data88 years old woman

DiagnosesI10 Arterial hypertensionZ8671 Personal history

of myocardial ischemiaI620 Non-traumatic subdural

hemorrhage

Medical proceduresABJA002Drainage of an acute

subdural hemorrhage, by craniotomy

FELF001Transfusion

Free-text reportsDischarge

letterSurgical report

Drugs

Laboratory results

AcetaminopheneVKAVitamin KStatinRed blood cells

INR

Hemo-globin

Material and Methods



Results

I. Example of decision tree & interpretation

II. The ADE Scorecards, a tool for automated ADE retrospective detection



2013-08-22

ResultsExample of decision tree (1)

f=0.1

f=0 f=0.25

VKAno yes

Butyrophenone discontinuationno yes

f=0.2Hypoalbuminemia

no yes f=0.4

f=0.05 f=0.5



VKA= vitamin K antagonists (anticoagulant)

INR= international normalized ratio. Evaluates VKA activity

INR>5 => risk of hemorrhage

The tree attempts to explain INR>5

2013-08-22


1. VKA & butyrophenone discontinuation P=0.4

2. VKA & no butyrophenone discontinuation & hypoalbuminemia P=0.5

3. VKA & no butyrophenone discontinuation & no hypoalbuminemia P=0.05

4. no VKA P=0

We obtain 4 rules. 2 of them are associated with increased risk of hemorrhage => 2 ADE detection rules

4

3 21



2013-08-22


Butyrophenone:neuroleptic drugs, may accelerate the intestinal transit

VKA

Step 1: normal VKA intake

INR

VKA

Step 2: butyrophenone=> transit acceleration

=> too low INR

Buty.

INR

VKA

Step 3: increased intake of VKA

=> normal INR

Buty.

INR

VKA

Step 4: buty. discontinuation

=> normal transit=> too high INR

INR



2013-08-22


Albumine = plasmatic protein to which VKA bind. Only the non-bound part is biologically active.

Serum albumin VKA

Normal state:99% of the VKA bind to albumin.Only 1% of VKA are biologically active. The intake is based on it.

Hypoalbuminemia:decrease of the bound fraction,increase of the non-bound fraction=> too high INR (with constant intake)



2013-08-22

Complete 2010 year of one hospital Number of stays : 14,747 Number of hyperkalemia cases : 117 (7.93‰) exhaustive review

Result Precision 39/75= 52.0% Recall 39/41= 95.1% Harmonic mean 67.2% Number of reported cases 0/41= 0%

52.0%

95.1%

Results

Evaluation of the ADE retrospective detection

ADE Not ADEADE 39 36

Not ADE 2 40

Experts

ADEScorecards



2013-08-22

Retrospective detection of ADEs

Retrospective identification of past ADEs, although no explicit signal exists in the dataInpatient stay

Prescriptio

nPrescriptio

nPrescriptio

n

Hospital information system, databases



2013-08-22

Prospective prevention of ADEs

Inpatient stay

Hospital information system

Prescriptio

n Prescriptio

n Alert method

Change of the drug prescription

Alert generation, before the ADE occurs, in order to prevent it.



2013-08-22

Statistiques sur les EIMdans la base de données PSIP

Nb of cases of outcome *

Nb of cases occurring during

the stay

Nb of potential ADEs (automated

detection)

Nb of confirmed ADEs (expert

review)** Hyperkalemia 1301

2.67% |||||| 703

2.84% |||||| 507

2.05% |||||| 271

1.1% ||||

Renal failure 2293 4.7%

|||||||||| 728 2.94%

|||||| 404 1.63%

|||| 189 0.76%

||

VKA overdose

625 1.28%

|||| 321 1.3% |||| 246 0.99%

|| 137 0.55%

||

Other kinds of outcomes

13936 28.56%

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 7171 28.97%

|||||||||||||||||||||||||||||||||||||||||||||||||||||||||| 1438 5.81%

|||||||||||| 380 1.53%

||||

Total 14454 (29.62%) 7624 (30.8%) 2196 (8.87%) 997 (4.03%)

*: le nombre d’événements est rapporté au nombre total de séjours, alors que les nombres suivants sont rapportés aux séjours de plus de 2 jours uniquement**: ces nombres sont extrapolés depuis un échantillon



2013-08-22



2013-08-22

Pour chaque règle, les statistiques contextualisées sont calculées dans chaque

service

X all departmentsX surgeryX gyneco-obstetricsX medicine AX medicine BX pneumologyY all departmentsY apoplexyY cardio & endocrinologyY geriatricsY gynecologyY intensive care unitY internal medicineY obstetricsY orthopedicsY rheumatologyY urologyZ all departmentsW all departments

X all dpts



Prévention prospective des EIM



2013-08-22

Prévention prospective des EIMApproche de PSIP

Ex : AVK & IPP risque hémorragique Implémentation classique :

Implémentation PSIP :

Service A Service B Service CAVK&IPP alerte

Service A Service B Service CProbabilité mesurée à 10%AVK&IPP alerte

Probabilité mesurée à 0.01%AVK&IPP silence

Circonstance inconnueAVK&IPP alerte



2013-08-22

Prévention prospective des EIMApproche de PSIP

Développement de plusieurs systèmes d’aide à la décision : Prototype IBM Prototype Medasys Simulation web d’ordonnance

Caractéristiques majeures : Alertes filtrées statistiquement, contextualisées

moins d’alertes, plus pertinentes Règles raffinées (segmentation statistique)

prédiction du risque plus pertinente Méthodes d’alerte moins interruptives, plus acceptables



Data reuse in insurance companies…

Routine activities:

Reuse of the data:

2013-08-22

Transactional activities.The company:

How much should Mr Smith pay for his

car insurance?Nearly-custom

database

Model for predicting

individual riskStatistic

al analysis

Daily feedingand updating

Accidents database

(outcomes)

Contributions data

(incomes)

Demographic data

Data transformation

Recruits and follows customers

Banks insurance premiums

Pays out claims

Personalized insurance premiums

Decision



“big data” is generally a property of the routinely collected data that can be reused

“Big” can be understood through 5 dimensions:

id age gender diagnosis… … … …

Definition of “big data”

2013-08-22

id age gender diagnosis123 23 M I10

125 78 M K37

245 13 F M61.2

278 24 M I41

324 65 F I48

350 34 F F20.2

1-Many records

……

…

…

…

…

…

…

2-Many variables

… … … …

… … … …

Id Par Val123 K+ 4.5123 K+ 4.8

123 K+ 5.2

3-Many possible values for qualitative variables

… … … …

4-Many tables & relationships

5-Variables with repeated measurements



Challenges in data reuse Where is the secret of a successful data reuse?

2013-08-22

Scientific question

Nearly-custom database

ResultsStatistical

analysis

Transactional database 3



Data transformation

InterpretationKnowledge

Here? Not really…Data mining techniques( statistical methods) are used, but not specific.

Here? Partially…Significant tests are nearly always observed in Big Data: correct the risk, consider the effect size. Cf. post.

Here? Yes, mainly!The decisions that are taken for the data transformation process have a critical effect.



Data reuse of EHR(Electronic Health Records)

2013-08-22

Demo. & AdminAge 80Man? 0Dead? 0Length of Stay 9 (…)

DiagnosesE119 DiabetesI251 AthérosclérosisI10 Arterial hypertensionN300 Cystitis

Lab resultsNPU03230 Potassium

Administered drugs

Free-text reportsMedical devices??

ProceduresZZBQ002Thorax radiography



Data quality assessment



Data quality assessment Getting reliable data is a challenge => an iterative quality control is

mandatory Extraction format, basic requirements (tables, fields) Single value validity. E.g.:

Incorrect type: Age=“old” Impossible value: age=141, diagnosis=“HHFA001” Out-of-terminology value: diagnosis=“B99.0”

Univariate validity. E.g.: Each value is possible, but Mean(age)=85 Contextual: mean(age)=21 in Pediatrics

Bivariate validity. E.g.: Length of stay=2, admission=“2013-05-14”, discharge=“2013-05-14” Age=21 in a Geriatrics Unit Age=21 with diagnosis=“Alzheimer disease”



Data transformation, data aggregation



Data transformation: what for?

2013-08-22

Mandatory before using statistical methods Enables to transform data into information: how would a

human comment/summarize the raw data? Suppresses 3 over 5 dimensions of “bigness” that are not

compatible with statistical analysis

id age gender diagnosis… … … …

id age gender diagnosis123 23 M I10

125 78 M K37

245 13 F M61.2

278 24 M I41

324 65 F I48

350 34 F F20.2

1-Many records

……

…

…

…

…

…

…

2-Many variables

… … … …

… … … …

Id Par Val123 K+ 4.5123 K+ 4.8123 K+ 5.2

3-Many possible values for qualitative variables

… … … …

4-Many tables & relationships

5-Variables with repeated

measurements



Data transformation, example 1:suppression repeated measurements (1)

2013-08-22

Potassium:

time0

Id stay Date Parameter Value123 0 Potassium 4

123 1 Potassium 4123 … … …123 0 Sodium 140123 … … …

528 0 Potassium 3.2528 … … …

Sodium:

time0

Potassium:

time0

Stayn°123

Stayn°528Several parameters may be measured

several times during a given inpatient stay.=> One curve per {id_patient*parameter}



Data transformation, example 1:suppression repeated measurements (2) Objective: 1 table with 1 row per stay Example of simple transformation (without a priori knowledge):

2013-08-22

Potassium:

time0Sodium:

time0Potassium:

time0

Stayn°123

Stayn°528

Id stay

Potassium.Min

Potassium.Max

Sodium.Min

Sodium.Ma

x

123 3.8 6.2 136 142528 2.8 3.2 NA NA



Data transformation, example 1:suppression repeated measurements (3) Another example of transformation, with knowledge Uses the range of normal values according to the parameters,

summarizes the anomalies

2013-08-22

Id stay

Hyperkalemia

Hypokalemia

Hypernatremia

Hyponatremia

123 1 0 0 0528 0 0 NA NA

Potassium:

time0Sodium:

time0Potassium:

time0

Stayn°123

Stayn°528



Data transformation, example 2:interpretation of a qualitative variable (1)

2013-08-22

Id stay Procedure.code Procedure.wording123 LMMC004 Bilateral treatment of inguinal hernia without

prosthesis, by video surgery123 GLHF001 Arterial blood collection for blood gas and pH

sampling528 LMMC020 Treatment of abdominal hernia with

prosthesis, by laparoscopy 528 ZZBQ002 Thorax radiography

Each stay may have 0, 1 or several procedures. The terminology used (CCAM) has more than 5,000 possible codes. In this case, we only interest on hernia treatment.



Data transformation, example 2:interpretation of a qualitative variable (2)

2013-08-22

Id stay Procedure.code123 LMMC004123 GLHF001528 LMMC020123 ZZBQ002

New qualitative variables are created:• Each variable has few possible values• A mapping is necessary (with overlaps)• Requires a strong medical knowledge about

codes

Id stay

H.type H.prosthesis

H.approach H.bilateral

123 Inguinal 0 Videosurgery 1528 Abdominal 1 Laparoscopy 0



Data transformation, example 2:interpretation of a qualitative variable (3) Example of mapping

used for hernia FROM (raw data):

18 codes TO (information):

Type (n=3) Prosthesis (n=2) Approach (n=6) Bilateral (n=2)

2013-08-22

Code type prosthesis approach bilateral

LMMA009 abdominal 0 direct 0

LMMA014 abdominal 0 direct 0

LMMC020 abdominal 1 laparoscopy 0

LMMA019 inguinal 0 inguinal 1




LMMC003 inguinal 0 videosurgery 0


LMMA006 inguinal 1 direct 0



LMMA002 inguinal 1 other 1

LMMA008 inguinal 1 other 0



LMMA011 other 0 inguinal 0

LLMA007 other 0 laparotomy 0



Problems of interpretation with big data

I. Many variables (columns)=> type I error correction

II. Many records (rows)=> effect size=> over fitting of the models



1- Type I error inflation with numerous statistical tests

Reminder: principle of statistical tests (1)

Example : we want to test whether variables A & B are independent.

A null hypothesis is supposed:H0: A & B are independent

Alternative hypothesis H1: A & B are not independent.

A test statistic is computed: Its behavior under H1 is unknown! Under H0, this test statistics follows a

known distribution

2013-08-22

Example : A & B are

qualitative Test

= Khi² test Test statistic

= ²



Example : The Test statistic ²

follows a ² distribution The reject zone is

represented in red:

Under H0, this test statistics follows a known distribution

Zone of reject of H0: extreme values of the test statistic, corresponding to a chosen probability (generally 5%)

2013-08-22

Distribution and zone of rejectof H0 in a Khi² test



95% 5%





Use of the test: we suppose that H0 is true. If the result of the test is an improbable value (associated probability p < ), we decide that H0 is not true: we reject H0.

But, if H0 is really true, there exists a risk to reject it wrongly: the risk = the Type 1 error (generally 5%)




What if several tests are realized? Scenario with several tests:

k independent tests are realized on a dataset A « Signal » is observed if at least one of the k tests is

significant (p<5%) Signal = Test1 Test2 … Testk

Let’s suppose that all the associated null hypothesis are TRUE, then what is the probability to observe a signal, if every test is realized with a indiv type 1 error?

total =1-(1- indiv)k

=> inflation of the risk.




What if several tests are realized? => inflation of the risk.

2013-08-22

0 20 40 60 80 100

0.0

0.2

0.4

0.6

0.8

1.0

a

b

Number of tests(0-100)

Total alpha risk if every test is

computed with a 5% individual

alpha risk




How to correct this inflation? In order to get the total wished, the individual threshold

indiv must be modified: Šidák’s correction:

Use indiv =1-(1- total)1/k

Reminder: ab = eb.ln(a)

Bonferroni’s correction, very popular: For usual values of total (1 to 10%), use simply:

indiv =(total)/kin practice: indiv =(0.05)/k

Threshold very close (and inferior to) Šidák’s threshold: more conservative, easy to compute

Usable even when the k tests are not independant!




Example of use (1) A dataset of inpatient stays contains X binary variables (e.g.

administration of Spironolactone, hyperkalemia, etc.). We want to discover associations by testing every couple of variables. For X=2, 10, 100, 1000, 10000:

How many tests are performed?

What is the value of total if indiv =0.05?

What indiv should be used to obtain total =0.05?Šidák: Bonferroni:

2013-08-22

2)1(1

1

xxinx

in

total 95.01

nindiv

/195.01nindiv05.0



Number of variables

Number of tests

total ifindiv=5%

Sidak’sindiv

Bonferroni’sindiv

2 1 0.05 5.000E-02 5.000E-0210 45 0.9006 1.139E-03 1.111E-03

100 4950 1.0 1.036E-05 1.010E-051000 499 500 1.0 1.027E-07 1.001E-07

10000 49 995 000 1.0 1.026E-09 1.000E-09


Example of use (2) A dataset of inpatient stays contains X binary variables (e.g. administration of

Spironolactone, hyperkalemia, etc.). We want to discover associations by testing every couple of variables.

For X=2, 10, 100, 1000, 10000:

2013-08-22

2)1(1

1

xxinx

i

ntotal 95.01

nindiv

/195.01

nindiv05.0



2- Significance versus size of the effect

Formulation of the result of a statistical test

A decision variable is first computed. Then it is tested. Two equivalent ways to formulate the

result (examples with =5%): The p value (significance) :

We test whether the decision variable is different from the value expected under H0 (generally 0 or 1)

H0 is rejected if p < 5% The confidence interval:

The test returns the 95% confidence interval of the decision variable

H0 is rejected if this confidence interval does not contain the value expected under H0 (generally 0 or 1)




Example: comparing proportions We want to compare 2 proportions with =5%.

Examples of equivalent ways: Way #1:

Δ= π1-π2 is the effect size H0 : Δ=0 Test => p value:

H0 is rejected if p < 0.05 Way #2:

Relative risk RR=P(A/B) / P(A/B)is the effect size

H0 : RR=1 Test => confidence interval CI0.95:

H0 is rejected if CI0.95 doesnot contain 1

2013-08-22

0 Δ

Significance

Effect size

1

RR

Confidence interval of RR

Significance

Effect size




Example of proportions comparison In a hospital database (n=500,000 patients) we want to test

whether the intake of Drug X is associated with the occurrence of cardiac insufficiency

Relative risk with 95% confidence interval:1.071 [1.008 ; 1.137]

=> statistically significant, but weak effect size (+7.1%)

2013-08-22

CI+ CI- (%CI+)drug+ 1,070 24,000 4.2%drug- 18,930 456,000 3.9%



3- Overfitting of predictive modelsWhy does it happen in Big Data? In Big Data and Data Reuse, it seems easy to

discover “good” predictive models: data-driven approaches: without hypothesis Absence of Bonferroni’s correction Big sample sizes => high significance

(even if poor size effect) Overfitting of the model:

The models seems to be good in the sample: it is “optimized” for this particular sample

But it could be fortuitous: perhaps another model would have been discovered on another sample



3- Overfitting of predictive models Which solutions?

Those solutions should be applied together when possible: Apply Bonferroni’s correction Consider the effect size as a stop criterion in some

algorithmes Trade-off between the complexity of the model and the

goodness of fit of the model using criterion such as Akaike Information Criterion

Build the model and test it on 2 different databases



3- Overfitting of predictive models

Which solutions? Example of procedure for predictive models:

2013-08-22

Original database

Random sampling

Learning set Evaluation set

20%80%

Predictivemodel

Statisticalprocedure

Evaluation

(prediction error

computation)

Learning phase Evaluation phase

Final predictive

model

Statisticalprocedure

100%

Final learning



Data reuse of HERSome data sources…

Drug (distinguish drug prescription and drug administration). Don’t forget to add some sources:

Some procedures to map, e.g.:Scanner with iodine => “iodine”Surgery with anesthesia => “anesthetic drug”

Perfusion, that are often provided without terminology code The drugs the patient brings with him…

Medical devices (implanted or not): Raw data: hudge amount Aggregated data Results of the interpretation automatically done by the device



Data reuse of HER Terminologies

Vocabulary, support of semantic interoprability: Couples of codes and wordings Codes are sometimes included in a hierarchy Enable to associate each concept to a code Contrary to free-text, unambiguous and usable for

statistical analysis Example :

Érysipèle = érésipèle = dermo-hypodermite = A46 in ICD10



2013-08-22

Data reuse of HER Drug terminologies

Commercial name Ex : Dafalgan®

CID Common international denomination Ex : Paracétamol / Acetaminophen

Various terminology Among them, the ATC classification ATC = Anatomical Therapeutic and Chemical

classification Codes, wordings and hierarchy (principally based on the

therapeutic indication)



2013-08-22

Example of the Aspirin in the ATC classification

A alimentary tract and metabolism A01AD other agents for local treatment

A01AD05 …aspirin…B blood and blood forming organs

B01AC platelet aggregation inhibitors B01AC06 …aspirin…

C cardiovascular system C10BX (…) other combinations

C10BX01 & C10BX02 …aspirin…

M musculo-skeletal system M01BA anti-inflammatory

M01BA03 …aspirin…

N nervous system N02BA salicylic acid and derivatives

N02BA01, N02BA51, N02BA71 …aspirin…



2013-08-22

Laboratory results Biochemistry, hematology, bacteriology, virology, immunology Examples terminologies:

LOINC (recommended) IUPAC …

But most of the time, each laboratory produces its own terminology:

No semantic interoperability: Impossible to pool 2 hospital databases In the same hospital, data are sometimes heterogeneous according to the

date of the measure Need for producing ad hoc mappings



2013-08-22

IntroductionLe projet européen PSIP

WP 1

HIS

CPOE

ADE

Exploitable Data structure

WP 2Data and

Semantic Mining

WP 3Knowledge Elicitation

WP 4Knowledge Engineering

WP 5Design and

Development of CDSS modules

WP 6Validation of the CDSS modules

WP 7Contextualisation of the CDSS modules

WP 8

Connectivity and integration

WP 9Support for Healthcare

Professionnals

WP 10Prototype Patient Support Service

WP 11Usability

Engineering

WP 12Prospective impact

assessment

If antidepresssor…

If antibiotic & sun…

Give potassium

Knowledge BaseSystem

Kernel

Data Banks

CDSS 1

CDSS 2

CDSS 3

Cx-CDSS for Physicians

Cx-CDSS for Pharmacist

Cx-CDSS for nurses

Cx-CDSS for patients

CDSS 1

CDSS 2

CDSS 3

CDSS 1

CDSS 2

CDSS 3

HIS

EHR

CPOE

• User center design

• Usability Inspection

• Fake integration evaluation

• Real site evaluation

WP 13Dissemination &

Exploitation

Workshop Website Papers

Folios Conferences (milestones)

CHU Lille

CHU Lille

Oracle

Oracle IDEEA AUTH Region H CHU Rouen

MedasysIBM-AcureAAUUMIT

KITE



2013-08-22

Patient XXXAge 80Homme ? 0Décédé ? 0Durée séjour 9 (…)

DiagnosticsE119 DiabèteI251 AthéroscléroseI10 HTAN300 Cystite

NPU03230 Potassium

Médicaments administrés

Courriers et compte-rendus

1,1

Physicalfiles

1-1 Keys1-2 Patient1-3 Intensive care1-4 Medical units1-5 Dates, duration1-6 Misc.

1-Stays

1-1 Keys1-2 Patient1-3 Intensive care1-4 Medical units1-5 Dates, duration1-6 Misc.

1-Stays

2-1 Keys2-2 Medical units2-3 Misc.

2-Steps or the stay

2-1 Keys2-2 Medical units2-3 Misc.

2-Steps or the stay

5-1 Keys5-2 Drugs

5- Drug prescriptions

5-1 Keys5-2 Drugs

5- Drug prescriptions

6-1 Keys6-2 Lab results

6- Lab results

6-1 Keys6-2 Lab results

6- Lab results

7-1 Keys7-2 Reports

7- Reports

7-1 Keys7-2 Reports

7- Reports

3-1 Keys3-2 Diagnoses

3- Diagnoses

3-1 Keys3-2 Diagnoses

3- Diagnoses

4-1 Keys4-2 Medical procedures

4- Medical procedures

4-1 Keys4-2 Medical procedures

4- Medical procedures

8-1 Keys8-2 Codes

8- Semantic mining

8-1 Keys8-2 Codes

8- Semantic mining

1,1 1,1

Administered drugs

Détection rétrospective des EIMModèle de données



2013-08-22

Vue générale des règles d’EIM

Kind of Outcome # Rules Coagulation disorders Hemorrhage (detected by the administration of haemostatic) 7 Heparin overdose (activated partial thromboplastin time>1.23) 5 VKA overdose (INR>4.9 or administration of vitamin K) 59 Thrombopenia (count<75,000) 24 Other coagulation disorders 23 Ionic and renal disorders Hyperkalemia (K+>5.3 mmol/l) 63 Renal failure (creatinine>135 µmol/l or urea>8 mmol/l) 8 Other ionic disorders 4 Miscellaneous Anemia (Hb<10g/dl) 2 Bacterial infection (detected by the administration of antibiotic) 4 Diarrhea (detected by the administration of an anti-diarrheal) 2 Fungal infection (detected by the administration of an antifungal) 10 Hepatic cholestasis (alk. Phos.>240 UI/l or bilirubins>22 µmol/l) 3 Hepatic cytolysis (ala. trans.>110 UI/l or asp. trans.>110 UI/l) 4 Hypereosinophilia (eosinophilocytes>109/l) 4 High level of pancreatic enzymes (amylase>90 UI/l or lipase>90 UI/l) 7 Neutropenia (count<1,500/mm3) 2 Others 5 Total 236

83

Routine use of the “ADE Scorecards”, an application for automated ADE detection, in a general hospital.

2013-08-22

Nombre croissant de séjours analysés



2013-08-22

Faculté des règles à détecter rétrospectivement des EIM

Revue de cas experte : évaluation de la précision des règles

Cas validés / cas revusPrécision

Intervalle de confiance à 95%

Hyperkaliémie(K+>5.3mmol/l) 54 / 101 53.5% [43.7;63.2]

Surdosage en AVK(INR>4.9) 5 / 9 55.6% [23.1;88.0]

Insuffisance rénale(creat.>135 µmol/l or urée>16.6 mmol/l) 35 / 75 46.7% [35.4;58.0]

Autres types d’événements 14 / 53 26.4% [14.5;38.3]

TOTAL 108 / 238 45.4% [39.1;51.7]



introduction

Documents