Deriving START

START was derived using the Emergency Department Data collection registry in New South Wales, Australia. Adult patients (age > or = 16 years) were included if they presented to a Level 5 or 6 Emergency Department in New South Wales, Australia between 2013 and 2014.

The outcome of interest was in-patient admission from the Emergency Department. This included all admissions to short stay and medical assessment units and being transferred out to another hospital. Analyses were performed using logistic regression. Discrimination was assessed using area under curve and derived risk scores were plotted to assess calibration.

Over 1.7 million presentations from twenty three level 5 or 6 hospitals were analysed. Of these 49.38% were male and the mean (sd) age was 49.85 years (22.13). Level 6 hospitals accounted for 47.70% of cases and 40.74% of cases were classified as an in-patient admission based on their mode of separation. The final multivariable model including age, arrival by ambulance triage category previous admission and presenting problem had an AUC of 0.82 (95%CI 0.81, 0.82).

Table 2. Multivariable model of in-patient admission with risk score using derivation set Akiake Information Criterion (intercept only 2326760, intercept and covariates 1768771) Area under Receiver Operator curve for validation dataset 0.82 (95%CI 0.81, 0.82). Hosmer-Lemeshow test statistic p<0.001.

Variable	Coefficient	Odds ratio (95%CI)	P value	Risk score
Age
16-19 yrs	ref	na		0
20-39 yrs	0.19	1.21 (1.19,1.23)	<0.001	+1
40-59 yrs	0.61	1.85 (1.82, 1.88)	<0.001	+3
60-79yrs	1.20	3.31 (3.25, 3.37)	<0.001	+6
≥80 yrs	1.79	6.01 (5.89, 6.13)	<0.001	+9
Ambulance arrival	0.77	2.17 (2.15, 2.19)	<0.001	+4
Triage category
1	4.47	87.13 (82.15, 92.60)	<0.001	+24
2	2.99	19.84 (19.28, 20.32)	<0.001	+16
3	2.08	7.97 (7.80, 8.15)	<0.001	+11
4	1.10	3.00 (2.94, 3.07)	<0.001	+5
5	ref	na		0
Admission within 30 days	0.66	1.93 (1.90, 1.96)	<0.001	+3
Hour of presentation
0800-1759	0.21	1.23 (1.22, 1.25)	<0.001	+1
1800-2259	0.01	1.01 (1.00, 1.02)	0.06	0
2300-0759	Ref	na		0
Presenting problem
Abdominal, gastrointestinal	0.33	1.39 (1.37, 1.41)	<0.001	+2
Cardiovascular	-0.71	0.49 (0.46, 0.48)	<0.001	-3
General symptoms	ref	Na		0
Febrile illness	0.65	1.91 (1.87, 1.96)	<0.001	+3
Injury	-0.75	0.47 (0.46, 0.49)	<0.001	-4
Respiratory	0.01	1.01 (0.99,1.03)	0.19	0
Musculoskeletal	-0.57	0.56 (0.55, 0.57)	<0.001	-3
Neurological	-0.25	0.78 (0.77, 0.79)	<0.001	-1
Mental health	-0.32	0.72 (0.71, 0.74)	<0.001	-2
Toxicological	-0.30	0.74 (0.72, 0.77)	<0.001	-2
ENT/eye/head and neck	-1.17	0.31 (0.30, 0.32)	<0.001	-6
Administrative	-0.57	0.57 (0.55, 0.59)	<0.001	-3
Genitourinary	-0.16	0.85 (0.83, 0.87)	<0.001	-1
Social	0.19	1.21 (1.07, 1.38)	0.004	+1
Endocrine	-0.03	0.97 (0.91, 1.05)	0.26	0
Obstetrics, Gynaecology	-0.55	0.58 (0.56, 0.59)	<0.001	-3
Skin, allergy	-0.30	0.74 (0.72, 0.76)	<0.001	-2
Other medical	0.99	2.70 (2.40, 3.04)	<0.001	+5

Fig.1 Mean predicted probability of in-patient admission based on all possible risk score totals

Fig 2. Calibration curve of actual admission rate by predicted mean probability  - dots denoting each risk score category (total risk score >40, 30-40, 20-30, 10-20, 1-10, <1). Dotted line denotes perfect calibration

Understanding NSW Emergency Data

The Emergency Department Data Collection Registry routinely collects patient level data on presentations to all designated Emergency Departments in NSW. Data collection includes, referral source (self referred, General Practice, Specialist, Nursing Home), mode of arrival (self referral, Ambulance), hospital facility, presenting problem, mode of separation (admitted to hospital, discharged or died). Triage categories were defined by the Australasian triage scale (1=immediately life-threatening, 2=potentially life threatening, 3=urgent, 4=semi-urgent and 5=non urgent). Presenting problems allocated by triage nurses and ED diagnoses entered by clinicians were categorised into broad clinical groups (see table1). Full data definitions for the EDDC are located at http://www.cherel.org.au/data-dictionaries#section2

About START

The main driver of this project is to improve ED patient flow and clinical decision making. The translational aspects of this project can therefore be divided into two broad domains

1. Evaluation of the effectiveness of large scale health policy changes within NSW and Australia
1. Examining trends in National Emergency Access Target performance across specific hospital groups together with ED models of care, such as team based care, fast track units designed to address this health policy change.
2. Changes in presentation and representation characteristics for specific population groups that may be affected by specific policy initiatives such as falls prevention  programs, hospital in the home, sepsis, Indigenous health and community mental health programs
3. Changes in ED patient acuity and presentation characteristics as a result of recent Federal Government proposals to implement General Practice visit co-payments
   
   
2. Generation and implementation of online tools to improve prediction of ED disposition and length of stay
1. A random sample of data from 2014-15 the DESTINY study will be used to derive and validate a prediction model for ED disposition based on patient age, acuity, presenting problem and representations. Models will be developed using logistic regression and machine learning techniques. Previous studies using logistic regression and the same variables have achieved areas under ROC curves of around 0.84-0.88 which indicates clinical usefulness of these models[1]. DESTINY investigators have experience with such modelling techniques and can access expertise in artificial neural networks at The University of Sydney School of IT and Engineering.
2. The model will be converted into an online "app" and web interface that will be used by front line ED clinicians to predict the following clinically useful outcomes based on initial variables derived on patient arrival
1. Probability of admission or discharge from the emergency department
2. Probability of admission to observation unit
3. Probability of meeting NEAT
4. Probability of ED discharge after arriving by ambulance
   
   
3. The interface will be trialled at three hospitals within Sydney Local Health District using real time prospective validation and ethnography with triage nurses and ED clinicians with respect to facilitating the disposition and patient flow within ED. The interface may also be trialled within the NSW Ambulance Service to provide an evidence base for decisions regarding transfer to hospitals
4. The Interface will be evaluated to assess usability, accuracy and effectiveness in assisting triage of patients, reducing unnecessary ambulance arrivals and expediting patient flow within these institutions.
5. Lessons from the project will be formally presented to the ACI and if the project is evaluated as a success based on point (d), a process of formal implementation will occur across a number of sites within NSW Health.

 

Translational Logic Behind Start Project

Evaluation study protocol

Objectives

Investigate whether use of the START score improves ED performance and decreases length of stay in ED

Methods

Design - Randomised control trial with unit of randomisation being day of the week

Setting - The study will be conducted in the Emergency Departments of Royal Prince Alfred Hospital (around 75,000 ED presentations per year), Canterbury Hospital, and Concord Hospital (both around 40,000 ED presentations per year). All three hospitals are within the Sydney Local Health District.

Patient population - Eligible adult (age>16 years) patients will be consecutive patients presenting at these hospital Emergency Departments between 1000 and 1400. Exclude planned representations, immediately life-threatening presentations (trauma calls, stroke calls, LifeNet (acute myocardial infarction) and cardiac arrest calls), transfers from other hospitals, expected admissions and those brought in by police

Intervention - The START score will be scored by a designated trained investigator at the point of triage. The investigator will observe each nursing triage encounter and enter relevant data fields (see START risk scoring sheet) that is being routinely collected by the triage nurse.  The score will be calculated using a paper scoring checklist or mobile app calculator if available. Risk scores and probabilities were determined using a previous derivation and validation paper.

The recommendation based on the score will range from very likely admission, likely admission, uncertain, likely discharge and very likely discharge.

For patients allocated to the intervention group a copy of the paper START risk scoring checklist will be attached to the clinical notes and used by the Nurse Unit Manager or patient flow Navigator in ED together with the treating clinician to assist with disposition decisions.

These decisions may include allocation of in-patient or short stay unit beds, or streaming to fast track units.  The patient will not be interviewed, approached or contacted for any information and no specific patient information will be recorded on the scoring app.

Control - Presentations allocated to control group will receive standard management in ED by clinicians without the assistance of the risk score tool. The study investigator will still score the triage encounter using the risk tool but the results of the risk scoring will not be made known to the clinician or included in the clinical notes.

Group allocation - Consecutive days of the week will be randomly allocated using a computer generated number sequence in an opaque sealed envelope drawn at the time of triage (OR START OF DAY?) by the study investigator.

Consent - Triage nurses and Nurse Unit Managers in ED will be asked to read a study information sheet and consent during regular weekly nursing in-service sessions. As the information does not directly affect patient care and will not involve direct interaction with ED patients, patient consent will not be obtained. Patients will be informed with signage at the triage office that a study investigator will be observing the triage process and collecting information that is being entered by the triage nurses.

Primary Outcome - The primary outcomes are proportion of patients with length of stay less than four hours and total length of stay in ED. These are routinely collected using existing patient information systems (FirstNet, Cerner Millenium) and reported by the ED data manager Ms Sook Lee Chai.

Secondary Outcome - Disposition time, (time that admission ready or discharge ready icons were activated on patient information system) Representation within 3 days of initial presentations, did not wait and hospital length of stay. These are also routinely collected and reported by the ED data manager.

Other data variables collected

The following variables will be collected at time of patient departure from ED using the patient information system.

• Designation of first treating doctor
• First location of patient
• Admitting team
• Emergency short stay unit (EMU) admission
• Age and sex of patient

Statistical Analysis

The hypothesis will be that patients allocated to the intervention groups are associated with reduced length of stay in ED. Descriptive statistics will be used to compare proportions and means between groups and (multi-level modelling used to account for day of week randomisation)

Specified subgroups

Pre specified subgroups will be admitted or discharged patients

Sample Size calculation

Based on length of stay in the derivation study, the mean (sd) for admitted patients was 7.2(12) and 3.3(16) for discharges. A 2 hour decrease is considered a clinically meaningful and it is estimated that around 500 patients in each arm. A total of 1200 presentations will need to be analysed assuming exclusions are factored.

This will provide enough power to detect a 10% improvement in proportion of patients staying in ED less than four hours with a power of 0.80 and a two tailed alpha of 0.05. Assuming around 5 presentations are allocated and studied each hour at each site, an estimated six months of active recruitment is required (two days a week at four hours a day).

Study period and timeline

• Ethics application July-August 2016
• Implementation study at RPA September-November 2016 - this study will implement the risk score tool for all triage encounters to assess processes and identify potential problems. This will likely take around three months
• Education and implementation across Sydney Local Health District November-December 2016
• Run in period January 2017
• Trial period - February to August 2017
• Analysis September to December 2017
• Presentations and publications - 2018

Ethics

An application will be made to the NSW Population and Health Services Research Ethics Committee with RPA Ethics committee acting as the lead site for all three single site applications. The trial will also be submitted for registration in the ANZ Clinical Trials Registry

Data storage

All risk tool forms and data collection sheets will be stored in paper form in a locked cabinet with the offices of the ED Executive. Patient details contained on data collection sheets will not be used for data analysis.

Limitations

The study will be limited to patients presenting during the hours of 1000 to 1400 on weekdays. This is seen as a necessary step to ensure study protocols are met prior to wider implementation across all hours.

Recent Publications

DESTINY protocol description

Low acuity presentations

Ambulance presentations

Representations to ED

HEAT maps

INJURY

START Derivation paper