This is a prospective, multi-centre study leading to the identification of a validated,
reliable and defensible tool for assessing ultrasound-guided regional anaesthesia (UGRA)
image interpretation.
This study will have 5 Phases:
Phase 1 - nerve block identification:
A panel of academic experts in regional anaesthesia will review published evidence from UGRA
international guidelines. They will determine which UGRA nerve blocks are relevant to
anaesthetic practice in the UK, and list the anatomical and clinical knowledge that is
relevant to performing these nerve blocks.
Phase 2 - content and question generation:
A panel of experts in regional anaesthesia, recognised for their involvement in regional
anaesthesia societies and publications, will review the UGRA nerve blocks and the relevant
anatomical structures identified in Phase 1. The anatomical structures that must be
identified on ultrasound for the safe and effective performance of these blocks will be
agreed upon following panel discussion. Each structure will be categorised as either:
Essential
Desirable
Expert
Not relevant
This information will be collated by the Primary and Co-investigators to generate a pilot
test of UGRA interpretation content and questions.
Phase 3 - preliminary testing:
The pilot test will be administered by local investigators via a computer-based application
to 35-50 anaesthetic trainees and consultants, recruited by email and flyers sent to schools
of anaesthesia across the UK. The cohorts will include an equal number of trainees and
consultants of varying regional anaesthesia and clinical experience. The pilot test will
contain short video clips and images of ultrasound guided regional anaesthesia with questions
in a constructed response question (CRQ) format to identify relevant structures, anatomy and
clinical knowledge, which will require the participant to write free-text responses to
questions. It is important to recognise that some participants will be familiar with the CRQ
format while others might be more familiar with other forms of assessment question.
Participants in the pilot will therefore be provided with sample questions and answers prior
to the test to familiarise themselves with the question format.
"Face validity" of the test (i.e. whether it measures what it claims to) will be assessed by
asking participants to comment on ease of understanding and to identify terms that appear
ambiguous. Participants will be able to comment in a text box after each item in the
questionnaire. They will also be asked to comment on how they would improve the test and any
relevant anatomy or clinical knowledge missing from the questions. The format of the test
will be amended based on the comments.
A "Difficulty Index" for each item will be calculated by dividing the number of correct
responses by the total number of responses. A higher value suggests a question is easier to
answer. Any item with a difficulty index of < 50% will be considered too difficult to answer
by the respondents and will be considered for removal. This will be compared to the
information gathered from the panel of experts on test items (that is, whether the anatomical
structure is essential, desirable, expert or not relevant to the specific ultrasound guided
nerve block).
The pilot study will be used to identify how well the answers for each test item discriminate
between the participant's level of ultrasound experience ("construct validity"). Differences
in ultrasound interpretation skills between study groups will be analysed using statistical
modelling. Items that discriminate well will be retained and those that discriminate poorly
will be removed from the main test.
Reliability of the test will be improved by presenting the pilot test items in a random order
to minimise the effect of cognitive fatigue diminishing performance on subsequent items. The
average time taken for participants to complete each item in the pilot test will also be
recorded to guide the optimal final study test length. It is important to balance the maximum
number of test items whilst minimising the risk of cognitive fatigue and participant drop
out. While the pilot test will necessarily have more items than the final test, pilot test
participants will be asked to comment on the point at which they felt fatigued to guide this
balance. It will also guide understanding of whether the level of regional anaesthesia
experience affects fatiguability.
Phase 4 - testing for content validity:
To ensure all items in the assessment tool contain content that tests the domain the
investigators intend to test, following the pilot the amended test will be presented to the
panel of 5 experts in regional anaesthesia, who will be asked to rate the suitability of
ultrasound images and questions to performance of the block. This will be on a 4-point Likert
scale:
= definitely not suitable
= probably not suitable
= probably suitable
= definitely suitable The "content validity" (i.e. the extent to which a measure fairly
represents a particular domain) will be derived by dividing the number of experts who
rated the item as probably suitable and definitely suitable (3 and 4 on the scale) by
the total number of experts. Items found to be appropriate by a sufficiently high
percentage of participants will be selected.
Phase 5: psychometric main study testing The final assessment tool will be administered to an
equal number of anaesthetists of varying regional anaesthesia experience. Participants will
be recruited using emails and flyers sent to schools of anaesthesia across the UK. Each
participant will answer a series of demographic questions, including age, gender, handedness,
as well as level of training and practical experience in UGRA to account for inconsistencies
in clinical exposure at different stages of training.
The exam will consist of constructed response questions (CRQs). Participants will be provided
with sample questions and answers before the test to familiarise themselves with the question
format. The construct validity of the tool will be tested by comparing the scores of
anaesthetists with their level of experience in regional anaesthesia. The level of experience
is a proxy outcome measure for their proficiency in ultrasound-guided regional anaesthesia.
Differences in ultrasound interpretation skills between study groups will be analysed using
statistical modelling. At the end of the assessment, there will be a short questionnaire to
ascertain ease of understanding and suitability of ultrasound images to support the face
validity of the test.
The "internal reliability" (i.e. the consistency of results across items in a test) of the
responses will be assessed using statistical modelling. A split-test method will also be used
for each participant to compare the first half of their assessment with the second half to
ensure internal consistency of their item responses. The difficulty index calculated for
items in the pilot study will be used to ensure a balanced set of test items, in terms of
their difficulty, in the first and second halves of the test. All items will be made
mandatory. The test will be administered by local investigators via a web-based application.
It will have an automatic system of marking to minimise inter-marker differences. It will
also be manually checked by investigators who are blinded to the level and identity of the
participant.
The investigators will aim to recruit a minimum of 250 participants, each of whom will be
allocated a participant number to maintain anonymity.
The results will be analysed only when recruitment has ended.