multiple methodologies, pools of qualified persons were identified
to participate in four separate advisory panels. Methods for identifying
and selecting participants assured that a wide and varied range
of abilities, disabilities, attitudes, philosophies, knowledge and
skills were reflected in the panel meetings. Each group included
a diverse array of 32 participants with expertise in the four areas
of disability: mobility, self-care, learning, and communication
limitations. Four draft instruments resulted, one from each group.
Each draft instrument was designed for use in a telephone or 'paper
pencil' survey that would be appropriate for general population
use, as well as applying to the full range of disability categories.
After review and critical assessment, a synthesis of the vital elements,
conceptualizations, and spirit of each draft was conducted, resulting
in a fifth or "next generation" survey that identified 25 key elements
of the environment.
forms of the draft instrument were proposed. Both had the same item
content, but two different metrics were used to assess environmental
impact. In one form, individuals were asked to indicate "how often"
a barrier is encountered using response categories of "daily, weekly,
monthly, less than monthly, and never". In the other form, individuals
were asked to assess the degree to which the environmental element
"facilitates or hinders" participation using response categories
of "big barrier, little barrier, no impact, little help, and big
help." The first form had the advantage of easier response categories,
while the second form had the advantage of identifying facilitators
as well as barriers.
forms were tested on a group of 97 people, 50 with disabilities
and 47 who indicated they did not have a disability. Results of
that pilot testing indicated:
The "frequency" response categories were strongly preferred by participants
over the "extent of barrier/facilitator" response categories.
The "frequency of barrier" response categories better differentiated
people with and without disabilities than the "extent of barrier/facilitator"
Correlations between the two response categories, while significant,
were relatively low.
of the results from the comparison of response categories with project
staff and representatives who attended advisory panel meetings,
yielded a consensus that all 25 items should be retained in the
draft instrument, but that a follow-up impact question needed to
be added since the correlation between frequency and impact was
not particularly high. These discussions led to adding a follow-up
question, "When this problem occurs, is it usually a big problem
or a little problem?" This question was added after each item where
the respondent indicated the frequency of the problem to be anything
other than never. The final draft instrument, the Craig Hospital
Inventory of Environmental Factors (CHIEF) was distributed to all
advisory panel members for review.
of the Environmental Instrument
- "CHIEF 400 Dataset"
sample of 409 individuals with disability was recruited for a validation
study to test the psychometric properties of the CHIEF (CHIEF 400
Dataset). In total, the sample included 124 participants with spinal
cord injury, 120 participants with traumatic brain injury and 165
participants with a wide variety of other disabilities. This included
55 persons with multiple sclerosis, 35 persons with amputations,
and others with auditory and visual impairments, developmental disabilities,
cerebral palsy and some with multiple impairments resulting in disability.
While the group of individuals with spinal cord injury was 80% male
with an average age of 41 and the group of individuals with traumatic
brain injury was 61% male with an average age of 41, the group of
individuals with a variety of other impairments was 62% female with
an average age of 48.
409 study participants were administered the CHIEF. In addition,
103 of the total 409 participants (46 with spinal cord injury, 44
with traumatic brain injury, and 13 with other impairments) were
interviewed using CHIEF a second time, approximately two weeks after
the first administration in order to assess test-retest reliability.
Finally, family members or friends of 125 subjects (46 with spinal
cord injury, 54 with traumatic brain injury, and 25 with other impairments),
not included in the test-reliability sub-study, were successfully
recruited and asked to complete the CHIEF as a proxy for the subject
in order to assess subject-proxy agreement.
of this data began by defining three methods of scoring each item:
A frequency score on a scale of 0-4 indicating the frequency with
which barriers were encountered (0=never, 1=less than monthly, 2=monthly,
3=weekly, and 4=daily).
A magnitude score on a scale of 0-2 indicating the size of the problem
which a barrier typically presented (0=no problem since the barrier
was never encountered, 1=a little problem, and 2=a big problem).
A frequency-magnitude product score on a scale of 0-8 calculated
as the product of the frequency score and the magnitude score, indicating
the overall impact of the barrier.
scores across the 25 items were calculated as the average frequency
score, the average magnitude score, and the average frequency-magnitude
product score across all of the non-missing scale items.
reliability of individual items and the total scale were calculated
using both the intra-class correlation coefficient (ICC) and the
percent of cases with exact agreement between both tests. Mean difference
scores between the test and retest were also calculated and significance
assessed. This process was repeated for frequency scores, magnitude
scores, and frequency-magnitude product scores. In general, the
product scores showed slightly higher reliability co-efficient and
they became the focus of additional psychometric analysis. These
data indicated a total scale score ICC reliability of .926, indicating
acceptable reliability for the instrument.
all disability sub-groups subject proxy interclass correlations
ranged from 0.494 to 0.618 with a total scale ICC of 0.618. These
data indicate that subject proxy agreement is marginal and result
in the recommendation that proxies not be asked to complete CHIEF
when subjects are unavailable to do so.
in response patterns were compared across impairment groupings in
an effort to determine if the instrument differentiated among impairment
groups in expected ways. The majority of items and sub-scales produced
statistically significant differences among the impairment groups.
For example, cases with traumatic brain injury scored dramatically
lower on physical barriers than the other groups. These data lend
support to the validity of CHIEF by indicating that the tool differentiates
scores among different disability groups in ways that are consistent
with the unique barriers faced by those groups.
Evaluation of the CHIEF
1999, the CHIEF was added to the Behavioral Risk Factor Surveillance
System (BRFSS) survey in Colorado in order to: 1) examine the underlying
dimensions that might exist within the context of the 25 items;
2) demonstrate its applicability to large-scale disability surveillance;
3) establish scoring norms; and 4) develop a CHIEF Short Form. The
survey was administered via telephone to 2,259 individuals. The
BRFSS data was weighted using the standard BRFSS weighting formula.
In addition, post-stratification weighting has been applied to account
for differences in age and gender between the sample and the population
of Colorado. All analyses have been performed using the weighted
data; therefore, the results can be generalized to the entire population
of Colorado, 18 years or older.
of CHIEF Subscales
analysis was used to identify underlying dimensions, or subscales,
within CHIEF. This analysis was performed on the 25 CHIEF items
with five factors accounting for 48% of the cumulative variance
across the 25 items. After varimax rotation, each item was assigned
to the factor with the highest positive loading. This resulted in
five factors with 3-7 items included in each factor. Descriptive
labels for the factors were assigned including "policy barriers",
"physical and structural barriers", "work and school barriers",
"attitude and support barriers", and "services and assistance barriers".
following indicates which items are contained in each subscale of
Subscale: Policies businesses, policies government, policies
employment/education & services community.
Subscale: Surroundings, natural environment, design home,
design community, design work/school, & technology.
Subscale: Attitudes work/school, help work/school &
Subscale: Attitudes home, discrimination, support community,
attitudes community & support home.
Subscale: Transportation, medical care, help home, information,
education/training, help community & personal equipment.
Differentiation Between Groups
items, subscales and total scores, the CHIEF was able to show differences
in reported frequency and magnitude of environmental barriers between
groups with a variety of impairments and activity limitations. Both
people with and without disabilities experienced environmental barriers.
However, those with disabilities reported an overall higher frequency
and magnitude of environmental barriers. Further, individuals with
traumatic brain injury reported greater barriers than those identified
as having a disability from the BRFSS data*, but fewer than individuals
with spinal cord injury. Individuals with other types of impairments
(i.e., multiple sclerosis, amputees, other auditory, visual and
multiple impairments, developmental disabilities, cerebral palsy)
reported the greatest barriers.
an examination of the mean scores for each CHIEF subscale and the
items on that subscale by disability status, the same general trend
is seen, however some items and subscales do vary by disability
status. This analysis confirms that the CHIEF has the ability to
differentiate between those with and without disability and between
different impairment groups.
as having an impairment or health problem that: limits ability to
work, learn, remember, concentrate, or any other activity or requires
special equipment or help
of the CHIEF Short Form (CHIEF-SF)
criteria were used to determine which items should be retained for
a "short form" version of the CHIEF. In general, these criteria
included items which: 1) had the highest frequency of barrier mean
scores; 2) had the highest magnitude of barrier mean scores; 3)
had the highest item score-subscale score correlations (using the
mean frequency-magnitude product score); 4) had the highest item
score-total score correlations (using the mean frequency-magnitude
product score); 5) were the most frequently reported barriers; and
6) best differentiated between people with and without disability.
In addition, taking all of the criteria into consideration, if an
item was to be excluded, but it was felt, conceptually should be
in the scale, it was retained. The following 12 items within the
original five subscales were retained:
Subscale: Policies businesses & policies government
Subscale: Surroundings & natural environment
Subscale: Attitudes work/school & help work/school
Subscale: Attitudes home & discrimination
Subscale: Transportation, medical care, help home &
CHIEF-SF retained the ability of the long form CHIEF to discriminate
among groups with differing disabilities (CHIEF 400 and BRFSS data).
has been an abbreviated description of the properties of the CHIEF.
For a more detailed description, please download the PDF file (CHIEF_Prop.pdf,
133K) that includes additional information on the development of the
CHIEF and the PDF file (CHIEF_Figs.pdf,
2416K and SF_Figs.pdf , 278K) that have
the supporting charts and figures.
(What is PDF and How do I
use PDF Files?)