Skip to main content

The AHRQ National Quality Measures Clearinghouse (NQMC, qualitymeasures.ahrq.gov) Web site will not be available after July 16, 2018 because federal funding
through AHRQ will no longer be available to support the NQMC as of that date. For additional information, read our full announcement.
  • Measure Summary
  • NQMC:010068
  • Feb 2015
  • NQF-Endorsed Measure

Patient experiences of psychiatric care: percent of patients who responded positively to the "Facility Environment" domain on the Inpatient Consumer Survey (ICS).

 National Quality Measures Clearinghouse (NQMC) measure submission: patient experience of psychiatric care as measured by the Inpatient Consumer Survey (ICS).  2015 Feb 20.  62 p. 

This is the current release of the measure.

Primary Measure Domain

Clinical Quality Measures: Patient Experience

Secondary Measure Domain

Does not apply to this measure

Description

This measure is used to assess the percent of patients at discharge or at annual review who responded positively to the "Hospital Environment" domain on the Inpatient Consumer Survey (ICS).

The individual questions of the survey that pertain to the "Hospital Environment" domain are:

During my hospital stay...:

  • Q19. I surroundings and atmosphere at the hospital helped me get better
  • Q20. I felt I had enough privacy in the hospital
  • Q21. I felt safe while in the hospital
  • Q22. The hospital environment was clean and comfortable

Note: The "Facility Environment" domain is one of six domains contained in the Inpatient Consumer Survey (ICS).

Scoring the domains: Each question is evaluated on a scale from strongly disagree to strongly agree. For purposes of computing averages, a number value is given to the qualities of the scale from 1 for strongly disagree to 5 for strongly agree. A patient must respond to a minimum of 2 questions in order for an average rating to be computed for the domain. Since there are only 3 to 4 questions in a domain, missing values are not inserted when a patient does not answer a question. When the average rating for the questions in the domain is greater than 3.5, the patient is considered to have "responded positively" to the domain. Then the proportion of patients who responded positively to the domain is determined as the ratio of the number of patients who responded positively to all patients who responded to the domain.

Rationale

The Inpatient Consumer Survey (ICS) includes 28 survey items that draw out the voice of the mental health patients in multiple dimensions of care, including being treated with dignity, patient rights, patient participation in treatment planning, medication, interactions with key hospital staff, and personal perception of outcomes. Direct feedback from patients in areas such as dignity and rights could inform the therapeutic interactions between staffs and patients which will ultimately produce better quality of life outcomes. These intermediate outcomes may foretell the likelihood that the patient may return to receive further services when needed and participate in services in other venues. Direct feedback on outcomes and participation in treatment could inform the treatment activities and greater involvement of patients in their own recovery. Each domain is comprised of distinct questions to enable the health care provider to target specific areas of performance.

Ultimately, outcome must be evaluated from the perspective of the person who receives services from the healthcare organization. The provision of services should be in an environment conducive to patients' feeling safe and enabling patients to focus on recovering from their illness. Patients' report of the effect of the environment on their care is an important dimension of the quality of care of the organization.

Evidence for Rationale

 National Quality Measures Clearinghouse (NQMC) measure submission: patient experience of psychiatric care as measured by the Inpatient Consumer Survey (ICS).  2015 Feb 20.  62 p. 

Primary Health Components

Patient experience of psychiatric care; hospital environment

Denominator Description

Number of patients completing at least 2 items in the "Hospital Environment" domain on the Inpatient Consumer Survey (ICS) (see the related "Denominator Inclusions/Exclusions" field)

Numerator Description

Number of patients who respond positively to the "Hospital Environment" domain (see the related "Numerator Inclusions/Exclusions" field)

Type of Evidence Supporting the Criterion of Quality for the Measure

  • A formal consensus procedure, involving experts in relevant clinical, methodological, public health and organizational sciences
  • Focus groups
  • One or more research studies published in a National Library of Medicine (NLM) indexed, peer-reviewed journal

Additional Information Supporting Need for the Measure

Unspecified

Extent of Measure Testing

Methods
To psychometrically test the structure of the Inpatient Consumer Survey (ICS), 34,878 survey results, submitted by 90 psychiatric hospitals in 2008, were extracted from the Behavioral Healthcare Performance Measurement System (BHPMS). Basic descriptive item-response and correlation analyses were performed for total surveys. Two datasets were randomly created for analysis. A random sample of 8,229 survey results was used for exploratory factor analysis (EFA). Another random sample of 8,261 consumer survey results was used for confirmatory factor analysis (CFA). This same sample was used to perform validity and reliability analyses.

Results
The item-response analysis showed that the mean range for a disagree/agree five-point scale was 3.10 to 3.94. Correlation analysis showed a strong relationship between items. Six domains (dignity, rights, environment, empowerment, participation, and outcome) with internal reliabilities between good to moderate (0.87 to 0.73) were shown to be related to overall care satisfaction. Overall reliability for the instrument was excellent (0.94). Results from CFA provided support for the domains structure of the ICS proposed through EFA.

Conclusion
The overall findings from this study provide evidence that the ICS is a reliable measure of consumer satisfaction in psychiatric inpatient settings. The analysis has shown the ICS to provide valid and reliable results and to focus on the specific concerns of consumers of psychiatric inpatient care. Scores by item indicate that opportunity for improvement exists across healthcare organizations.

Refer to Psychometric Evaluation of an Inpatient Consumer Survey Measuring Satisfaction with Psychiatric Care for further details.

Evidence for Extent of Measure Testing

 National Quality Measures Clearinghouse (NQMC) measure submission: patient experience of psychiatric care as measured by the Inpatient Consumer Survey (ICS).  2015 Feb 20.  62 p. 

Ortiz G, Schacht L. Psychometric evaluation of an inpatient consumer survey measuring satisfaction with psychiatric care. Patient. 2012;5(3):163-73. PubMed External Web Site Policy

State of Use

Current routine use

Current Use

Accreditation

Collaborative inter-organizational quality improvement

Internal quality improvement

Monitoring and planning

National reporting

Measurement Setting

Behavioral Health Care

Hospital Inpatient

Professionals Involved in Delivery of Health Services

Does not apply to this measure (e.g., measure is not provider specific)

Least Aggregated Level of Services Delivery Addressed

Single Health Care Delivery or Public Health Organizations

Statement of Acceptable Minimum Sample Size

Does not apply to this measure

Target Population Age

Age greater than or equal to 13 years

Target Population Gender

Either male or female

National Quality Strategy Aim

Better Care

National Quality Strategy Priority

Person- and Family-centered Care
Prevention and Treatment of Leading Causes of Mortality

IOM Care Need

Getting Better

Living with Illness

IOM Domain

Effectiveness

Patient-centeredness

Case Finding Period

At time of discharge or during the patient annual review

Denominator Sampling Frame

Patients associated with provider

Denominator (Index) Event or Characteristic

Institutionalization

Denominator Time Window

Does not apply to this measure

Denominator Inclusions/Exclusions

Inclusions
Number of patients completing at least 2 items in the "Hospital Environment" domain on the Inpatient Consumer Survey (ICS)

Included populations: Patients who were discharged or had an annual review during the period and completed at least 2 questions in the "Hospital Environment" domain

Exclusions
None

Exclusions/Exceptions

Does not apply to this measure

Numerator Inclusions/Exclusions

Inclusions
Number of patients who respond positively to the "Hospital Environment" domain

Included populations: Patients who are discharged or have an annual review during the period, complete at least 2 questions in the "Hospital Environment" domain, and average a positive rating for those questions

Note: When the average rating for the questions in the domain is greater than 3.5, the patient is considered to have "responded positively" to the domain.

Exclusions
None

Numerator Search Strategy

Fixed time period or point in time

Data Source

Patient/Individual survey

Type of Health State

Does not apply to this measure

Instruments Used and/or Associated with the Measure

Inpatient Consumer Survey©

Measure Specifies Disaggregation

Does not apply to this measure

Scoring

Composite/Scale

Rate/Proportion

Interpretation of Score

Desired value is a higher score

Allowance for Patient or Population Factors

Analysis by subgroup (stratification by individual factors, geographic factors, etc.)

Description of Allowance for Patient or Population Factors

Stratification reports provide separate performance measure rates for each patient group to assist hospitals with utilizing comparison groups based on patients with similar characteristics at other hospitals.

The stratification reports provide graphs of monthly data points for 6 age stratifications and 2 forensic stratifications for each domain measure chosen by a hospital. The comparison data provide only the national weighted average; no confidence intervals are provided in these reports.

Standard of Comparison

External comparison of time trends

Internal time comparison

Original Title

Patient perception of the hospital environment.

Measure Collection Name

Inpatient Consumer Survey (ICS)

Submitter

National Association of State Mental Health Program Directors Research Institute, Inc. - Nonprofit Research Organization

Developer

National Association of State Mental Health Program Directors Research Institute, Inc. - Nonprofit Research Organization

Funding Source(s)

National Association of State Mental Health Program Directors Research Institute, Inc. (NRI)

Composition of the Group that Developed the Measure

National Association of State Mental Health Program Directors Research Institute, Inc. (NRI) staff invited consumers and the Mental Health Statistics Improvement Program (MHSIP) Policy Group to assist NRI in formulating an Inpatient version of the MHSIP Consumer Survey. The workgroup was formed consisting of a representative from these two groups, a research consultant, and NRI-BHPMS staff. The outcome of a series of meetings was an instrument consisting of 43 total items organized around six conceptual domains and a plan for implementation and analysis.

An original workgroup that created a 43-item pilot survey and signed-off on the original 28-item survey included the following external consultants:

  1. Cindy Hopkins (Consumer group/MHSIP, Texas)
  2. Mary Smith (MHSIP Policy Group, Illinois)
  3. Jack Wackowitz (Contractor NRI, Colorado)

An assessment workgroup created by NRI that modified and finalized the 28-item survey included:

  1. Vijay Ganju (Texas)
  2. Doug Hancock (Texas)
  3. Tom Muller (Georgia)
  4. Randy Koch (Virginia)
  5. Mary Smith (Illinois)
  6. Jack Wackowitz (Contractor NRI, Colorado)
  7. Huyi Hines (Contractor NRI, University of Kentucky-BHPMS Staff)
  8. Robert Littrell (Contractor NRI, University of Kentucky-BHPMS Staff)
  9. Lucille Schacht (NRI)
  10. Ted Lutterman (NRI)

Financial Disclosures/Other Potential Conflicts of Interest

No potential conflicts of interest need to be disclosed

Endorser

National Quality Forum

NQF Number

0726

Date of Endorsement

2015 Jan 7

Adaptation

This measure was not adapted from another source.

Date of Most Current Version in NQMC

2015 Feb

Measure Maintenance

The developer reviews/updates the measure on an annual basis

Date of Next Anticipated Revision

April 2016

Measure Status

This is the current release of the measure.

Source(s)

 National Quality Measures Clearinghouse (NQMC) measure submission: patient experience of psychiatric care as measured by the Inpatient Consumer Survey (ICS).  2015 Feb 20.  62 p. 

Measure Availability

Source not available electronically.

For more information, contact the National Association of State Mental Health Program Directors Research Institute, Inc. (NRI) at 3141 Fairview Park Drive, Suite 650, Falls Church, VA 22042; Phone: 703-738-8160; Fax: 703-738-8185; E-mail: NRI@nri-inc.org; Web site: www.nri-inc.org External Web Site Policy.

NQMC Status

This NQMC summary was completed by ECRI Institute on August 3, 2015. The information was verified by the measure developer on September 24, 2015.

Copyright Statement

This NQMC summary is based on the original measure, which is subject to the measure developer's copyright restrictions.

The Patient Experience of Psychiatric Care as Measured by the Inpatient Consumer Survey (ICS)© is a proprietary survey tool, copyrighted to the National Association of State Mental Health Program Directors Research Institute, Inc. (NRI), but NRI allows individual psychiatric hospitals to use the survey under a Terms of Use Agreement External Web Site Policy. For additional information, contact NRI at 703-738-8160 or NRI@nri-inc.org.

NQMC Disclaimer

The National Quality Measures Clearinghouse™ (NQMC) does not develop, produce, approve, or endorse the measures represented on this site.

All measures summarized by NQMC and hosted on our site are produced under the auspices of medical specialty societies, relevant professional associations, public and private organizations, other government agencies, health care organizations or plans, individuals, and similar entities.

Measures represented on the NQMC Web site are submitted by measure developers, and are screened solely to determine that they meet the NQMC Inclusion Criteria.

NQMC, AHRQ, and its contractor ECRI Institute make no warranties concerning the content or its reliability and/or validity of the quality measures and related materials represented on this site. Moreover, the views and opinions of developers or authors of measures represented on this site do not necessarily state or reflect those of NQMC, AHRQ, or its contractor, ECRI Institute, and inclusion or hosting of measures in NQMC may not be used for advertising or commercial endorsement purposes.

Readers with questions regarding measure content are directed to contact the measure developer.

About NQMC Measure Summaries

NQMC provides structured summaries containing information about measures and their development.

Measure Summary FAQs


Measure Summaries

New This Week

View more and sign up for our Newsletter

Get Adobe Reader