Objective To verify the reliability and validity of a self-developed satisfaction evaluation questionnaire for outpatient department employees in public hospitals, and to provide suitable tools for conducting such surveys. Methods Two anonymous surveys were conducted on all employees of the Outpatient Department of West China Hospital of Sichuan University in July 2019 and November 2021, respectively. Questionnaire items were screened using methods such as item distribution, coefficient of variation, and decision value, and the reliability and validity of the questionnaire were evaluated using Spearman-Brown coefficient and Cronbach’s α coefficient, exploratory factor analysis, and confirmatory factor analysis. Results The final questionnaire retained 14 items, which could be divided into two dimensions: work conditions and interpersonal environment, and the overall fit index of structural equation model were as follows: χ2/ν=6.957, the standardized root mean square residual was 0.061, the root mean square error of approximation was 0.147, the goodness-of-fit index was 0.796, the adjusted goodness-of-fit index was 0.719, the normed fit index was 0.849, the relative fit index was 0.819, the incremental fit index was 0.868, the Tucker-Lewis Index was 0.841, and the comparative fit index was 0.867. The combined reliability of the two factors in the questionnaire was 0.94 and 0.91, respectively. The average variance extraction was 0.67 and 0.76, respectively, and the square root of the average variance extraction was 0.82 and 0.87, respectively, both of which were greater than the correlation coefficient of 0.71 between the two factors. The Spearman-Brown coefficient of the final questionnaire was 0.913, and the Cronbach’s α coefficients for the overall and two dimensions were 0.953, 0.937, and 0.910, respectively. Conclusion The reliability and validity of the satisfaction evaluation questionnaire for outpatient department employees in public hospitals are good and the questionnaire can be applied to practical surveys.
Measurement properties studies of patient-reported outcome measures (PROMs) aims to validate the measurement properties of PROMs. In the process of designing and statistical analysis of these measurement properties studies, bias will occur if there are any defects, which will affect the quality of PROMs. Therefore, the COSMIN (consensus-based standards for the selection of health measurement instruments) team has developed the COSMIN risk of bias (COSMIN-RoB) checklist to evaluate risk of bias of studies on measurement properties of PROMs. The checklist can be used to develop systematic reviews of PROMs measurement properties, and for PROMs developers, it can also be used to guide the research design in the measurement tool development process for reducing bias. At present, similar assessment tools are lacking in China. Therefore, this article aims to introduce the primary contents of COSMIN-RoB checklist and to interpret how to evaluate risk of bias of the internal structure studies of PROMs with examples.
ObjectiveTo translate the King’s Brief Interstitial Lung Disease (K-BILD) to Chinese, so as to provide an well reliability and validity assessment instrument for health status of patients with interstitial lung disease.MethodsBrislin’s transition model, six expert’s panel and pre-survey were used for initial Chinese version of K-BILD. Items analysis, exploratory factor analysis (EFA), confirmatory factor analysis (CFA), internal consistency reliability and test-retest reliability were used for validity and reliability test with 122 respondents.ResultsTen-item Chinese version of K-BILD were proved to have great psychometric qualities, two factors were extracted by EFA, which could explain 63.35% of the total variance. Furthermore, the CFA demonstrates the fit indices of two-factors mode: χ2/df=0.797, RMSEA=0.000, NFI=0.848, IFI=1.048, CFI=1.000, TLI=1.071. Cronbach’s α and Guttman Split-half were 0.893 and 0.861, respectively. Besides, the test-retest reliability of the scale was 0.805.ConclusionThe Chinese version of K-BILD scale has good validity and reliability, which is applicable for health status assessment in patient with interstitial lung disease.
ObjectiveTo compare and evaluate the discrimination, validity, and reliability of different data envelopment analysis (DEA) models for measuring the effectiveness of models by selecting different input and output indicators of the model.MethodsData from health statistical reports and pilot program of diagnosis-related groups of tertiary hospitals in Hubei Province from 2017 to 2018 were used to analyze the discrimination, content and structure validity, and reliability of the models. Six DEA models were established by enriching the details of input and output on the basis of the input and output indicators of the conventional DEA model of hospitals.ResultsFrom the view of discrimination, the results of all models were left-skewed, the cost-efficiency model had the lowest left-skewed degree (skewness coefficient: -0.14) and was the flattest (kurtosis coefficient: -1.02). From the view of structure validity, the results of the cost-efficiency model were positively correlated with total weights, outpatient visits, and inpatient visits (r=0.328, 0.329, 0.315; P<0.05). From the perspective of content validity, the interpretation of model was more consistent with theory of production after revision of input and output indicators. From the view of reliability, the cost efficiency model had the largest correlation coefficient between the data of 2017 and 2018 (r=0.880, P<0.05).ConclusionsAfter refining the input and output indicators of the DEA model, the discrimination, validity, and reliability of the model are higher, and the results are more reasonable. Using indicators such as discrimination, validity, and reliability can measure the effectiveness of the DEA model, and then optimize the model by selecting different input and output indicators.
Patient-reported outcome measures (PROM) measure attribute studies refer to studies conducted by investigators to validate the measurement attributes of PROM. The consensus-based standards for the selection of health measurement instruments (COSMIN), an international consensus standard for the selection of health measurement instruments, divides this attribute into three aspects: reliability, validity and responsiveness, and adds interpretability as an additional important feature for evaluating PROM. The purpose of this paper is to introduce the verification methods and principles of the three major measurement attributes in the COSMIN consensus, as well as the significance and direction of interpretability evaluation, and to provide international methodological experience and reference for the development of high-quality PROM psychometric attribute verification in China.
Objective To develop a behavioral assessment scale for medication management plans in women of childbearing age with epilepsy and to test its reliability and validity. Methods Based on the Theory of Planned Behavior, a pool of questionnaire items was initially drafted through literature review and focus group discussions. A two-round Delphi expert consultation was conducted with 15 experts to form a test version of the behavioral assessment scale for medication management plans in women of childbearing age with epilepsy (including 27 items and 5 dimensions). Convenience sampling was used to conduct surveys among women of childbearing age with epilepsy in some tertiary hospitals in Chuxiong, Shenzhen and Wuhan from February to May 2024 (the first time) and from June to October 2024 (the second time). ResultsThe effective recovery rates of the two rounds of questionnaires were 95.5% and 94.6%, respectively. The final scale included 24 items and 5 dimensions, with good reliability and validity: the content validity index (S-CVI) was 0.934, Cronbach's α coefficient was 0.876, split-half reliability was 0.819, and test-retest reliability was 0.901; exploratory factor analysis extracted 5 factors (cumulative variance explained rate 73.97%, item load 0.42~0.85), and confirmatory factor analysis showed that the model had good fit (χ2/df=1.849, RMSEA=0.075, CFI, GFI, AGFI, IFI, TLI all>0.85). Conclusion The scale meets the reliability and validity standards and can be used to assess the medication management plans and behaviors of women of childbearing age with epilepsy.
Objective We aimed to develop a self-management assessment scale for children with epilepsy and test its reliability and validity. Methods A research group was established, and the items were revised through literature review, group discussion and pre-investigation, and 280 patients with epilepsy in children were included, and the reliability and validity of the scale were tested. Results 28 items in 4 dimensions were developed to form the scale, namely, knowledge and belief of diseases and medication, compliance of medication and treatment, self-efficacy of medication and obstacles of medication. Confirmatory factor analysis extracted four common factors with characteristic roots greater than 1, and the cumulative variance explanation rate was 65.639%. The factor load of all items is > 0.5. The overall Cronbach’s alpha is 0.880, and the coefficients in seven measurement dimensions are all greater than 0.8. Conclusion The self-management assessment scale for children’s epilepsy drugs has good reliability and validity, and can provide a measuring tool for the drug management of children’s epilepsy diseases.
Objective To develop an evaluation tool for the screening of high risk population for oral complications in critically ill patients, which can be performed accurately and scientifically. Methods Basing on the related foreign oral assessment scale, combined with the method of brainstorming, expert consultation, method of clinical status and so on, the item pool of the assessment scale was determined. Five nursing experts and two oral experts assessed the content validity and 50 ICU nurses were tested. Then, the screening accuracy of the assessment scale was proved by application in 100 critically ill patients selected randomly. Results The Cronbach’s a coefficient of final version of the High Risk Assessment Scale for Oral Complications in Critically Ill Patients (including seven parts contents of oral health assessment and oral pH value test) was 0.815, the content validity index (Sr-CVI/Ave) was 0.932. The results of 50 nurses to the 91.2% assessment items of the assessment scale were very important and important. For screening related indicators of oral complications in high-risk patients, the sensitivity of the assessment scale was 97.53%, the specificity was 94.11%, the positive predictive value was 98.75%, the negative predictive value was 88.89%, and the crude agreement was 95%. Conclusion There are good reliability, validity and a high accuracy of screening test in the High Risk Assessment Scale for Oral Complications in Critically Ill Patients. It can be used for screening patients at high risk for oral complications in critically ill patients, and help clinical nurses to complete the oral health status of the critically ill patients quickly.
Evidence synthesis is the process of systematically gathering, analyzing, and integrating available research evidence. The quality of evidence synthesis depends on the quality of the original studies included. Validity assessment, also known as risk of bias assessment, is an essential method for assessing the quality of these original studies. Currently, there are numerous validity assessment tools available, but some of them lack a rigorous development process and evaluation. The application of inappropriate validity assessment tools to assessing the quality of the original studies during the evidence synthesis process may compromise the accuracy of study conclusions and mislead the clinical practice. To address this dilemma, the LATITUDES Network, a one-stop resource website for validity assessment tools, was established in September 2023, led by academics at the University of Bristol, U.K. This Network is dedicated to collecting, sorting and promoting validity assessment tools to improve the accuracy of original study validity assessments and increase the robustness and reliability of the results of evidence synthesis. This study introduces the background of the establishment of the LATITUDES Network, the included validity assessment tools, and the training resources for the use of validity assessment tools, in order to provide a reference for domestic scholars to learn more about the LATITUDES Network, to better use the appropriate validity assessment tools to conduct study quality assessments, and to provide references for the development of validity assessment tools.
ObjectiveTo evaluate the reliability and validity of the Quality of Working Life Scale (QWL7-32). MethodsThe QWL7-32 scale was used to survey 487 drilling workers. The presence of chronic diseases was regarded as an effector for evaluating physical health, and the result of SCL-90 measurement was regarded as an effector for evaluating psychological health. The reliability and validity of the scale were statistically analyzed. ResultsThe results of the Pearson correlation coefficient was 0.713, the Cronbach's alpha coefficient was 0.920, and the Splithalf reliability coefficient was 0.942. The result of confirmatory factor analysis showed that the construct validity of scale was good, and the accumulative rate of 7 variances was 62.59%. The results of correlation analysis and t test showed that the validity of scale criterion was also good. In QWL7-32 scale, each dimension showed a good correlation with its relevant item but poor correlation with any other items. ConclusionThe QWL7-32 has a good reliability and validity.