Introduction

The Department of Health-Hospital Statistical Reports (DOH-HSRs) are reports on quality management, hospital operations, staffing patterns, and finances submitted annually by hospitals nationwide to the Department of Health-Health Facilities and Services Regulatory Bureau (HFSRB). Ensuring the full implementation of HSRs will help DOH monitor the performance of health facilities that are supposed to ensure good quality care, aligned with DOH requirement that the HSRs be submitted in compliance with annual licensing, and as a data platform for improving the service delivery network in the country. The study aims to assess the online Hospital Statistical Report (OHSR) in: health information resources, indicators, data management, and information products.

Methods

Initially, the plan was to use descriptive statistics to summarize key hospital performance indicators, as well as Pabon Lasso (PBL) and Data Envelopment Analysis (DEA) as measures of hospital efficiency. However, initial sampling of the submitted hospital statistical reports showed poor data quality ranging from poor compliance to the prescribed format with significant amount of missing submissions. Hence, analysis using Pabon Lasso (PBL) and Data Envelopment Analysis (DEA) were done as proof of concept instead. Additionally, business process assessment using lean six sigma tools, Supplies-Input-Process-Output-Customers (SIPOC) and Voice of Process (VOP), was employed to capture the process of acquiring information of the nine specific indicators in 6 health facilities in NCR. Thematic areas to define operational and policy gaps were described using the Ishikawa (Fishbone) Cause and Effect analysis and the 6M model -Man, Machine, Material, Method, Measurement and Milieu (Environment).

Results

Of 1195 registered hospitals in the country, 73% submitted their HSR on time and in proper format in 2015, the highest in 5 years. Due to the poor data quality, missing data and inconsistent submissions, the sample of records included for PBL and DEA analysis was reduced to less than 180. According to the demonstration analysis by Pabon Lasso, 31% of hospitals were placed in Zone 3 and thus considered “efficient”, while 54% were in Zone 1 and thus considered “inefficient” (n=118). According to DEA, 30.45% of hospitals were found to be technically efficient (n=82), 53.57% were efficient in terms of input effectiveness (n=28) and 19.51% were efficient in terms of service effectiveness (n=41).

The process outcomes of achieving submission compliance and data validation were investigated in six hospitals in NCR. The business process context of the generation of the nine (9) indicators for the annual hospital statistical report was assessed to determine the “as is” hospital state of operations conducted at six (6) characteristically different hospital profiles. The SIPOC process mapping and VOP findings across the hospitals included both policy and operational gaps such as different indicator formula definitions used and process variation of assigned process owners of the OHSR and data management practices.

On the regulator side, the process of OHSR submission monitoring and data validation were not yet strictly enforced in full effect based on both policy and operational gaps found during the assessment. Regulatory compliance of the OHSR was hampered by the lack of direct participation of the licensing offices.

Discussion

Fluctuations in proper HSR submission compliance rate coincided with and can be attributed to changes in the submission form and change in the way of transmission (from paper to online format). Interpretation of PBL and DEA became difficult to attain because of diminished sample size and poor data quality, which emphasized the importance of obtaining complete and accurate data for analysis, before further re-analysis and interpretation.

The feedback from the process mapping sessions conducted in both the six (6) hospitals and in HFSRB were analyzed thematically using the 6M Ishikawa Framework. The more salient problem areas that were highlighted were of the “Measurement” and “Method” themes.

For the “Measurement” theme, results that showed varying indicator definitions used in the different hospitals casted doubt on the validity of the whole OHSR dataset since no valid conclusion can be made from the inconsistent data. To address problems under this theme, the researchers recommended the use and implementation of a standard quick reference manual that includes uniform indicator definitions and prescribed formulas to be used.

For the latter theme, accountability was a recurring point of discussion in the investigation of causes for the low rate of compliance. With process owners differing from hospital to hospital, and process ownership being assumed only on an “as and when” basis, OHSR compliance tends to fall behind other more urgent concerns at the hospitals. Also, a salient finding under “Method” theme is the lack of a standard procedure in handling OHSR submission non-compliance, which has a negative implication on the incentive of hospitals to comply with OHSR submission. For this, the researchers did an additional memo trail exercise to look into the history of the logic and actual implementation of the OHSR documented in various Department Circulars, Department Memos and Administrative Orders. It was found that unclear implementing rules and regulations, particularly on delineation of roles as to OHSR submission monitoring and handling of OHSR non-submissions should be looked into and rectified as a first step in an organizational re-alignment effort moving forward.

Conclusion

DOH-HFSRB should engage in quality improvement efforts in data collection and validation before utilizing analyses such as Pabon Lasso and DEA as health efficiency measures. Strategic alignment of roles and responsibilities with HFSRB must be done to address the gaps in effective implementation of the OHSR. This is done by way of documented delineation of responsibilities, work schedules as well as performance management mechanism for those to be held accountable for the implementation of the OHSR. Next, HFSRB must clearly communicate its specifications for the data to be gathered. In particular, standardization by provision of standard process guidelines including uniform indicator definitions and enforcement of a defined OHSR process owner that will serve as liaison must be effected to foster data quality, accountability and compliance. Reorientation of hospitals following provision of the standard process is also critical.

Process efficiency should be looked into by way of a deeper dive analysis of process behavior. This is typically done by actually observing the process as it happens. This way, Time and Motion (T&M) analysis could be applied to determine the optimal staffing requirements given the context of current HFSRB operations. Furthermore, providing definitions for levels of compliance as well as corresponding penalties shall be done to ensure strict reinforcement. Quality improvement studies are again recommended following process standardization to gauge outcomes and better control. After improvements in data quality, efficiency measures such as Pabon Lasso and DEA may be used to identify inefficient facilities and employ efficiency interventions. Finally, effectiveness studies and assessment of compliance to standard of care are also recommended to ensure both efficiency and effectiveness of health facilities.

Executive Summary

Policy Brief

Full Infographic

For more information on the study, email us at contact@epimetrics.com.ph

John Q. Wong, MD, MSc

Principal Investigator

Johnny Raymund Tamon, MD-MBA

Co-Investigator

Nico Angelo Rabang

Research Associate

Jean Alexis Go

Research Assistant

Christine Ingrid Espinosa

Research Assistant

Abigail Lim

Qualitative Specialist

Krizelle Cleo Fowler

Statistician