Background: Pharmacy interventions are a subset of public health interventions and its research is usually performed within the scope of a trial. The economic evaluation of pharmacy interventions requires certain considerations which have some similarities to those of public health interventions and to economic evaluations alongside trials. The objective of this research is to perform an overview of systematic reviews of economic evaluations of pharmacy services and triangulate results with recommendations for economic evaluations of both public health interventions and alongside trials. Methods: (
Keywords: Overview; Umbrella review; Systematic review; Economic evaluation; Cost-effectiveness; Community pharmacy; Pharmacy interventions; Pharmacy services; Public health interventions; Methods
Pharmacy-based public health interventions can be defined as complex health interventions, provided by pharmacists to patients in the community pharmacy setting, which are beyond, but do not necessarily exclude, the medication supply role.
These interventions include health promotion and support on self-monitoring, disease prevention, screening, and disease and medication management and cover a wide spectrum of areas, including the main public health areas of interest as defined by the National Institute for Health and Care Excellence (NICE): cardiovascular disease; chronic illness; diabetes; drugs; mental health; obesity; physical activity; screening; sexual health; smoking and tobacco; and vaccine preventable diseases [[
Complex health interventions require several interacting components, including behavioral changes from providers and patients, factors influencing multiple levels, and some degree of flexibility of interventions [[
The strategies used to operate pharmacy-based complex health interventions at multiple levels seem to be consistent with the diffusion of innovation theory [[
Financing prevention interventions requires economic evaluations which should be a routine and consistent part of all public health interventions [[
In fact, the original definition of pharmacoeconomics clearly identified the description and analysis of the costs and consequences of pharmacy services as a domain of pharmacoeconomics [[
In addition, there are also methodological recommendations for conducting economic evaluations alongside clinical or pragmatic trials, which may be useful to explore [[
The first economic evaluations of clinical pharmacy services, a subset of public health interventions provided by pharmacists to optimize medication therapy and promote health, wellness, and disease prevention, were published in the late 1970s. As the profession progressed beyond dispensing, an increasing number of evaluations were published [[
A landmark economic study demonstrated the important healthcare savings of medication management services provided by pharmacists [[
Hence, the economic evaluation of such services in collaborative care environments may contribute to informing decisions made regarding the successful expansion, continuation, or justification of such services [[
The overall aim of this research is to perform an overview of systematic reviews of economic evaluations of pharmacy services to better understand their successes and downsides and to assist in improving the methods for future research, following the early work of Schumock [[
Since the term "pharmacy services" is more frequently used by researchers in pharmacy practice and the term "public health interventions" is, however, used more frequently in the context of health technology assessment and economic evaluation, we will use the terms pharmacy-based public health interventions and pharmacy services interchangeably.
There are already a few systematic reviews on the economic evaluation of pharmacy services, which support the rationale to conduct this overview of reviews. We found two overviews of reviews on pharmacy services [[
This overview is, to the best of our knowledge, the first centered on methods of economic evaluations exclusively in the pharmacy setting and triangulating results with recommendations for economic evaluations of public health interventions and for economic evaluations alongside trials.
The objectives of this paper are as follows: to review the methods and issues in systematic reviews of economic evaluations of pharmacy services compared with usual care or other alternatives in multiple population groups; to compare these with recommendations on the economic evaluation of public health interventions and conducted alongside clinical and pragmatic trials; to propose a system for the measurement and valuation of costs and health effects feasible for the economic evaluation studies of pharmacy services; and to contribute to the methods in the economic evaluation of pharmacy services and of public health interventions.
We selected four policy documents that addressed the specifics and issued recommendations on the economic evaluation of public health interventions: Kelly et al. [[
This was a selected search on existing recommendations on the economic evaluation of public health interventions with the purpose of obtaining key insights for triangulation. This selected search was performed on MEDLINE® (via PubMed) in titles/abstracts till July 2017. A first search used the following terms in titles/abstracts: "review" and "economic evaluation" and "public health interventions" and "methods" which returned 15 titles. After reviewing title and abstract, we included five relevant articles [[
We selected three academic publications that issued specific recommendations for conducting economic evaluations alongside clinical or pragmatic trials: O'Sullivan et al. [[
This was a selected search on existing recommendations on the economic evaluation alongside clinical or pragmatic trials with the purpose of obtaining key insights for triangulation. This selected search was first performed on MEDLINE® (via PubMed) till July 2017 using the term "economic evaluation" in title/abstract and the following terms in title: "alongside" and ("clinical trials" or "randomized controlled trials"). This first search returned 14 titles of which we just selected the two most recent relevant titles available for free [[
This overview considered recommendations by Cochrane, by Smith et al., and by the Joanna Briggs Institute (JBI) [[
We have followed PRISMA Checklist in reporting this overview. See supplementary completed PRISMA Checklist, Additional file 1.
The protocol of this overview was registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 7 January 2016. Reference number PROSPERO 2016: CRD42016032768 (
Reviews were included if they met the following inclusion criteria: systematic review as per stated by authors or implied by methods; describing or containing individual economic evaluation studies (economic evaluation and/or equivalent terms "economic" OR "economic evaluation" OR "cost-effectiveness" OR "cost-utility" OR "cost-benefit" explicitly stated in search terms of systematic review); describing pharmacist-provided patient care interventions as per protocol definition published in PROSPERO (complex public health interventions, in health promotion, disease prevention, and disease/medication management, provided by pharmacists to patients in the community pharmacy setting, with the aim of preventing disease, promoting health, and prolonging life, which are beyond, but not necessarily excluding, the medication supply role); and describing or containing at least one economic study in community pharmacy setting.
No restriction on the types of populations, comparators, nor outcomes was considered, as stated in protocol, as this was an inclusive review to critique methods and aid future research design.
A comprehensive search was performed in the following databases through July 2017: MEDLINE®In-Process & Other Non-Indexed Citations, MEDLINE® (from 1946) and EMBASE (from 1980), via the OVID SP interface; Cochrane Database of Systematic Reviews (CDSR); Database of Abstracts of Reviews of Effects (DARE), NHS Economic Evaluation Database (NHS EED) and HTA, via the CRD database; Tufts CEA Registry; and Web of Science to identify existing systematic reviews of economic evaluation studies of pharmacy services.
In addition, a further search was performed in Google Scholar and snowballing from reference lists of retrieved reviews. An additional search for gray literature was performed in the ISPOR Database and OpenGrey.
Finally, a search for ongoing systematic reviews was performed in PROSPERO.
Searches used the following terms: "systematic review" and "economic evaluation" (or "cost-effectiveness" or "cost-utility" or "cost-benefit") and "pharmacy" (or "pharmacist") and "intervention" (or "service" or "program" or "management") with slight changes, according to the conventions of each database, in combination with database-specific filters for systematic reviews and economic evaluations.
See supplementary detailed search strategies, Additional file 2.
Citations that resulted from searches were downloaded, and duplicates were removed. Two researchers (SC and MC) reviewed all potentially relevant titles against the inclusion criteria and reviewed the abstracts associated with retrieved titles. Finally, full-text articles of retrieved abstracts were reviewed for eligibility by two review team members (SC and CM). Disagreements were resolved through discussion with additional review team members (CM and DKH). We recorded the reasons for exclusion at screening.
See supplementary list of excluded reviews, Additional file 3.
We have followed the PRISMA flowchart in reporting the study selection [[
A template adapted from the JBI Data Extraction Form for Reviews of Systematic Reviews [[
Extracted data (SC) of eight randomly selected reviews were audited by two other review team members (CM and DKH). Existing discrepancies were resolved through discussion. None of the review team members was blind to journal titles or to study authors or institutions.
Since primary studies are often included in more than one review, the degree of overlap was determined using the corrected covered area (CCA) method of Pieper et al. A CCA value lower than 5% is a slight overlap [[
The following items were extracted: title; first author/year of publication; journal; objectives; no. and sources searched; date range of reviews; no. of included economic evaluation community pharmacy (CP) studies/total no. of studies; countries of origin of CP studies; populations in CP studies; interventions in CP studies; comparators in CP studies; outcomes in CP studies; study designs in CP studies; types of economic evaluation in CP studies/total; cost and resource use categories in CP studies; cost year/discount rates in CP studies; data sources in CP studies; perspectives in CP studies; key findings in CP studies; uncertainty in included studies; assessment of quality of evidence for deriving effectiveness; assessment of risk of bias; assessment of economic quality; method of data synthesis; key findings of SR; significance/direction; assessment of heterogeneity; process indicators; equity considerations; sustainability of interventions; context; source of funding; conflict of interest; methodological challenges identified by authors; comments.
The list of references in the primary studies in the community pharmacy setting included in the systematic review was also recorded in this extraction form. Primary studies were not consulted. We extracted data of primary community pharmacy studies provided data were reported in the systematic review in accordance with guidelines on overviews.
A threefold critical appraisal of the methodology was performed as per protocol and further detailed.
We used the 16-item Assessment of Multiple Systematic Reviews (AMSTAR 2) questionnaire to measure the general methodological quality of each included systematic review [[
We assessed the following: the overall quality of evidence for deriving effectiveness reported in systematic reviews assisted by the Cochrane guidelines "What study designs should be included in an EPOC review" [[
Quality assessment of primary studies relies on reported information in included reviews.
All systematic reviews were then collectively assessed in terms of applicability and transferability of interventions, as described by Mossialos et al. [[
Quality assessment was performed by the lead reviewer (SC). Quality of all included reviews, applicability, and transferability were reviewed by another review team member (MC). Quality of evidence in eight randomly selected included reviews was reviewed by two other review team members (CM and DKH). Disagreements were resolved through discussion.
Information on study characteristics from systematic reviews was drawn from the data extraction form and reported in evidence summary tables. A narrative synthesis was performed, but no additional statistical analysis was performed. Whenever results described multiple settings, we limited ourselves to the subset in the pharmacy setting.
Based on the findings, we proposed a methodological approach for the measurement and valuation of costs and health effects and for the types of analysis for economic evaluations of pharmacy interventions.
First, a systematic review or, at least, an in-depth review of the best available evidence of the effectiveness and cost-effectiveness should be performed prior to the economic evaluation [[
First, most authors consider that such economic evaluations should be based on well-designed pragmatic trials with fewer strict protocols [[
Electronic searches until July 2017 identified 761 potential citations, of which 45 duplicates were found and removed, leaving 716 potential titles. The initial title screening excluded 594 (588 not matching inclusion criteria, 4 duplicates, and 2 protocols) titles, leaving a total of 122 potentially relevant titles. Abstract assessment resulted in the further exclusion of 100 studies and 22 potentially relevant abstracts were retrieved. The full-text assessment process resulted in 14 articles being retrieved.
OpenGrey and ISPOR Databases returned 1 and 77 potential titles, respectively, but all were excluded after abstract assessment. Hence, no references were obtained from gray literature.
Snowballing did not identify further reviews. A total of 14 reviews were included [[
PROSPERO registry returned 28 potential titles of ongoing reviews which were excluded after abstract assessment.
Figure 1 illustrates the study selection process.
Graph: Fig. 1 Study selection process
Fourteen systematic reviews containing 118 included publications (including double counting) corresponding to 75 index publications on economic evaluation in community pharmacy were included.
A CCA value of 4.4% was obtained, indicating only a slight degree of overlap.
Search dates of systematic reviews range from 1988 to 2015 (28 years).
Reviews were published in 11 different journals: 5 pharmacy journals (Pharmacotherapy, American Journal of Health-System Pharmacists, Journal of Managed Care Pharmacy, Journal of Clinical Pharmacy and Therapeutics, Annals of Pharmacotherapy); 2 health journals (BMJ Open, Health Policy); 2 pharmacoeconomic journals (Pharmacoeconomics, Expert Review of Pharmacoeconomics & Outcomes Research); and 2 disease-specific journals (Diabetic Medicine, Hellenic Journal of Cardiology).
Only two reviews exclusively focused on both economic evaluation studies and the community pharmacy setting [[
Reviews contain studies from 12 different countries, of which 7 are from Europe. The origin of studies in reviews published until 2013 was mostly the USA and other countries outside of Europe. After 2014, we see an increase in the proportion of studies from Europe.
With a few exceptions [[
Eight reviews allowed randomized trials, nonrandomized trials, and other controlled designs. Only 3 reviews reported RCT as inclusion criteria for included studies [[
Table 1 summarizes the characteristics of included reviews.
Characteristics of included reviews
First author (year SR) No. databases Search until No. included studies (CP/total) Countries of origin in CP studies Populations in CP studies Interventions in CP studies Comparators in CP studies Outcomes in CP studies Study design in trial-based CP studies Schumock GY (1996) [ 2 1988–1995 2/104 (1.9%) USA (2) Patients requiring therapeutic monitoring Therapeutic monitoring None Drug costs avoided and no. interventions; avoided medical care costs per intervention – Schumock GY (2003) [ 2 1996–2000 6/59 (10.1%) USA (5), Australia (1) Patients individuals: requiring therapeutic monitoring; DS; with drug related problems; for flu immunization. Therapeutic monitoring; DSM; patient education or cognitive service; flu immunization Yes (4) Changes in physician office visits, prescriptions and charges; medical and prescription utilizations and costs; costs per prescription; healthcare utilization cost savings 4 (controlled) trials Perez A (2008) [ 2 2001–2005 16/93 (17%) USA (5), Australia (4), UK (3), Canada (3), Multicentric (1) Total sample 13,304 patients in 15 studies (median 181): requiring therapeutic monitoring; DS; with dose related problems; for flu immunization; smokers. MTM (1); therapeutic monitoring (4); DSM (9); dose optimization (1); flu immunization (1); smoking cessation (1). Median length of follow-up: 9 months. Yes (8) Changes in medical costs (most studies); adherence, knowledge and satisfaction; QALYs 4 randomized trials + 1 (controlled) + 3 nonrandomized Chisholm Burns MA (2010) [ 13 Jan/09 1/20 (126 total) (5%) USA (1) NR Diabetes care No Changes in HbA1c and in cholesterol – Touchette DR (2014) [ 4 2006–2010 8/25 (32%) UK (5), USA (3) Total sample 7134 patients in 7 studies (median 760): DS; requiring therapeutic monitoring > 75 years. DSM (6); therapeutic monitoring (1). Median length of follow-up: 12 months. Yes (6) Changes in no. medicines, compliance; antiplatelet drug prescribing, CV disease visits; BP, LDL-C, HDL, TG, CV rate; % patients with asthma action plans, ED visits, hospital admissions; HbA1c, LDL-C, BP, influenza vaccination rate, eye and foot exam rate; MAI score, number of drugs 3 randomized + 1 multiple interrupted time series + 1 before/after Altowaijri A (2013) [ 13 Feb 2011 7/8 (87.5%) UK (3), Canada (2), Australia (1), Thailand (1) Total sample size NR. Patients: DS; smokers. CV: DSM (4) and CV risk smoking cessation (3). Median length of follow-up reported in 3 studies: 9 months. Yes (all) Changes in CHD patients' outcomes; CV risk; no. quitters; HbA1c, glucose; probability of events Most seem to be trials (2 randomized + 1 nonrandomized, 1 controlled) Elliott RA (2014) [ 2 2003–2013 14/31 (45%) UK (8), The Netherlands (3), USA (1), Australia (1), Canada (1) Total sample > 7000 patients in 7 studies: DS; elderly/on high no. medicines; smokers. DSM (6); MTM (8); smoking cessation (2); screening (2). Follow-up interventions in 5 studies: 6 months (2), 6–12 months (2), 1–2 years (1). Yes (13, 7 well described) Changes in adherence; prescribing errors/inappropriate prescribing; medication changes; infection, disease, quit rates); CV indicators; frequency of ED or hospital admissions; utility 4 randomized + 5 cluster randomized + 1 nonrandomized + 1 multiple interrupted time series Brown TJ (2016) [ 10 May 2014 4/19 (21%) UK (3), Australia (1) Total sample size 2791 smokers > 21 cigarettes/day in 4 studies. Mean age 24–44. Females 54–68.7%. SES variables collected in some studies. Smoking cessation. Follow-up per patient 26 or 52 weeks. Some studies reported Stages of Change Model. Yes, usual care (advice + NRT) and other settings Quit rate (self-reported or CO measurement or Fagerström Test) 2 randomized, 1 nonrandomized Wang Y (2016) [ 6 2006–2014 10/25 (40%) USA (7), Canada (1), Australia (1), Bulgaria (1) Total sample 1238 patients in 10 studies (median 68): diabetic patients (5 studies in type 2 diabetics). Diabetes: DSM (8); medication review (2). Median length of follow-up: 12 months. Yes (all) Changes in medical costs (healthcare use) 3 randomized trials + 2 nonrandomized trials + 3 nonconcurrent cohorts + 1 retrospective cohort Peletidi A (2016) [ 7 1990–2014 2/6 (33.3%) UK (2) Total sample 3764 in 2 studies: majority female smokers ≥ 21 cigarettes/day. Smoking cessation. Length of follow-up: 4 weeks or 12 week; measurement at 4 and at 52 weeks after quitting. Yes, self-quit rate Changes in CO-validated quitters drop in CO levels – Perraudin C (2016) [ 5 2004–2015 21/21 (100%) UK (13), The Netherlands (3), Spain (2), Belgium (1), France (1), Denmark (1) Total sample size NR. Patients: at risk of serious medication errors; elderly on multiple medicines; new to therapy; DS; with minor ailments; smokers. MTM (5); DSM (3); adherence (5); smoking cessation (5); screening (2); minor ailment (1). Follow-up 6 or 12 months. Yes, usual care. No intervention in some. Changes in QALY; score errors, healthcare resources, or disease avoided. No. patients on appropriate treatment/controlled/adherent/quitters 6 randomized trials + 2 cluster randomized trials + 1 multiple interrupted time series + 1 before/after Loh ZWR (2016) [ 5 Aug 2015 3/25 (12%) Multicountry (1), Spain (1), Canada (1) Total sample 3992 patients in 3 economic studies: average age approx. 75. Medication review, patient education on drug-related problems. Follow-up 6 or 18 months. Yes (all) Changes in QoL. % of recommendations accepted by physician All 3 randomized trials (inclusion criteria) Malet-Larrea A (2016) [ 7 Sept 2015 13/13 (100%) Multicentric (1), UK (4), Australia (2), Canada (2), Spain (2), The Netherlands (1), Belgium (1) Total sample 11,491 in 13 studies (median: 675): new to therapy DS patients; new to therapy elderly. MTM (5); DSM (4); adherence/compliance (4). Median length of follow-up: 6 months. Yes, usual care Changes in adherence, risk/disease symptoms/severity; BP, BMI, lipids, HbA1c, PEFR; medication; use of healthcare resources; EQ-5D or other QoL; patient satisfaction All 13 randomized trials (inclusion criteria) Gammie T (2016) [ 6 2010–2015 10/14 (71%) UK (4), Spain (2), Brazil (2), France (1), Australia (1) Total sample size NR. Patients: DS; elderly; with medication errors; at risk of apnea; smokers. DSM (5); MTM (3); screening (1); smoking cessation (1) Yes, usual care Changes in adherence, disease/severity; clinical proxy outcomes; medication use; unscheduled use of healthcare resources; QoL NR, presumably all 10 controlled
SR systematic review, CP community pharmacy, DS disease state, NR not reported, DSM disease state management, MTM medication therapy management, CV cardiovascular risk, NRT nicotine replacement therapy, BP blood pressure, TG triglycerides, ED emergency department, MAI medication appropriateness index, CHD coronary heart disease, QoL quality of life, BMI body mass index, PEFR peak expiratory flow rate
Most reviews report medication costs, supplies/tests costs, and healthcare utilization costs. More recent reviews also report indirect costs. Very few reviews report intervention costs charged.
Only half of the reviews report cost year and discount rates of included studies. Nine reviews report the perspectives of included studies.
Almost all reviews include trial-based studies. Four reviews also report synthesis/model-based studies.
Summary measures, namely, incremental ratios, are reported in eight recent reviews. Five reviews report benefit-to-cost ratio as a summary measure.
Six reviews report uncertainty of included studies but only two of them explicit sensitivity analysis methods [[
Table 2 summarizes the major economic findings of included reviews.
Economic findings of included reviews
First author (Year SR) Types of economic evaluation in CP studies Resource use and cost categories in CP studies Cost year/discount rates in CP studies Data sources in CP studies Perspectives in CP studies Key findings in CP studies Schumock GY (1996) [ No full economic evaluation NR NA Trial-based NA No B/C for CP studies. Cost avoided per prescription and avoided care costs per intervention Schumock GY (2003) [ No full economic, 1 CCA (1) NR (3), program, drug and healthcare costs (1), fees but lumped with drug costs savings (1), drug and advertising (1) Not discounted (1) Trial-based NA No B/C for CP studies. Lower mean total charges; lower medical and Rx costs; lower prescription costs; cost savings for interventions; costs exceed benefits (but error reported); costs per vaccination Perez A (2008) [ CMA (4), CBA (2), CEA (2) Program costs in most studies, staff time/wages/fees in some studies Reported Most trial-based, 1 trial/model Reported (15) 3 B/C from CP studies (1.17; 9.47; 7.67). Decrease 57% in overall health direct and indirect costs; cost savings per patient; lower incremental cost per quitter; no significant changes in 2 studies Chisholm Burns MA (2010) [ NR Direct medical costs, Indirect costs NR Trial-based NR Improvements in HbA1c and cholesterol and decreased medical direct costs per patient per year and decreased no. of sick days every year Touchette DR (2014) [ CMA (1), CUA (1) NR NR 6 trial-based, 1 model-based Reported (4) 1 ICER cost-effective: 10,000£/QALY; no difference in outcomes, costs increased in 2 studies; increase in prescribing antiplatelet drug use, no cost difference at 1 year; CV medical costs decreased; direct and indirect cost savings; healthcare costs/patient/year reduced Altowaijri A (2013) [ CEA (3), CUA (1), CBA (1), CMA (1) NR NR 5 trial-based, 2 model-based Health system (4), society (2), not clear (1) 5 CP studies cost-effective, 1 not cost-effective (but CMA used), and 1 not full economic evaluation. Cost increase, no changes in outcomes, use of CMA questionable; cost-effective; reduction of HbA1c, cost-saving on a longer term; cost saving, gain in life years; cost-effective; program seems promising in improving patient blood pressure Elliott RA (2014) [ CEA (5), CUA (3), CMA (1), CCA (6) Patient resource use (12), costs of intervention (13), partial costs (3), costs borne by patient (1), indirect costs (2) Reported (11) 11 trial-based, 3 model-based Health system (12), societal (2) This review looked at methods. 4 studies CEA, 2 CUA, 1 CEA/CUA, 1 CMA, 6 CCA. Incremental analysis used in 8 full economic evaluations: cost per error avoided, cost per extra adherent patient; cost per % increase in patient adherence; cost per quitter; cost per pelvic inflammatory disease avoided; cost per QALY Brown TJ (2016) [ CEA (3), CUA (1) Direct costs of intervention, fee charged (1), travel costs (1) Reported 3 trial-based, 1 trial/model Health system (3), societal (1) All 4 economic evaluation studies reported being cost-effective ranging from 181£ to 772£ per life-year saved, ICUR 2600£; studies used ICER and 1 used ICUR. Wang Y (2016) [ CEA (3), CBA (3) Labor costs (4), cost of intervention (3), fees charged (1), transportation costs (1) Reported 9 trial-based, 1 model-based Payer (7), provider (3), patient (1) Costs increased in both groups; B/C ratios favorable for 2 studies; no difference in 1 study; ICER cost-effective; costs avoided per person per year Peletidi A (2016) [ CEA/CUA (1), CEA (1) NR NR Trial-, model-based NR Both studies report incremental ratios ICER per quitter; ICUR per QALY, and cost-effective. Perraudin C (2016) [ CEA (12), CUA (10), CMA (2) Labor costs (15), costs of intervention, training costs (12), fixed costs, productivity loss OR fees charged (3) Some: lower rates for effects 10 trial-based, 11 model-based Payer (17); societal (5). Some both All 21 studies are economic evaluations. ICER for most studies. CEAC ranging from 59 to 97% prob. of being C/E. Uncertainty very low for screening (chlamydia and sleep apnea) and smoking cessation. Some degree of uncertainty for remainder medication or disease interventions. Loh ZWR (2016) [ CUA (1), not stated in other two. NR NR All 3 trial-based NR 1 study was 100% cost-effective when WTP threshold €30,000/QALY– €45,000/QALY; 2 studies no summary measures Malet-Larrea A (2016) [ CEA/CUA (3), CUA (3), CEA (2), CCA (4), CMA (1) Labor costs, hospital use, GP visits, medication, supplies, productivity loss Reported for all 10 trial-based, 3 trial/model-based Health payer (9), societal (2), govn (1), both (1). Few patient/provider Incremental analysis performed in 9 studies and calculated for 3: 4 dominant; 7 cost-effective; 1 not cost-effective Gammie T (2016) [ CUA (8), CEA (2) NR NR NR Reported (2) ICERs performed for 9 studies: 8 are cost-effective
SR systematic review, CCA cost-consequence analysis, CMA cost-minimization analysis, CBA cost-benefit analysis, CEA cost-effectiveness analysis, CUA cost-utility analysis, B/C benefit-to-cost ratio, ICER incremental cost-effectiveness ratio, ICUR incremental cost-utility ratio
Nearly 60% of CP primary studies (index publications) are from the USA and the UK.
Figure 2 illustrates countries of origin of economic evaluation of community pharmacy primary studies.
Graph: Fig. 2 Countries of origin of economic evaluation of community pharmacy primary studies
Disease management is the most frequent intervention category in primary studies (40 studies), namely, in diabetes, hypertension, hyperlipidemia, asthma, and smoking cessation, followed by medication management (13 studies) and high cost/case management, especially for the elderly on multiple medication (10 studies). Adherence is the intervention category in 5 studies for patients new to therapy, screening in 4 studies, and disease prevention (immunization) in 2 studies. The intervention for one study was not reported.
Thirty-one studies are controlled trials. Seven studies are nonrandomized trials, and one is a multiple interrupted series design. Twelve studies (16%) are model counterfactual. Twelve studies are reported as not controlled, 3 are nonconcurrent cohort studies, 2 are before/after designs, 1 is a retrospective cohort study, and no report exists for 6 studies.
CEA is the most frequent type of economic analysis, followed by CUA. However, 19 studies are not full economic analyses (Fig. 3).
Graph: Fig. 3 Types of economic analysis of economic evaluation CP primary studies
Health payer is the most frequent perspective reported (43 studies), societal perspective is reported for 8 studies, 2 studies seem to adopt both payer and societal perspectives, and provider perspective is reported for 6 studies. The perspective is not reported in reviews for 16 studies.
When reported, the time horizon most frequently reported is up to 12 months. Sensitivity or scenario analysis is reported for 19 studies, currency year for 50, and discount rate for 6 studies only (3%, 3.5%, and 4%), of which 2 studies present lower discount rates for consequences (1.5%). Summary measures are reported for 27 studies, 16 of which are ICERs.
Overall, reviews reported favorable economic findings for 37 of 52 studies (71%) with full economic evaluations or CCA or cost-minimization analysis (CMA).
See supplementary characteristics of primary studies as reported in included reviews, Additional file 4.
In accordance with recommended approach in AMSTAR 2, we identified (four) critical domains; we assessed all 16 items for each included review but did not use responses to derive an overall score; and we rated overall confidence on the four critical domains as high (green), moderate (yellow), or low (red). The two critical quality domains that seem to be absent from almost all reviews are protocol registered before commencement of the review and no justification for excluding individual studies.
Table 3 summarizes the results of quality assessment of included reviews in the considered four critical domains of AMSTAR 2.
Quality of included reviews in critical domains (AMSTAR 2)
No. AMSTAR 2 item Critical domains 2 Explicit statement—review methods established prior to review and justification for deviations from protocol 2 (14%) 4 Used comprehensive literature search strategy 14 (100%) 7 Provided list of excluded studies and justification for exclusions 0 (0%) 13 Accounted for risk of bias in individual studies when interpreting/discussing results of review 10 (71%)
See supplementary complete quality assessment of included reviews in all 16 items of AMSTAR, Additional file 5.
We assessed the quality of evidence of CP primary studies reported in Reviews in five areas: quality of evidence for deriving effectiveness; risk of bias; quality of economic evaluation; heterogeneity; wider research issues.
Quality of evidence of CP primary studies for deriving effectiveness
. Three reviews defined RCTs as one inclusion criteria [[
Risk of bias of CP primary studies
. Three recent reviews used an instrument to assess the risk of bias of included studies [[
Brown et al. used the six-item Effective Public Health Practice Project Quality Assessment Tool for Quantitative Studies [[
Loh et al. used the seven-item Cochrane Collaboration's tool for assessing risk of bias in randomized trials [[
Malet-Larrea et al. used the nine-item Risk of Bias for EPOC Reviews Tool [[
Six reviews used no instrument but reported risk of bias in included studies [[
One review assessed but reported no significant bias [[
Quality of economic evaluation of CP primary studies
. Five reviews used an instrument to assess the quality of economic evaluations of included studies [[
Perez et al. used an instrument developed by the authors [[
Wang et al. also used CHEERS [[
Malet-Larrea et al. used the Evers Checklist Tool used for all economic evaluations in addition to the Phillips instrument for three combined trial and model-based economic evaluations [[
Six other reviews did not use an instrument but performed some assessment of economic evaluation in the included studies [[
Heterogeneity (populations, interventions, outcomes) of CP primary studies
. Ten reviews include studies addressing diverse populations, interventions, outcomes, and even settings, and most report this variability, which does not allow synthesizing results or generalizing [[
Four reviews do not assess or report the impact of heterogeneity in results [[
Wider research issues of CP primary studies
. Twelve reviews do not report equity considerations—e.g., whether socioeconomic variables had differential effect on intervention—an important aspect in public health interventions [[
Nine reviews provide some meager considerations on the sustainability of interventions [[
Eight reviews provide some considerations on the context of the included studies [[
Assisted by Wang's framework, nine reviews briefly discuss the importance of process dimensions for applicability: political environment, resource implications, and organizational structure of pharmacies and skill of pharmacists. The social acceptability by the target population, cultural adaptability, and the impact of the educational level of the target population are not reported in reviews.
Reviews do not seem to discuss the importance of the following outcome dimensions for transferability: baseline prevalence of the condition, differences in populations, and capacity to implement the intervention. However, despite variations in populations and in the capacity to implement the intervention, we observe some consistency in favorable findings for some intervention categories across different countries.
Economic evaluations of pharmacy-based public health interventions include various elements, additional dimensions, and challenges which are summarized in Table 4.
Major key methodological findings
Most frequent risk of biases Random sequence generation, allocation concealment, baseline characteristics, baseline outcome measurements similar, and contamination between intervention and comparator Study designs Not restricted to RCTs or cluster RCTs Economic quality criteria Most criteria are met but there are some issues: not detailing target population; not describing "usual care" comparators; not including patient costs, indirect costs or intervention fees; analytical methods poorly described; incremental costs and outcomes sometimes not reported; not accounting for uncertainty Heterogeneity In populations, interventions and some outcomes Equity Not assessed Process dimensions Process dimensions that impact dissemination and external validity poorly described.
RCT randomized controlled trial
Favorable economic findings appear for 71% of studies with full economic evaluations or CCA or CMA.
In triangulation with recommendations for economic evaluations of both public health interventions and alongside trials, we assessed all ten items of these recommendations for each included review and we rated collective overall agreement of reviews with each item as high (green), moderate (yellow), or low (red).
Triangulation with recommendations for economic evaluations both of public health interventions and alongside trials reveals poor evidence of the following: societal perspective, costs and benefits falling on nonhealth sectors, lower discounting rates (vs medicines), equity assessment, valuation of costs, and methods for costs and outcome analysis.
The following recommendations are present but not well detailed: prior review of evidence, linking intermediate to long-term outcomes, wider spectrum of research methods, selection of subjects and sites seeking real-world target population and providers, current practice or standard of care, cost data collection, and detailed reporting.
In contrast, the following recommendations are usually present and well detailed: economic appraisal is linked to effectiveness, most study designs are RCTs but include other controlled designs, the types of economic evaluation are diverse, study designs tend to be pragmatic, sample size is based on an important clinical outcome correlated with economic outcomes, and single direct measures are usually preferred.
Tables 5 and 6 summarize the triangulation with recommendations.
Triangulation with key recommendations on the economic evaluation of public health interventions
# Key recommendations Overview findings 1 Review of evidence: Systematic review or in-depth review of evidence prior to the economic evaluation of PHI 2 Effectiveness and economic appraisal: Economic appraisal linked to the appraisal of effectiveness of PHI 3 Study designs: When randomized trial not feasible, quasi-experimental designs or econometric techniques 4 Perspective: Societal perspective (public sector may be used where appropriate) 5 Time horizon: Trial data may need modeling but requires reliable link between intermediate and long-term outcomes 6 Types of economic evaluation preferred: CBA and CCA preferred but CUA and CEA also recommended whenever health is the sole benefit 7 Nonhealth costs and benefits of PHI: Need to capture costs and benefits falling on nonhealth sectors 8 Discounting rates: Lower discounting rates for PHI (1.5% for NICE, 3% for CDC) if costs and health effects accrued > 1 year 9 Equity considerations: Compare differences in health status changes between different health economic groups 10 Wider spectrum of research methods: Understand contextual and process indicators affecting behavior change and other variables
PHI public health interventions, CBA cost-benefit analysis, CCA cost-consequence analysis, CUA cost-utility analysis, CEA cost-effectiveness analysis, NICE National Institute for Health and Care Excellence, CDC Centers for Disease Control and Prevention
Triangulation with key recommendations on the economic evaluation alongside clinical or pragmatic trials
# Key recommendations Overview findings 1 Study design: Based on well-designed pragmatic/naturalistic trials with fewer strict protocols 2 Selection of subjects and sites: Seek for proximity to real-world target population and less restrictive patient inclusion criteria 3 Sample size: Based on important clinical outcome correlated with economic outcome, previous pilot or wider CI for ICER/CEAC 4 Estimates beyond trial: Appropriate length of follow-up, estimates beyond trial require survival analysis, link to final outcomes or regression 5 Comparator: Current practice or standard of care should be the comparator, although there may be different standards of care 6 Measures of outcomes: Direct, single measures are preferred. Utilities collected directly from study subjects at regular intervals 7 Data collection (resource use and costs): Relevant resource use and cost measures collected with clinical data (case report forms, patient records, patient diaries, interviews, computerized record linkage) 8 Valuation of costs: May include: microcosting; unit costing; and gross costing 9 Methods for cost and outcome analysis: Arithmetic mean cost differences, bootstrapping, OLS or GLM for between group comparison; multivariable methods for outcomes; confidence intervals, 10 Reporting: General description of trial and major findings; economic data collected alongside trial; missing and censored data; methods to construct, compare and project costs and outcomes; statistical methods; results on resource use, costs and outcomes; results within and beyond time horizon of trial.
CI confidence intervals, ICER incremental cost-effectiveness ratio, CEAC cost-effectiveness acceptability curve
Economic evaluations of pharmacy-based public health interventions seem to follow most economic quality criteria, but there are still some issues in certain key areas to improve.
This overview sought to include all relevant systematic reviews complying with inclusion criteria, with no restrictions on the period covered, populations, interventions, comparators, or outcomes, in attempt to be inclusive and to critique methods.
We have defined four of seven AMSTAR 2 critical domains for an overall critique on how well included reviews performed in these domains, as recommended. Risk of bias from individual studies being included in the review was not considered, since the most important instruments for nonrandomized studies included in a systematic review were released in 2016 and 2017. Appropriateness of meta-analytical methods was not considered, since included reviews were narrative. Assessment of presence and likely impact of publication bias was also not considered, since funnel plot asymmetry requires a minimum of ten studies and included reviews each have eight studies on average.
The findings of this overview are consistent with methodological issues described by Elliot et al., Perraudin et al., Malet-Larrea et al., Jommy, and Whitty for the economic evaluation of pharmacy-based interventions [[
This review only included those studies containing the selected search terms. Hence, it is possible we might have missed relevant systematic reviews that may have used a different terminology.
We used a specific filter recommended by the Centre for Reviews and Dissemination (CRD) for economic evaluation in Embase via Ovid, as stated in Additional file 2.
We did not find a specific filter for economic evaluation for other databases used in this review published in ISSG Search Filter Resource. We restricted the search strategy to the definition of full economic evaluation, that is, comparing costs and health outcomes of two or more interventions in which three generic types of economic evaluation are used: cost-effectiveness, cost-utility, or cost-benefit analyses. We deliberately excluded search terms used in partial economic evaluations. However, this restriction might have implicated on the sensitivity of these searches.
In addition, it is possible we might have missed relevant recommendations on economic evaluations of public health interventions and/or alongside trials due to the selected exploratory approach used for these reviews with the sole purpose of obtaining key insights for triangulation.
In recent years, governments in various countries have introduced profound changes to pharmacy remuneration systems in the components related to equitable, safe, and quality access to medicines and to efficiency and/or quality incentives. These two components are the most important in pharmacy remuneration systems. However, the network of pharmacies provides a unique opportunity for governments to implement relevant public health interventions, that fit within national, regional and local health policies, in close collaboration with primary care. In several countries, governments have already contracted with pharmacies to pay for relevant interventions. However, as for any other health technology, public health interventions provided by community pharmacists must seek to demonstrate effectiveness and economic benefits to be reimbursed by public payers.
The economic evaluation of pharmacy interventions presents challenges.
We hope the findings of this overview may assist in improving the design, implementation, and assessment of pilot trials; hence, the robustness of evidence to justify payers' investment, which requires the endorsement of community pharmacists to participate in trials and an informed understanding of policy makers in negotiations.
Based on the findings of this overview and in addition to the methodological considerations for economic evaluations of pharmacist interventions by Elliot et al. [[
Methodological approach for the economic evaluation of pharmacy-based public health interventions
Step Methodological approach 1 Performing a systematic or in-depth review of existing evidence prior to economic evaluation 2 Planning and conducting well-designed trial(s) for the assessment of effectiveness using PICO framework for a clear definition of population, intervention, comparator and outcomes EPOC study design to assist in the selection of the best possible study design Risk of Bias for EPOC Reviews Tool to assist in strategies to minimize the most frequent risk of bias 3 Planning and conducting an economic evaluation using: CHEERS Checklist to perform economic evaluations according to accepted standards Recommendations for economic evaluations of public health interventions and for economic evaluations alongside trials to assist in adjustments: using a societal perspective; reliable linking of intermediate to long-term outcomes; choice of health outcomes may not allow for QALY; if costs and benefits also fall on nonhealth sectors, CBA approach may also be required; intervention costs must consider a retail price (as this would be the case if reimbursed); use of lower discount rates; other summary measures may be required if CEA nor CUA are used; and performing equity assessment 4 Using a wider spectrum of research methods to understand: Contextual and process indicators affecting the behavior change of patients and providers and other variables
PICO population, intervention, comparator, outcomes, EPOC Effective Practice and Organization of Care, CHEERS Consolidating Health Economic Evaluation Reporting Standards, QALY quality-adjusted life years, CBA cost-benefit analysis, CEA cost-effectiveness analysis, CUA cost-utility analysis
As the research corpus continues to expand following practice and policy requirements, it will become important to build a multidisciplinary expert consensus around a specific guidance for the economic evaluation of pharmacy-based public health interventions.
This research received no funding and was undertaken as part of a Ph.D. thesis research project of the first author on the economic evaluation of selected pharmacy interventions in a collaborative care model in Portugal (alongside a trial) at Escola Nacional de Saúde Pública (National School of Public Health), Universidade NOVA de Lisboa, Portugal.
We gratefully acknowledge the contribution of Daniel Touchette, PharmD, MA, FCCP, Professor of Pharmacy at University of Illinois, USA, for clarifications provided concerning his coauthored review, and of Ricardo Fernandes, MD, PhD, Assistant Professor and Clinical Researcher at the Faculty of Medicine and Instituto de Medicina Molecular of the University of Lisbon and Co-Director, Cochrane Child Health Field, for valuable comments on a previous draft of this paper.
CM proposed the main research question. CM and JP assisted in planning of the work. SC conceptualized the review, conducted search, selected reviews, extracted data, performed quality assessment and data synthesis, and drafted the article. SC and MC independently reviewed titles and abstracts; CM and DKH resolved disagreements. SC and CM reviewed and selected full-text articles for eligibility. CM and DKH reviewed data extraction. MC reviewed the quality of included reviews, heterogeneity, wider research issues, applicability, and transferability. All authors contributed to the discussion of the findings and conclusions, revised the paper, and approved the final version. CM acts as overall guarantor.
All data generated or analyzed during this research are included in this published article. Additional supporting information may be found in the following:
• Completed PRISMA Checklist (Additional file 1)
• Search strategies (Additional file 2)
• List of excluded reviews (Additional file 3)
• Characteristics of primary studies as reported in included reviews (Additional file 4)
• Quality of included reviews in all 16 items (AMSTAR 2) (Additional file 5).
This article does not contain any studies with human participants performed by any of the authors. No identifying information about participants is available in the article.
Not applicable.
SC was employed by the National Association of Pharmacies (ANF) at the time of this research. MC is employed by the ANF. DKH has given talks on US innovative pharmacist-led interventions at conferences in Portugal for which travel and accommodation costs have been borne by the ANF. The ANF had no role in the concept or design of this paper, in the collection, analysis, and interpretation of data, or in the writing of or commenting on this paper or previous versions of it. The other coauthors declare that they have no known conflicts of interest. None of the authors was involved in any of the included reviews or in individual studies of this overview.
Graph: Additional file 1. Completed PRISMA Checklist.pdf (Completed PRISMA Checklist) – this file contains the completed PRISMA items checklist.
Graph: Additional file 2. Search Strategies.pdf (Search strategies) – this file contains the search terms and search strategies used for electronic databases.
Graph: Additional file 3. List of Excluded Reviews.pdf (List of excluded reviews) – this file contains the list of excluded reviews during the study selection process.
Graph: Additional file 4. Characteristics of Primary Studies.xls (Characteristics of primary studies as reported in included reviews) – this file contains a table of all 75 primary studies and its main characteristics as reported in included reviews.
Graph: Additional file 5. Quality of included reviews in all 16 items (AMSTAR 2) – this file contains the complete quality assessment of included reviews in all 16 items of AMSTAR 2.
- AMSTAR-2
- Assessing the Methodological Quality of Systematic Reviews
• CBA
- Cost-benefit analysis
• CCA
- Corrected covered area
• CCA
- Cost-consequence analysis
• CDC
- Centers for Disease Control and Prevention
• CDSR
- Cochrane Database of Systematic Reviews
• CEA
- Cost-effectiveness analysis
• CHEERS
- Consolidated Health Economic Evaluation Reporting Standards
• CMA
- Cost-minimization analysis
• CP
- Community pharmacy
• CRD
- Centre for Reviews and Dissemination
• CRF
- Case report forms
• CUA
- Cost-utility analysis
• DARE
- Database of Abstracts of Reviews of Effects
• EPOC
- Effective Practice and Organization of Care
• EU
- European Union
• GLM
- Generalized linear model
• ICER
- Incremental cost-effectiveness ratio
• ISPOR
- International Society for Pharmacoeconomics and Outcomes Research
• JBI
- Joanna Briggs Institute
• MCDA
- Multiple-criteria decision analysis
• MRC
- Medical Research Council
• NHS EED
- National Health Service Economic Evaluation Database
• NICE
- The National Institute for Health and Care Excellence
• OECD
- Organization for Economic Co-operation and Development
• OLS
- Ordinary least squares
• PRISMA
- Preferred Reporting Items for Systematic Reviews and Meta-Analyses
- PROSPERO
- International Prospective Register of Systematic Reviews
• QALY
- Quality-adjusted life years
• QHES
- Quality of Health Economic Studies
• RCT
- Randomized controlled trials
Supplementary information accompanies this paper at 10.1186/s13643-019-1177-3.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
By Suzete Costa; Maria Cary; Dennis K. Helling; João Pereira and Céu Mateus
Reported by Author; Author; Author; Author; Author