PQ Program Evaluation

May'26

Jun

Jul

Aug

Sep

Oct

Nov

Dec

Jan'27

Feb

Mar

Apr

May

Jun

Jul

IRB expedited review

Engage RA + ARC1h

Acceptability instrument0.5h

MyCAP: instruments5h

MyCAP: control + triggers25h

Recruitment materials2h

Manuscript outline4h

Pilot-test MyCAP5h

Recruitment ramp4h

6-wk program + T0/T1

Analysis → draft → submit64h

Milestones

Pre-launch build ≈ 54h total · ~30h (MyCAP instruments + control build) offloadable to the research assistant → PI-level lift ≈ 24h spread across June–Sept.

Active Pending Milestone Slavin report Today

Study at a glance

QuestionIs a 6-wk app-based mental-fitness program feasible & acceptable in anesthesiology residents, and what effect sizes does it produce?

DesignPilot RCT, 2-arm parallel, 1:1, multi-site, unblinded

ComparatorActive control (MyCap), matched for time + attention

Remuneration$40 (T0) + $60 (T1) = $100; lifetime PQ access

TargetJEPM original article (3000 words, 30 refs); present at SEA

PurposeFeasibility + effect-size data to power a future multi-site efficacy trial

Outcomes & measures

Primary Feasibility

Outcome	Specific measure
Recruitment rate	% of eligible PGY4 residents who consent
Dyadic enrollment rate	% of consenting residents who enroll a loved one
Retention rate	% of enrolled who complete the T1 assessment
Adherence	Module completion + daily engagement, from app/platform backend

Primary Acceptability, appropriateness & feasibility recommended set

Replaces the draft items (which mixed acceptability with perceived efficacy) with the validated Weiner et al. triad: three 4-item scales, each rated 1 (completely disagree) – 5 (completely agree), administered at T1 for both arms. Keeps acceptability cleanly separable from the efficacy signal and is instantly interpretable to reviewers.

Measure (4 items, 5-pt)	Item stems — "The PQ program…"
AIM — Acceptability	…meets my approval · is appealing to me · I like it · I welcome it
IAM — Appropriateness	…seems fitting · seems suitable · seems applicable · seems like a good match
FIM — Feasibility	…seems implementable · seems possible · seems doable · seems easy to use
NPS (retained)	"How likely are you to recommend this program to a colleague or fellow resident?" (0–10)
Open-ended (retained)	Most beneficial? · Suggested improvements? · Single biggest barrier to completion?

Why this is better: acceptability, appropriateness, and feasibility are distinct implementation outcomes (Proctor 2011). The Weiner scales measure them separately with validated items, so a reviewer can't accuse us of conflating "did residents like it" with "did it work." Adherence (objective backend data) and retention live under Feasibility above.

References. Weiner BJ, Lewis CC, Stanick C, et al. Psychometric assessment of three newly developed implementation outcome measures. Implement Sci. 2017;12:108. PMID 28851459. · Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2011;38(2):65–76. PMID 20957426.

Secondary Preliminary psychological — T0→T1 change by arm (Cohen's d)

Measure	Construct	Format
SCS-SF	Self-compassion (planned primary outcome of the future trial)	12 items, 5-pt
MBI-GS (abbrev.)	Burnout: exhaustion, cynicism, professional efficacy	Likert frequency
PHQ-8	Depressive symptoms	8 items, 0–3
GAD-7	Anxiety symptoms	7 items, 0–3

Analysis is descriptive: feasibility/acceptability as percentages; psychological change as effect sizes (Cohen's d, 95% CI) via independent t-tests / ANCOVA (Mann-Whitney if non-normal). p-values exploratory only — no a priori power analysis (pilot).

Who's who

Johanna Lee, MD

PI & first author

MGH Anesthesia, Assistant PD. Owns IRB, MyCAP build, recruitment, analysis, and manuscript. Slavin grant PI.

Jimin Kim, MD

BWH site PI

Incoming BWH Anesthesiology PD (7/1/26). Advising + BWH recruitment; limited bandwidth for execution.

Dan Saddawi-Konefka, MD, MBA

Senior author / co-I

Mentor. Methodology, framing, writing, and air cover for resident recruitment via the APD team.

Ariel Mueller

ARC Director

MGH Anesthesia Research Center. Route to research coordinator + biostatistician (both Slavin-funded). Engage post-IRB.

Positive Intelligence, Inc.

Vendor

Provides the PQ platform free for this and future studies. Needs an executed Data Use Agreement (post-IRB).

Shirzad Chamine

External stakeholder

PQ creator. Not on protocol (kept independent). Milestone-only updates; Dan owns that channel.

PQ Program Evaluation — Pilot RCT

What's due when

Timeline (Gantt)

Study at a glance

Outcomes & measures

Who's who