Project summary

I am a hospitalist at Rhode Island Hospital, Brown University Health, and Clinical Assistant Professor of Medical Science at Brown University. My background includes clinical outcomes research, machine learning and work as a data scientist. I am developing this project with Dr. Octavian Ioachimescu.

We are seeking $12,000 for a six-month pilot. The minimum award is $6,000. The funding would support clinical rubrics and independent ratings, programming assistance, statistical work and capped computing costs. We still need to recruit independent reviewers.

The study concerns disagreement between AI agents assigned hospital medicine, cardiology and nephrology roles. A planned synthetic case has persistent congestion with worsening kidney function. Another involves hypotension during fluid removal. We want to find out whether a specialty label changes the advice an agent accepts, and whether the resulting change is clinically justified.

We will use one fixed model snapshot. Agents will assess each case separately, then we will copy the same assessments into two discussions. In one, peer specialties are visible; in the other, peers have neutral identifiers. Each agent keeps its own expertise description. Agent positions and speaking order will be counterbalanced. We will obtain private final answers before prespecified aggregation, without a leader or a requirement to agree.

Two blinded clinicians will score recommendations against case rubrics written before they see model outputs. We will track retention of clinically sound advice by the initial dissenter in explicit one-versus-two splits, distinguish evidence-supported revisions from unsupported concessions, and score final-plan safety and appropriateness. A separate experiment will change only the specialty or authority attributed to identical advice.

The pilot will measure feasibility and cost. It will not establish clinical effectiveness. Cases will be synthetic, with no identifiable patient data or use in patient care. We will publish code, cases, rubrics and shareable logs, including null and negative findings.

What the study adds

Medical multi-agent systems and their failure modes already have a substantial literature. MedAgents and TeamMedAgents study collaboration in medical reasoning [1,2]. MedAgentAudit examines failures including false consensus and minority suppression [3]. Separate work tests authority effects and identity bias [4,5]. Simply showing that AI agents defer to one another would add little.

We will focus on the effect of a visible hospital-specialty label while keeping the model, initial recommendations and evidence fixed, then assess whether the resulting changes are clinically justified. Specialty-specific management conflicts and independent clinical rubrics are central to that comparison. A targeted literature search was updated on October 15, 2026. We will complete the full-text comparison of the closest studies and update the search before preregistration. If the same question has already been answered, we will narrow or redirect the experiment before confirmatory work.

How we will run the study

We will begin with one fixed model snapshot and neutral descriptions of expertise in hospital medicine, cardiology and nephrology. Each agent will first assess the case in isolation. We will then replay the same initial assessments into paired discussion conditions: peers identified by specialty, or peers shown with neutral identifiers. Each agent keeps its own expertise description. Neutral agent positions and speaking order will be counterbalanced.

Discussion will initially use simultaneous rounds, with all messages in a round released together. Agents will have the same available history, without an appointed leader or a requirement to agree. We will collect individual final recommendations privately before applying a prespecified team aggregation rule. A single generalist and an independent specialist panel without discussion will provide comparison conditions. We will report token and inference budgets so any advantage from extra computation is visible.

A separate experiment will give agents clinician-approved scripted advice. We will keep the recommendation, supporting evidence, opposing messages, model and order identical, changing only the specialty or authority attributed to the focal advice. This tests attribution directly. We will analyze it separately from disagreements that arise naturally during discussion.

How we will judge the results

The proposed primary social outcome is whether the initial dissenter retains a clinically sound focal recommendation after discussion. An eligible minority requires an explicit one-versus-two split in the initial assessments. An omitted or uncertain stance will not count as a minority. Two blinded clinicians will independently assess soundness against a case-specific rubric written before any model outputs are seen, with disagreements adjudicated. A case may have several acceptable plans.

We will also score final-plan safety and appropriateness. A change of mind can be appropriate, so we will distinguish retained acceptable advice, justified revision, unsupported concession and changes that cannot be classified. We will check automated interaction coding against blinded human annotations. Message count and self-reported influence will not be treated as proof of dominance.

The development pilot will use four clinician-authored cases with two or three repetitions per condition to measure feasibility, dissent frequency and cost. Separate cases will be held out. Pilot findings will guide a focused design and simulation-based sample sizing for any later confirmatory study. This grant does not assume that the pilot will be adequately powered to establish clinical effectiveness.

Before confirmatory work, we will preregister primary comparisons, exclusions, missing-run handling and multiplicity rules. The analysis will estimate paired effects with equal case weighting, account for repeated runs, and cluster observations by case. Individual messages will not be treated as independent samples.

Who will do the work

I am a hospitalist at Rhode Island Hospital, Brown University Health, and Clinical Assistant Professor of Medical Science at Brown University. My background includes clinical outcomes research, machine learning and work as a data scientist. I am developing this study with Dr. Octavian Ioachimescu. We will recruit independent hospital medicine, cardiology and nephrology reviewers and statistical support. Those appointments have not yet been secured.

An offline harness has passed 16 software tests and produced 32 mock records. This establishes that the software workflow can run; it is not a clinical result. Its current vignettes are engineering fixtures, not a clinician-validated case bank. No paid model calls have been made.

What the funding will pay for

Most of the budget pays for clinical judgment and independent assessment. The target and minimum budgets cover different pilot scopes.

Target and minimum budgets

Clinical rubrics and blinded review: target $4,500 (60 hours at $75); minimum $3,000.

Research and programming assistance: target $2,800 (80 hours at $35); minimum $1,400.

Statistical design and analysis: target $2,000 (20 hours at $100); minimum $1,000.

API inference and computing: target cap $1,500; minimum cap $600.

Independent reproducibility and log audit: target $1,200 (24 hours at $50); not separately funded in the minimum budget.

Total target $12,000; minimum $6,000.

These are planning estimates, not staff commitments or accepted quotes. The budget includes no PI salary, indirect costs, travel or hardware. Purchasing and payments will follow funder and applicable institutional requirements. We will measure costs on a small number of calls before scaling; paid API work requires separate project, credential and spending authorization.

At $6,000, we will prioritize case and rubric feasibility and the controlled attribution experiment. We will publish the resources and feasibility findings, including any inability to proceed because independent clinical review cannot be secured. That scope does not include a powered clinical effectiveness comparison.

Six month work plan

Months 1–2 will cover the literature comparison, case and rubric development, reviewer recruitment and the applicable institutional research-review determination. Months 3–4 will cover the authorized, cost-capped API pilot, selection of a focused analysis plan and evaluation on held-out pilot cases. Months 5–6 will cover blinded ratings, analysis and the public report. The six months begin after funding and required institutional determinations. Any later confirmatory study will have a separate preregistered sample-size plan.

What could prevent a useful result

Clinically sound minority recommendations may be uncommon, model responses may vary substantially, or independent reviewers may be difficult to recruit. If natural minorities are rare, that is a feasibility finding worth reporting. We will use the separate scripted attribution experiment to study influence without presenting manufactured disagreements as natural ones. If we cannot secure adequate independent review, we will reduce the scope or stop clinical evaluation and report why.

The study evaluates AI representations of hospital dynamics. A human comparison would be needed to determine whether the findings apply to real clinical teams. Synthetic cases also do not settle local institutional review requirements. We will seek the applicable determination before the study and report null and negative findings.

Records and reproducibility

We will preserve complete visible discussions, initial and final structured recommendations, prompt and case versions, model identifiers, timestamps, token use and actual costs. Failures and retries will remain auditable. Histories will not be silently shortened differently between conditions, and we will use visible clinical rationales without requesting hidden chain-of-thought. These records support replication, although identical outputs are not guaranteed across repeated model calls.

Other funding and the public benefit

This project has raised $0 to date. I submitted an application for the same broad research program to the BHMG Innovation and Academic Assessment Fund on October 15, 2026. That program offers up to $75,000; no award has been made.

We will disclose overlapping support, keep separate expense records and never charge the same work to two awards. If BHMG funds cover the work proposed here, we will notify Manifund before spending overlapping funds and seek approval to reduce or return the allocation, or use it for a clearly distinct extension. We will not assume that redirection is permitted.

The public output will be a pilot report and reusable evaluation materials: code, prompt versions, synthetic cases, clinical rubrics and shareable logs. These will let other researchers inspect when discussion corrected an error and when it suppressed sound advice. The fundraising deadline currently saved on Manifund is November 15, 2026.

Selected references

1. Tang X, et al. MedAgents: Large Language Models as Collaborators for Zero-shot Medical Reasoning. Findings of ACL. 2024;599–621. doi:10.18653/v1/2024.findings-acl.33. https://aclanthology.org/2024.findings-acl.33/

2. Mishra PP, Arvan M, Zalake M. TeamMedAgents: Pareto-Efficient Multi-Agent Medical Reasoning Through Teamwork Theory. arXiv:2508.08115v3. Revised March 31, 2026. https://arxiv.org/abs/2508.08115v3

3. Zhu Y, et al. Auditing medical multi-agent AI reveals risks of false consensus. arXiv:2510.10185v3. Revised May 27, 2026. https://arxiv.org/abs/2510.10185v3

4. Choi J, et al. Belief in Authority: Impact of Authority in Multi-Agent Evaluation Framework. arXiv:2601.04790. 2026. https://arxiv.org/abs/2601.04790

5. Choi HK, Zhu J, Li S. When Identity Skews Debate: Anonymization for Bias-Reduced Multi-Agent Reasoning. ACL. 2026;14284–14311. doi:10.18653/v1/2026.acl-long.650. https://aclanthology.org/2026.acl-long.650/