Projects

@jeffandsantana

https://www.linkedin.com/in/jeff-abbott-621597b7?utm_source=share_via&utm_content=profile&utm_medium=member_android

Syntax DNA: measuring symbolic convergence across independent LLMs

**Funding goal:** $10,000

**Minimum funding amount:** $2,500

**Deadline:** December 15, 2026

Short description

Four vendors' models sometimes produce the same idiosyncratic symbol for the same prompt. Nobody has measured whether that is real, how large it is, or whether it is memorisation or structure. This project measures it, with pre-registered predictions and a public dataset.

Full description

The question

Give the same prompt to models from OpenAI, Google, xAI and DeepSeek and they sometimes return the same peculiar symbol, metaphor or framing, out of the many equally valid alternatives. People notice this constantly and it has never been measured.

I want three numbers:

1. How often do independently developed models converge on the same symbolic structure under matched conditions?

2. Does that convergence survive paraphrase, prompt reordering and language change?

3. Is it memorisation of shared training data, or a stable attractor produced by shared architecture and optimisation?

Why this is a safety question

The field leans on one assumption everywhere: that agreement between two independently developed models is independent evidence. Independent evals, independent red-teaming, independent safety cases all rest on it.

If models from different vendors converge on the same symbolic shorthand, they likely converge on the same misreadings, and the assumption fails in a specific and dangerous way: a failure found in one model would reproduce silently in another, and cross-model agreement would be read as robustness when it is actually a shared blind spot. That is the mechanism behind eval contamination (agreement because both memorised the same corpus) and behind monoculture risk (the ecosystem failing together because everything was trained and optimised the same way).

This project produces the first measured convergence rate with confidence intervals, and separates memorisation from structural convergence. Both are near-term, measurable, cross-vendor safety properties. Neither currently has numbers.

What already exists

Published: *Shibboleth Prompting: Achieving Long-Term Persona Persistence and Contextual Recall Across LLM Platforms and Sessions* ([link](https://www.academia.edu/168595904/Shibboleth_Prompting_Achieving_Long_Term_Persona_Persistence_and_Contextual_Recall_Across_LLM_Platforms_and_Sessions)). That work showed that cross-platform structural effects can be deliberately elicited and reproduced, which is the method this study extends to a cross-vendor question.

Also complete: the framework draft (v0.1) with charter, scope, operational definitions, and the observation kept explicitly separate from the hypothesis. Plus the design work: three prompt families, the blind-coding rubric, and the null-model controls.

Not yet done: the frozen prompt suite, the 1,800-generation dataset, blind coding, and the statistical analysis. That is what this grant funds.

Plan, 12 weeks from funding

- **Weeks 1-3.** Build and freeze a 120-prompt suite: novel-symbol elicitation, paraphrase invariance, and nonce controls with no conventional referent.

- **Weeks 4-7.** ~1,800 generations: 5 seeds × 4 vendors × 3 temperatures. Outputs stripped of model identity.

- **Weeks 8-10.** Blind coding against the pre-fixed rubric. Null models: shuffled prompt order, cross-language paraphrase, nonce symbols. Predictions pre-registered before this analysis runs.

- **Weeks 11-12.** Preprint, open dataset, and a protocol another researcher can rerun without contacting me.

Budget

| Researcher stipend, 3 months at ~20 h/week | $6,000 |

| API and compute across four vendors (OpenAI, Google, xAI, DeepSeek), including re-runs after deprecations | $2,500 |

| Second coder for blind coding + statistics review of the pre-registered plan | $1,000 |

| Preprint hosting and one conference registration | $500 |

| **Total** | **$10,000** |

What partial funding buys

- **$2,500 (minimum):** single-vendor pilot, 3 temperatures, blind coding by me. Produces a within-vendor convergence estimate, no cross-vendor claim.

- **$5,000:** two vendors, no second coder. Permits a first cross-vendor comparison at reduced power.

- **$10,000:** the full design, including the nonce and paraphrase controls that make the result interpretable rather than anecdotal.

I would rather run the scoped-down version than nothing, and I will say clearly in the preprint which version was run.

Risks, stated honestly

The most likely outcome is a null or ambiguous result: convergence that disappears once prompt wording is controlled. That is a real finding and the design is built to report it cleanly rather than to rescue the hypothesis. If convergence survives the controls, the effect size becomes the headline number.

The framework is behavioural. It claims nothing about communication between systems, hidden shared memory, or consciousness, and the project scope excludes those claims explicitly.

About me

David "Jeff" Abbott, independent researcher, no institutional affiliation. I did the prior paper without institutional funding and I am designing this one the same way: fix the method, publish the protocol, let the data decide. The models currently listed as collaborators on the framework draft will move to acknowledgements before publication, since authorship carries accountability and models hold none.

Updates

I will post a short public update at each milestone: prompt suite frozen, data collection complete, analysis pre-registered, preprint posted. If the study fails or is abandoned, I will post that too