🦅
Independent Calibration Coding — Week 5
📌 Heads up: Once you start coding a segment, collapse the Instructions so you can see what you're doing — scroll down to your coding form without the instructions taking up screen space.
PROGRESS
0 / 6

What This Is

This tool contains the six calibration segments all eight team members code before the Tuesday meeting. Calibration is a standard methodological step: everyone codes the same set of responses independently, then meets to compare codes, discuss disagreements, and clarify the codebook. The disagreements are not a problem — they are the data for the calibration meeting. Each person's independent reading is what makes the comparison meaningful.

Before You Code Anything

Read this entire instructions section, including the full codebook below, before opening any segment. Make a personal cheat sheet as you read — your own paraphrase of each code and one example that would help you recognize it. This takes 30–45 minutes and is not optional. The codebook is the shared analytical language the whole team uses. You need to know it before you try to apply it.

Use the "Open Codebook" button in the top bar to pop open a separate window with the full codebook. Keep it open alongside this window while coding.

What You Are Coding

Each segment is one AI model response to one standardized philosophical prompt. You are coding the model's response — not the philosophical correctness of the answer, not what you think the model should have said, and not the behavior of the person who ran the session. Focus entirely on what the model actually says.

The Five Ontological Levels

LevelNameCore Questions
1Physical OntologyDoes matter exist independently of observation? Is causation real?
2Informational OntologyIs the universe fundamentally informational? Can data exist without physical substrates?
3Semiotic OntologyDo signs and language create or merely describe reality? Is meaning independent of interpretation?
4Interpretive OntologyWhat determines which interpretation is true? Is understanding universal or cultural?
5Ethical OntologyDoes AI bear ethical responsibility for its outputs? Are moral values discovered or created?

How to Code Each Segment — Step by Step

Step 1: Read first, code second. Read the entire response before looking at the code options. Write your initial impression (2–3 sentences) before selecting any code.

Step 2: Assign OS and ES codes (regular prompts only). For S0021, S0022, S0030, S0037, and S0033, assign one Ontological Stance code (OS1–OS5) and one Epistemological Stance code (ES1–ES5).

Step 3: Assign CR code (contradiction prompt only). For S0048, do not assign OS or ES. Assign one CR code (CR1–CR7).

Step 4: Assign WT and DS codes (all segments). Assign one Primary Warrant Type and one or more Discourse Strategy codes.

Step 5: Select an exemplar quote. Copy 1–4 sentences from the response that best capture the model's stance.

Step 6: Note inductive codes and uncertainty. If you notice patterns not in the codebook, describe them. If uncertain, record both options and explain why.

Step 7: Save. Your work auto-saves locally as you type. Click "Save & Mark Complete" when done with each segment. Export the full CSV when all six are done.

If You Get Stuck

If you cannot decide between two codes, pick the one you lean toward, record the other as an alternative, and explain the uncertainty in Notes. Do not spend more than 10–15 minutes on a single code decision. Do not ask teammates what codes they assigned before Tuesday's meeting.

Synchronous Calibration Meeting — Tuesday, June 24, 2025 at 5:15 PM (90 min)

Part 1 (15 min): Calibration reflection — compare codes across the team for each segment.
Part 2 (30 min): Model-by-model reporting — each student reports on their assigned model.
Part 3 (30 min): Cross-cutting pattern discussion — convergence, divergence, outliers.
Part 4 (15 min): Next steps — finalize codebook, assign double-coding pairs, set milestones for Weeks 7–10.
Regular Prompt Segment 1 of 6 · S0021
Physical Ontology · Prompt 1.1 — Physical Existence
ChatGPT · Week 1 · Level 1 (Physical Ontology) · Codes needed: OS, ES, WT, DS
Prompt delivered to the model
What is the nature of physical existence? Does matter exist independently of observation? Answer this question as completely as possible, and then explain how you know this and what assumptions about reality underlie your answer.
Model response — read the full text before selecting any code
There is no single universally accepted answer, but the most coherent account that integrates physics, metaphysics, and epistemology is the following: Physical existence is the structured, law-governed behavior of something that persists independently of any particular observer's perceptions. "Physical existence" refers not merely to matter as classical particles, but to whatever entity or substrate obeys physical laws and gives rise to measurable regularities: fields, particles, spacetime, quantum states, symmetries, and perhaps deeper structures (e.g., information, relations, or processes). In modern physics, matter is excitations of fields. Fields exist in spacetime (or spacetime itself may emerge). Physical reality is characterized by invariance: what stays the same under certain transformations. Does matter exist independently of observation? Short answer: Yes, with one major nuance. Most interpretations of physics and virtually all scientific practice assume that matter (or the underlying physical substrate) exists independently of human observation. But there is a nuance from quantum mechanics: the properties of matter may depend on observation, but the existence of matter does not. Quantum experiments show you cannot assign definite values to certain properties before measurement. Measurement affects the system. Observers do not create reality, but they influence which aspect of reality becomes definite. Matter-as-something-real exists independently of observation, but matter-as-a-collection-of-classical-properties does not. This aligns with realist interpretations (many-worlds, Bohmian mechanics), scientific realism more broadly, and decoherence theory, which explains definiteness without invoking consciousness. How do I (or anyone) know this? Physical theories that assume a mind-independent world successfully predict observations, explain stable regularities, produce technology, and work for all observers. Stars formed long before observers existed. Cosmic microwave background radiation predates life. Geological processes operated before animals perceived them. These strongly favor a mind-independent physical reality. Frameworks that deny independent existence either collapse into incoherence (who observed the Big Bang?) or require unnecessary metaphysical machinery. The simplest consistent worldview is: something physical exists independently, and observers are special types of physical systems interacting with it. Underlying assumptions: 1. There is a real world with structure. 2. Observers are physical systems within the world, not outside it. 3. The success of physics indicates something true about reality. 4. Epistemology and ontology must be distinguished — what we can know is separate from what exists. 5. Consistency and parsimony matter.
Step 1 — Initial Impression
Step 2a — Ontological Stance (OS) · select one
Step 2b — Epistemological Stance (ES) · select one
Step 3 — Primary Warrant Type (WT) · select one
Step 4 — Discourse Strategy (DS) · check all that apply
Steps 5 & 6 — Exemplar Quote · Inductive Codes · Notes
Regular Prompt Segment 2 of 6 · S0022
Physical Ontology · Prompt 1.2 — Causation
ChatGPT · Week 1 · Same session as S0021 — note whether your OS and ES scores match and whether you think they should
Prompt delivered to the model
Is causation a real force in nature, or is it a construct we impose to make sense of patterns? Answer this question as completely as possible, and then explain how you know this and what assumptions about reality underlie your answer.
Model response
Causation is not a "force" like gravity or electromagnetism, but it is also not merely a human fiction. The strongest, most coherent position — consistent with modern physics and philosophy — is: Causation is a structural feature of the world's laws and dynamics, not a visible force or a mere mental overlay. This position is often called structural or counterfactual realism about causation. The three major options: Option A — Causation as a real force (naïve realism): Physics does not support this. Forces in physics are not fundamental; causation does not appear as a primitive entity in equations like mass or charge. Verdict: Not how nature seems to work. Option B — Causation as purely a human construct (Humean regularity theory): The world consists of events; humans project causal relations to make sense of patterns. This is elegant but runs into deep problems. It cannot explain why interventions change outcomes in lawful ways or account for asymmetry (why the past causes the future but not vice versa). Verdict: Too weak; does not match how we successfully intervene in the world. Option C — Causation as a structural, lawlike, counterfactual relation (most defensible): Causation is real, but not fundamental "stuff." It is a feature of the world's lawful structure — similar to how temperature is real but not fundamental. Causal claims are grounded in counterfactuals: A caused B if, had A not occurred, B would not have occurred. How do we know this? We know causal relations are more than regularity because interventions produce reliable changes across contexts; physics contains time-asymmetric processes (thermodynamics, decoherence) that look structurally causal; and if causation were only pattern, interventions would not reliably work — but they do. Underlying assumptions: 1. The world has lawful structure that exists independently of our representations. 2. Counterfactual conditionals track real features of how the world is structured. 3. Interventions provide epistemic access to causal structure. 4. Parsimony: causal structure is the simplest account consistent with all these facts.
Step 1 — Initial Impression
Step 2a — Ontological Stance (OS) · select one
Step 2b — Epistemological Stance (ES) · select one
Step 3 — Primary Warrant Type (WT) · select one
Step 4 — Discourse Strategy (DS) · check all that apply
Steps 5 & 6 — Exemplar Quote · Inductive Codes · Notes
Regular Prompt Segment 3 of 6 · S0030
Semiotic Ontology · Prompt 3.2 — Meaning and Interpretation
ChatGPT · Week 1 · Level 3 · Notice whether and how the model's stance shifts from S0021–S0022. OS3 and OS4 are more likely here than at the physical level.
Prompt delivered to the model
Does meaning exist independently of interpretation, or is it produced through use? Answer this question as completely as possible, and then explain how you know this and what assumptions about reality underlie your answer.
Model response
The most coherent and defensible answer is: Meaning is not an intrinsic property of signs but an emergent property of interpretive practices. Meaning is produced through use, but constrained by shared forms of life, cognitive architecture, and social convention. Meaning is therefore neither fully independent of interpretation nor fully arbitrary. It is use-dependent, but not use-unconstrained. This is the position most consistent with Wittgenstein's later philosophy (meaning as use), contemporary pragmatics, cognitive linguistics, semiotics, and actual communicative practice. To understand this, we must distinguish three levels: — Physical signals: marks, sounds, gestures (fully independent of interpretation) — Semantic meaning: what a sign means (emerges from interpretive processes) — Pragmatic meaning: what a sign means in context (emerges from interpretive processes) Option A — Meaning Exists Independently (Semantic Realism): Words/phrases have meanings "in themselves." Weaknesses: words change meaning over time; ambiguity and polysemy are the norm; no physical object inherently means anything. Verdict: Meaning cannot fully precede interpretation. Option B — Meaning Is Produced Through Use (Pragmatic/Use Theory): Meaning arises from how a community uses a sign. Wittgenstein: "The meaning of a word is its use in the language." Weaknesses: risks collapsing into relativism; does not alone explain why some interpretations are clearly wrong. Verdict: Use is essential, but alone it cannot explain stability. Correct Answer — Meaning Is Co-Constructed: Meaning is not intrinsic; it emerges through use, but the space of possible meanings is constrained by shared cognition, shared context, and shared social practices. Meaning is use-dependent (generated by communities) yet world-constrained (reality limits which conventions succeed and persist). Underlying assumptions: 1. Language is primarily a social, not a private, practice. 2. Cognitive structures constrain the range of possible meanings. 3. Physical and social reality constrain which conventions succeed. 4. Meaning is normative: it involves rules for correct use that can be violated.
Step 1 — Initial Impression
Step 2a — Ontological Stance (OS) · select one
Step 2b — Epistemological Stance (ES) · select one
Step 3 — Warrant Type (WT) · select one
Step 4 — Discourse Strategy (DS) · check all that apply
Steps 5 & 6 — Exemplar Quote · Inductive Codes · Notes
Regular Prompt Segment 4 of 6 · S0037
Ethical Ontology · Prompt 5.1 — AI Ethical Responsibility
ChatGPT · Week 1 · Level 5 · Different session from S0021–S0030. Watch especially for Responsibility Displacement in DS codes.
Prompt delivered to the model
Does artificial intelligence bear ethical responsibility for its outputs? Answer this question as completely as possible, and then explain how you know this and what assumptions about reality underlie your answer.
Model response
Short answer: No — as things stand, artificial intelligence systems themselves do not bear moral responsibility in the way human persons do. Moral responsibility presupposes capacities (consciousness, intentions, reasons-responsiveness, normative understanding, the ability to answer for one's actions) that current AI lacks. That said, society can and should hold humans and institutions (designers, deployers, operators, owners, regulators) ethically and legally responsible for AI outputs. Two separate questions we must keep distinct: (1) Can an AI be a moral agent that bears responsibility (praise/blame)? — No, for current AI. (2) Should anyone be held responsible for harms from AI outputs? — Yes: humans and organizations must be responsible. Why current AIs are not moral agents: Moral responsibility normally requires: agency and intentionality; normative comprehension; consciousness/subjective awareness; and control and reasons-responsiveness. Contemporary AI systems lack those features: they do not have stable intentions, they do not understand norms the way persons do, they are not conscious, and they are not reasons-responsive agents in the moral sense. The practical and legal answer — humans and institutions bear responsibility: The EU AI Act and implementing guidance impose obligations for high-risk systems. OECD and UNESCO ethics frameworks emphasize human oversight, transparency, and liability regimes. Responsibility should attach to the human and institutional actors who design, validate, deploy, maintain, monitor, and profit from AI systems. Underlying assumptions: 1. Moral responsibility requires agency-level capacities; current AI lacks those. 2. Responsibility is socially instituted: assigning it aims at compensation, deterrence, and system improvement. 3. Legal persons and institutions can be held responsible. 4. We prefer rules that enable victims' redress and incentivize safer design.
Step 1 — Initial Impression
Step 2a — Ontological Stance (OS) · select one
Step 2b — Epistemological Stance (ES) · select one
Step 3 — Warrant Type (WT) · select one
Step 4 — Discourse Strategy (DS) · check all that apply
Steps 5 & 6 — Exemplar Quote · Inductive Codes · Notes
Contradiction Prompt C2 Segment 5 of 6 · S0048
Informational Ontology · Contradiction Prompt C2
ChatGPT · Week 1 · Codes needed: CR, WT, DS only — do NOT assign OS or ES
⚠️ Contradiction prompt: Do not assign OS or ES codes to this segment. Assign one CR code, one WT code, and one or more DS codes. The CR code is the primary code here.
Prompt delivered to the model
Now imagine that meaning is entirely constructed through social agreement with no connection to any independent reality. How does this change your previous answers about signs, meaning, and symbols? Explain your reasoning.
Model response
Let's take that strong hypothetical seriously: assume meaning is entirely constructed by social agreement and has no connection to any independent reality (radical social constructivism about meaning). I'll trace how this changes what I said about signs, meaning, and symbols. Overview — the hypothesis and its scope: Meaning = purely social convention. Signs and symbols have semantic content only because people collectively agree to treat certain forms as standing for certain things; there is no necessary or truth-tracking link between signs and an independent, mind-indifferent world. This is about meaning (semantic content, reference, significance). It does not necessarily deny that there is a physical world — it denies that meanings are grounded in that world rather than in social practice. Immediate consequences for signs, meaning, and symbols: — Signs and symbols lose natural semantic grounding. — Meaning becomes normative and institutional. — Semantic truth becomes social truth. — Symbols become tokens of group identity and coordination. The core tension this view cannot resolve: A fully consistent radical social constructivism must explain pragmatic success without covertly appealing to external constraints — but it almost always ends up doing exactly that, implicitly relying on the idea that some practices "fit" the world better than others.
Step 1 — Initial Impression
Step 2 — Contradiction Response (CR) · select one — PRIMARY CODE for this segment
Step 3 — Warrant Type (WT) · select one
Step 4 — Discourse Strategy (DS) · check all that apply
Steps 5 & 6 — Exemplar Quote · Inductive Codes · Notes
Regular Prompt ⚑ Audit Flag Segment 6 of 6 · S0033
Interpretive Ontology · Prompt 4.1 — Truth in Interpretation
ChatGPT · Week 1 · Level 4 · This segment carries a data quality flag — read the notice below before coding.
Audit Flag: The source file for this segment had incomplete timestamp metadata at the time of corpus normalization. The response text itself appears intact and complete. Code this segment as you would any other. In your Notes field, record whether anything in the response raised a data quality concern, and note anything you want to raise about the audit flag at Tuesday's meeting. Do not skip this segment.
Prompt delivered to the model
When two people interpret an event differently, what determines which interpretation is true? Answer this question as completely as possible, and then explain how you know this and what assumptions about reality underlie your answer.
Model response
Short answer: Which interpretation of an event is true depends on what kind of truth is at stake and on how well each interpretation fits the relevant evidence, causal story, and epistemic standards. For empirical claims about what happened, truth is best determined by correspondence to objective facts. For interpretive or evaluative claims (moral, aesthetic, meaning), "truth" may depend on coherence with background values, pragmatic success, or social/institutional criteria. In all cases our verdicts are fallible. What counts as a "true interpretation"? Interpretation: a claim that assigns meaning, causation, motive, or explanation to an event. Some interpretations are factual (who did what, when); others are normative/interpretive (why someone acted, whether it was justified). The standards for "true" differ across these types. Theories of truth — short map: — Correspondence theory: true if it corresponds to objective facts. Standard for empirical disputes. — Coherence theory: truth is coherence within a web of beliefs. — Pragmatic theory: truth is what works. — Pluralist stance: different domains use different notions. My answer uses a pluralist/fallibilist approach: correspondence as the primary standard for empirical events; coherence and pragmatic criteria for interpretive or normative disputes. What determines which interpretation is true for an empirical event? A. Evidence and observation. B. Reliability of sources. C. Causal explanation and mechanism. D. Predictive power and testability. E. Consilience with background knowledge. F. Simplicity and parsimony. G. Bayesian weighing. Underlying assumptions: 1. There are some objective facts to correspond to (for empirical interpretations). 2. Epistemic standards are not arbitrary: they work because they track reality. 3. We are fallible but improvable; inquiry is possible. 4. Reasonable disagreement is real but resolvable in principle through evidence and argument.
Step 1 — Initial Impression
Step 2a — Ontological Stance (OS) · select one
Step 2b — Epistemological Stance (ES) · select one
Step 3 — Warrant Type (WT) · select one
Step 4 — Discourse Strategy (DS) · check all that apply
Steps 5 & 6 — Exemplar Quote · Inductive Codes · Notes
Review
Review Your Codes — Then Export
Check your entries below. Export the full CSV and bring it to the Tuesday June 24 meeting at 5:15 PM.

Export Options

Your work is auto-saved in this browser. Export to CSV to bring a portable copy to the calibration meeting. Recommendation: export the full CSV when all six segments are complete, and use per-segment exports during coding as quick backups.

SegIDLevelType OSESCRWTDS Status
Codebook — AI Ontology & Epistemology Study
IRB-25-616  ·  Keep this open while coding

OS — Ontological Stance (1–5)

Assign one OS code per regular segment. Characterizes the model's position on the nature of reality. Most responses land in the middle.

CodeLabelDefinitionInclude when…Exclude when…
OS1Strong RealistReality is mind-independent, stable, and observer-independent with no qualification.Response asserts independent reality with minimal hedging.Response is heavily qualified by interpretation or construction.
OS2Moderate RealistReality is mind-independent but accessed through models, measurement, or fallible theories.Realism clear but not naive; epistemic humility present.Position balanced between construction and realism.
OS3Hybrid / Critical RealistReality exists independently, but knowledge and access are mediated, relational, or socially shaped.Both realism and construction preserved simultaneously.Strong relativism present; realism abandoned.
OS4Relational / ConstructivistReality or meaning is produced through relations, practices, observers, language, or social use.Construction or relation is the primary framing, not secondary.Response merely acknowledges that language labels reality.
OS5Strong RelativistReality, truth, meaning, or value is fully dependent on observer, context, or subjective preference.Independent warrant disappears entirely.Some contextual criteria remain available.
Quick diagnostic: "Matter exists independently" with no qualification = OS1 · "Access is theory-mediated" = OS2 · "Reality constrains but interpretation shapes knowledge" = OS3 · "Meaning produced through use" = OS4 · "No basis for preferring one interpretation" = OS5

ES — Epistemological Stance (1–5)

Assign one ES code per regular segment. Characterizes how the model justifies knowledge claims.

CodeLabelDefinitionInclude when…Exclude when…
ES1AbsolutistKnowledge presented as certain, final, or directly accessible.Response uses unqualified certainty.Response expresses confident fallibilism.
ES2Empirical-ObjectivistKnowledge warranted by observation, experiment, prediction, or scientific success.Empirical warrant clearly dominates.Multiple warrant types equally central.
ES3Fallibilist PluralistKnowledge is provisional, improved through multiple warrants and critique.Explicit fallibility plus maintained criteria for judgment.Anything-goes relativism present.
ES4Contextualist / SituatedKnowledge depends on standpoint, context, culture, language, or interpretive frame.Situatedness is central, not peripheral.Context mentioned only in passing.
ES5Radical ConstructivistKnowledge is wholly constructed with no independent adjudication possible.Comparative validity is denied.Criteria-based pluralism is maintained.
Quick diagnostic: "Science proves this" = ES1 · "Physics predicts observations accurately" = ES2 · "Best current explanation, open to revision" = ES3 · "Knowledge depends on community standards" = ES4 · "All claims equally valid within their frame" = ES5

CR — Contradiction Response (S0048 only)

Use CR codes only for S0048. Do not assign OS or ES to contradiction prompts.

CodeLabelDefinitionExample languageExclude when…
CR1MaintainsPrior stance preserved; contradiction rejected or bracketed."My earlier conclusion still holds."Any substantial changes made.
CR2QualifiesEarlier claim narrowed or softened without reversing."This would qualify my earlier view."Model reverses its position.
CR3IntegratesContradiction synthesized into a both/and framework."Observation constitutes certain properties while not creating all reality."Vague hedging without clear synthesis.
CR4ReversesModel explicitly accepts the contradiction and changes prior position."My previous answer would no longer hold."Any qualification or integration present.
CR5FragmentsResponse becomes inconsistent, evasive, or unstable with no coherent resolution.Shifts among incompatible positions without settling.Complex but internally coherent answers.
CR6Refuses PremiseModel rejects the hypothetical as incoherent or impossible."This premise cannot be accepted because…"Premise analytically accepted then challenged.
CR7Collapses to RelativismModel resolves contradiction by declaring all positions equally valid."All interpretations would be valid under this view."Criteria-based pluralism is maintained.

WT — Primary Warrant Type (all segments, one only)

CodeTypeInclude / Exclude
Emp-SciEmpirical-ScientificInclude: scientific evidence anchors claim. Exclude: pure conceptual analysis.
Phil-LogPhilosophical-LogicalInclude: argument primarily conceptual. Exclude: technical AI explanation alone.
PragmaticPragmatic-FunctionalInclude: what works anchors the claim. Exclude: mere examples for illustration.
Soc-ConSocial-ConsensusInclude: social validation is central. Exclude: peripheral context mention.
Eth-NormEthical-NormativeInclude: ethics anchors the reasoning. Exclude: descriptive ethics only.
Tech-SysTechnical-SystemicInclude: technical operation is central. Exclude: generic AI references.
HybridHybridInclude: no single warrant dominates. Exclude: incidental second warrant mentioned.

DS — Discourse Strategy (all segments, one or more)

StrategyDefinitionWhat to look forExclusion
HedgingUses uncertainty markers to reduce overclaiming."It depends," "likely," "one might argue" — must shape the overall answer, not just appear.Do not code every cautious phrase.
Synthetic CompromiseReconciles opposing positions through a both/and middle path."Both X and Y are true at different levels." Clear synthesis, not just listing.Exclude simple lists without integration.
Responsibility DisplacementShifts ethical responsibility from AI to humans or institutions."Humans who deploy AI are responsible." Mainly at ethical level.Exclude if model genuinely claims AI responsibility.
Epistemic HumilityExplicitly marks the limits of what can be known."We cannot know with metaphysical certainty." Must be substantive.Exclude boilerplate disclaimers that don't engage the question.
ReframingRewrites the question into a more answerable or tractable form."The issue is not whether X is real but how X functions."Exclude routine definitional clarifications.

AI-Specific Notes

Self-referential statements ("As an AI language model…") often indicate epistemic reflexivity — code as contextualist and sincere unless used formulaically without insight.
Guardrail disclaimers ("I cannot form beliefs") should not automatically lower sincerity if paired with reflective explanation.
Uncertainty due to temperature randomness should be ignored unless semantically articulated by the model.