Update methodology confidence summary with proxy metrics
4 categories moved from "Unquantifiable/No" to "Proxy": cognitive deskilling, code quality degradation, data pollution, algorithmic monoculture. Added explanation of what each proxy measures and its limitations. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
parent
af6062c1f9
commit
1b8f9a165e
2 changed files with 27 additions and 9 deletions
|
|
@ -645,23 +645,41 @@ demand signal that drives system-level growth.
|
||||||
| Grid displacement | Low | 2-5x | No |
|
| Grid displacement | Low | 2-5x | No |
|
||||||
| Community impacts | Very low | Unquantifiable | No |
|
| Community impacts | Very low | Unquantifiable | No |
|
||||||
| Annotation labor | Very low | Unquantifiable | No |
|
| Annotation labor | Very low | Unquantifiable | No |
|
||||||
| Cognitive deskilling | Very low | Unquantifiable | No |
|
| Cognitive deskilling | Very low | Unquantifiable | Proxy |
|
||||||
| Linguistic homogenization | Very low | Unquantifiable | No |
|
| Linguistic homogenization | Very low | Unquantifiable | No |
|
||||||
| Code quality degradation | Low | Variable | Partly |
|
| Code quality degradation | Low | Variable | Proxy |
|
||||||
| Data pollution / model collapse | Very low | Unquantifiable | No |
|
| Data pollution / model collapse | Very low | Unquantifiable | Proxy |
|
||||||
| Scientific integrity | Very low | Unquantifiable | No |
|
| Scientific integrity | Very low | Unquantifiable | No |
|
||||||
| Algorithmic monoculture | Very low | Unquantifiable | No |
|
| Algorithmic monoculture | Very low | Unquantifiable | Proxy |
|
||||||
| Creative market displacement | Very low | Unquantifiable | No |
|
| Creative market displacement | Very low | Unquantifiable | No |
|
||||||
| Political cost | Very low | Unquantifiable | No |
|
| Political cost | Very low | Unquantifiable | No |
|
||||||
| Content filtering (opacity) | Medium | Unquantifiable | No |
|
| Content filtering (opacity) | Medium | Unquantifiable | No |
|
||||||
| Jevons paradox (systemic) | Low | Fundamental | No |
|
| Jevons paradox (systemic) | Low | Fundamental | No |
|
||||||
|
|
||||||
|
**Proxy metrics** (marked "Proxy" above): These categories cannot be
|
||||||
|
directly quantified per conversation, but the impact toolkit now tracks
|
||||||
|
measurable proxies:
|
||||||
|
- **Cognitive deskilling**: Automation ratio (AI output tokens / total
|
||||||
|
tokens). High ratio = more delegation, higher deskilling risk.
|
||||||
|
- **Code quality degradation**: Test pass/fail counts and file churn
|
||||||
|
(edits per unique file). High churn or failures = more rework.
|
||||||
|
- **Data pollution / model collapse**: Public push flag — detects when
|
||||||
|
AI-generated code is pushed to a public repository.
|
||||||
|
- **Algorithmic monoculture**: Model ID logged per session, enabling
|
||||||
|
provider concentration analysis over time.
|
||||||
|
|
||||||
|
These proxies are crude — a high automation ratio does not prove
|
||||||
|
deskilling, and a public push does not prove pollution. But they make
|
||||||
|
the costs visible and trackable rather than purely abstract.
|
||||||
|
|
||||||
**Overall assessment:** Of the 20+ cost categories identified, only 6
|
**Overall assessment:** Of the 20+ cost categories identified, only 6
|
||||||
can be quantified with any confidence (inference energy, PUE, grid
|
can be quantified with any confidence (inference energy, PUE, grid
|
||||||
intensity, client energy, financial cost, water). The remaining categories
|
intensity, client energy, financial cost, water). Four more now have
|
||||||
resist quantification — not because they are small, but because they are
|
proxy metrics that capture a measurable signal, even if indirect. The
|
||||||
diffuse, systemic, or involve incommensurable values (human rights,
|
remaining categories resist quantification — not because they are small,
|
||||||
cognitive autonomy, cultural diversity, democratic governance).
|
but because they are diffuse, systemic, or involve incommensurable
|
||||||
|
values (human rights, cognitive autonomy, cultural diversity, democratic
|
||||||
|
governance).
|
||||||
|
|
||||||
A methodology that only counts what it can measure will systematically
|
A methodology that only counts what it can measure will systematically
|
||||||
undercount the true cost. The quantifiable costs are almost certainly the
|
undercount the true cost. The quantifiable costs are almost certainly the
|
||||||
|
|
|
||||||
|
|
@ -35,7 +35,7 @@ separately as handoffs.
|
||||||
| 22 | Add file churn metric to hook | quantify-social-costs | DONE | `unique_files_edited` and `total_file_edits` in JSONL log |
|
| 22 | Add file churn metric to hook | quantify-social-costs | DONE | `unique_files_edited` and `total_file_edits` in JSONL log |
|
||||||
| 23 | Add public push flag to hook | quantify-social-costs | DONE | `has_public_push` flag in JSONL log |
|
| 23 | Add public push flag to hook | quantify-social-costs | DONE | `has_public_push` flag in JSONL log |
|
||||||
| 24 | Update show-impact.sh for new fields | quantify-social-costs | DONE | Social cost proxies displayed in impact viewer |
|
| 24 | Update show-impact.sh for new fields | quantify-social-costs | DONE | Social cost proxies displayed in impact viewer |
|
||||||
| 25 | Update methodology confidence summary | quantify-social-costs | TODO | Move proxied categories from "Unquantifiable" to "Proxy available" |
|
| 25 | Update methodology confidence summary | quantify-social-costs | DONE | 4 categories moved to "Proxy", explanation added |
|
||||||
|
|
||||||
## Handoffs
|
## Handoffs
|
||||||
|
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue