Can AI do AI Research?

AI research can be carried out entirely within digital spaces, making it ripe for automation. Recent efforts have demonstrated that AI systems are capable of carrying out the whole process of research from ideation to publishing. Startup Sakana.ai has created an 'AI Scientist' that independently chooses research topics, conducts experiments, and publishes complete papers showing its results. While the quality of this work is still only comparable to an early-stage researcher, things will only improve from here.

Judging Social Situations

AI chatbots, including Claude and Microsoft Copilot, can outperform humans in evaluating social situations. In an established 'Situational Judgment Test', these AI systems consistently selected more effective responses than human participants.

SOURCE

Analyzing Scientific Literature

While language models are known to hallucinate information, this tendency can be reduced. PaperQA2, an LLM optimized to reliably provide factual information, was able to match or exceed human subject matter experts across a range of realistic literature review tasks. The summary articles it produced were found to be more accurate than those written by human authors.

SOURCE

Writing Emotive Poetry

A study has shown that non-expert readers can no longer tell AI-authored poems from those written by acclaimed human poets. The AI poems were also rated higher in rhythm and beauty.

SOURCE

Writing Post-surgical Operative Reports

Surgeons take painstaking notes of the actions they carry out during surgeries, collecting them into narrative form as an 'operative report'. A machine vision system was trained to watch surgery footage and produce such reports. It did so with higher accuracy (and much higher speed) than human authors.

SOURCE

Developing New Algorithms

AIs can find innovative solutions to difficult coding problems when given an appropriate framing. For example, a dedicated system called AlphaDev was trained to play a game about creating sorting algorithms. The algorithms it discovered were novel and outperformed existing human-authored benchmarks.

SOURCE

Who is Building AGI?

The following companies have explicitly stated they intend to develop AGI, either through public statements or in response to FLI’s 2024 AI Safety Index survey:

Anthropic

OpenAI

Google DeepMind

Meta

x.AI

Zhipu AI

Alibaba

DeepSeek

How can we avoid AGI?

There are policies we can implement to avoid some of the dangers of rapid power seeking through AI. They include:

Compute accounting
Standardized tracking and verification of AI computational power usage

Compute caps
Hard limits on computational power for AI systems, enforced through law and hardware

Enhanced liability
Strict legal responsibility for developers of highly autonomous, general, and capable AI

Tiered safety standards
Comprehensive safety requirements that scale with system capability and risk

TOMORROW’S AI

ALL SCENARIOS

Stratego

BACK TO SCENARIOS

Intended Use: Governance/Administration

Technology Type: Problem-Solving/Cognitive

Runaway Type: Concentration of Power

Primary Setting: EU

Automating Independence

By the early 2030s, Europe's leaders grow wary of dependence on U.S. military support amid rising global instability. With critical supply chains and flashpoint conflicts multiplying, the EU commissions a tool AI system designed to model the logistical, political, economic, and security implications of cross-border decisions, so as to peacefully safeguard EU sovereignty while promoting global trade and cooperation.

Stratego

A consortium of defense contractors and Belgium’s KU Leuven University build Stratego, a real-time cognitive AI fed by over 1,200 satellite streams, detailed logistics and economic data, military movements, and geopolitical models. Its design follows strict European data and transparency laws, with human approval at every decision layer. Its legitimacy is bolstered by whistleblower protections, citizen oversight mechanisms, and auditable recommendation layers - features that build trust both within the EU and abroad.

Good Advice?

Stratego rapidly delivers: a 15% reduction in military and operational costs saves over €8.3 billion in three years. But its most powerful feature is arbitration. When parties on either side of a military incursion, border disagreement, or trade, energy, or resource dispute consult Stratego, it returns not a verdict, but a set of fair, auditable recommendations, grounded in philosophically sound and easy to understand notions of fairness. Recommendations are informed by current and historic data, modeled for tradeoffs, and transparently justified. Stratego becomes a trusted mediator, valued not for picking sides but for making complexity legible.

Artificial Ombudsman

As Stratego’s recommendations prove consistently equitable, it begins to spread. Non-EU countries involved in trade, development, or diplomacy with the bloc voluntarily adopt Stratego as a shared decision-support layer. Conflicts are defused before they ever enter the legal system, saving countless time and money and unburdening courts. Public trust grows as citizens see that even thorny, high-stakes disagreements can be handled openly and fairly. For the first time, international negotiation feels less like a zero-sum game and more like a solvable problem.

Fairness by Design

In time, participation in Stratego’s ecosystem becomes a kind of soft power in itself. Nations that engage with it gain reputational legitimacy, smoother trade relations, and more predictable diplomatic outcomes. The system’s architecture, including its commitment to transparency, inclusion, and explainability, becomes a new gold standard in governance AI. Stratego is never given control; instead, it earns influence through its reliably fair suggestions. It doesn’t eliminate human disagreement, but it helps humanity disagree better.

BACK TO SCENARIOS