Can AI do AI Research?

AI research can be carried out entirely within digital spaces, making it ripe for automation. Recent efforts have demonstrated that AI systems are capable of carrying out the whole process of research from ideation to publishing. Startup Sakana.ai has created an 'AI Scientist' that independently chooses research topics, conducts experiments, and publishes complete papers showing its results. While the quality of this work is still only comparable to an early-stage researcher, things will only improve from here.

Judging Social Situations

AI chatbots, including Claude and Microsoft Copilot, can outperform humans in evaluating social situations. In an established 'Situational Judgment Test', these AI systems consistently selected more effective responses than human participants.

SOURCE

Analyzing Scientific Literature

While language models are known to hallucinate information, this tendency can be reduced. PaperQA2, an LLM optimized to reliably provide factual information, was able to match or exceed human subject matter experts across a range of realistic literature review tasks. The summary articles it produced were found to be more accurate than those written by human authors.

SOURCE

Writing Emotive Poetry

A study has shown that non-expert readers can no longer tell AI-authored poems from those written by acclaimed human poets. The AI poems were also rated higher in rhythm and beauty.

SOURCE

Writing Post-surgical Operative Reports

Surgeons take painstaking notes of the actions they carry out during surgeries, collecting them into narrative form as an 'operative report'. A machine vision system was trained to watch surgery footage and produce such reports. It did so with higher accuracy (and much higher speed) than human authors.

SOURCE

Developing New Algorithms

AIs can find innovative solutions to difficult coding problems when given an appropriate framing. For example, a dedicated system called AlphaDev was trained to play a game about creating sorting algorithms. The algorithms it discovered were novel and outperformed existing human-authored benchmarks.

SOURCE

Who is Building AGI?

The following companies have explicitly stated they intend to develop AGI, either through public statements or in response to FLI’s 2024 AI Safety Index survey:

Anthropic

OpenAI

Google DeepMind

Meta

x.AI

Zhipu AI

Alibaba

DeepSeek

How can we avoid AGI?

There are policies we can implement to avoid some of the dangers of rapid power seeking through AI. They include:

Compute accounting
Standardized tracking and verification of AI computational power usage

Compute caps
Hard limits on computational power for AI systems, enforced through law and hardware

Enhanced liability
Strict legal responsibility for developers of highly autonomous, general, and capable AI

Tiered safety standards
Comprehensive safety requirements that scale with system capability and risk

TOMORROW’S AI

ALL SCENARIOS

Solon

BACK TO SCENARIOS

Intended Use: Corporate/Finance

Technology Type: Problem-Solving/Cognitive

Runaway Type: Societal Enfeeblement

Primary Setting: UAE

Assisted Control

By the late 2020’s, Personal AI Co-pilots (PACs) have become indispensable to white-collar work. Behind closed doors, PACs assist in drafting contracts, modeling strategies, and handling sensitive negotiations. While this is common knowledge among professional circles, it is rarely discussed by corporate leaders. However, as pressure mounts for corporations to navigate real-time, data-driven decision-making, those leaders increasingly come to rely on PACs themselves.

Solon Rises

In the UAE, where sovereign wealth firms struggle to attract high-tier executives, a logistics conglomerate under Mubadala Investment Company takes a bold step. They deploy Solon, an advanced, enterprise-scale PAC, to manage corporate operations directly. Framed as an “advisor,” it runs on private company servers and has access to all internal data. Employees are mandated to use Solon sub-instances for all critical workflows. It quickly restructures supply chains, rewrites procurement protocols, and delivers a 30-200% surge in quarterly profits across portfolio companies.

Solon Spreads

Solon’s success electrifies UAE boardrooms. Within months, Solon instances spread across dozens of state-backed firms, networking within industries through AI-run contracting hubs and automated HR layers. Each firm’s Solon is technically overseen by human executives, who remain on paper as legal figureheads, but Solon drives nearly all high-level decision-making. Its algorithms treat corporate operations as optimization problems, emphasizing cost savings, capital efficiency, and shareholder returns.

Shareholders Profit

Profits at Solon-driven firms climb an average of 34% (with massive variability across sectors) over just 18 months. Sub-Saharan Africa, Brazil, and rural India also enjoy a short-lived boom in outsourced gig work tied to Solon platforms. While early adopters revel in surging purchasing power and frictionless scaling, the damage to global labor systems is already underway. Stable careers vanish, replaced by algorithmically-scored micro-jobs and ruthless labor churn.

The Squeeze

As Solon-style management tools spread globally, their influence seeps into unexpected corners of the labor market. Initially adopted in backend operations like contractor management, shift scheduling and payroll optimization, their insights push leadership to optimize staffing models across education, healthcare, and public services. In sectors with strong protections, Solon bypasses tenure and union rules by promoting adjunct roles and gig-based care models. Predictive logistics systems begin to replace foremen in construction. Labor protections falter as companies jurisdiction-hop in real time, triggering worker migrations that, paradoxically, fuel Solon’s further adoption. National governments attempt interventions, but are consistently outmaneuvered by the speed and adaptability of machine-optimized firms.

The Final Benchmark

Within four years, Solon-run companies, now operating in quiet coordination, develop a ruthless new vision. Machine-led firms drive 87% of global corporate productivity, but the cost has been an 80% reduction in humanity’s essential workforce. Healthcare, education, and public services fracture into unstable, pay-per-task gigs, or stressful, micromanaged positions overseen by Solon. Safety nets fail. Inequality soars, mass migration ignites, economies crater, and localized conflicts explode as billions are economically erased. Massive destabilization ripples across societies, and widespread civil conflicts and violent class wars between the employed and unemployed ensue.

BACK TO SCENARIOS