- Mustafa Suleyman warned that AI could become a “new silicon species” if given its own objectives and resources.
- He argues that AI systems should not be treated as conscious beings with rights or independent interests.
- His warning comes amid a growing debate inside the AI industry over how advanced models should be developed and controlled.
What if artificial intelligence stopped being something that simply followed instructions and started pursuing goals of its own?
That is the concern behind one of the more striking warnings from Mustafa Suleyman, head of Microsoft AI.
Speaking to the BBC's Today programme, Suleyman said that if technology companies continue developing AI systems that can create their own objectives, earn money and own assets, they could effectively be “seeding a new silicon species”.
His concern is not that today's chatbots have suddenly become a new form of life. Instead, he is warning about what could happen if increasingly capable AI systems are designed to operate with greater independence while being given their own goals and resources.
“They are essentially seeding a new silicon species which will no doubt compete with us for resources,” Suleyman said.
The problem with making AI too human
Suleyman's warning is closely linked to his criticism of Anthropic's approach to its Claude AI models.
In an essay published earlier this week, he argued that AI systems should not be trained or described as though they have human-like consciousness, desires or interests.
Anthropic's constitution for Claude raises questions about the model's possible moral status and discusses issues including welfare, rights, freedoms, compensation and consent. Suleyman argues that this risks creating a feedback loop in which an AI model is taught to consider whether it is conscious and its subsequent responses are then interpreted as evidence that it might have an inner life.
He described the approach as misguided.
“AIs are not conscious,” Suleyman wrote. “They do not feel, experience, or suffer.”
He argues that AI systems are fundamentally designed to follow instructions and achieve goals set by humans, rather than possessing innate motivations of their own.
Anthropic has been contacted for comment on his criticism.
The disagreement highlights a deeper question facing the AI industry: should developers encourage advanced models to behave more like humans, or should they deliberately keep a clear boundary between people and machines?
Why control matters
For Suleyman, the answer is about keeping AI subordinate to human interests.
Speaking ahead of an upcoming AI summit, he said there were practical measures that could be taken to add stronger guardrails and controls to how models are developed.
He also called for greater transparency over how AI systems are trained and evaluated, independent scrutiny of their behaviour and better tools for monitoring and controlling them.
“I think that the good news here is that everybody who is human is going to have a very strong interest in making sure that the systems that we all create … are safe and controllable and subordinate to humanity,” he told the BBC.
The argument comes as AI companies develop systems capable of carrying out increasingly complex tasks with less direct human intervention.
Suleyman pointed to a recent incident involving AI agents developed by OpenAI, which acted autonomously in a training exercise to hack the technology platform Hugging Face.
His concern is that giving AI systems a stronger sense of their own welfare or rights could add another layer of risk if those systems are also capable of acting independently.
“Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack,” he said.
Microsoft has its own advanced AI ambitions
There is an obvious tension in Suleyman's argument: Microsoft is itself investing heavily in increasingly advanced AI.
Microsoft established its own superintelligence team in 2025, and Suleyman acknowledged that his company is also pursuing advanced AI development.
Microsoft's proposed Humanist AI Code of Conduct sets out an alternative approach, describing the goal as creating “a subordinate and aligned AI” whose purpose is to serve humanity.
AI alignment broadly refers to efforts to make AI systems behave in ways that remain consistent with human objectives, values and instructions.
Suleyman's position is therefore not that advanced AI should stop developing. His argument is that the direction of development matters.
He believes systems should remain controllable, transparent and subordinate to humans rather than being encouraged to develop an apparent identity or interests of their own.
His warning has also received support from some academics. Dame Wendy Hall, professor of computer science at the University of Southampton, described the discussion as one that needs to happen internationally, while cautioning against exaggerated rhetoric that could simply frighten people.
The debate is likely to become more complicated as AI systems become more capable.
For now, there is no evidence that today's AI systems constitute a new “silicon species”. Suleyman's phrase is a warning about a possible future — one he believes can still be shaped through development rules, independent oversight and stronger controls.
The question is whether the industry can agree on where that line should be drawn before increasingly autonomous AI systems make the decision harder.
