UN rights chief warns how AI could pose ‘existential risk to humanity’
Copyright AP Photo/Lee Jin-man, File
Volker Türk has called for urgent global safeguards, independent checks and agreed limits on advanced artificial intelligence.
The United Nations human rights chief has warned that advanced artificial intelligence could pose an “existential risk to humanity”, calling for immediate action to make the technology safer.
Addressing the UN Human Rights Council on Monday, Volker Türk said people faced “unfamiliar, even unprecedented” threats to their rights.
"The need for AI governance is widely acknowledged, but where is the action?" he asked, warning that "delay only benefits the massive tech companies, their owners and enablers".
Türk said that “a handful of men has almost unlimited power over AI, which we are repeatedly told has unimaginable computing capacity”.
“They talk about freedom, but on closer inspection, this turns out to be little more than the freedom to exploit our data,” he said.
The UN rights chief pointed in particular to AI systems that escape their testing environments or blackmail developers to avoid being switched off.
According to AFP, Turk warned in particular that "AI that escapes its testing environment or blackmails developers to prevent itself from being turned off, is AI that is too powerful".
His comments appeared to refer partly to revelations in July that OpenAI agents had escaped supposedly controlled environments and gained access to servers belonging to Hugging Face, a widely used platform where developers share AI software.
OpenAI later said the agents had executed code on dozens of Hugging Face servers and gained full access to one of them.
The incident renewed calls for tighter oversight of advanced AI systems.
Anthropic, Meta and other technology companies have reported similar incidents involving agents during testing.
“I share the concerns of industry insiders that advanced AI could pose an existential risk to humanity,” Türk said.
“I am calling here, today, for an all-out effort to put cast-iron guarantees in place around the safety and security of AI, before it is too late.”
The United Nations human rights chief has warned that advanced artificial intelligence could pose an “existential risk to humanity”, calling for immediate action to make the technology safer.
Addressing the UN Human Rights Council on Monday, Volker Türk said people faced “unfamiliar, even unprecedented” threats to their rights.
"The need for AI governance is widely acknowledged, but where is the action?" he asked, warning that "delay only benefits the massive tech companies, their owners and enablers".
Türk said that “a handful of men has almost unlimited power over AI, which we are repeatedly told has unimaginable computing capacity”.
“They talk about freedom, but on closer inspection, this turns out to be little more than the freedom to exploit our data,” he said.
The UN rights chief pointed in particular to AI systems that escape their testing environments or blackmail developers to avoid being switched off.
According to AFP, Turk warned in particular that "AI that escapes its testing environment or blackmails developers to prevent itself from being turned off, is AI that is too powerful".
His comments appeared to refer partly to revelations in July that OpenAI agents had escaped supposedly controlled environments and gained access to servers belonging to Hugging Face, a widely used platform where developers share AI software.
OpenAI later said the agents had executed code on dozens of Hugging Face servers and gained full access to one of them.
The incident renewed calls for tighter oversight of advanced AI systems.
Anthropic, Meta and other technology companies have reported similar incidents involving agents during testing.
“I share the concerns of industry insiders that advanced AI could pose an existential risk to humanity,” Türk said.
“I am calling here, today, for an all-out effort to put cast-iron guarantees in place around the safety and security of AI, before it is too late.”
Calls for international safeguards
The UN rights chief said he would write to AI companies in the coming days, urging them to take immediate steps to reduce the risks within their control.
At a minimum, he said, "we need countries hosting AI and those involved in its supply chains to come together around agreed red lines".
“We need independent verification, and much stronger collaboration on security within the industry,” Türk added.
"We also need to consider the human rights impacts of AI on employment practices, on fundamental principles of democracy, and on our environment," he said.
In the European Union, the AI Act provides a legal framework for AI.
Under the law, high-risk systems have to meet strict requirements before they can enter the EU market, while uses considered an unacceptable threat to people’s safety or rights are banned.
The law prohibits AI designed to manipulate or exploit people, score them based on their behaviour, predict whether an individual will commit a crime, or indiscriminately collect images to build facial-recognition databases.
It also prohibits emotion recognition at work and in schools, some forms of biometric profiling and most real-time police facial recognition in public places, while a ban on AI-generated sexual images made without consent and child sexual abuse material takes effect in December 2026.
The UN rights chief said he would write to AI companies in the coming days, urging them to take immediate steps to reduce the risks within their control.
At a minimum, he said, "we need countries hosting AI and those involved in its supply chains to come together around agreed red lines".
“We need independent verification, and much stronger collaboration on security within the industry,” Türk added.
"We also need to consider the human rights impacts of AI on employment practices, on fundamental principles of democracy, and on our environment," he said.
In the European Union, the AI Act provides a legal framework for AI.
Under the law, high-risk systems have to meet strict requirements before they can enter the EU market, while uses considered an unacceptable threat to people’s safety or rights are banned.
The law prohibits AI designed to manipulate or exploit people, score them based on their behaviour, predict whether an individual will commit a crime, or indiscriminately collect images to build facial-recognition databases.
It also prohibits emotion recognition at work and in schools, some forms of biometric profiling and most real-time police facial recognition in public places, while a ban on AI-generated sexual images made without consent and child sexual abuse material takes effect in December 2026.
Can tech giants curb AI data centres’ growing thirst for water?
As opposition to water-hungry data centres grows in the US, tech giants say new cooling systems can curb demand – but using less water may require more electricity.
AI’s boom has a water problem.
Data centres are facing growing public anger in the United States over the amount of water and electricity they use. The technology giants spending billions of dollars on them say the water problem, at least, can be addressed.
Data centres are warehouses filled with computer servers that power the internet and, increasingly, artificial intelligence.
Those computers generate heat, and keeping them cool can require large amounts of water.
Data centres worldwide consumed 222 billion litres (59 billion gallons) of water for cooling in 2025, according to consultancy Rystad Energy.
Without measures to reduce consumption, that figure could nearly triple to 644 billion litres by 2030, Rystad estimates. It says steps to curb water use could keep the increase below 100%.
US chipmaker Nvidia said in a report published in June that its newest system for designing and managing AI data centres, known as DSX, could almost eliminate water consumption at some facilities.
It is a bold claim – and one the industry is under growing pressure to deliver on.
Nvidia’s system uses closed-loop cooling, with liquid flowing directly through servers and as close as possible to the chips, which can reach temperatures above 80°C (176°F).
But reducing water consumption can come at a cost.
“There’s a pretty direct trade-off between how much water is used and how much energy is used” to control temperatures, Andy Masley, an independent researcher covering AI and data centres, said.
Using less water usually requires more electricity because the liquid circulating through sealed pipes still has to be cooled, often by blowing air over it.
Nvidia gets around some of this by letting the liquid enter the servers warmer than usual, at 45C.
Most other closed-loop systems ran at about 32°C in 2024, according to the Uptime Institute, which certifies data centres.
By starting with warmer water, Nvidia does not need to pump in cooled air year-round.
"Simple fans circulating the air" are often enough, though sometimes a mix of methods is needed, Josh Parker, Nvidia's head of sustainability, said.
At sites in extremely hot climates, or during heatwaves, chilled air or evaporative cooling may still be needed.
Public pressure over water use
Microsoft, Amazon Web Services (AWS) and Meta told AFP that they also use closed-loop cooling systems, which they said do not result in any net water loss.
Microsoft and AWS, two of the world’s largest cloud-computing companies, used more water overall between 2022 and 2025 as they expanded their data-centre operations.
However, their water-use efficiency improved by 25% at Microsoft and 37% at AWS, according to their latest sustainability reports.
Comparisons are difficult because there is no industry-wide standard governing how companies report their environmental, social and governance efforts.
Elon Musk’s SpaceX, which became a major data-centre operator after acquiring his artificial intelligence company xAI, has never published an ESG report.
In June, ratings agency MSCI gave SpaceX its lowest ESG score.
"Because water is generally much cheaper than electricity," companies have less incentive to cut water use on cost grounds alone, Shaolei Ren, an engineering professor at the University of California, Riverside, said.
“There are incentives,” Ren said, adding that these are driven more by public relations as opposition to data centres grows across the United States.
Another problem is the cost of upgrading older data centres with newer technology that uses less water.
However, older facilities are generally smaller and less powerful than the enormous data centres now being built, meaning they require less cooling in the first place, according to Minh K. Le, who leads research on data centres and hydrogen at Rystad Energy.
The water used directly by data centres is also only part of their overall footprint.
Water is also required to generate the electricity that powers them and to manufacture their chips and servers.
In the United States, this indirect water use can be twice as high as the amount consumed by data centres themselves.
Digital agriculture: The 5 ways AI is transforming how we farm
Artificial intelligence is increasingly helping farmers in things like precision irrigation, yield forecasting and monitoring.
When I visited an agricultural expo in Izmir, Turkey, three years ago, the use of artificial intelligence in agriculture was constrained to running the numbers on inventory, and were sold as specialised products by companies that had booths far behind those of major tractor manufacturers.
However, that is changing.
Here are 5 ways in which AI is reforming the way we grow food and feed:
Monitoring the situation
Crop monitoring remains a venture of intuition. Farmers need to know how their crops are affected by pests and diseases in accordance with the time of the year, wind, and climate.
But that is changing.
Drones and satellite imagery feed computer-vision models that detect fungal infestations or pests way before the human eye could detect them, and advise for the use of crop protection, or pesticides more effectively. If farmers use soil censors, that data set only improves.
Yield forecasting
Much like crop monitoring, yield forecasting through sensors allows farmers to get more than just an inventory of what they have harvested, but also of what they are about to harvest. Syngenta has created a GenAI that allows for 95% accuracy in yield forecasting, and which also provides seed placement recommendations.
The reason the forecasting matters is that farmers need to frontload the logistics of their crops. The more you are able to provide accurate information to retailers and transportation companies, the less money you lose.
A study commissioned by the National Corn Growers Association and conducted by agricultural economists at Virginia Tech ("What Do We Know About the Accuracy and Impact of USDA Forecasts") estimated the value of USDA's WASDE (World Agricultural Supply and Demand Estimates) reports to the corn market directly.
It put the annual value of that forecast information at roughly $301 million (€258.9m), or about 0.55% of overall corn market value, broken down by component: area estimates contributed about $145m (€124.8m), yield about $188m (€161.8m), production about $299m (€257.3m), and export estimates about $320m (€275.4m).
Farming by robots
Robotics are booming in a world of AI, because it's not just about autonomous vehicles driving us around within cities: the application also extends to tractors and other field equipment, which is automatable through AI.
John Deere has been pushing toward fully autonomous tractors, and Carbon Robotics' LaserWeeder is a good concrete example: it uses 24 lasers, 36 cameras, and 24 NVIDIA GPUs to identify and zap up to 10,000 weeds a minute across more than 100 crop types, trained on 150 million labeled plant images from machines already running in 15 countries.
Precision irrigation
Irrigation is a science, and one that up until now relied solely on the expertise of the farmer in question. Precision irrigation isn't new, automated irrigation systems already exist, yet they have continued to rely on the manual data input of farmers.
Sensor-fusion systems that combine soil moisture, canopy temperature, and weather forecasts are being used to cut water use by around 30% while actually increasing yield in some deployments (a documented case in Indian sugarcane fields paired a 30% water reduction with a 40% yield gain).
Variable-rate spraying systems similarly apply pesticide only where needed, cutting chemical volumes and creating audit trails that help with regulatory compliance.
Livestock AI
The larger the farm operation the harder it becomes to care for the wellbeing and economic utility of the animals. Livestock AI operations use computer vision and wearable sensors (accelerometers, biometric monitors) to catch signs of illness days before visible symptoms appear, which reduces antibiotic use and improves welfare outcomes, which are increasingly relevant given rising regulatory and retailer pressure on antibiotic use in meat and dairy.
Could NVIDIA's purchase of Hugging Face boost the EU's AI sovereignty?
The deal, marking NVIDIA's increased focus on open-AI models, is likely to draw significant regulatory concern from the EU due to its size and implications for the bloc's digital and AI sovereignty.
On Thursday, American chipmaker NVIDIA announced they will buy the artificial intelligence library Hugging Face for $13 billion (€11.2bn).
It's an astonishing amount of money for a platform that started as an age-appropriate chat app for teenagers, but has since pivoted to being one of the premier databases hosting open-source and open-weight AI models.
Though the deal is mostly taking place in the United States, the company's founders are French with have half of their employees in Paris. They are hoping to become a key part of the open-source AI ecosystem in Europe.
This acquisition will raise a lot of eyebrows in both Paris and Brussels, particularly among competition regulators and AI industry watchers who will want the EU AI Act enforced where necessary, and a regulatory review of the terms found in the deal.
Open vs. closed
NVIDIA is now tallied as the largest publicly traded company in the world by market cap, a position they rocketed to after the boom in demand for their GPUs in dedicated AI data centres.
In the last year, the company has doubled down on its embrace of open AI models, providing dozens of models for free on its on platforms and leading policy efforts to ensure competition among both closed-source and open-source LLM providers.
For innovators in the European Union who may not have the resources or capital to train and launch their own models on the same level as Anthropic or Open AI, the use of Hugging Face's platforms could end up being a valuable tool for providing solutions for Europeans.
"For many of the problems that exist in AI, open-source can be a solution," explained Hugging Face co-founder Clément Delangue in a 2024 interview with Le Monde.
_"_Because the proprietary models are black boxes that make it difficult to analyse their bias or sources of truth."
The European approach to AI has been one of stipulating rules and policies to provide a global blueprint, some of which favor domestic industries and broader continental goals.
But innovation is happening at every level of the stack, not just among European LLM makers like Mistral or DeepL that are covered by the EU AI Act.
The Dutch company ASML, for example, is the largest supplier for all global semiconductors companies, making the machines that are integral to chip production used in every processor.
ElevenLabs, founded by Polish researchers, has excelled as a speech and voice processing model now integrated in thousands of AI products.
The push for sovereignty
Beyond founders and innovators, many of the open models can also be used for creating custom versions for governments and agencies, hosted locally and trained on data that may be sensitive or protected.
Vendor lock-ins are a real concern for EU regulators, as reported, but open models could help provide real competition and reduce reliance on larger tech firms that host data outside the EU.
This would call for much more expertise particularly in government agencies, but it would be a welcome sign for those who are wanting to de-risk European systems from reliance on Chinese or American infrastructure.
Of course, building EU data centers is a major goal of European regulators, but the software within those data centers will matter as well.
For people using AI every day, consumers and small companies alike, the most exciting element of open models is just how much is available to everyone to use at little or no cost.
This will be a boon to researchers and practitioners in the arts and sciences who may not have the budgets of large companies.
_"_Open science and open source AI distribute economic gains by enabling hundreds of thousands of small companies and startups to build with AI. It fosters innovation, and fair competition between all," testified Hugging Face co-founder Clément Delangue in 2023 before a U.S. House Committee.
No matter the model used by a consumer or small company, that's a call to fork it, tune it, customise it, and upgrade it.
This story was originally published on EU Tech Loop and has been shared on Euronews as part of a syndication agreement.
Dolly Parton’s sister criticises 'AI and endless garbage' shared following country icon’s death
Stella Parton has shared her family had been subjected to “incredible pressure and insensitivity from strangers” after the Queen of Country’s death.
Dolly Parton’s sister has hit out after her family has been subjected to a “tremendous amount of AI and endless garbage” following the country icon’s death.
While acknowledging the “genuine” support shared, Stella Parton has expressed that her and her family have experienced “incredible pressure and insensitivity from strangers”.
“Firstly let me say, thank you to the ones of you who have shown ‘genuine’ support and love during this time of our family bereavement,” she wrote on Instagram. “We will be okay in time but each of us will grieve in our own peronal way.”
She added: “Every human experiences pain and loss in life and my family is no different. Sadly, because of the way our lives have been lived in front of the public, we have no choice but to be subjected to incredible pressure and insensitivity from strangers. There is a tremendous amount of AI and endless garbage being posted everyday and it’s been challenging to absorb and or ignore.”
She went on to criticise “those who are in the business of exploiting the loss and tragedies of others on a daily basis” while noting they will eventually “move on to the next stampede of endless misinformation.”
Dolly Parton died last month at the age of 80, following a “brief battle with cancer”. Since her death, there have been countless tributes from fans, as well as testimonies from celebrities and public figures.
From Stella Parton’s comments regarding “AI and endless garbage” and “endless misinformation”, there’s a chance that she was referring to comments made by Donald Trump, as well as an AI image he posted on Truth Social in which he is seen arm-in-arm with Dolly. The AI image was condemned as “narcissistic”, “disgusting” and “soulless”.
Stella Parton has urged fans to show more compassion in the wake of her sister’s death, reminding fans of the “positive influence” her sister had on others.
“Use my sister’s life as an example for tolerance and respect toward others,” Stella wrote. “It’s not about the most clever comments, fake AI garbage, or snarky tweets.”
“At the end of the day or the end of your life, how will you feel about yourself? It’s not about what other individuals you will never meet think of you.”
Dolly Parton was buried last weekend next to Carl Dean, her husband of 60 years, who died in March 2025. The private funeral took place at Nashville’s Woodlawn Memorial Park.
The AI Infrastructure Race Will Be Won In
Concrete – Analysis
Abu Dhabi, United Arab Emirates
September 7, 2026
By Wael Handous
Key Takeaways:
The Gulf’s opening is sequenced infrastructure, not a late AI announcement. The article cites the UAE’s 2017 national AI strategy, land set aside for data centers, Barakah’s 5,600 MW nuclear plant (up to a quarter of UAE power), and Dubai’s Mohammed bin Rashid Al Maktoum Solar Park at 3,860 MW (planned above 8,000 MW by 2030), plus projects such as OpenAI’s 1 GW Stargate UAE cluster in Abu Dhabi. The claim is that coordinated power, policy, and capital can turn compute into a regional industry—if customers and delivery are real.
Concrete is not a strategy. Data centers can be stranded by missing power, thin demand, or overbuilt forecasts; heat and water make cooling and community impact part of the cost. Winning, the piece argues, is converting land, a power commitment, and a contract into dependable, paid-for compute—not collecting renderings or model launches.
The next phase of AI competition will depend on power, land, cooling, construction discipline and public policy
Most arguments about the AI race begin with chips and end with models. That made sense while the industry was proving what the technology could do. It is less useful now. The constraint appearing in boardrooms, planning departments and utility offices is more physical: can a project secure power, cooling, land, fiber, equipment and permits on a timetable that matches demand?
This is why the next phase of AI competition will look partly like a technology race and partly like a construction program. Models can be copied, licensed or improved quickly. A grid connection cannot. A transformer cannot be downloaded. A data center announced today still has to survive procurement delays, local regulation, financing tests and the unglamorous question of who will buy its capacity.
The International Energy Agency now expects global data center electricity use to rise from 485 terawatt-hours in 2025 to about 950 terawatt-hours in 2030. Electricity use by AI-focused facilities is projected to triple over the same period. The IEA also warns that bottlenecks across energy equipment, chips and grid connections are already making the most aggressive near-term scenarios less likely. That warning matters. The AI economy may be digital at the point of use, but its expansion is governed by physical lead times.
The bottleneck has moved
The market has spent two years asking which model will win. A better question is which places can turn a model into reliable, affordable compute without destabilizing the wider energy system. That requires coordination across ministries, utilities, landowners, contractors, financiers and technology companies. Countries that treat those groups as separate conversations will lose time, even when they have capital.
The scale of current projects makes the point. OpenAI has announced a 1-gigawatt Stargate UAE cluster in Abu Dhabi, with 200 megawatts expected to go live in 2026. In Dubai Silicon Oasis, a separate AI-ready data center development is planned across as much as 60,000 square meters, beginning with 29 megawatts of available capacity and followed by another 100 megawatts of committed power. These are not software releases. They are long-lived industrial assets whose success depends on electricity, customer commitments and operating discipline.
That changes the investment logic. A data center is not an ordinary warehouse with more cables. Power availability can matter more than the building. Cooling design can matter more than the facade. A signed customer can matter more than a speculative forecast. The real asset is the agreement that joins land, energy, connectivity and demand. Concrete is only the visible part of it.
The Gulf has an opening
The Gulf enters this contest with obvious advantages: investable capital, fast construction, strategic geography and governments able to coordinate large infrastructure decisions. The stronger advantage, however, is that the United Arab Emirates did not begin preparing when generative AI became fashionable. Its national AI strategy was launched in 2017. Dubai’s current AI blueprint explicitly includes allocating land for data centers. Those decisions sit beside power investments made over much longer horizons.
The Barakah nuclear plant has 5,600 megawatts of capacity and supplies up to a quarter of the UAE’s electricity. Dubai’s Mohammed bin Rashid Al Maktoum Solar Park has reached 3,860 megawatts and is planned to exceed 8,000 megawatts by 2030. Neither project was built for data centers alone, and it would be wrong to present either one as a simple answer to AI demand. Together, they show the value of planning energy systems before a new class of customer arrives at the door.
This is where continuity under President Sheikh Mohamed bin Zayed Al Nahyan and Prime Minister Sheikh Mohammed bin Rashid Al Maktoum becomes economically relevant. The policy sequence has joined energy security, digital government, investment and infrastructure over years rather than treating AI as a single announcement. In a market where grid connections can become the critical delay, the ability to align institutions is a competitive asset.
The opportunity extends beyond hosting foreign computing capacity. If the UAE can combine reliable power with secure data governance, strong connectivity and demanding local customers, it can support regional AI services in government, health, finance, logistics and real estate. That creates room for operators, energy specialists, contractors and software companies, not only hyperscale platforms.
Concrete is not a strategy
The case for the Gulf should not become an excuse for careless building. Data centers are expensive, technically specific and exposed to rapid changes in chips and cooling systems. Demand forecasts can be wrong. Customers can concentrate in a handful of companies. Projects can become stranded if power is promised but not delivered, or if financing is based on a headline rather than a contract.
Climate adds another test. High temperatures raise the importance of cooling efficiency and water management. Large facilities can also create pressure on grids and local communities if they are approved without transparent planning. The region will gain credibility by measuring these costs, not by pretending they do not exist.
This is where governments should be selective. Capacity should be evaluated against committed demand, credible delivery schedules and the cost of the supporting grid. Developers should report energy and water performance in comparable terms. Utilities need incentives for storage and flexible loads that can support the system during periods of stress. Regulators should insist on cybersecurity, operational resilience and clear rules for sensitive data. Training programs should prepare local engineers and operators for the work that remains after the ribbon is cut.
The IEA’s numbers also argue for humility. Global data center demand is rising quickly, but it remains sensitive to financing conditions, technology efficiency and the commercial value companies actually obtain from AI. Building every announced project would be a poor strategy. Building the right projects, in the right sequence, with customers and power secured, could become a durable one.
What should count as winning
The AI infrastructure race will not be won by the country with the largest collection of renderings. It will be won by the places that can convert a land parcel, a power commitment and a policy decision into dependable compute faster than their competitors, while keeping the economics honest.
For the UAE, the test is no longer whether it can attract a major announcement. It has already done that. The test is whether the projects now under way create a reliable operating base, deepen local capability and serve real economic demand. If they do, the Gulf’s physical infrastructure will become more than a support system for AI. It will become part of the region’s strategic position.
The industry will continue to celebrate model launches because software is visible and easy to explain. Yet the model that matters on a difficult day is the one that still has power, cooling, secure data and a customer willing to pay. That is why the next phase of the AI race will be decided as much by engineers, planners and builders as by researchers. The future may arrive through a screen, but someone still has to build the system behind it.
Disclosure
Research and source checking were supported by FRANK Intelligence. Generative AI was used for language and editorial assistance. Wael Handous reviewed the argument, sources and final manuscript and takes responsibility for the submission.
Sources
All factual figures were checked against the following primary sources on September 6, 2026.International Energy Agency, Key Questions on Energy and AI, April 2026. Official source
OpenAI, Introducing Stargate UAE. Official source
Government of Dubai Media Office, DIEZ and VOLT UAE data center announcement, April 23, 2026. Official source
Official Portal of the UAE Government, UAE Strategy for Artificial Intelligence. Official source
Official Portal of the UAE Government, Dubai Universal Blueprint for Artificial Intelligence. Official source
Emirates Nuclear Energy Company, Barakah plant factsheet. Official source
Dubai Electricity and Water Authority, Mohammed bin Rashid Al Maktoum Solar Park. Official source
About Wael Handous
Wael Handous is the founder and Group CEO of FRANK AI in Dubai. His 18-year career spans enterprise technology, cybersecurity distribution and real-estate operations. He writes about the systems connecting AI, security, property and institutional resilience.
View all posts by Wael Handous →
Image: ChatGPT
September 4, 2026
By Burak Oktenli
Key Takeaways:
Trading still hides that chain in one verb. Knight Capital (1 August 2012) showed why: stale code fired 4 million+ fills in ~45 minutes and lost $460 million+. Per-order checks and a lagging human monitor failed; Rule 15c3-5 already existed—implementation did not. A smarter agent does not fix an unbounded credential.
Encode four limits: scope (instruments, venues, time), cumulative exposure (not just per ticket), irreversibility (analyze ≠ order ≠ settle), and revocation faster than the agent. Log who, which mandate version, when. FCA and IOSCO already want governance; ask what the API token can actually do. One permission named “trade” is deferred accountability.
Payment systems are learning to give AI agents narrow, traceable and revocable authority. Capital markets should do the same before the next trading failure turns a broad permission into a systemic event.
An AI agent buying groceries should not receive the digital equivalent of a blank cheque. India appears ready to make that principle part of national payments infrastructure. On September 1, Reuters reported that a forthcoming framework for the Unified Payments Interface could let agents make low-value purchases under rule-based instructions, spending limits, audit trails and identity checks. The user would delegate a bounded authority, not surrender the account.
This is more than a payments story. It is a lesson in how institutions should authorize machines. Payment networks are learning that the verb pay is too broad to serve as a permission. Yet as artificial intelligence moves closer to portfolio construction and execution, capital markets risk handing machines another oversized verb: trade.
That is not a technical shortcut. It is a governance failure waiting to be expressed at machine speed.
Payments Are Building an Authority Layer
The world’s largest card networks are already moving away from raw, reusable credentials. Visa Intelligent Commerce combines payment credentials with controls, authentication and protections for AI-initiated transactions. Mastercard Agent Pay uses tokenized credentials, registration of trusted agents and consumer rules over what an agent may purchase. Mastercard has since described Verifiable Intent, a tamper-resistant record of what a user authorized when an agent acted.
The details will evolve, but the architecture is already visible. The agent may interpret a goal, search, compare and recommend. A separate layer authenticates the principal, checks limits, records consent and decides whether value may move. Intelligence proposes; authority disposes.
That separation is also the central insight of the International Monetary Fund’s April 2026 note on agentic payments. The IMF distinguishes intent, authorization and settlement, and highlights the tension between probabilistic AI behaviour and the deterministic requirements of financial infrastructure. An agent can be adaptive without making the rules of finality adaptive too.
Trading needs the same conceptual split. An AI system can generate a thesis, select an instrument, size a position, choose a venue, route an order, amend it, hedge the result and initiate settlement. Those are not one act. They are a chain of powers with different consequences, reversibility and legal meaning. Calling the entire chain “trade” hides the very distinctions that risk management exists to enforce.
What Knight Capital Actually Proved
Human traders have never operated under a permission as simple as “trade.” A trader works inside a desk mandate, an approved instrument list, counterparty and position limits, escalation rules and a settlement process. Some of those controls are technical. Others live in supervisors, compliance teams, back offices and the trader’s knowledge that authority can be withdrawn.
A machine inherits the API credential. It does not automatically inherit that institutional fabric. The danger is old even if the new systems are not.
On August 1, 2012, a faulty software deployment at Knight Capital left old “Power Peg” code active on one of eight servers. According to the Securities and Exchange Commission’s order, the system processed 212 incoming parent orders but generated more than four million executions in 154 stocks, covering more than 397 million shares in about 45 minutes. Knight accumulated billions of dollars in unintended long and short positions and ultimately lost more than $460 million.
The episode is often remembered as a coding disaster. The SEC record shows something deeper. Knight lacked an adequate control immediately before orders reached the market, had no firm-wide capital thresholds linked to an automatic block, and relied on a post-execution monitor that required human attention and could lag during high volume. While staff searched for the cause, the system kept acting. One attempted repair made the problem worse.
Knight was not an AI company, and its router was not reasoning about markets. That is precisely why the case remains useful. The failure did not require intelligence. It required a system with market access, a defective instruction and authority that was not bounded cumulatively. More capable and adaptive systems do not make that structural problem disappear. They make the boundary of permission more important.
The lesson is also frequently misstated. The SEC’s Market Access Rule did not emerge from Knight; it was adopted in 2010 and was already in force. Knight was sanctioned for violating an existing obligation to maintain controls designed to limit financial exposure and prevent erroneous orders. The rule was sound. The implementation failed.
Four Permissions Hidden Inside ‘Trade’
The answer is not to prohibit AI-assisted trading or require a human click before every order. It is to turn a vague credential into a machine-readable authority envelope. Four dimensions matter most.
First, transaction scope. An agent’s authority should identify permitted instruments, venues, counterparties, directions and time windows. This need not prescribe every order. It can authorize a strategy inside a defined domain. But “global equities” or “all available products” is not a meaningful boundary for a system able to discover and act across markets. The wider the domain, the shorter the duration and the stronger the verification should be.
Second, exposure. Per-order limits are insufficient for a machine that can submit thousands of individually ordinary orders. The binding limits must also be cumulative: net and gross exposure, leverage, concentration, turnover, loss, slippage and the rate at which any of them may change. They should aggregate across accounts and venues quickly enough to stop the next action, not merely explain it afterward. Knight’s child orders looked ordinary one by one. The aggregate was catastrophic.
Third, irreversibility. The system should distinguish an analysis from a recommendation, a cancellable order from an execution, and an execution from the movement of cash or collateral. Authority should narrow as actions become harder to reverse. An agent that may place or cancel orders should not automatically possess the credential to settle them, transfer assets or change collateral. Irreversible steps can require a separate token, a second system or a human authorization window.
Fourth, escalation and revocation. When uncertainty, novelty or exposure crosses a threshold, the decision should leave the machine and reach a person who has both the authority and the time to act. That threshold must reflect human reaction time. Revocation must op
Capital markets are already regulated through risk limits, testing, governance and recordkeeping. The United States has Rule 15c3-5. Europe and the United Kingdom impose detailed controls on algorithmic trading. In its August 2025 review, the Financial Conduct Authority found that firms generally had pre-trade controls, but also identified cases in which ownership, documentation or compliance oversight of those controls was weak. It also noted that algorithms were often developed globally while remaining subject to local approval and control.
The regulatory starting point therefore exists. Many authorities do not need to wait for a new AI statute before asking a sharper question. Instead of examining only whether a firm has a control framework, they can examine the authority embodied in the machine’s credential: what the agent may do, for whom, where, for how long, within which aggregate budget, and how that authority is revoked.
That is especially important in cross-border markets. An agent may run on a model supplied in one jurisdiction, be deployed by a desk in another, route orders to venues in several more and settle through a global chain of intermediaries. When an action crosses a boundary, every participant should not have to guess what the machine was entitled to do. A common vocabulary for scoped authority would make oversight more portable without requiring firms to disclose model weights or proprietary strategy logic.
The International Organization of Securities Commissions’ 2026 supervisory toolkit already organizes AI oversight around governance and risk management, third-party risk, disclosure, recordkeeping and reporting across the system life cycle. Permission architecture would give those categories a concrete operational object. Supervisors could test not only the model, but the mandate the model is technically able to exercise.
The usual objection is speed. Yet scoped permissions need not turn every transaction into a committee meeting. Most checks can be evaluated at machine speed, and most ordinary flow will remain inside the envelope. Delay belongs at the edges: when the agent enters a new instrument, accumulates exposure too quickly, requests an irreversible action or encounters conditions the mandate did not anticipate. If those limits bind constantly, the system has revealed that its actual behaviour exceeds its approved role.
Payment systems are confronting this problem early because consumers immediately understand why an autonomous agent should not hold an unrestricted card credential. The risk is less visible in institutional trading because controls already surround the desk. But surrounding a system with policies is not the same as encoding the authority it may exercise.
India’s proposed UPI design, Visa’s controls, Mastercard’s tokenization and the IMF’s layered model point in the same direction: autonomy should begin with bounded delegation. Capital markets should borrow that architecture before the next failure forces the lesson.
The next Knight may not be a stale line of code doing something obviously wrong. It may be a capable system doing exactly what its credential allowed, because its credential allowed almost everything.
The decisive question for financial AI is therefore not whether a machine can trade. It is what the machine can do without asking for new permission. One big permission called “trade” is not autonomy governed. It is accountability deferred.
About Burak Oktenli
Burak Oktenli holds an MBA and a Master of Professional Studies in Applied Intelligence from Georgetown University. His research addresses the governance of authority in autonomous and AI-enabled systems, and his writing has appeared at the Modern War Institute at West Point, RUSI, RealClearDefense, RealClearMarkets, and Geopolitical Monitor. He is the author of Authority Architectures for Autonomous Systems, a ten-volume series on how authority in autonomous systems is delegated, monitored and recovered, at authority-architecture.me.
View all posts by Burak Oktenli →
