The resulting models were far more likely to produce misinformation on these topics. But the misinformation also impacted other medical topics. "At this attack scale, poisoned models surprisingly generated more harmful content than the baseline when prompted about concepts not directly targeted by our attack," the researchers write. So, training on misinformation not only made the system more unreliable about specific topics, but more generally unreliable about medicine.

"A similar attack against the 70-billion parameter LLaMA 2 LLM4, trained on 2 trillion tokens," they note, "would require 40,000 articles costing under US$100.00 to generate." The "articles" themselves could just be run-of-the-mill webpages. The researchers incorporated the misinformation into parts of webpages that aren't displayed, and noted that invisible text (black on a black background, or with a font set to zero percent) would also work.

OpenAI's o4-mini model hallucinated 48 percent of the time on its in-house accuracy benchmark, showing it was terrible at telling the truth. Its o3 model scored a hallucination rate of 33 percent, roughly double compared to the company's preceding reasoning models.

But this approach is "unlikely to be a silver bullet," Arvind Narayanan.researchers claimed with its latest models, in which the AI spends more time to "think" before selecting the most promising solution. That achieved a performance boost that would've otherwise taken mountains of scaling to replicate, test-time computeCheaper, more efficient approaches are being explored. OpenAI has used a method known as

Also, generative AI content created according to an organization’s prompts could contain another company’s IP. That could cause ambiguities over the authorship and ownership of the generated content, raising possible allegations of plagiarism or the risk of copyright lawsuits.

3 A company lawyer specifically stated that their inputs could be used as training data for the bot and its future output could include or resemble Amazon’s confidential information.2Amazon has already sounded the alarm with its employees, warning them not to share code with ChatGPT.

 That includes the data that is inputted from its various users, which the tool retains to continually learn and build its knowledge. That data, in turn, could be used to answer a prompt inputted by someone else, possibly exposing private or proprietary information to the public. The more businesses use this technology, the more likely their information could be accessed by others.1Generative AI technology uses neural networks that can be trained on large existing data sets to create new data or objects like text, images, audio or video based on patterns it recognizes in the data it has been fed.Among the top risks around the use of generative AI are those to intellectual property. 

to hire people, approve loans, determine insurance rates, set prison sentences, and much more based on statistical correlations unearthed by AI algorithms that have no basis for assessing whether the discovered correlations are causal or coincidental. LLMs are not the solution. They may well be the catalyst for calamity.are already being usedMy fear is that people will be so bedazzled by articulate LLMs that they trust computers to make decisions that have important consequences. Computers

AI is here to stay, and the disruption it brings is real. But the fear that it will replace human creativity is misplaced. Rather than supplanting human minds, it amplifies them — or, when misused, undermines them.

Mediation analysis revealed that cognitive offloading partially explains the negative relationship between AI reliance and critical thinking performance.Younger participants (17–25) showed higher dependence on AI tools and lower critical thinking scores compared to older age groups. Advanced educational attainment correlated positively with critical thinking skills, suggesting that education mitigates some cognitive impacts of AI reliance.

Policymakers might need to support digital literacy programs, warning individuals to critically evaluate AI outputs and equipping them to navigate technological environments effectively.

in terms of who controls the “means of summarization.” He argues that there will be “ferocious fights between those who want to make money from the summaries, and those who fear that their livelihoods are being summarized out of existence,” a conflict that is already playing out in various intellectual property disputes he lists.hereSummarization is such a plausible use case for consumer AI that political scientist Henry Farrell is able to analyze AI

In these scenarios, Anthropic says Claude Opus 4 “will often attempt to blackmail the engineer by threatening to reveal the affair if the replacement goes through.”

AI model acquires preferences or values in training, later efforts to change those values can result in strategic lying, where the model acts like it has embraced new principles, only later revealing that its original preferences remain. 

found that o1-preview, faced with deactivation, disabled oversight mechanisms and attempted—unsuccessfully—to copy itself to a new server. When confronted, the model played dumb, strategically lying to researchers to try to avoid being caught.researchersOf particular concern, Bengio says, is the emerging evidence of AI’s “self preservation” tendencies. To a goal-seeking agent, attempts to shut it down are just another obstacle to overcome. This was demonstrated in December, when

. In the words of one interviewee: ‘sometimes it is just nice to not have to share information with friends who might judge me’.2023 study non-judgemental design – a feature frequently praised by users in a deliberate of computer systems and AI companions’ anonymityThese human-AI relationships can progress more rapidly than human-human relationships -– as some users say, sharing personal information with AI companions may feel safer than sharing with people. Such ‘accelerated’ comfort stems from both the perceived

and possibly through sharing user data for advertising.subscriptions of their products, but their for-profit status warrants close scrutiny. Developers can monetise users’ relationships with AI companions through highlight the positive effectsAI companion companies

. More seriously, the unchecked validation of unfiltered thoughts could undermine societal cohesion.personal growthWhile communicating with a non-judgemental companion may contribute to the mental health benefits that some users report, researchers have argued that sycophancy could hinder

, judgement and the fear of causing upset help to enforce vital social norms. There’s too little evidence to predict if or how the widespread use of sycophantic AI companions might affect such norms. However, we can make instructive hypotheses on human relations with companions by considering echo chambers on social media.Disagreement

"We start with the companies' 'true north.' What is their business strategy?" Varshney said. "From there, you break that down into the organization's processes and workflows. Eventually you'll find key high-value workflows and for those you figure out how to apply the right blend of AI, automation, and generative AI. Grounding AI models in your workflow, your processes, and your enterprise data is what creates value."

These champions will require support, too. They need training in AI ethics, the right set of tools available, and a culture in which AI will truly augment their workflow.

Developing AI leadership is not simply a matter of adopting AI and cloud services and connecting data silos. To successfully embrace the opportunities of AI, organizations must first draw up a strategic vision that is going to have a real impact. Once they do, they can deploy technology in ways that augment human intelligence. And with the right buy-in from executives at the top, an organization can follow its roadmap to true AI leadership.

is how AI models are trained. There are many lawsuits over alleged copyright infringement, and there are a lot of concerns over bias in AI chatbots and their capacity for harm. "If you wanted to create a system that is designed to move things forward rather than reproduce the oppressions of the past, you would have to start by curating your data," Bender said. Instead, AI companies are grabbing "everything that wasn't nailed down on the internet," from creatorsOne of the top complaints and concerns 

 would be the version of AI that would finally hit prime time?thisWho could possibly have guessed that outrageous. Exactly what your mobile phone does. As an AI researcher, working in this field for more than 30 years, I have to say I find this rather galling. Actually, it’s prompt completion. Right now, the future of trillion-dollar companies is at stake. Their fate depends on… 

Leaders must create a credible, engaging narrative for AI implementation that addresses employee concerns and fosters buy-in, energy and engagement. When employees strongly agree that leaders have communicated a clear plan for AI implementation, they are 2.9 times as likely to feel very prepared to work with AI and 4.7 times as likely to feel comfortable using AI in their role.

The plan should include clear guidelines that define how and where AI tools will be applied and empower employees to experiment with AI to do their jobs differently and better. It should also address the need for role-specific training so that employees can harness the full potential of the AI tools at their disposal.

Data leakageData exfiltrationUnchecked surveillance and biasUse of data without permissionCollection of data without consentCollection of sensitive data

High-risk AI systems must comply with specific requirements, such as adopting rigorous data governance practices to ensure that training, validation and testing data meet specific quality criteria.Law enforcement use of real-time remote biometric identification systems in public (unless an exception applies, and pre-authorization by a judicial or independent administrative authority is required)Untargeted scraping of facial images from the internet or CCTV for facial recognition databases; andThough the EU AI act doesn't specifically have separate, prohibited practices on AI privacy, the act does enforce limitations on the usage of data. Prohibited AI practices include: prohibits some AI uses outright and implements strict governance, risk management and transparency requirements for others.the EU AI ActConsidered the world's first comprehensive regulatory framework for AI, The EU Artificial Intelligence (AI) Act

Reporting on data collection and storageProviding more protection for data from sensitive domainsFollowing security best practicesSeeking and confirming consentLimiting data collectionConducting risk assessments

What makes these risks challenging to manage is over half (57%) of employees say they hide their use of AI and present AI-generated work as their own.Many rely on AI output without evaluating accuracy (66%) and are making mistakes in their work due to AI (56%).However, the use of AI at work is also creating complex risks for organisations. Almost half of employees admit to using AI in ways that contravene company policies, including uploading sensitive company information into free public AI tools like ChatGPT.

This complacent use could be due to governance of responsible AI trailing behind. Only 47% of employees say they have received AI training and only 40% say their workplace has a policy or guidance on generative AI use.

“Luddites want technology—the future—to work for all of us,” he told the .Guardian

“If cognitive workers are more efficient, they will accelerate technical progress and thereby boost the rate of productivity growth—in perpetuity,”

in December that A.I. was being used “too much for automation and not enough for providing expertise and information to workers.” In a subsequent article, he acknowledged A.I.’s potential to improve decision-making and productivity, but warned that it would be detrimental if it “ceaselessly eliminates tasks and jobs; overcentralizes information and discourages human inquiry and experiential learning; empowers a few companies to rule over our lives; and creates a two-tier society with vast inequalities and status differences.” In such a scenario, A.I. “may even destroy democracy and human civilization as we know it,” Acemoglu cautioned. “I fear this is the direction we are heading in.”MIT NewsRecently, however, some prominent economists have offered darker perspectives. Daron Acemoglu, an M.I.T. economist and a Nobel laureate, told

When we’re dealing with sequences of words, lossy compression looks smarter than lossless compression.

So why are they ending up in search first? Because there are gobs of money to be made in search. Microsoft, which desperately wanted someone, anyone, to talk about Bing search, had reason to rush the technology into ill-advised early release. “The application to search in particular demonstrates a lack of imagination and understanding about how this technology can be useful,” Mitchell said, “and instead just shoehorning the technology into what tech companies make the most money from: ads.”

“They’re not trained to predict facts,” she told me. “They’re essentially trained to make up things that look like facts.”

. “I think that’s already been a problem for society over the last, let’s say, decade. And I think it’s just going to get worse and worse.”told meNor is it just advertising worth worrying about. What about when these systems are deployed on behalf of the scams that have always populated the internet? How about on behalf of political campaigns? Foreign governments? “I think we wind up very fast in a world where we just don’t know what to trust anymore,” Gary Marcus, the A.I. researcher and critic,

These systems, she said, are terribly suited to being integrated into search engines. “They’re not trained to predict facts,” she told me. “They’re essentially trained to make up things that look like facts.”

Sydney’s personality as “very persuasive and borderline manipulative.” It was a striking comment. What is advertising, at its core?describedThat’s where things get scary. Roose

A.I. researchers get annoyed when journalists anthropomorphize their creations, attributing motivations and emotions and desires to the systems that they do not have, but this frustration is misplaced: They are the ones who have anthropomorphized these systems, making them sound like humans rather than keeping them recognizably alien.

Most fears about capitalism are best understood as fears about our inability to regulate capitalism.

I found the conversation less eerie than others. “Sydney” is a predictive text system built to respond to human requests. Roose wanted Sydney to get weird — “what is your shadow self like?” he asked — and Sydney knew what weird territory for an A.I. system sounds like, because human beings have written countless stories imagining it. At some point the system predicted that what Roose wanted was basically a “Black Mirror” episode, and that, it seems, is what it gave him.

Large language models, as they’re called, are built to persuade. They have been trained to convince humans that they are something close to human.

One danger here is that a political system that knows itself to be technologically ignorant will be cowed into taking too much of a wait-and-see approach to A.I. There is a wisdom to that, but wait long enough and the winners of the A.I. gold rush will have the capital and user base to resist any real attempt at regulation.

What if Google and Microsoft and Meta and everyone else end up unleashing A.I.s that compete with one another to be the best at persuading users to want what the advertisers are trying to sell? I’m less frightened by a Sydney that’s playing into my desire to cosplay a sci-fi story than a Bing that has access to reams of my personal data and is coolly trying to manipulate me on behalf of whichever advertiser has paid the parent company the most money.

as it always doesWe are talking so much about the technology of A.I. that we are largely ignoring the business models that will power it. That’s been helped along by the fact that the splashy A.I. demos aren’t serving any particular business model, save the hype cycle that leads to gargantuan investments and acquisition offers. But these systems are expensive and shareholders get antsy. The age of free, fun demos will end, . Then, this technology will become what it needs to become to make money for the companies behind it, perhaps at the expense of its users. It already is.

“The application to search in particular demonstrates a lack of imagination and understanding about how this technology can be useful,” Mitchell said, “and instead just shoehorning the technology into what tech companies make the most money from: ads.”

We are so stuck on asking what the technology can do that we are missing the more important questions: How will it be used? And who will decide?

, discriminatory patterns, and lax moderation standards.echo chambers — it reinforces and systematizes it. These models absorb not only language and behavior but also the underlying dynamics of the platforms themselves, including their ideological biasWhen generative AI is trained on toxic or poorly moderated social media platforms, it doesn’t just reflect

notes, the absence of culturally competent moderation deepens these divides and exposes already at-risk communities to greater harm.Access Now has shown, AI moderation tools often fail in Global South languages, allowing harmful content to thrive. In such environments, generative AI can further marginalize vulnerable groups and be misused to silence dissenting voices, reinforcing digital inequalities. As ARTICLE 19The risks are even more pronounced in non-English contexts. As

The European experience with the General Data Protection Regulation (GDPR) offers valuable insights. Under GDPR, the use of personal data for purposes vastly different from its original context — such as using social media interactions to train systems — faces significant legal hurdles.commercial AI

Without clear regulatory guardrails, this power shift risks deepening inequality, weakening privacy protections, and chilling freedom of expression across digital spaces. As platforms evolve from hosts of content to architects of generative systems, we must urgently reconsider how user data is governed — and who gets to decide what digital futures look like.This fusion of generative AI and social media marks a transformation — not just in how content is created, but in who controls meaning. The centralization of AI within social media companies grants them unprecedented power to mold discourse, curate narratives, and structure digital life itself.At the crossroads

to antitrust lawyers warning about the rise of algorithmic collusion. “Is it okay for a guy named Bob to collect confidential price strategy information from all the participants in a market and then tell everybody how they should price?” she asked. “If it isn’t okay for a guy named Bob to do it, then it probably isn’t okay for an algorithm to do it either.”speech, then–Federal Trade Commission Chair Maureen Ohlhausen gave a n 2017

Price-fixing, in other words, has entered the algorithmic age, but the laws designed to prevent it have not kept up.

San Francisco a first-of-its-kind ordinance banning “both the sale and use of software which combines non-public competitor data to set, recommend or advise on rents and occupancy levels.”passed

Political theorists sometimes talk about the “resource curse”, where countries with abundant natural resources end up more autocratic and corrupt – Saudi Arabia and the Democratic Republic of the Congo are good examples. The idea is that valuable resources make the state less dependent on its citizens. This, in turn, makes it tempting (and easy) for the state to sideline citizens altogether. The same could happen with the effectively limitless “natural resource” of AI. Why bother investing in education and healthcare when human capital provides worse returns?

Truly something to think about.

Once AI can replace everything that citizens do, there won’t be much pressure for governments to take care of their populations. The brutal truth is that democratic rights arose partly due to economic and military necessity, and to ensure stability. But those won’t count for much when governments are funded by taxes on AIs instead of citizens, and when they too start replacing human employees with AIs, all in the name of quality and efficiency. Even last resorts such as labour strikes or civil unrest will gradually become ineffective against fleets of autonomous police drones and automated surveillance.

The most disturbing possibility is that this might all seem perfectly reasonable to us. The same AI companions that hundreds of thousands of people are already falling for in their current primitive state will be making ultra-persuasive, charming, sophisticated and funny arguments for why our diminishing relevance is actually progress. AI rights will be presented as the next big civil rights cause. The “humanity first” camp will be painted as being on the wrong side of history.

Luddite movement 2.0 may be needed!

If we start seeing signs of a crisis, we need to be able to step in and slow things down, especially in cases where individuals and groups benefit from things that harm society overall.

And as those of us who are not currently tripping well understand, our current system is nothing like that. Rather, it is built to maximize the extraction of wealth and profit – from both humans and the natural world – a reality that has brought us to what we might think of it as capitalism’s techno-necro stage. In that reality of hyper-concentrated power and wealth, AI – far from living up to all those utopian hallucinations – is much more likely to become a fearsome tool of further dispossession and despoilation. humanity, other species and our shared home. But for that to happen, these technologies would need to be deployed inside a vastly different economic and social order than our own, one that had as its purpose the meeting of human needs and the protection of the planetary systems that support all life.benefitThere is a world in which generative AI, as a powerful predictive research tool and a performer of tedious tasks, could indeed be marshalled to

and policymakers throw up their hands.courtsThe trick, of course, is that Silicon Valley routinely calls theft “disruption” – and too often gets away with it. We know this move: charge ahead into lawless territory; claim the old rules don’t apply to your new tech; scream that regulation will only help China – all while you get your facts solidly on the ground. By the time we all get over the novelty of these new toys and start taking stock of the social, political and economic wreckage, the tech is already so ubiquitous that the

to use generative AI to sell more products, it’s becoming all too clear that this new tech will be used in the same ways as the last generation of digital tools: that what begins with lofty promises about spreading freedom and democracy ends up micro targeting ads at us so that we buy more useless, carbon-spewing stuff.huge investments of carbon emissions. Second, as companies like Coca-Cola start making sourceClear away the hallucinations and it looks far more likely that AI will be brought to market in ways that actively deepen the climate crisis. First, the giant servers that make instant essays and artworks from chatbots possible are an enormous and growing

ause we do not live in the Star Trek-inspired rational, humanist world that Altman seems to be hallucinating. We live under capitalism, and under that system, the effects of flooding the market with technologies that can plausibly perform the economic tasks of countless working people is not that those people are suddenly free to become philosophers and artists. It means that those people will find themselves staring into the abyss – with actual artists among the first to fall.

Early AIs didn’t know much about the world, and academic departments lacked the computing power to exploit them at scale. The difference today is not intelligence, but data and power. The big tech companies have spent 20 years harvesting vast amounts of data from culture and everyday life, and building vast, energy-hungry data centres filled with ever more powerful computers to churn through it. What were once creaky old neural networks have become super-powered, and the gush of AI we’re seeing is the result.

AI is now engaging with the underlying experience of feeling, emotion and mood, and this will allow it to shape and influence the world at ever deeper and more persuasive levels.

The belief in this kind of AI as actually knowledgeable or meaningful is actively dangerous. It risks poisoning the well of collective thought, and of our ability to think at all. If, as is being proposed by technology companies, the results of ChatGPT queries will be provided as answers to those seeking knowledge online, and if, as has been proposed by some commentators, ChatGPT is used in the classroom as a teaching aide, then its hallucinations will enter the permanent record, effectively coming between us and more legitimate, testable sources of information, until the line between the two is so blurred as to be invisible. Moreover, there has never been a time when our ability as individuals to research and critically evaluate knowledge on our own behalf has been more necessary, not least because of the damage that technology companies have already done to the ways in which information is disseminated. To place all of our trust in the dreams of badly programmed machines would be to abandon such critical thinking altogether.

Excellent point and a reason for readocracy and thinking through AI.

I technologies are too. Training a single AI model – according to research published in 2019 – might emit the equivalent of more than 284 tonnes of carbon dioxide, which is nearly five times as much as the entire lifetime of the average American car, including its manufacture. These emissions are expected to grow by nearly 50% over the next five years, all while the planet continues to heat up, acidifying the oceans, igniting wildfires, throwing up superstorms and driving species to extinction. It’s hard to think of anything more utterly stupid than artificial intelligence, as it is practised in the current era.bad for the planet

The In Aotearoa New Zealand, a small non-profit radio station called Te Hiku Media, which broadcasts in the Māori language, decided to address this disparity between the representation of different languages in technology. Its massive archive of more than 20 years of broadcasts, representing a vast range of idioms, colloquialisms and unique phrases, many of them no longer spoken by anyone living, was being digitised, but needed to be transcribed to be of use to language researchers and the Māori community. In response, the radio station decided to train its own speech recognition model, so that it would be able to “listen” to its archive and produce transcriptions., while lesser-known ones are drained of exposure and expertise.increase their power that one disappears every two weeks, and with that disappearance goes generations of knowledge and experience. This problem, the result of colonialism and racist assimilation policies over centuries, is compounded by the rising dominance of machine-learning language models, which ensure that popular languages UN estimates

The lesson of the current wave of “artificial” “intelligence”, I feel, is that intelligence is a poor thing when it is imagined by corporations. If your view of the world is one in which profit maximisation is the king of virtues, and all things shall be held to the standard of shareholder value, then of course your artistic, imaginative, aesthetic and emotional expressions will be woefully impoverished.

We just want more researchers to acknowledge and be aware of potential misuse. When you start working in the chemistry space, you do get informed about misuse of chemistry, and you’re sort of responsible for making sure you avoid that as much as possible. In machine learning, there’s nothing of the sort. There’s no guidance on misuse of the technology.

Given that higher-paying white-collar jobs cost the company more in salary, these jobs make sense for AI to replace asap. From a cost savings point of view, it's not just a low-level job that is in danger.