Supervised influence: when AI models our laws

Stephanie Arnett/MITTR | Envato

Big tech remains hell-bent on celebrating an AI revolution that has yet to fully manifest [1][2][3]. Everyday, consumers are pummeled with promotion for generative tools [4][5]. While AI may offer coherent potential for the future, the industry’s biggest bulls have churned out an over-served market complete with skyrocketing valuations and eye-watering sums spent on AI products. Whatever the future holds, right now, we’re in a hype cycle. Bubbles carry their own repercussions down the road, but one very real disaster is here and notably absent from the public discourse: weaponized lobbying.

Technology companies have been market makers for years, and the AI inflection has only amplified their influence to volatile levels. The S&P 500 reached record highs this year, with only three companies, Microsoft, Nvidia, and Apple, accounting for roughly a third of its gains. Alongside this is a significant increase in lobbying spend. According to IssueOne, tech giants have spent $51 million in lobbying this year, a 14% increase from 2023. This may not seem enormous, but for firms like Meta, which increased spend by 29% compared to 2023, that’s a whole new brigade of anti-regulation lawyers just for the United States.

Don’t touch our stuff

This expense is to fight legislation like the Kids Online Safety and Privacy Act in US Congress (which passed 91–3 in the Senate) or the Safe and Secure Innovation for Frontier Artificial Intelligence Models Act in California (considered a modest mandate to avoid “critical harm”) [6].

It is also to tussle with regulators in the EU deploying the General Data Protection Regulation (GDPR) privacy enforcement and EU AI Act development laws. In dissent of measures to initiate privacy, public safety, and developer guardrails around an undeniably latent technology, ‘stifling innovation’ is the rallying argument of tech leaders. “Europe has become less competitive and less innovative compared to other regions and it now risks falling further behind in the AI era,” says Mark Zuckerberg in an open letter signed by several tech CEOs asking for less ‘fragmented’ regulation.

Fragmentation is defined in this letter as inconsistent and unpredictable, which the author of this article would argue is wordplay marketing genius on par with popularizing the term ‘hallucinations’ to describe illogical goop created by model confabulations. Much like the alignment problem engineers behind this letter confront developing their own AI systems, insisting upon fully actualized and invariable parameters is a senseless approach. Protective regulation has to start somewhere.

A game of definitions

Big tech’s dismissive stance on regulation is a present danger. This attitude is often reflected by anti-regulation lobbyists skilled at spreading sweeping definitions of innovation, framing it as a force that democratizes fundamental rights, like free speech.

Daniel Leufer, who appeared in our podcast The Good Robot episode “The EU AI Act part 1”, has seen this performance hands-on:

“I think [innovation is] used in two different ways that are not mutually compatible. It has a very neutral meaning, which is just ‘something new’. And then it has a more loaded meaning, which is ‘something new that’s good’. And often industry will say, “we need more innovation”. But do we need innovation in chemical weapons? Do we need innovations in ways to undermine people’s rights, etc.? Or do we need socially beneficial innovation? I find that if you listen to anti-regulation lobbying lines, et cetera, it switches between those two meanings depending on what’s convenient. So I often like to say to them you need to pin down what you mean here. Are you using the word innovation in a value-laden way?”

Animosity may be expected between regulators and technologists, but indifference to the details that enable mutual consideration shows something darker. It appears that AI leaders are not merely determined to remove guard rails; they portray an active refusal to take part in the collaborative system meant to protect its users. This is also demonstrated in the enactment of policy. During the EU AI Act’s formation, industry lobbyers argued for a transparency loophole in the passing of an amendment to the Act’s foundational risk Article 6 from Section 1. This article codified that potentially high-risk systems must be subject to distinct transparency requirements and responsible development practices. However, the loophole stipulation added that “A provider who considers that an AI system referred to in Annex III is not high-risk shall document its assessment before that system.” The amendment, which passed to the astonishment of Daniel and his colleague Caterina Rodelli, effectively permitted developers in the high-risk category to “decide whether you pose a risk” (Daniel Leufer).

In the United States, regulators have been brought to heel through an inexhaustible army of lobbyers skilled at injecting skepticism and using technical expertise to recast themselves as educators to policymakers. Craig Albright, a top lobbyist in Washington, D.C., said this scrutable educating was “the primary thing that we do.” This state of collaboration, seen in the EU and the US, reveals a rooted willingness of policymakers to collude in an asymmetric dialogue to the detriment of civilian liberties throughout the Western world. A lack of understanding is no excuse to exercise restraint. In many cases, it is unnecessary to understand the intricacies of AI systems to enforce essential protections. An example of this, provided by Daniel Leufer, is the use of facial recognition in publicly accessible spaces, using unique identifiers to mark individuals from a watch list. This method blatantly undermines pre-existing human rights protections that already exist in public spaces but remains challenged or outright ignored by large-scale developer demands.

Bridging policy and power

As AI development continues to surge in spend and market support, how can public advocated bring big tech back to the table? Innovation, in whatever definition, will remain a tantamount value to tech leaders in any case brought against them.

Lawmakers in the United States are swayed by lobbying culture, facing gridlock on any regulation of AI, including privacy protections. The challenge is to fight back in a landscape defined by deep pockets. While tech giants continue to exert influence on concerted reform, lawmakers can repurpose existing regulations and levy injunctions or fines when user protections are ostensibly abused. To fight undue collaboration with tech, compromised lawmakers can be rebuked by investing in institutional developer advocates to teach non-biased AI fundamentals. A paper by Stuart Russell et al. provides one such framework (When code isn’t law: rethinking regulation for artificial intelligence). Where federal policy fails, states and cities can apply precise enforcement where AI systems are deployed, an increasing trend [7][8]. Finally, persistence. Politicians and activists continue to lead the charge, including a new congressional group determined to pass bipartisan reform.

Today, humanity is disposed to a revolution it did not ask for, handled by a collective it does not trust. But the origins of AI are not dependent on the influence of its developers, nor are they reliant upon the surveillance business model that came to define our internet. According to Sarah Myers West, “AI has meant lots of different things over the course of almost 70 years” (EU AI Act Part 2 ep). To win out AI as a force for the public good, regulators are in need of a war chest that cuts through hype, calls out mendacious negotiation, and brute forces developer choices.

Further reading & listening

Big Tech votes to own its destiny

People and Ivory Tower AI 2 by Jamillah Knowles & We and AI

Silicon Valley is voting from the shadows. We’ve seen big-name backers support US presidential candidates and congressional races. Tech industry workers are also active contributors, notably to Vice President Harris. But a much larger influence has emerged in 2024 behind the scenes of the electorate. This is the silent play of Big Tech to both US Presidential campaigns.

The tech industry has a history of progressive members, but has recently blurred this paradigm with several high-profile endorsements and campaign contributions. It was hard to miss the outsized support for Donald Trump by Elon Musk, followed by other prominent technologists, including Mark Andresson, Peter Thiel, and Former Sequoia Capital head Douglas Leone [1][2]. Outspoken on the Democratic ticket for Vice President Kamala Harris include LinkedIn Founder Reid Hoffman, Mark Cuban, and Bill Gates. It’s a toss-up in cash and signaling. One thing that’s clear, however, is the willingness of tech leaders to convey a kind of neutrality, even if that includes removing any former progressive sentiment (such as the tech industry’s support of Biden in 2020). To this end, several Big Tech CEOs have made offerings to former President Trump along the campaign trail.

Meta CEO Mark Zuckerberg called Trump after his assassination attempt to commend his heroism. Google CEO Sundar Pichai allegedly praised his McDonalds visit. And Apple CEO Tim Cook apparently called in to discuss his frustration with EU fines levied on Apple. These reports are unclear and may be examples of flattery tailored to this President’s distinct ego. But they would align to an unusual courting of presidential candidates compared to previous election cycles. Vice President Harris has a history of interactions with the tech community in her home state of California, but in response to her recent declarations of tech regulation and anti-trust busting, active impartiality may be the best strategy for its leaders. On this note, we also saw Amazon founder Jeff Bezos remove the Washington Post’s longstanding tradition of endorsing presidential candidates, citing a “perception of bias” as a core concern.

Why the impartiality? A unifying theme is that tech leaders are more hostile to regulatory interference than ever. At the beginning of the year, the DoJ took aim at large mergers with Big Tech and its increasing acquisition activity of AI outfits, including Meta’s Instagram and Whatsapp purchases. In August, Google lost a major antitrust case in its search business. Antitrust lawsuits have also been filed between the DoJ and FCC against Amazon, Apple, Google, and Meta. Amidst this crackdown, tech giants have been battling fiercely to defend their moats and regulation to improve user privacy and safety laws. These include the Algorithmic Accountability Act, Federal Artificial Intelligence Risk Management Act , Kids Online Safety Act, and California’s Safe and Secure Innovation for Frontier Artificial Intelligence Models Act. A record number of legislation (120 AI bills in Congress) has put the tech industry on its toes, causing a groundswell of political spending action to safeguard an unimpeded future. Loyalties are being calculated on this condition to the detriment of fundamental protections and privacy for millions of Americans.

At the time of this article’s writing, the 2024 US Presidential Election was not called for either candidate. That has since changed. Donald Trump’s sweeping victory across the seven swing states has put the tech industry’s strategic positioning into sharper focus. Despite anti-regulatory positions in his previous term, Trump has shown disdain for Big Tech’s size and manufacturing developments offshore. It appears likely that Trump’s biggest backers, like Elon Musk, gain greater influence on policy by leveraging their close relationships with the president. It will also introduce considerable volatility due to his unpredictable nature regarding regulatory scrutiny. The tech landscape remains uncertain under a Trump administration 2.0, but it appears that direct relationships to the president’s ear are the surest bet to dictate what restrictions or anti-trust investigations are carried out or scrapped.

We saw personal appeals by tech leaders to Trump throughout the election, and we will likely see more of it in the coming years as tech companies seek to protect unmitigated AI development, acquisitions, and muted policy over their handling of the digital landscape. Whatever regulatory posture Trump takes toward Big Tech in 2025, the departments and bureaus he presides over are more likely to execute his will directly. This election has laid bare the lengths to which technocrats will go to ensure their interests are untouched. As trust in these tech organizations continues to plummet, the question remains: how much of our political future is up for determination by the few controlling our digital infrastructures? Or are we willing to fight back against the encroachment of our fundamental rights and, increasingly, democratic principles?

Further reading:

The scientific community’s embrace of AI

Safety Precautions by Yasmin Dwiputri & Data Hazards Project

This year, the Nobel committee decided to award two prizes to research advanced by AI systems. The prize in Chemistry was given to David Baker of the University of Washington, as well as Demis Hassabis and John Jumper of Google DeepMind for developing an AI model capable of accurately predicting proteins’ complex structures. The Nobel Prize in Physics was awarded to John Hopfield of Princeton University and Geoffrey Hinton of the University of Toronto for their artificial neural networks and data pattern reconstruction work developed in the 1980s. This work gave rise to the machine learning revolution that began around 2010.

While both groups of recipients are recognized for outstanding contributions to their fields, what does this mean? Why did the Nobel Committee recognize AI-driven research in two separate categories this year? Perhaps it is to signal that models have proven adept enough to perform under the rigorous standards demanded of world-class scholarship. However, given the enormous amount of computational power and data necessitating these models, it may also reveal an olive branch to the technology companies underpinning their use.

Scientists across the academic world are embracing the use of AI for their research and, with it, the incredible risks of a nascent technology enabled by large corporations. Consumer class LLMs like Chat GPT are known to hallucinate, and models made for scientific inquiry are no different. More pressingly, machine learning systems are deeply opaque. This means that whatever outcomes scientists using these systems may find statistically significant, they know nothing of the underlying mechanisms that made them. If they are able to contribute to the model’s training, unraveling their behavior post-training is detached from certainty. This has not stopped the progression of AI-enabled research, with a 270% increase in related publications from 2010 (from 88,000 to 240,00 in 2024). Data generated from these research models attempts to capture the natural world indirectly, which threatens to compromise the interpretability and reliability of results pursued by academic disciplines that normalize their use.

A downstream impact of this momentum is more fundamental to the disciplines themselves: originality. LLMs offer something we cannot do ourselves, which is finding patterns in tremendous complexity. But that is still data that we assemble from the observable universe.

Original inference reaches for nuance beyond what data can reveal on its own. Researchers must employ unique rationale instead of relying upon raw pattern analysis to be groundbreaking, especially for achievements as high as the Nobel Prize. Their hypothesis testing must stretch a field’s known boundaries. While generative AI supports creativity, it often discourages true originality, creating a culture that values compelling results but stifles genuine discovery.

In the competitive and funding-scarce environment that is academia, reckless AI adoption into research is an urgent issue. But as powerful models struggle to guarantee basic causal relationships, to say nothing of the argument that these data-dependent systems stand to make the scientific method obsolete, any scientist ought to tread lightly for their own sake of credibility.

Further reading:

Where the Hype Don’t Shine: the promise and perils of AI for developing countries

Clarote & AI4Media / Better Images of AI / Labour/Resources / CC-BY 4.0

In an interview with The Economic Times, a prestigious Indian newspaper, Microsoft CEO Satya Nadella asked about the distinction between what is a ‘developed’ and ‘developing’ country, of which the latter term faces controversy for its hierarchical implications globally: “At the end of the day, what’s the difference between being a developed country and a developing country? It’s just the rate of growth over long periods of time”. Mr. Nadella is, as Damien Vreznik of The Economist concluded, “haunted by the fact that the Industrial Revolution left behind India, his country of birth.” Indeed, many countries’ development spanning the first waves of industrialization to the rise of the internet lacked both participation and crucial investment until decades later. A similar pattern exists in the emergence of mass AI technology, with the West dominating much of the innovation and market share. Developing countries (or developing regions therein, such as India or Nigeria) appear slated once again to be passive recipients of this new technology, or the “fourth revolution” more broadly.

However, the nature of this technology is not akin to prior eras of mass innovation. Instead, the advent of AI may exist as the accelerant needed to lower traditionally steep barriers to entry for mass adoption and participation in development. AI-powered language models (LLMs) are advancing the democratization of information and learning, building upon the foundation laid by the internet by offering more personalized, interactive, and context-aware access to knowledge. Moreover, popular AI tools currently on the market claim to improve productivity within public information, financial and legal services, marketing and advertising, and software development. The AI ‘revolution’ is distinct from others in that the majority of its disruption has been across white-collar workers. According to the IMF, 30% of jobs in advanced economies are at risk for replacement by AI, compared to 20% in emerging markets and 18% in low-income countries.

Fortunately, the rate of expansion enabled by widespread internet access also separates the AI moment from the pack of innovation. As for ownership, model deployment is increasingly democratized, and the design of large model training accommodates localized data to fine-tune existing foundation models that originated in advanced tech markets. While compute and energy remain a necessity, developers no longer require scale-enabling resources to reinvent generative approaches according to their needs. To this end, an awakening of startups in India, Indonesia, Kenya, Nigeria, and elsewhere is helping to realize the potential of bespoke datasets to enrich models for their markets. As the cost of training AI models reduces (to the benefit of these nations), smaller and cheaper models in these countries can extend AI tools to domestic needs that would overlooked by big developers. For instance, agricultural monitoring models for smaller farmers with limited water supply and software knowledge; medical predictive models to detect region-specific diseases like malaria in sub-Saharan Africa; AI microfinance tools for largely unbanked populations; telemedicine AI to alleviate the strain on hospital; and freely-accessible educational apps for reskilling large youth populations dealing with teacher shortages. Past revolutions have disappointed many million members of nations who have stood to gain the most. While AI remains an ambiguous insurgence, there is, by its character, undeniable opportunity.

Further reading:

originally published in July 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

Elections Spotlight: what we’ve learned from India and South Africa

Jamillah Knowles & We and AI / Better Images of AI / People and Ivory Tower AI 2 / CC-BY 4.0

In the 2024 election supercycle, two nations encountered a spread of AI-enabled features for the very first time. In South Africa, AI played a positive role in some respects. Two AI chatbots in particular, named Thoko the Bot, have been used to provide voter education and information on the voting process, making it more accessible. Cybersecurity measures have also been implemented to protect against hacking and a vast influx of digital threats. According to Galix -a cybersecurity firm- in their report entitled “Check Point’s Threat Report for South Africa”, South Africa’s elections were likely to be barraged by a host of deepfakes, misinformation and disinformation. In the past, these have mostly targeted government and military organizations, “which receive over double the average amount of weekly attacks.” Among its 25 million online mobile users, South Africa is one of the most susceptible electorates to misinformation at scale in the world.

The main cultural concern of voter misinformation in South Africa concerns race, where there have already been eruptions of divisive protest in the past. In 2016, a firm called Bell Pottinger was hired by a wealthy family known as the Guptas to implement state capture in advancing the reputation of an investment company with connections to former South African President Jacob Zuma. Bell Pottinger ran a campaign on social media to stir up ideas of ‘white monopoly capital’ and ‘economic apartheid’ by spreading incorrect information to incite racial tension towards wealthy, white South Africans. During the Cape Town taxi protests in 2023, misinformation spread rapidly among the wider Black population, causing division. False reports circulated about unfair vehicle impounding, looting incidents, and the closure of public spaces like malls. Social media, amplified by AI algorithms, has the potential to exacerbate such misinformation campaigns, leading to even more widespread synthetic content and heightened tensions.

India’s electorate has seen swaths of deepfakes and misinformation campaigns swarm the country. Prime Minister Narendra Modi’s BJP faces off against major opposition parties, including the Indian National Congress (INC) and regional allies, in an election marred by allegations of voter bribery, concerns about vote counting integrity, and widespread misinformation, all of which are eroding public trust in the electoral process. While Modi’s BJP has won the election, they lost an outright majority, requiring a coalition. Each party will now need to work together more than ever before, despite reports of their widespread use of misinformation to target the leaders of each respective party. One shocking example was two deepfakes made of figures who had passed away before their production. In January, a video circulated of Muthuvel Karunanidhi, an Indian actor seen congratulating his son on his successful leadership of the Indian state Tamil Nadu. The same was seen of a woman named Duwaraka, daughter of a Tamil Tiger militant chief Velupillai Prabhakaran, giving a speech on their rights to freedom outside Indian administration. These deepfakes were created to leverage the emotional appeal and political influence of deceased figures, aiming to sway public opinion and garner support for specific political agendas.

Modi and his BJP have deployed a harsh and divisive campaign against his opponents and some minorities across the country, such as “demonizing” Muslims. The BJP is firmly rooted in a Hindu nationalist ideology called “Hindutva,” a doctrine that seeks to “establish a Hindu hegemony at the expense of religious minorities.” Following this rhetoric has been a flurry of synthetic media spread to millions of the Indian electorate. According to Nature, researchers using a sample of roughly two million WhatsApp messages from users in India found urgent concerns about the spread and prevalence of AI-generated political content. While it is still early days for AI-propagated political misinformation, the groundwork for its influence at scale is already being demonstrated in the world’s largest democracy. Following one of the most staggering parliamentary shakeups in the UK’s history on July 4th, how will AI media play a role in the transition of governments or in the strategies of rising groups such as Reform UK? In the United States, where candidate favorability is at an all time low, how will the electorate respond to anger-inducing synthetic media like content seen circulating across India?

Further reading:

originally published in July 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

Healthcare Foundation Models: where’s the breakthrough?

Illustration by Yarek Waszul

The medical world has a finicky relationship with AI and ML systems, and for good reason. As industries worldwide tinker with foundation models leveraging specialized data to find a bespoke machine-driven edge, healthcare researchers must grapple with the highest privacy standards and user (patient) procedures. This itself creates additional problems, such as a dearth in shared evaluation frameworks of budding AI that are crucial to machine learning fields everywhere. By the extremely delicate nature of their services, healthcare ML developers cannot enjoy the same try/fail process as their colleagues in other domains; however, they are in the midst of model creation that may push boundaries beyond the medical realm.

Since the beginning of the AI hype cycle two years ago, many researchers look at the lackluster predictive accuracy and factual inconsistency of LLMs and question whether such services would ever be worth investment into a safe and reliable healthcare service. Scholars at Stanford’s Human-Centered Artificial Intelligence Institute reviewed more than 80 clinical foundation models, including Clinical Language Models (CLaMs) and Foundation Models for Electronic Medical Records (FEMRs). Each model displayed considerable difficulty in evaluating but showed the potential to handle complex medical data without extensive labeling efforts. “The authors propose a new evaluation paradigm to better align with clinical value, emphasizing the need for clearer metrics and datasets in healthcare applications of LLMs. They acknowledge the risks, including data privacy concerns and interpretability issues, but remain optimistic about the potential of foundation models in addressing healthcare challenges.” More and more researchers are supporting this view, warranting new strategies for collecting medical datasets that can permit AI devices to classify and respond better. Michael Moor et al. from Stanford put forward a paper arguing for the feasibility of a generalist medical AI (GMAI) in 2023 to interpret an exceptional breadth of medical modalities (e.g., “imaging, electronic health records, laboratory results, genomics, graphs or medical text”) without little or any use of specialized data labeling. Such a sweeping model was unheard of only a couple of years ago in the medical community.

If these developments are adequate, new multipurpose foundation models can deliver on the medical intelligence revolution in three core areas: medical diagnosis, treatment planning, and drug discovery. AI diagnosis has already shown promise in oncology. Last March, The Economist broke the story of Barbara, a woman undergoing a mammogram last year in the UK whose doctors had disregarded a small but aggressive growth. A new AI diagnosis system named Mia correctly identified the six-millimetre patch as Stage 2 breast cancer, which would have evolved quickly between then and her next routine checkup. Outside precision oncology, these models have also been significant in diminishing hospital admission time, new drug development, and improving critical trial design to test the efficacy of a new product. Significant issues remain with these advances, however, with many researchers grappling with moral and accountability concerns in the AI-enabled drug development culture.

The Food and Drug Administration (FDA) in the United States continues to approve drug discovery and diagnostic services AI systems provide. After mitigating extensive data intake needs as discussed, the fundamental concern is whether the training and output of these AI systems is backed by effective oversight. Regulating a nascent technology is inherently fraught with more reactiveness than preemptive accuracy. Still, as NYT reporter Christina Jewett has learned covering the FDA: “In medicine, the cautionary tales about the unintended effects of artificial intelligence are already legendary.” Noticing the warning signs, ranging from overzealous development from commercial upstarts to opaque research from top institutions, is essential to promoting realistic dialogue outside the hype. Healthcare is drowning in a tsunami of data, and the urgency to wrangle it comes with the same intensity to protect it. Because of this, and as new models emerge, perhaps medical foundation models are one participant of the greater AI arc that have the best chance to get it right.

Further reading:

Artificial Systems in Warfare

US Department of Defense / Sgt. Cory D. Payne, public domain

As we enter the second half of 2024, the realm of warfare is undergoing a dramatic transformation. Lethal autonomous weapons systems (LAWS) guided by artificial intelligence have emerged on battlefields worldwide. This development has sparked intense debates among researchers, legal experts, and ethicists about how to control and regulate these high-tech killing machines. As these technologies become more accessible, their proliferation is accelerating. Israel, Russia, South Korea, and Turkey have reportedly deployed weapons with autonomous capabilities, while Australia, Britain, China, and the United States are investing heavily in LAWS development.

The Good Robot January newsletter covered one such transformation within the Israel Defense Force and their targeting of Hamas fighters with an AI-led geospatial classification system, “Gospel.” Concerns loom over the immense risks posed by putting responsibility into the hands of a machine learning classifier deployed for bombing decisions over one of the densest population areas in the world (Gaza). Soon, Gospel began to increase the frequency of strikes in Gaza, extending concerns into shock that surveillance data may be the primary training set used in the database for Gospel, of which its accuracy towards distinguishing civilians from Hamas would be critically flawed.

The Russia-Ukraine conflict has brought the use of AI weapons into sharp focus. Reports suggest that both sides have employed drones with varying degrees of autonomy. Russia has allegedly deployed the KUB-BLA, a suicide drone that uses AI targeting to attack ground targets. Meanwhile, Ukraine has utilized Turkish-made Bayraktar TB2 drones with some autonomous capabilities, as well as the US-designed Switchblade drones, capable of loitering over targets and identifying them using algorithms. Regrettably, the sheer volume of drones being used along the front lines is driving both sides towards greater automation in their weapons systems. With no internationally agreed norms on these weapons systems, these first field-based autonomous weapons may needlessly normalize a form of combat with weak humanitarian guardrails.

In a significant move, the United Nations is placing LAWS on the upcoming UN General Assembly meeting agenda this September. Secretary-General António Guterres is pushing for a ban on weapons that operate without human oversight by 2026, underscoring the urgency of addressing the ethical and legal implications mounting behind AI-powered weapons. The international community faces the challenge of balancing potential military advantages with ethical concerns and the need for human control. As it remains unmitigated, the next 2–3 years will be crucial in shaping the future of warfare and the role of AI within it.

Further reading:

EU AI Act: final release, response, and outlook

Anne Fehres and Luke Conroy & AI4Media / Better Images of AI / Hidden Labour of Internet Browsing / CC-BY 4.0

The highly anticipated EU AI Act was released in draft this January and has already encountered a series of concerns. The final draft of the Act was leaked to the public, displaying new definitions and requirements that differ substantially from its original proposal from April 2021. The tenants of the AI Act surround matching regulatory measures to the proportionate risk of a given AI system. The European Commission’s “AI Office” is tasked with enforcing and supervising the policies and was launched officially on February 21st with a mandate to engage with the scientific community as closely as possible. However, there remain worrying gaps where researchers need to take action:

(1) Self-policing risk assessment of AI developers, where the deployment of under-tested AI systems are often neglected by the community until they achieve an observable defect at a much higher adoption scale. This is where the AI Office may fail to identify unacceptable risk in AI systems on account of their failure to disclose internal design or information. According to the Center for Security and Emerging Technology, this unacceptable risk is characterized as “those that have a significant potential for manipulation either through subconscious messaging and stimuli, or by exploiting vulnerabilities like socioeconomic status, disability, or age. AI systems for social scoring, a term that describes the evaluation and treatment of people based on their social behavior, are also banned.”

(2) Classification of risk by the AI Office. The four risk levels defined in the Act are minimal risk, high risk, unacceptable risk, and specific transparency risk. Definitions for specific generative models or machine learning use cases are needed for better enforcement, which many researchers are concerned are too vague even as a starting point. Stuart Russell spoke on the draft: “They sent me a draft, and I sent them back 20 pages of comments. Anything not on their list of high-risk applications would not count, and the list excluded ChatGPT and most A.I. systems.”

(3) Enforcement within EU geography. A severe limitation to enforcement is unrestricted development outside of the European Union. Transmission of models, as seen by a host of foundation models from the United States as the basis for development abroad, is an irreversible trickle. While open source has been a key cultural feature in the AI community, their exclusion from regulation (notably the secrecy of the underlying software) ensures vulnerabilities that must be policed retroactively.

(4) Pace. Lack of rules have left a vacuum for high tech companies to expand AI development for existing products and invest in new programs, which may continue at similar scale in Europe from US big tech despite the AI Act. Whether the Act’s design and deployment regulations can match AI’s speed of improvement (known to have broken Moore’s law), its provisions remain the most comprehensive legislation in the world for general-purpose models. Its impact remains to be seen but will likely serve as the best available answer for an exploding experiment.

(5) Competition. The AI Act targets providers and deployers of AI systems, with providers held to the strictest requirements. Substantial financial penalties and limitations have some policymakers concerned about stifling innovation. While the law comes with an exemption for models used purely for research, small AI companies translating this research in Europe face difficulties in growth under these laws. “To adapt as a small company is really hard,” says Robert Kaczmarczyk, a physician at the Technical University of Munich in Germany and co-founder of LAION (Large-scale Artificial Intelligence Open Network).

Closing out February, Mistral AI drew deep scrutiny from European Parliament members following its newest LLM unveiling, Mistral Large, alongside an investment partnership with Microsoft Azure. Mistral had lobbied for looser AI regulation laws preceding this partnership, and as Europe’s foremost LLM software company, signals a rejection of comprehensive oversight for unrestrained growth objectives.

Further reading

originally published in February 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

AI-generated sexism and “the synthetic mirror”

Clarote & AI4Media / Better Images of AI / User/Chimera / CC-BY 4.0

When Melissa Heikkilä, a Senior Tech Reporter for MIT Tech review, image prompted the AI avatar app Lensa, she encountered distinctive results from her male counterparts. While the men received digital portraits as explorers or inventors, her generated images were ‘cartoonishly pornified.’ Generative visual AI models have a reoccurring history of bias and often extremely stereotypical or dangerous training. Stable Diffusion powers Lensa and thousands of other image generators and was built using LAION-5B, an enormous open-source data set. LAION-5B, like all generative trainer data sets, was constructed by amassing images from the internet. This points to a larger online crisis surrounding the proliferation of biases involving sexist and racist stereotypes. Heikkilä notes that while not all image generative models use open-sourcing data (e.g. Google’s Imagen and Open AI’s DALL-E), “they are built in a similar way”.

What efforts can be made to detoxify the data fueling generative AI? The outstanding consideration is how LAION and other training sets hold up a mirror to the world and our unspoken biases. Melanie Mitchell, a Professor at the Santa Fe Institute, charged that the associations made from this data within generative models are concerning as much as they are sweepingly generic. Quoted in Bloomberg, “When they start then generating new language, they rely on those associations to generate the language, which itself can be biased in racist, sexist and other ways.”

This last month, however, Google Gemini hinted at the concerns of overcorrecting for these biases. According to Wired, Gemini produced historically inaccurate images of ‘excessive’ representation, such as “exclusively Black people in traditional Viking garb” or “Indigenous people in colonial outfits.” Google has paused Gemini’s people generation feature until they could correct this issue. Jack Krawczyk, a senior director at Gemini, posted on X: “We design our image generation capabilities to reflect our global user base, and we take representation and bias seriously. We will continue to do this for open-ended prompts (images of a person walking a dog are universal!) Historical contexts have more nuance to them and we will further tune to accommodate that.” Gemini’s data training corrections, while shown to exhibit inaccuracies, is one approach to a more significant representational conflict of internet image training data. In the coming months, this conversation will indeed unfold into further efforts to find a curated picture that represents humanity and to remove our ugliest, deep-seated projections.

Shanon Vallor, a philosopher of technology at the Edinburgh Futures Institute and our most recent podcast guest, coined The AI Mirror as her recent book title, advancing the idea that AI’s potential is marred by its influence of the past in making associations from existing data. “…today’s powerful AI technologies reproduce the past. Forged from oceans of our data into immensely powerful but flawed mirrors, they reflect the same errors, biases, and failures of wisdom that we strive to escape.” While generative AI (and other robust emerging systems) offers an extraordinary recipe to enrich human flourishing, we must also take ownership of its ingredients. Challenging open-source as the best available approach for development is one step under which compelling research continues. While we are predisposed to the internet data we have, increasing the public conversation online to take ownership of our content will allow us to turn the tide.

Further listening + reading:

originally published in February 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

Davos 2024: key takeaways and concerns in responsible technology

AI dominated conversations and panels at this year’s World Economic Forum in Davos (Jan 15–19). From economic enclaves and science corners to policy and governance debates across conference domains, executives and global voices were seemingly all compelled to address the AI moment.

originally published in January 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

While many of these dialogues were meshed in commercial opportunities, ethics discussions also had their day. “There’s a chance to find some ethical solutions to the AI dilemmas we’re facing,” said Oleg Lavrovsky (VP of Opendata.ch), “I hope that people here will be inspired to open up to each other and find some good bridges across science and industry and governance.”

A central topic here is the concern of whether AI would help or hamper structural inequality across the world. Economic experts weighed in on the tightening of financial conditions and regional development opportunities encountered by widespread AI and recent tech advances at a challenging moment. Al Jazeera reported on a WEF-Davos survey of chief economists, saying, “Economists predict weakened global economic conditions while technology will help promote differences across regions.”

A new report this month by the IMF signaled that 60% of advanced economy jobs would be impacted by AI, whereas low-income countries would realize a 26% impact. As AI tools’ earliest adoption and learning allow economies and governments more time to adapt, AI’s slower uptake for productivity improvements in these countries is a serious inequality concern. The report considered, “This could exacerbate the digital divide and cross-country income disparity.”

At the end of Davos, the World Economic Forum’s AI Governance Alliance released three papers to confront equitable access concerns and proposals that will minimize “widening existing digital divides”.

Here’s four of the Key Takeaways from Davos according to the WeForum:

  1. Leaders need to ‘pull together’

  2. ‘Projections are not destiny’

  3. ‘Humans are going to have better tools’

  4. ‘Urgency is our only saviour’

Further listening + reading from Davos:

AI in Gaza: “The Gospel”, Red Wolf and more

originally published in January 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

According to NPR, Israeli forces have struck more than 22,000 targets in Gaza leading into 2024, which President Joe Biden as characterized as “indiscriminate bombing” (AP News). Israeli Defense Forces state that all strikes are carried out with precision, a claim they increasingly validate with their usage of an AI systems for targeting.

“The Gospel” is the primary AI system used by the Israeli military, which takes in enormous quantity of surveillance data, crunches it together and makes recommendations about where the military should strike. Surveillance data includes the likes of drone footage and satellite images, as well as people’s cell phone conversations and locations.

A growing concern among researchers is that while such an AI system is good at sorting through surveillance data, there is disagreement on whether it can deliver accurate targeting results. The Gospel can be efficient in its training and outperform human analysts, but the central danger is that this AI system was only trained on correct targets and not incorrect examples, such as civilians living in Gaza. NPR spoke with Heidi Khlaaf, an AI expert who runs Trail of Bits, who is “very critical of the idea of putting AI in charge of targeting, in part because she thinks the training data just won’t be good enough to tell a Hamas fighter from a civilian”. Heidi stated, “You’re ultimately looking at biased and imprecise targeting automation that’s really not far from indiscriminate targeting”.

Since the AI system trains itself, this also threatens to delude the point of intention and accountability for targeting decisions made by Gospel. “it then becomes impossible to trace decisions to specific design points that can hold individuals or military accountable for AI’s actions”, said Heidi.

Furthermore, a report called “Automated Apartheid” by Amnesty International found that Israel has been deploying facial recognition software, known as “Red Wolf,” to control Palestinian locations and restrict movement. This is an early example of a government or institution using AI-enabled surveillance against an ethnic group.

As 2024 unfolds, keep a lookout for growing usage of AI surveillance systems in defense targeting, discriminatory movement restrictions, as well as recognition software for large public gatherings, notably, protests.

Further reading:

A concrete and actionable plan for making yourself smarter

To most-effectively determine an actionable plan to increase intelligence, one must first consider the best of standards by which it can be measured and coordinated into mental gains. The most promising encouragement of this task comes from Richard Nesbit in Intelligence and How to Get It, where, when deliberating the relevance of hereditary intelligence, proclaims “the degree of heritability of IQ places no constraint on the degree of modifiability that is possible” (pg. 38). Intuitively, I imagine a structure set about some kind of conditioning for the mind, such as IQ puzzles or memory tests, but it is important not to overlook the easy fixes as well. For instance, if we are defining the word ‘smarter’ comprehensively, one can educate themselves on the most effective external mechanisms that allow him/her to solve problems (mental or not) more swiftly. As evidenced by Fisher, Counts, and Kittur in “Distributed Sensemaking: Improving Sensemaking by Leveraging the Efforts of Previous Users”, one such mechanism would be using others (friends, computers, or anything with additional brain power) to make more efficient sensemaking and deduction.

Then there is the question of defining intelligence. As our prompt asks for actionable representations of growth from our minds, or “evidence on practical outcomes of intelligence differences” (202, Deary, Penke, Johnson), we should prioritize the most transparent methods of demonstration, which include those discernable via paper result, positron emission tomography, regional cerebral blood flow analysis, and functional MRI (fMRI).

Information recall is one such demonstration. The understanding of an individual can be trialed via trivia, memorization, and potentially even multi-tasking examinations that can observably probe at their base intelligence measured over a period and any change in performance.

To this end, one should first adhere to the architecture of a transactive memory system as mentioned by Fisher, Goddu, and Keil in “Searching for Explanations: How the Internet Inflates Estimates of Internal Knowledge”, wherein one designates roles so as to maximize the ultimate output. A system rooted upon specialization can eliminate redundancy and other loud problems permeating intellectual performance. This can best be accomplished by establishing a consistent initiative to learn and retain internally (as opposed to misguiding confidence in one’s reliance on knowledge-access-mediums like the Internet).

Operating by this system, the person should then focus in on each competency underlying general g. For Verbal, they can practice articulation and the ability to recite and apprehend complex material. With Spatial, they can work with shape configurations and navigation. For mathematical intelligence, they should round out their current math acumen and work to become comfortable with abstract proofs and other ideas. One could diversify this further and follow Gardner’s model of multiple intelligences but should exclude those less demonstratable, such as existential intelligence.

To make these more implementable, one should form an unbreaking routine of drilling and exercise throughout each of the intellectual competencies, working during the most productive moments of the day and ensuring adequate time for sleep and subconscious processing. One will have to balance this among the demands of societal and professional life, making an inquisitive effort to inject learning or subconscious activation where relevant. Given the scale of tangible intelligence to develop, the durations of learning (weeks, months, years) are up for participant discretion, but should not be so long or short to hamper the reliability of measurement.

Before closing, it is important to recognize the less-readily observable, those prone naturally to the bedrock of intelligence in humans, such as aptitude and heritable traits, and the less significant variances elsewhere, like height. These divergences among people threaten the concreteness of a growth plan for intelligence but do not rule out its potential. Venturing to sophisticate one’s most fundamental competencies and prioritizing the monitoring of those with the most transparent evaluation methods is, too, the clearest means of making the self smarter.

Sources:

1.     Nesbit, Richard Intelligence and How to Get It

2.     Fisher, Counts, Kittur, “Distributed Sensemaking: Improving Sensemaking by Leveraging the Efforts of Previous Users”

3.     Deary, Penke and Johnson, The neuroscience of human intelligence differences

4.     Powerpoint Lecture Slides for 88-230, Danny Oppenheimer