Tech Affairs & Macro Dev

Where the Hype Don’t Shine: the promise and perils of AI for developing countries

Clarote & AI4Media / Better Images of AI / Labour/Resources / CC-BY 4.0

In an interview with The Economic Times, a prestigious Indian newspaper, Microsoft CEO Satya Nadella asked about the distinction between what is a ‘developed’ and ‘developing’ country, of which the latter term faces controversy for its hierarchical implications globally: “At the end of the day, what’s the difference between being a developed country and a developing country? It’s just the rate of growth over long periods of time”. Mr. Nadella is, as Damien Vreznik of The Economist concluded, “haunted by the fact that the Industrial Revolution left behind India, his country of birth.” Indeed, many countries’ development spanning the first waves of industrialization to the rise of the internet lacked both participation and crucial investment until decades later. A similar pattern exists in the emergence of mass AI technology, with the West dominating much of the innovation and market share. Developing countries (or developing regions therein, such as India or Nigeria) appear slated once again to be passive recipients of this new technology, or the “fourth revolution” more broadly.

However, the nature of this technology is not akin to prior eras of mass innovation. Instead, the advent of AI may exist as the accelerant needed to lower traditionally steep barriers to entry for mass adoption and participation in development. AI-powered language models (LLMs) are advancing the democratization of information and learning, building upon the foundation laid by the internet by offering more personalized, interactive, and context-aware access to knowledge. Moreover, popular AI tools currently on the market claim to improve productivity within public information, financial and legal services, marketing and advertising, and software development. The AI ‘revolution’ is distinct from others in that the majority of its disruption has been across white-collar workers. According to the IMF, 30% of jobs in advanced economies are at risk for replacement by AI, compared to 20% in emerging markets and 18% in low-income countries.

Fortunately, the rate of expansion enabled by widespread internet access also separates the AI moment from the pack of innovation. As for ownership, model deployment is increasingly democratized, and the design of large model training accommodates localized data to fine-tune existing foundation models that originated in advanced tech markets. While compute and energy remain a necessity, developers no longer require scale-enabling resources to reinvent generative approaches according to their needs. To this end, an awakening of startups in India, Indonesia, Kenya, Nigeria, and elsewhere is helping to realize the potential of bespoke datasets to enrich models for their markets. As the cost of training AI models reduces (to the benefit of these nations), smaller and cheaper models in these countries can extend AI tools to domestic needs that would overlooked by big developers. For instance, agricultural monitoring models for smaller farmers with limited water supply and software knowledge; medical predictive models to detect region-specific diseases like malaria in sub-Saharan Africa; AI microfinance tools for largely unbanked populations; telemedicine AI to alleviate the strain on hospital; and freely-accessible educational apps for reskilling large youth populations dealing with teacher shortages. Past revolutions have disappointed many million members of nations who have stood to gain the most. While AI remains an ambiguous insurgence, there is, by its character, undeniable opportunity.

Further reading:

originally published in July 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

Elections Spotlight: what we’ve learned from India and South Africa

Jamillah Knowles & We and AI / Better Images of AI / People and Ivory Tower AI 2 / CC-BY 4.0

In the 2024 election supercycle, two nations encountered a spread of AI-enabled features for the very first time. In South Africa, AI played a positive role in some respects. Two AI chatbots in particular, named Thoko the Bot, have been used to provide voter education and information on the voting process, making it more accessible. Cybersecurity measures have also been implemented to protect against hacking and a vast influx of digital threats. According to Galix -a cybersecurity firm- in their report entitled “Check Point’s Threat Report for South Africa”, South Africa’s elections were likely to be barraged by a host of deepfakes, misinformation and disinformation. In the past, these have mostly targeted government and military organizations, “which receive over double the average amount of weekly attacks.” Among its 25 million online mobile users, South Africa is one of the most susceptible electorates to misinformation at scale in the world.

The main cultural concern of voter misinformation in South Africa concerns race, where there have already been eruptions of divisive protest in the past. In 2016, a firm called Bell Pottinger was hired by a wealthy family known as the Guptas to implement state capture in advancing the reputation of an investment company with connections to former South African President Jacob Zuma. Bell Pottinger ran a campaign on social media to stir up ideas of ‘white monopoly capital’ and ‘economic apartheid’ by spreading incorrect information to incite racial tension towards wealthy, white South Africans. During the Cape Town taxi protests in 2023, misinformation spread rapidly among the wider Black population, causing division. False reports circulated about unfair vehicle impounding, looting incidents, and the closure of public spaces like malls. Social media, amplified by AI algorithms, has the potential to exacerbate such misinformation campaigns, leading to even more widespread synthetic content and heightened tensions.

India’s electorate has seen swaths of deepfakes and misinformation campaigns swarm the country. Prime Minister Narendra Modi’s BJP faces off against major opposition parties, including the Indian National Congress (INC) and regional allies, in an election marred by allegations of voter bribery, concerns about vote counting integrity, and widespread misinformation, all of which are eroding public trust in the electoral process. While Modi’s BJP has won the election, they lost an outright majority, requiring a coalition. Each party will now need to work together more than ever before, despite reports of their widespread use of misinformation to target the leaders of each respective party. One shocking example was two deepfakes made of figures who had passed away before their production. In January, a video circulated of Muthuvel Karunanidhi, an Indian actor seen congratulating his son on his successful leadership of the Indian state Tamil Nadu. The same was seen of a woman named Duwaraka, daughter of a Tamil Tiger militant chief Velupillai Prabhakaran, giving a speech on their rights to freedom outside Indian administration. These deepfakes were created to leverage the emotional appeal and political influence of deceased figures, aiming to sway public opinion and garner support for specific political agendas.

Modi and his BJP have deployed a harsh and divisive campaign against his opponents and some minorities across the country, such as “demonizing” Muslims. The BJP is firmly rooted in a Hindu nationalist ideology called “Hindutva,” a doctrine that seeks to “establish a Hindu hegemony at the expense of religious minorities.” Following this rhetoric has been a flurry of synthetic media spread to millions of the Indian electorate. According to Nature, researchers using a sample of roughly two million WhatsApp messages from users in India found urgent concerns about the spread and prevalence of AI-generated political content. While it is still early days for AI-propagated political misinformation, the groundwork for its influence at scale is already being demonstrated in the world’s largest democracy. Following one of the most staggering parliamentary shakeups in the UK’s history on July 4th, how will AI media play a role in the transition of governments or in the strategies of rising groups such as Reform UK? In the United States, where candidate favorability is at an all time low, how will the electorate respond to anger-inducing synthetic media like content seen circulating across India?

Further reading:

originally published in July 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

Healthcare Foundation Models: where’s the breakthrough?

Illustration by Yarek Waszul

The medical world has a finicky relationship with AI and ML systems, and for good reason. As industries worldwide tinker with foundation models leveraging specialized data to find a bespoke machine-driven edge, healthcare researchers must grapple with the highest privacy standards and user (patient) procedures. This itself creates additional problems, such as a dearth in shared evaluation frameworks of budding AI that are crucial to machine learning fields everywhere. By the extremely delicate nature of their services, healthcare ML developers cannot enjoy the same try/fail process as their colleagues in other domains; however, they are in the midst of model creation that may push boundaries beyond the medical realm.

Since the beginning of the AI hype cycle two years ago, many researchers look at the lackluster predictive accuracy and factual inconsistency of LLMs and question whether such services would ever be worth investment into a safe and reliable healthcare service. Scholars at Stanford’s Human-Centered Artificial Intelligence Institute reviewed more than 80 clinical foundation models, including Clinical Language Models (CLaMs) and Foundation Models for Electronic Medical Records (FEMRs). Each model displayed considerable difficulty in evaluating but showed the potential to handle complex medical data without extensive labeling efforts. “The authors propose a new evaluation paradigm to better align with clinical value, emphasizing the need for clearer metrics and datasets in healthcare applications of LLMs. They acknowledge the risks, including data privacy concerns and interpretability issues, but remain optimistic about the potential of foundation models in addressing healthcare challenges.” More and more researchers are supporting this view, warranting new strategies for collecting medical datasets that can permit AI devices to classify and respond better. Michael Moor et al. from Stanford put forward a paper arguing for the feasibility of a generalist medical AI (GMAI) in 2023 to interpret an exceptional breadth of medical modalities (e.g., “imaging, electronic health records, laboratory results, genomics, graphs or medical text”) without little or any use of specialized data labeling. Such a sweeping model was unheard of only a couple of years ago in the medical community.

If these developments are adequate, new multipurpose foundation models can deliver on the medical intelligence revolution in three core areas: medical diagnosis, treatment planning, and drug discovery. AI diagnosis has already shown promise in oncology. Last March, The Economist broke the story of Barbara, a woman undergoing a mammogram last year in the UK whose doctors had disregarded a small but aggressive growth. A new AI diagnosis system named Mia correctly identified the six-millimetre patch as Stage 2 breast cancer, which would have evolved quickly between then and her next routine checkup. Outside precision oncology, these models have also been significant in diminishing hospital admission time, new drug development, and improving critical trial design to test the efficacy of a new product. Significant issues remain with these advances, however, with many researchers grappling with moral and accountability concerns in the AI-enabled drug development culture.

The Food and Drug Administration (FDA) in the United States continues to approve drug discovery and diagnostic services AI systems provide. After mitigating extensive data intake needs as discussed, the fundamental concern is whether the training and output of these AI systems is backed by effective oversight. Regulating a nascent technology is inherently fraught with more reactiveness than preemptive accuracy. Still, as NYT reporter Christina Jewett has learned covering the FDA: “In medicine, the cautionary tales about the unintended effects of artificial intelligence are already legendary.” Noticing the warning signs, ranging from overzealous development from commercial upstarts to opaque research from top institutions, is essential to promoting realistic dialogue outside the hype. Healthcare is drowning in a tsunami of data, and the urgency to wrangle it comes with the same intensity to protect it. Because of this, and as new models emerge, perhaps medical foundation models are one participant of the greater AI arc that have the best chance to get it right.

Further reading:

Artificial Systems in Warfare

US Department of Defense / Sgt. Cory D. Payne, public domain

As we enter the second half of 2024, the realm of warfare is undergoing a dramatic transformation. Lethal autonomous weapons systems (LAWS) guided by artificial intelligence have emerged on battlefields worldwide. This development has sparked intense debates among researchers, legal experts, and ethicists about how to control and regulate these high-tech killing machines. As these technologies become more accessible, their proliferation is accelerating. Israel, Russia, South Korea, and Turkey have reportedly deployed weapons with autonomous capabilities, while Australia, Britain, China, and the United States are investing heavily in LAWS development.

The Good Robot January newsletter covered one such transformation within the Israel Defense Force and their targeting of Hamas fighters with an AI-led geospatial classification system, “Gospel.” Concerns loom over the immense risks posed by putting responsibility into the hands of a machine learning classifier deployed for bombing decisions over one of the densest population areas in the world (Gaza). Soon, Gospel began to increase the frequency of strikes in Gaza, extending concerns into shock that surveillance data may be the primary training set used in the database for Gospel, of which its accuracy towards distinguishing civilians from Hamas would be critically flawed.

The Russia-Ukraine conflict has brought the use of AI weapons into sharp focus. Reports suggest that both sides have employed drones with varying degrees of autonomy. Russia has allegedly deployed the KUB-BLA, a suicide drone that uses AI targeting to attack ground targets. Meanwhile, Ukraine has utilized Turkish-made Bayraktar TB2 drones with some autonomous capabilities, as well as the US-designed Switchblade drones, capable of loitering over targets and identifying them using algorithms. Regrettably, the sheer volume of drones being used along the front lines is driving both sides towards greater automation in their weapons systems. With no internationally agreed norms on these weapons systems, these first field-based autonomous weapons may needlessly normalize a form of combat with weak humanitarian guardrails.

In a significant move, the United Nations is placing LAWS on the upcoming UN General Assembly meeting agenda this September. Secretary-General António Guterres is pushing for a ban on weapons that operate without human oversight by 2026, underscoring the urgency of addressing the ethical and legal implications mounting behind AI-powered weapons. The international community faces the challenge of balancing potential military advantages with ethical concerns and the need for human control. As it remains unmitigated, the next 2–3 years will be crucial in shaping the future of warfare and the role of AI within it.

Further reading:

EU AI Act: final release, response, and outlook

Anne Fehres and Luke Conroy & AI4Media / Better Images of AI / Hidden Labour of Internet Browsing / CC-BY 4.0

The highly anticipated EU AI Act was released in draft this January and has already encountered a series of concerns. The final draft of the Act was leaked to the public, displaying new definitions and requirements that differ substantially from its original proposal from April 2021. The tenants of the AI Act surround matching regulatory measures to the proportionate risk of a given AI system. The European Commission’s “AI Office” is tasked with enforcing and supervising the policies and was launched officially on February 21st with a mandate to engage with the scientific community as closely as possible. However, there remain worrying gaps where researchers need to take action:

(1) Self-policing risk assessment of AI developers, where the deployment of under-tested AI systems are often neglected by the community until they achieve an observable defect at a much higher adoption scale. This is where the AI Office may fail to identify unacceptable risk in AI systems on account of their failure to disclose internal design or information. According to the Center for Security and Emerging Technology, this unacceptable risk is characterized as “those that have a significant potential for manipulation either through subconscious messaging and stimuli, or by exploiting vulnerabilities like socioeconomic status, disability, or age. AI systems for social scoring, a term that describes the evaluation and treatment of people based on their social behavior, are also banned.”

(2) Classification of risk by the AI Office. The four risk levels defined in the Act are minimal risk, high risk, unacceptable risk, and specific transparency risk. Definitions for specific generative models or machine learning use cases are needed for better enforcement, which many researchers are concerned are too vague even as a starting point. Stuart Russell spoke on the draft: “They sent me a draft, and I sent them back 20 pages of comments. Anything not on their list of high-risk applications would not count, and the list excluded ChatGPT and most A.I. systems.”

(3) Enforcement within EU geography. A severe limitation to enforcement is unrestricted development outside of the European Union. Transmission of models, as seen by a host of foundation models from the United States as the basis for development abroad, is an irreversible trickle. While open source has been a key cultural feature in the AI community, their exclusion from regulation (notably the secrecy of the underlying software) ensures vulnerabilities that must be policed retroactively.

(4) Pace. Lack of rules have left a vacuum for high tech companies to expand AI development for existing products and invest in new programs, which may continue at similar scale in Europe from US big tech despite the AI Act. Whether the Act’s design and deployment regulations can match AI’s speed of improvement (known to have broken Moore’s law), its provisions remain the most comprehensive legislation in the world for general-purpose models. Its impact remains to be seen but will likely serve as the best available answer for an exploding experiment.

(5) Competition. The AI Act targets providers and deployers of AI systems, with providers held to the strictest requirements. Substantial financial penalties and limitations have some policymakers concerned about stifling innovation. While the law comes with an exemption for models used purely for research, small AI companies translating this research in Europe face difficulties in growth under these laws. “To adapt as a small company is really hard,” says Robert Kaczmarczyk, a physician at the Technical University of Munich in Germany and co-founder of LAION (Large-scale Artificial Intelligence Open Network).

Closing out February, Mistral AI drew deep scrutiny from European Parliament members following its newest LLM unveiling, Mistral Large, alongside an investment partnership with Microsoft Azure. Mistral had lobbied for looser AI regulation laws preceding this partnership, and as Europe’s foremost LLM software company, signals a rejection of comprehensive oversight for unrestrained growth objectives.

Further reading

originally published in February 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

AI-generated sexism and “the synthetic mirror”

Clarote & AI4Media / Better Images of AI / User/Chimera / CC-BY 4.0

When Melissa Heikkilä, a Senior Tech Reporter for MIT Tech review, image prompted the AI avatar app Lensa, she encountered distinctive results from her male counterparts. While the men received digital portraits as explorers or inventors, her generated images were ‘cartoonishly pornified.’ Generative visual AI models have a reoccurring history of bias and often extremely stereotypical or dangerous training. Stable Diffusion powers Lensa and thousands of other image generators and was built using LAION-5B, an enormous open-source data set. LAION-5B, like all generative trainer data sets, was constructed by amassing images from the internet. This points to a larger online crisis surrounding the proliferation of biases involving sexist and racist stereotypes. Heikkilä notes that while not all image generative models use open-sourcing data (e.g. Google’s Imagen and Open AI’s DALL-E), “they are built in a similar way”.

What efforts can be made to detoxify the data fueling generative AI? The outstanding consideration is how LAION and other training sets hold up a mirror to the world and our unspoken biases. Melanie Mitchell, a Professor at the Santa Fe Institute, charged that the associations made from this data within generative models are concerning as much as they are sweepingly generic. Quoted in Bloomberg, “When they start then generating new language, they rely on those associations to generate the language, which itself can be biased in racist, sexist and other ways.”

This last month, however, Google Gemini hinted at the concerns of overcorrecting for these biases. According to Wired, Gemini produced historically inaccurate images of ‘excessive’ representation, such as “exclusively Black people in traditional Viking garb” or “Indigenous people in colonial outfits.” Google has paused Gemini’s people generation feature until they could correct this issue. Jack Krawczyk, a senior director at Gemini, posted on X: “We design our image generation capabilities to reflect our global user base, and we take representation and bias seriously. We will continue to do this for open-ended prompts (images of a person walking a dog are universal!) Historical contexts have more nuance to them and we will further tune to accommodate that.” Gemini’s data training corrections, while shown to exhibit inaccuracies, is one approach to a more significant representational conflict of internet image training data. In the coming months, this conversation will indeed unfold into further efforts to find a curated picture that represents humanity and to remove our ugliest, deep-seated projections.

Shanon Vallor, a philosopher of technology at the Edinburgh Futures Institute and our most recent podcast guest, coined The AI Mirror as her recent book title, advancing the idea that AI’s potential is marred by its influence of the past in making associations from existing data. “…today’s powerful AI technologies reproduce the past. Forged from oceans of our data into immensely powerful but flawed mirrors, they reflect the same errors, biases, and failures of wisdom that we strive to escape.” While generative AI (and other robust emerging systems) offers an extraordinary recipe to enrich human flourishing, we must also take ownership of its ingredients. Challenging open-source as the best available approach for development is one step under which compelling research continues. While we are predisposed to the internet data we have, increasing the public conversation online to take ownership of our content will allow us to turn the tide.

Further listening + reading:

originally published in February 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

Davos 2024: key takeaways and concerns in responsible technology

AI dominated conversations and panels at this year’s World Economic Forum in Davos (Jan 15–19). From economic enclaves and science corners to policy and governance debates across conference domains, executives and global voices were seemingly all compelled to address the AI moment.

originally published in January 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

While many of these dialogues were meshed in commercial opportunities, ethics discussions also had their day. “There’s a chance to find some ethical solutions to the AI dilemmas we’re facing,” said Oleg Lavrovsky (VP of Opendata.ch), “I hope that people here will be inspired to open up to each other and find some good bridges across science and industry and governance.”

A central topic here is the concern of whether AI would help or hamper structural inequality across the world. Economic experts weighed in on the tightening of financial conditions and regional development opportunities encountered by widespread AI and recent tech advances at a challenging moment. Al Jazeera reported on a WEF-Davos survey of chief economists, saying, “Economists predict weakened global economic conditions while technology will help promote differences across regions.”

A new report this month by the IMF signaled that 60% of advanced economy jobs would be impacted by AI, whereas low-income countries would realize a 26% impact. As AI tools’ earliest adoption and learning allow economies and governments more time to adapt, AI’s slower uptake for productivity improvements in these countries is a serious inequality concern. The report considered, “This could exacerbate the digital divide and cross-country income disparity.”

At the end of Davos, the World Economic Forum’s AI Governance Alliance released three papers to confront equitable access concerns and proposals that will minimize “widening existing digital divides”.

Here’s four of the Key Takeaways from Davos according to the WeForum:

  1. Leaders need to ‘pull together’

  2. ‘Projections are not destiny’

  3. ‘Humans are going to have better tools’

  4. ‘Urgency is our only saviour’

Further listening + reading from Davos:

AI in Gaza: “The Gospel”, Red Wolf and more

originally published in January 2024 under the University of Cambridge newsletter “The Good Robot Podcast”

According to NPR, Israeli forces have struck more than 22,000 targets in Gaza leading into 2024, which President Joe Biden as characterized as “indiscriminate bombing” (AP News). Israeli Defense Forces state that all strikes are carried out with precision, a claim they increasingly validate with their usage of an AI systems for targeting.

“The Gospel” is the primary AI system used by the Israeli military, which takes in enormous quantity of surveillance data, crunches it together and makes recommendations about where the military should strike. Surveillance data includes the likes of drone footage and satellite images, as well as people’s cell phone conversations and locations.

A growing concern among researchers is that while such an AI system is good at sorting through surveillance data, there is disagreement on whether it can deliver accurate targeting results. The Gospel can be efficient in its training and outperform human analysts, but the central danger is that this AI system was only trained on correct targets and not incorrect examples, such as civilians living in Gaza. NPR spoke with Heidi Khlaaf, an AI expert who runs Trail of Bits, who is “very critical of the idea of putting AI in charge of targeting, in part because she thinks the training data just won’t be good enough to tell a Hamas fighter from a civilian”. Heidi stated, “You’re ultimately looking at biased and imprecise targeting automation that’s really not far from indiscriminate targeting”.

Since the AI system trains itself, this also threatens to delude the point of intention and accountability for targeting decisions made by Gospel. “it then becomes impossible to trace decisions to specific design points that can hold individuals or military accountable for AI’s actions”, said Heidi.

Furthermore, a report called “Automated Apartheid” by Amnesty International found that Israel has been deploying facial recognition software, known as “Red Wolf,” to control Palestinian locations and restrict movement. This is an early example of a government or institution using AI-enabled surveillance against an ethnic group.

As 2024 unfolds, keep a lookout for growing usage of AI surveillance systems in defense targeting, discriminatory movement restrictions, as well as recognition software for large public gatherings, notably, protests.

Further reading: