IT and Society | Ethics in IT

Slide 1 of 35  |  ETH-W3B  |  Week 3 of 8  |  Chapter 8
IT and
Society
Productivity  •  AI Workforce Impact  •  Digital Divide  •  Algorithmic Bias  •  IBM Watson
In 2013, MD Anderson Cancer Center partnered with IBM to deploy Watson for Oncology -- an AI system that would help oncologists identify cancer treatment options. By 2017, internal documents obtained by STAT News showed Watson was producing "unsafe and incorrect" treatment recommendations. The project was quietly canceled. The question is not whether AI can assist medicine -- it can. The question is what obligations exist when we deploy systems we do not yet fully understand in domains where errors kill people.
35 Slides ETH-W3B Week 3 of 8 Chapter 8
Slide 2 of 35
IT and Productivity
The relationship between information technology investment and economic productivity is more complicated than the popular narrative suggests.
The Productivity Paradox
Robert Solow observed in 1987: "You can see the computer age everywhere but in the productivity statistics." Despite massive IT investment throughout the 1970s and 1980s, aggregate productivity growth was slow. Later analyses suggested benefits did materialize in the 1990s -- but with a significant lag between investment and measurable economic output.
Measurement Problems
GDP and traditional productivity measures may undercount IT's benefits. A word processor eliminated secretarial labor but also enabled individuals to do more. Email eliminated letter-writing time but also created new time demands. The real productivity impact may be in quality improvements and new categories of value that GDP does not capture well.
Uneven Distribution
Productivity gains from IT are not distributed uniformly. Organizations with strong complementary investments -- training, process redesign, management practices -- capture more benefit than those that merely install technology. The productivity effect of IT is contingent on organizational capability, not a property of the technology itself.
The Ethical Dimension
If IT productivity gains accrue primarily to capital owners and high-skill workers while displacing lower-skill workers, the technology is not neutral. The distribution of productivity gains -- not just their aggregate size -- is an ethical question about who bears the cost of technological transition and who captures its benefits.
Slide 3 of 35
Automation: Historical Pattern
Every major wave of automation created new jobs while destroying old ones. Whether this pattern holds for AI is genuinely contested.
Agricultural Mechanization
The mechanical harvester and tractor displaced millions of farm laborers over the 19th and 20th centuries. US farm employment fell from 40% of the workforce in 1900 to under 2% today. Workers displaced from agriculture moved to manufacturing and services. The transition took generations and involved enormous human hardship that is largely absent from the optimistic historical narrative.
Industrial Automation
Computer Numerical Control (CNC) machines, assembly line robots, and automated quality inspection eliminated manufacturing jobs that had sustained the postwar middle class. Manufacturing employment as a percentage of US workforce peaked in the 1950s and declined steadily. The geographical concentration of these losses -- in the Rust Belt, in former industrial cities -- produced lasting social and political consequences.
Information Technology
ATMs did not eliminate bank tellers -- they actually increased teller employment, because lower branch costs enabled more branches. Word processors displaced typists but created desktop publishing. The internet displaced travel agents but created entire industries in search, e-commerce, and digital media. New categories of work emerged. But they were not the same jobs, in the same places, for the same people.
Slide 4 of 35
AI and Robotics: The Current Wave
What makes the current wave of AI-driven automation potentially different from previous waves of technological displacement.
Cognitive Work Is Now Automatable
Previous automation waves displaced physical and routine cognitive tasks. AI systems can now perform non-routine cognitive tasks -- legal research, medical image analysis, financial forecasting, written communication. The category of "safe from automation" has shrunk significantly and continues to shrink. For the first time, highly educated knowledge workers face displacement alongside manual workers.
Speed of Transition
Agricultural displacement took generations. Industrial displacement took decades. AI capabilities are doubling in capability metrics on a two-to-three year cycle. Social and educational institutions that adapted to previous transitions over generations do not adapt this fast. The transition speed may outpace societies' capacity to respond without significant harm to displaced workers.
Geographic and Demographic Concentration
AI benefits accrue primarily to owners of AI systems and workers who use AI effectively. Displacement falls on workers in roles AI replaces. These two groups are not randomly distributed -- they differ systematically by education level, age, geographic location, and industry. The distributional pattern of AI's impact is not an accidental side effect; it is a predictable consequence of where AI capability currently sits.
The Augmentation Counter-Argument
MIT economist David Autor and others argue that AI is more likely to augment workers than replace them -- raising productivity in existing roles rather than eliminating them. Radiologists using AI can read more scans with higher accuracy. Lawyers using AI can review more documents faster. The question is whether the augmentation benefits are captured by workers or redirected to owners through reduced headcount and wages.
Slide 5 of 35
Workforce Ethics
Organizations and technologists who deploy automating AI systems have ethical obligations to the workers whose livelihoods are affected.
A regional bank uses AI to automate loan processing, eliminating 200 back-office positions. The AI is more accurate and faster than the human processors. The bank's profits increase. The 200 displaced workers -- average age 47, with specialized skills not easily transferable -- face unemployment in a labor market that does not have equivalent positions available. Is the decision to deploy ethically neutral because the AI performs better? Does better performance justify the transition cost to workers who did not choose it?
The Efficiency Justification
Standard economic argument: automation increases efficiency, reduces costs, lowers prices for consumers, creates new industries. The displaced workers will retrain and find new employment. This argument is true in aggregate and over long time horizons. It is less compelling as an ethical justification for specific organizations choosing to displace specific workers, particularly when the benefits accrue to shareholders and the costs are externalized to workers and the social safety net.
Responsible Transition Practices
Organizations making automation decisions can provide: advance notice (not required in the US beyond WARN Act minimums), retraining assistance (AT&T's "Future Ready" program invested $1 billion in reskilling 100,000 employees), severance packages above legal minimums, and internal mobility programs that match displaced workers to new roles before external hiring. These practices are not legally required -- they are ethically driven.
Policy and Collective Responsibility
Individual organizations cannot solve structural displacement. Proposals include: expanded unemployment insurance, education and retraining programs funded by automation taxes, portable benefits not tied to specific employment, and updated labor laws that reflect gig and platform work realities. The ethics of automation cannot be fully addressed at the organizational level -- it requires collective policy responses that distribute transition costs more equitably.
Slide 6 of 35
Healthcare IT
Information technology in healthcare creates extraordinary opportunities to improve patient outcomes -- and equally extraordinary opportunities to cause harm.
Electronic Health Records
EHR systems enable care coordination across providers, reduce duplicate testing, provide allergy and medication interaction alerts, and create data infrastructure for population health analytics. The transition from paper to EHR has genuinely improved care in documented ways. It has also created new categories of error: copy-paste errors, alert fatigue, and physician burnout from administrative burden.
Interoperability and Data Silos
Despite decades of standards work (HL7, FHIR), healthcare data remains fragmented across incompatible systems. A patient who sees providers at different health systems has fragmented records. Emergency physicians treating unconscious patients often cannot access their medical history. This fragmentation is not a technical problem -- it is a combination of commercial competition, regulatory failure, and insufficient political will to mandate interoperability.
Medical Device Security
Connected medical devices -- infusion pumps, ventilators, cardiac monitors -- represent an attack surface in the most vulnerable environments. Cybersecurity vulnerabilities in medical devices have been documented by researchers and acknowledged by the FDA. The intersection of patient safety and cybersecurity creates obligations that are both technical and ethical: healthcare organizations cannot treat medical device security as an optional expenditure.
Telemedicine and Access
The COVID-19 pandemic accelerated telemedicine adoption dramatically. Telemedicine can extend care to rural areas, reduce barriers for patients with transportation or mobility limitations, and enable more frequent monitoring of chronic conditions. But it requires broadband access, digital literacy, and appropriate devices -- all of which are unevenly distributed. Telemedicine's benefits are most reachable by patients who need them least.
Slide 7 of 35
Case Study: IBM Watson for Oncology
The gap between AI marketing claims and deployed system performance, and its consequences in a life-or-death domain.
IBM Watson won Jeopardy! in 2011 and became the face of AI's potential to transform knowledge work. IBM marketed Watson for Oncology as capable of processing millions of pages of medical literature to assist oncologists. Dozens of major cancer centers deployed it. Internal IBM documents, obtained by investigative reporting in 2017, described Watson as producing treatment recommendations that were "unsafe and incorrect." Watson had been trained primarily on hypothetical patient cases created by a small number of oncologists at Memorial Sloan Kettering -- not on actual patient outcomes data.
The Overpromise Problem
IBM's marketing for Watson used language -- "cognitive computing," "understanding," "learning" -- that implied capabilities the system did not have. The gap between the marketed capability and the deployed capability was not a minor technical shortfall. It was the difference between "reads medical literature to assist oncologists" and "was trained on hypothetical cases created by a handful of doctors at one hospital." Misleading performance claims in a medical context is an ethical violation with life-or-death consequences.
Training Data Failures
The system was trained on synthetic patient cases curated by oncologists at Memorial Sloan Kettering, not on actual patient outcome data. When deployed in healthcare systems with different patient populations, different drug availability, and different clinical guidelines (particularly overseas), the recommendations were systematically misaligned. The training data was not representative. This is not a Watson-specific failure -- it is a systematic AI deployment failure that repeats across domains.
What Healthcare Providers Should Have Known
Healthcare organizations that purchased Watson for Oncology bear partial responsibility. A rigorous procurement process for AI systems in clinical settings should include: independent validation on representative patient populations, review of training data provenance, clinical trial evidence of outcome improvement, and defined escalation protocols when the AI recommendation conflicts with physician judgment. These due diligence steps were not uniformly applied.
Slide 8 of 35
IBM Watson: Broader Lessons
Watson for Oncology is a template for AI deployment failures that recurs across domains.
1 AI systems must be validated on data that is representative of the deployment population. Training on one hospital's cases and deploying globally is an epistemological and ethical failure, not just a technical miscalculation.
2 Impressive performance on benchmarks (Jeopardy!) does not predict clinical utility. The capabilities required for game-show question answering are fundamentally different from those required for oncology treatment planning.
3 The reputational and commercial incentives to maintain an AI narrative are strong enough to delay honest reporting of failures even when patient safety is at risk. IBM did not proactively disclose the system's limitations.
4 Healthcare organizations that deploy AI systems have an independent obligation to validate performance before clinical use. "The vendor said it works" is not adequate due diligence in a domain where errors can kill patients.
5 The FDA's regulation of AI-based medical devices as software as a medical device (SaMD) is relevant. Pre-market review requirements exist. Whether they are sufficient to catch Watson-type failures is an ongoing policy debate.
6 The failure does not mean AI in oncology is impossible -- it means the Watson deployment failed. AI assistance in radiology, pathology image analysis, and drug discovery has produced documented, validated improvements. The lesson is about validation standards, not about AI's potential in medicine.
Slide 9 of 35
The Digital Divide
Unequal access to information and communication technology, and its consequences for economic opportunity and social participation.
Access Divide
Whether someone has access to devices, internet connectivity, and software. In the US, 21 million people lack broadband access -- concentrated in rural areas, low-income urban neighborhoods, and tribal lands. Global disparities are far more pronounced. As more essential services migrate online (benefits applications, job applications, education, healthcare), lack of access is not a minor inconvenience -- it is exclusion from modern civic and economic life.
Skills Divide
Having a device and connection does not guarantee the ability to use them effectively. Digital literacy -- searching effectively, evaluating source credibility, using productivity software, protecting personal data, avoiding phishing -- requires skills that are unevenly distributed. The skills divide correlates with age, education, and socioeconomic status. Bridging access without bridging skills does not close the divide.
Quality Divide
Even among those with access, the quality of that access varies enormously. A household with a 10 Mbps connection shared among five family members, with older devices and a single smartphone, has fundamentally different capability than a household with gigabit fiber and multiple modern computers. Remote learning during COVID-19 made the quality divide visible in ways that access statistics alone obscured.
Slide 10 of 35
Digital Divide: Dimensions
The divide is not a single gap -- it has multiple intersecting dimensions that reinforce each other.
Geographic
Rural broadband infrastructure has historically been underinvested because low population density makes commercial deployment unprofitable. The FCC's E-Rate program and recent Infrastructure Investment and Jobs Act allocations have begun to address this, but deployment timelines for rural fiber are measured in years. In the interim, rural students, job seekers, and telemedicine patients remain disadvantaged.
Socioeconomic
Broadband at $60-80/month is not affordable for households at the federal poverty level. The Affordable Connectivity Program (ACP) was providing subsidies to 23 million households before its funding lapsed in 2024. When the program ended, an estimated 23 million households faced the choice of cutting internet access or cutting other necessities. Policy continuity for digital access programs is not guaranteed.
Age
Adults over 65 are significantly less likely to use the internet, use smartphones, or have home broadband than younger adults. The shift of government services, healthcare management, financial services, and social connection to digital platforms creates systematic disadvantage for older adults who have not made that transition. This is not a fixed demographic characteristic -- it will shift as digitally native generations age.
Disability
Accessibility is an underappreciated dimension of the digital divide. Screen readers, voice control, and captioning technology enable digital participation for people with visual, motor, and hearing impairments -- when implemented. The Web Content Accessibility Guidelines (WCAG) provide standards, but compliance is inconsistent. Digital inaccessibility is not just inconvenience -- for people with certain disabilities, it is the difference between independence and dependence on others for basic tasks.
Slide 11 of 35
Digital Divide: Ethical Obligations
Who is obligated to address digital inequality -- and what does that obligation require in practice?
Government Obligations
As governments migrate essential services -- benefits, taxes, healthcare, voting -- to digital platforms, they incur obligations to ensure all citizens can access those platforms. A government that requires digital filing but does not ensure digital access has transferred its administrative costs onto citizens who cannot bear them. Universal service obligations that apply to telephone service should arguably extend to broadband.
Technology Company Obligations
Companies that profit from digital services profit more as the population of digital users expands. This creates a commercial alignment between access expansion and revenue growth that supports investment in connectivity programs. Google's Project Loon (stratospheric balloon internet), Starlink's low-earth-orbit broadband, and Meta's internet.org initiative all reflect this alignment -- as well as the complications of commercial entities controlling public communication infrastructure.
IT Professional Obligations
Software and system designers make choices that determine who can effectively use technology. Designing for minimum viable accessibility -- rather than best achievable accessibility -- systematically excludes people who cannot use the dominant interface. ACM's Code of Ethics (Principle 1.4) explicitly requires that computing professionals ensure technology is accessible to all potential users, including those with disabilities and limited resources.
Slide 12 of 35
Algorithmic Bias
When automated systems produce outcomes that systematically disadvantage certain groups, often without explicit discriminatory intent.
Amazon built a machine learning tool to screen resumes for software engineering positions, trained on ten years of hiring data. In 2018, the company discovered the system was systematically downgrading resumes containing the word "women's" (as in "women's chess club") and downgrading graduates of all-women's colleges. The system had learned that successful candidates in the historical data were predominantly male. It then replicated that historical pattern -- labeling it meritocracy. Amazon scrapped the tool.
Historical Bias in Training Data
ML systems trained on historical decisions inherit the biases of those decisions. If historical lending data reflects discriminatory lending practices, a model trained to predict creditworthiness will predict "bad credit risk" for demographics that were discriminated against historically -- and it will call this prediction objective, because it is based on "data." The bias is real; only its source and visibility have changed.
Proxy Discrimination
Laws prohibit using protected characteristics (race, gender, religion) in lending, hiring, and criminal justice. But variables that are not protected characteristics can serve as proxies for protected ones. Zip code correlates with race due to historical housing segregation. Certain hobbies and vocabulary patterns correlate with gender. Using these proxies in automated systems produces discriminatory outcomes without using the protected characteristic directly -- and thus evades existing legal frameworks.
The Automation Bias Problem
Humans tend to trust algorithmic outputs more than equivalent human judgments, even when the algorithm is demonstrably wrong. A judge who receives an algorithmic risk score is more likely to follow it than to override it. A hiring manager who receives an AI ranking is less likely to question it than a human recruiter's list. Automation bias means algorithmic errors propagate with less challenge than human errors -- increasing, not decreasing, the harm from bias.
Slide 13 of 35
Case: COMPAS and Criminal Justice
Algorithmic risk assessment tools in criminal justice and their documented racial disparities.
What COMPAS Is
COMPAS (Correctional Offender Management Profiling for Alternative Sanctions) is a proprietary risk assessment tool used by courts in multiple US states to assess the likelihood of a defendant reoffending. The score is used in bail, sentencing, and parole decisions. A 2016 ProPublica analysis of Broward County, Florida data found that the tool incorrectly flagged Black defendants as high-risk at nearly twice the rate it incorrectly flagged White defendants as high-risk.
The Accuracy vs. Fairness Tension
Northpointe (COMPAS's creator) responded that the algorithm is equally accurate across racial groups as measured by overall accuracy. This is technically true -- but ProPublica's analysis showed the error types were distributed unequally. The algorithm was equally accurate in the aggregate by making more false-positive errors for Black defendants and more false-negative errors for White defendants. Mathematical definitions of fairness can conflict: a system cannot simultaneously satisfy all definitions when base rates differ between groups.
The Transparency Problem
COMPAS is proprietary. Defendants whose bail or sentencing was influenced by a COMPAS score cannot obtain the algorithm or its weights for independent review. In State v. Loomis (2016), the Wisconsin Supreme Court upheld the use of COMPAS scores in sentencing while acknowledging that the defendant could not challenge the algorithm's calculations directly. The due process implications of secret algorithms making decisions about liberty remain an active area of litigation and policy debate.
Possible Responses
Options that have been proposed: mandatory disclosure of algorithm details for systems used in public decision-making, algorithmic impact assessments before government deployment, bias audits conducted by independent third parties, prohibition of high-stakes algorithmic decision-making without human review, and shifting the burden of proof so that algorithms must demonstrate fairness before deployment rather than after harm is documented.
Slide 14 of 35
Facial Recognition and Bias
A domain where algorithmic bias intersects directly with police power and civil liberties.
The MIT Media Lab Findings
Joy Buolamwini's Gender Shades study (2018) tested facial recognition systems from IBM, Microsoft, and Face++ on a benchmark dataset. Error rates for dark-skinned women were up to 34 percentage points higher than for light-skinned men. The systems were most accurate for the demographic most represented in training data (white men) and least accurate for the least represented demographic (dark-skinned women). This is not a theoretical risk -- it is a documented, measured disparity.
False Identification Consequences
Robert Williams was arrested in Detroit in January 2020 based on a facial recognition match. He was innocent. The technology had identified the wrong person. He spent 30 hours in jail before investigators acknowledged the error. Williams and attorneys noted that the detective was using a blurry still from surveillance video and accepted the automated match without independent corroboration. Facial recognition in law enforcement functions as an investigative lead -- but it is frequently treated as evidence.
The Ban vs. Regulate Debate
Cities including San Francisco, Boston, and Portland have banned government use of facial recognition. The EU AI Act classifies real-time remote biometric identification in public spaces as a prohibited AI practice with narrow exceptions. Arguments for regulation rather than prohibition: the technology can be validated to a higher standard for specific use cases. Arguments for prohibition: the harms from error in a law enforcement context are severe, irreversible, and disproportionately fall on already-marginalized communities.
Slide 15 of 35
What Would You Do?
Algorithmic bias scenarios. Consider both the technical response and the ethical obligations.
What Would You Do? Scenario A
You are a data scientist at a bank. During model validation, you discover that your credit scoring model has a disparate impact on Latino applicants -- not because the model uses ethnicity, but because it uses zip code, and the zip codes with the lowest scores map almost entirely onto historically Latino neighborhoods. The model's overall accuracy is strong. Do you deploy it? What do you do? What law may be relevant?
What Would You Do? Scenario B
Your company sells a sentiment analysis AI to a large employer for use in employee feedback surveys. Six months after deployment, HR data shows that negative feedback is being flagged at a higher rate for employees who write in non-standard English. The employer is using this data in performance reviews. You suspect the model was not trained on sufficiently diverse language. What is your obligation to the employer? To the employees who are being harmed?
What Would You Do? Scenario C
A city government wants to use your predictive policing software, which flags high-risk neighborhoods for increased patrol deployment. Your internal analysis shows the system perpetuates existing patrol distribution rather than identifying genuinely higher-risk areas. The city's police chief says it will improve public safety. What do you say? Do you accept the contract?
Slide 16 of 35
AI in Hiring
Automated hiring tools can scale screening dramatically -- and scale bias just as dramatically.
Resume Screening Tools
AI resume screening tools are used by the majority of Fortune 500 companies. They filter applicants before any human sees a resume. When these tools are trained on historical hiring data from organizations with documented diversity gaps, they perpetuate and scale those gaps. The Amazon case is the most publicized example, but the underlying dynamic is widespread. Many organizations do not audit their screening tools for disparate impact.
Video Interview Analysis
Tools that analyze facial expressions, vocal tone, and word choice during recorded video interviews to score candidates on personality traits and "culture fit" have been marketed to employers. Multiple researchers have found no validated evidence that these analyses predict job performance. When applied at scale, tools that assess body language and accent introduce systematic barriers for neurodivergent candidates, non-native speakers, and people from cultures with different norms for eye contact and emotional expression.
NYC Local Law 144
New York City's Local Law 144 (effective 2023) requires employers using automated employment decision tools to conduct bias audits performed by independent third parties and to disclose to candidates that such tools are being used. This is the first US law directly regulating AI in hiring. It provides a template that other jurisdictions are considering. The law's enforcement teeth are limited -- penalties are modest and compliance is self-reported -- but it establishes a disclosure norm.
Toward Fair Hiring AI
Best practices emerging from research and advocacy include: auditing for disparate impact before deployment, using diverse training data, providing explainability for adverse outcomes (why was this candidate rejected?), maintaining human review of automated decisions, and specifically testing tools on populations that differ from the majority training population. These practices exist. They are not universally applied. The gap between available practice and common practice is an ethical failure of the industry.
Slide 17 of 35
Content Moderation at Scale
The impossible problem of moderating billions of pieces of content while balancing free expression, user safety, and commercial interests.
The Scale Problem
Facebook processes over 100 billion pieces of content per day. No human review system can operate at that scale. Automated systems must do the initial filtering. But automated systems make errors -- false positives (removing permitted content) and false negatives (allowing harmful content). Both types of errors have real-world consequences. At Facebook's scale, even a 0.1% error rate represents 100 million incorrect decisions per day.
The Human Cost of Moderation
Human content reviewers are exposed to the most disturbing content on the internet -- child exploitation, graphic violence, genocide documentation, animal abuse -- at high volume, for low pay, with inadequate psychological support. Facebook's content moderators in Nairobi, Kenya filed a class action lawsuit in 2022 documenting rates of PTSD consistent with combat veterans. The business model of large platforms depends on this labor. Its human cost is largely invisible to users.
Differential Enforcement
Content moderation systems consistently perform differently across languages, geographies, and communities. Arabic-language political content is moderated at higher rates than equivalent English-language content. Academic research on systemic violence is moderated more aggressively than the actual content being studied. LGBTQ+ health information is moderated as sexually explicit in some configurations. These differential enforcement patterns have documented consequences for information access by marginalized communities.
Slide 18 of 35
Surveillance Capitalism
The economic model of collecting behavioral data at scale and converting it into predictive products sold to advertisers.
What Surveillance Capitalism Is
Shoshana Zuboff's term for the dominant business model of major platforms: collect behavioral data from users (what they click, read, where they go, who they know, when they are awake), build predictive models of future behavior, sell access to those predictions to advertisers and other third parties. The user is not the customer -- they are the raw material from which the product (behavioral prediction) is manufactured.
The Consent Fiction
Terms of service that users "agree to" by continuing to use a platform are not meaningful consent in any morally relevant sense. Documents that are thousands of words long, written in legal language, and require agreement as a condition of accessing a service that has become socially and economically essential do not meet the informed, voluntary, specific, and unambiguous consent standard that GDPR and ethical theory require. The consent framework is formal without being substantive.
Behavioral Modification
Zuboff argues that surveillance capitalism does not merely predict behavior -- it seeks to modify it. If platforms can predict that showing you content X makes you more likely to click an ad for product Y, they have an incentive to show you content X. Recommendation algorithms optimized for engagement time converge on emotionally stimulating content, because emotionally stimulated users spend more time on platform. The modification of behavior is built into the optimization function -- not as a side effect, but as a design goal.
Regulatory Responses
GDPR (EU) requires lawful basis for processing personal data, purpose limitation, and data minimization. The California Consumer Privacy Act (CCPA) gives California residents the right to know what data is collected, to opt out of its sale, and to request deletion. These frameworks impose real compliance costs and have produced real changes in practice -- cookie consent banners are the most visible symptom. Whether they address surveillance capitalism's structural incentives or merely regulate its surface is debated.
Slide 19 of 35
AI in Healthcare: Beyond Watson
Where AI assistance in healthcare is delivering validated results -- and where the ethical obligations remain the same.
Medical Imaging
Google's DeepMind demonstrated that its AI system could detect over 50 eye diseases from retinal scans with accuracy matching expert ophthalmologists. Multiple studies have validated AI systems in pathology, radiology, and dermatology that match or exceed specialist accuracy on specific defined tasks. The key distinction from Watson: these systems were validated on prospective clinical trials, not on hypothetical training cases. The validation standard is what distinguishes ethical AI deployment from irresponsible hype.
Drug Discovery
AI systems have successfully identified drug candidates that traditional screening would have missed. DeepMind's AlphaFold2 solved the protein folding problem with accuracy that has accelerated structural biology and drug target identification across the field. These applications have limited direct patient harm potential (decisions are made with extensive subsequent validation) and represent appropriate uses of AI capabilities in the research pipeline rather than in direct clinical decision-making.
The Persistent Ethical Obligations
Even for validated AI healthcare applications, ethical obligations persist: training data must be representative of the patient population; performance must be monitored after deployment for distribution shift; clinicians must maintain independent judgment rather than deferring to algorithmic outputs; patients must be informed when AI-assisted tools are used in their care; and failures must be reported and investigated under the same framework as other medical errors. Validation at deployment is not a permanent certification.
Slide 20 of 35
Robotics in the Workplace
Physical automation in warehouses, manufacturing, and logistics -- and the ethical questions it raises about human dignity and labor.
Amazon and Warehouse Automation
Amazon's fulfillment centers use Kiva robots (now Amazon Robotics) for inventory movement, reducing the walking workers do. But workers report increased pace-of-work demands, mandatory productivity rates that require near-constant motion, and injury rates that exceed industry averages. The robots did not replace the workers -- they changed the work into something more physically demanding and more algorithmically monitored. Automation in physical labor does not necessarily make work safer or more humane.
Algorithmic Management
Algorithmic management systems set productivity targets, monitor performance in real time, issue warnings, and make termination recommendations -- with limited human review. Amazon warehouse workers have reported receiving automated termination notices for missing productivity targets without human supervisors being meaningfully involved in the decision. This is the employment equivalent of algorithmic criminal sentencing: consequential decisions about people's lives made by systems with no accountability structure and no mechanism for individual circumstances to be considered.
Human Dignity in Automated Workplaces
Labor ethics has long held that work should respect human dignity -- that people are not instruments to be optimized but persons with autonomy, rest needs, and the right to be treated as ends rather than means. An automated workplace that monitors bathroom break length, requires workers to wear biometric trackers, and issues disciplinary actions based on algorithm output without human judgment raises genuine questions about whether the efficiency gains are purchased at the cost of human dignity that workplaces are obligated to preserve.
Collaborative Robotics
Cobots -- collaborative robots designed to work alongside humans rather than replace them -- represent a different design philosophy. Cobot systems are designed to handle physically hazardous, repetitive, or ergonomically stressful tasks while leaving the judgment-intensive, variable, and social dimensions of work to humans. Whether this design philosophy dominates depends on economic incentives that do not automatically favor it. Design choices for automation are ethical choices.
Slide 21 of 35
Generative AI and Creative Work
The emergence of AI systems that generate text, images, music, and code raises ethical questions about authorship, compensation, and cultural value.
Training Data and Consent
Large language models and image generation systems are trained on vast datasets scraped from the internet without explicit consent from creators. Authors, artists, photographers, and programmers whose work is in the training data did not agree to have their work used to build systems that compete with them commercially. Multiple class action lawsuits are pending. The legal question of whether training on copyrighted works constitutes infringement is unresolved in most jurisdictions.
Labor Displacement in Creative Industries
Stock illustration markets have been significantly disrupted by AI image generation. Entry-level writing, copywriting, and content work is increasingly handled by AI systems. Voice actors have seen work displaced by voice synthesis. These are not hypothetical future risks -- they are current market realities being experienced by workers in the creative economy right now. The affected workers did not consent to having their work used to train the systems displacing them.
The Compensation Question
Proposals for addressing the training data consent and compensation problem include: opt-in rather than opt-out consent for training data, a levy on AI system revenues distributed to creators whose work was used in training, licensing agreements between AI developers and creator collectives, and technical systems that track attribution at the training level. None of these solutions has been widely adopted. The current default -- train on everything, pay no one -- may not survive legal challenge.
Slide 22 of 35
AI Environmental Impact
The energy and resource costs of large-scale AI training and inference, and the ethical obligations they create.
Training Costs
Training a large language model consumes energy equivalent to hundreds of transatlantic flights. GPT-3's training run was estimated to require approximately 1,287 MWh of electricity, producing roughly 552 tons of CO2 equivalent. Larger subsequent models have higher costs. These costs are borne by the global environment but are not priced into the commercial value of the resulting models. This is a classic externality problem with an ethical dimension.
Water Consumption
Data center cooling requires enormous amounts of water. Microsoft's AI infrastructure consumed 1.7 million gallons of fresh water for cooling purposes in a single month in 2023, in a region experiencing drought conditions. Google's data centers used approximately 5.6 billion gallons of water in 2022. The siting of data centers in regions with water stress is an environmental justice issue: communities that already face water scarcity bear the marginal burden of AI infrastructure without capturing its economic benefits.
E-Waste
The hardware required for AI training (GPUs, TPUs, specialized accelerators) has short useful lifespans at the frontier -- newer architectures are substantially more efficient, creating pressure to retire working hardware. AI-grade hardware contains rare earth elements, lead, cadmium, and mercury. When disposed of improperly -- which the majority of global e-waste is -- these materials leach into soil and water in the communities where informal recycling occurs, primarily in low-income countries.
The Efficiency Counter-Trend
AI can also reduce environmental impact: optimizing grid energy routing, reducing food waste through supply chain prediction, accelerating materials discovery for batteries and solar cells, and improving building energy management. The net environmental impact of AI is not fixed -- it depends on where and how AI capability is applied. The ethical question is not whether to use AI but how to ensure its environmental costs are counted and its environmental applications are prioritized.
Slide 23 of 35
Algorithmic Decisions in Government
When public agencies use automated systems to make consequential decisions about citizens, due process and equal protection obligations apply.
Child Welfare Prediction
Multiple US counties use predictive risk scoring to inform child abuse investigation and removal decisions. A 2021 analysis of the Allegheny Family Screening Tool found it assigned higher risk scores to Black children controlling for poverty. Removal from a family is one of the most consequential actions government can take. An algorithmic input into that decision that is racially biased -- even as a proxy -- raises serious constitutional and ethical concerns.
Public Benefits Administration
Indiana deployed an IBM automated benefits eligibility system in 2009 that automatically denied claims for minor documentation issues. Over 1 million people had their food assistance, Medicaid, or cash assistance terminated without meaningful human review. The state eventually canceled the contract. Arkansas deployed an algorithmic Medicaid allocation system that significantly cut home care hours for people with disabilities without explanation. Courts ordered the state to provide explanations -- a due process requirement that algorithmic systems by default fail to satisfy.
The Explainability Requirement
Due process requires that when government takes adverse action against an individual, the individual receives an explanation and an opportunity to contest it. Algorithmic systems that deny benefits or flag risks without explainable reasoning -- "the algorithm said so" -- fail this requirement. The EU AI Act's prohibition on unexplainable decisions in high-stakes public contexts is a response to this problem. US law addresses it less systematically, but courts have found due process violations in specific algorithmic benefit denial cases.
Slide 24 of 35
Responsible AI Frameworks
Emerging frameworks for evaluating and governing AI systems in high-stakes contexts.
EU AI Act
First comprehensive AI regulation enacted by a major jurisdiction (2024). Classifies AI systems by risk level: unacceptable (prohibited -- social scoring by governments, real-time biometric surveillance in public), high-risk (requires conformity assessment, bias auditing, human oversight -- medical devices, hiring, credit, critical infrastructure), limited risk (transparency requirements), and minimal risk (no requirements). Sets a global compliance template for multinationals.
NIST AI Risk Management Framework
Voluntary US framework for managing AI risks across the AI lifecycle. Four core functions: Govern (organizational culture and accountability), Map (identify and categorize risks), Measure (analyze and assess risks), Manage (prioritize and respond to risks). Analogous to the NIST Cybersecurity Framework in structure. Increasingly referenced in government AI procurement requirements. Voluntary status limits its impact outside government contracting.
Algorithmic Impact Assessments
Modeled on environmental impact assessments. Before deploying an AI system in a high-stakes context, conduct a structured analysis of: who is affected, what harms could occur, how likely and severe those harms are, what groups are disproportionately exposed, what mitigation is in place, and who should have oversight. The Canadian Directive on Automated Decision-Making requires AIAs for federal government systems. Several US cities and states have proposed or enacted similar requirements.
AI Ethics Principles Comparison
IEEE, ACM, Google, Microsoft, IBM, and the European Commission have all published AI ethics principles. Despite different sources, they converge on similar themes: fairness and non-discrimination, transparency and explainability, accountability and human oversight, privacy and security, and safety. The challenge is not lack of principles -- it is the gap between published principles and deployed systems. Principles without enforcement mechanisms are aspirational documents, not operational constraints.
Slide 25 of 35
What Would You Do?
AI deployment scenarios. The decisions you make as a practitioner have consequences for real people.
What Would You Do? Scenario A
You are an ML engineer at a company that sells predictive analytics to insurance companies. Your model predicts health insurance risk with high accuracy. A civil rights organization contacts your company claiming the model has disparate impact on Black applicants. Your legal team says you are not in violation of current law because you do not use race as a variable. What is your ethical obligation beyond legal compliance?
What Would You Do? Scenario B
Your company is considering deploying a generative AI chatbot to handle customer service inquiries for a bank. The chatbot will handle questions about account balances, dispute resolution, and loan applications. It occasionally hallucinates -- confidently states incorrect information. Management says the error rate is below the threshold for human agents in the same role. Do you deploy it? What safeguards are required? What disclosures are owed to customers?
What Would You Do? Scenario C
You are a senior IT manager at a hospital. A vendor is offering an AI sepsis prediction tool that flags patients at risk of sepsis 6 hours before standard clinical indicators appear. Independent studies show it reduces sepsis mortality by 18%. It also generates 40% false positive alerts. Nurses report alert fatigue is affecting their response to all alerts, including correct ones. The harm from alert fatigue may offset the benefit from early detection. How do you decide? What additional information do you need?
Slide 26 of 35
The Future of Work
Competing scenarios for AI's impact on employment -- and the ethical obligations that each scenario implies.
The Augmentation Scenario
AI serves as a productivity multiplier for all workers, raising wages by increasing output per worker, creating new high-value tasks that require human judgment, and enabling workers to focus on more meaningful work while AI handles routine cognitive tasks. McKinsey estimates this scenario requires successful redeployment of displaced workers into new tasks within their existing organizations or new ones. The transition requires active investment in worker adaptation.
The Displacement Scenario
AI replaces workers at sufficient scale to create structural unemployment that the labor market does not absorb through new job creation. The benefits accrue to owners of AI systems, while workers bear costs through unemployment, wage suppression, and precarious employment. This scenario requires policy responses -- expanded social insurance, education investment, possibly universal basic income -- that exceed current political will in most countries.
The Polarization Scenario
The most commonly documented actual outcome of automation: AI is most likely to automate middle-skill, routine tasks -- both cognitive and physical. High-skill, non-routine cognitive tasks (design, strategy, complex judgment) are less automatable currently. Low-skill, non-routine physical tasks (cleaning, care work, skilled trades) are expensive to automate physically. The middle is hollowed out. This polarization has been documented as a driver of income inequality growth in OECD countries over the past 30 years.
Slide 27 of 35
Broadband and Civic Participation
When civic life migrates online, the digital divide becomes a participation gap with democratic consequences.
COVID-19 as a Case Study
During the pandemic, remote work, remote education, telehealth, and government service access all migrated primarily or entirely online. Households without reliable broadband were excluded from the dominant channels for employment, education, healthcare, and social connection simultaneously. The pandemic did not create the digital divide -- it made its consequences catastrophic at a moment when everyone's ability to function in society depended on connectivity.
Online Government Services
The IRS Free File program for tax preparation, state unemployment insurance systems, Medicaid enrollment, Social Security applications, SNAP benefits management -- all have primary or exclusive digital interfaces. These are not optional services for most recipients. When the government migrates services to digital platforms without ensuring universal access, it has effectively used the digital divide as a mechanism to reduce program utilization -- a consequential policy choice that is rarely described as such.
Online Voter Registration and Misinformation
Most US states offer or require online voter registration. This is a genuine access improvement for most citizens and a barrier for citizens without consistent internet access. Separately, digital information environments are the primary channel for political misinformation -- and the groups most vulnerable to misinformation (less digital literacy, less exposure to diverse sources) are systematically different from those who created and benefit from the systems. Democratic participation has an information infrastructure that is not equally accessible.
Proposals for Universal Broadband
The Infrastructure Investment and Jobs Act (2021) allocated $65 billion for broadband deployment and affordability. Rural broadband cooperatives, municipal broadband networks, and tribal broadband programs provide models for non-commercial deployment. FCC redefinition of "broadband" to require 100/20 Mbps speeds is designed to raise the floor of what counts as adequate access. Whether the current investment level is sufficient to close the access gap -- let alone the skills and quality gaps -- is contested among researchers.
Slide 28 of 35
Key Cases: Summary
The cases covered in this module and the ethical principles they illustrate.
1 IBM Watson for Oncology: AI marketed beyond its validated capabilities, trained on non-representative data, deployed in a high-stakes clinical setting without adequate independent validation. Principle violated: honesty in performance claims and obligation to validate before deployment in safety-critical contexts.
2 Amazon Resume Screening: Historical bias in hiring data reproduced and scaled by ML. Principle violated: algorithmic systems must be audited for disparate impact before deployment in consequential decision contexts.
3 COMPAS: Proprietary risk scoring in criminal justice with documented racial error disparities and no transparency for defendants. Principles violated: explainability and due process in consequential automated decisions.
4 Robert Williams: Wrongful arrest based on facial recognition misidentification. Principle violated: human oversight requirements for AI systems in law enforcement applications with severe consequences for error.
5 Amazon Warehouse: Physical automation that changed working conditions without eliminating workers, with algorithmic management reducing human judgment in employment decisions. Principle violated: human dignity and due process in employment decisions.
Slide 29 of 35
Digital Divide: Global Dimension
The domestic digital divide is significant. The global digital divide is an order of magnitude larger and more consequential.
Global Connectivity Gap
As of 2023, approximately 2.6 billion people globally lack internet access -- nearly a third of the world's population. Sub-Saharan Africa has internet penetration below 40%. The benefits of digital economic participation -- remote work, e-commerce, access to global knowledge -- are unavailable to these populations. AI systems trained predominantly on data from connected populations will systematically underperform for populations not represented in that data.
AI and Language Representation
Large language models are trained primarily on English text, with significant representation of other European languages and Mandarin. Languages spoken primarily by populations with low internet penetration are dramatically underrepresented. AI systems deployed for translation, information access, or decision-making in these languages perform substantially worse -- creating and reinforcing a two-tier information environment where AI benefits are accessible to high-resource language speakers and not to others.
AI Governance as a Power Dynamic
The organizations that set AI standards, train frontier models, and define AI ethics frameworks are concentrated in the US, EU, and China. Countries in the Global South -- which will bear significant effects of AI deployment -- have minimal voice in governance frameworks. This is not a unique problem to AI: it replicates patterns from internet governance, financial regulation, and trade policy. It is worth naming as an ethical problem even when its solutions are not obvious.
Slide 30 of 35
What Good AI Governance Looks Like
The principles that distinguish responsible AI deployment from irresponsible deployment are identifiable and actionable.
1 Validate before you deploy. Performance claims must be supported by evidence from populations representative of the deployment context. "It worked in our testing environment" is not sufficient for high-stakes deployment.
2 Audit for disparate impact. Before deployment in any context affecting people's access to employment, credit, healthcare, housing, or legal status, conduct a bias audit that specifically tests for differential performance across protected and vulnerable groups.
3 Maintain human oversight for consequential decisions. Algorithmic recommendations should inform human judgment, not replace it, in contexts where errors have severe consequences and where the affected person has a right to individualized consideration.
4 Disclose when AI is being used. People affected by AI-assisted decisions have an interest in knowing that an automated system was involved. This enables meaningful challenge and builds warranted (or undermines unwarranted) trust.
5 Monitor after deployment. Distribution shift, emerging bias patterns, and unintended consequences appear over time. Post-deployment monitoring is not optional -- it is part of the obligation that comes with deploying systems that affect people's lives.
6 Create accountability structures. Someone must be identifiably responsible for the performance and impact of deployed AI systems. "The algorithm decided" is not an accountable answer. Name the person responsible for the system and the decision to deploy it.
Slide 31 of 35
Ethics vs. Determinism
The framing of technological change as inevitable is itself an ethical and political choice -- not a description of reality.
The Inevitability Narrative
Technological change is often described as inevitable: "AI will automate X" and "the digital divide will close as technology spreads" as if these are natural phenomena rather than outcomes of choices made by people with power and interests. This framing is ethically important because it removes agency -- and with it, responsibility. If displacement is inevitable, no one is responsible for the harm to displaced workers. If the divide will close on its own, no one is obligated to accelerate it.
What Is Actually Chosen
Which tasks AI replaces vs. augments is a design choice. Which workers receive transition support is a policy choice. Whether training data is representative is a procurement choice. Whether hiring algorithms are audited for bias is a management choice. None of these outcomes is technologically determined. All are chosen. Calling them inevitable is a way of declining responsibility for the choice.
The Role of Political Will
Most technology policy gaps are not knowledge gaps -- researchers and practitioners understand what good practice looks like. They are political will gaps. Algorithmic impact assessments, bias auditing requirements, worker transition programs, and AI liability frameworks all exist as proposals. Their absence from law is not a knowledge problem. It is a reflection of which interests are successfully represented in policy-making processes.
The Practitioner's Agency
Individual IT professionals have more agency than technological determinism implies. Design choices, data choices, validation choices, and deployment recommendations are all made by real people. A practitioner who says "this is how the technology works" when the technology is a system they designed has obscured their own agency. Reclaiming that agency is the precondition for exercising ethical responsibility.
Slide 32 of 35
Chapter 8: The Intersection
IT impact, the digital divide, algorithmic bias, and workforce displacement are not separate issues -- they intersect and reinforce each other.
Displacement and the Divide
Workers displaced by automation who lack digital skills face a double barrier: their jobs were automated, and they cannot easily access the digital resources -- online job boards, online training, digital benefits applications -- that should assist their transition. The digital divide is not just a separate problem from automation displacement. It is a mechanism that makes displacement more harmful for workers who are already disadvantaged.
Bias and the Divide
Algorithmic bias disproportionately affects populations that are also more likely to be on the wrong side of the digital divide -- lower income, older, less educated, more geographically isolated. These populations are less likely to have the technical knowledge to challenge algorithmic decisions, less likely to have lawyers who can litigate, and more dependent on the services (benefits, public housing, healthcare) where algorithmic decision-making is most prevalent. Bias and access disadvantage compound each other.
Healthcare IT and Equity
AI diagnostic tools trained primarily on data from well-resourced health systems perform less well on patients from underserved communities -- who have different disease presentations, different comorbidities, and different access to prior care documentation. Deploying these tools in underserved contexts without validation on representative data worsens health equity. The Watson Oncology failure is not an outlier -- it is representative of a systematic pattern.
The Policy Coherence Challenge
Addressing these intersecting challenges requires policy coherence across domains that are institutionally separated: labor policy, technology regulation, education policy, healthcare policy, and civil rights enforcement are all relevant but housed in different agencies with different mandates. Building policy responses that address the intersection rather than the individual problems in isolation is politically and institutionally difficult -- which partly explains why comprehensive responses have not emerged despite clear problem identification.
Slide 33 of 35  |  Discussion Questions
Discussion Questions
Prepare written or oral responses. These themes appear in the final exam.
1 The Watson for Oncology case involved misleading marketing, inadequate validation, and deployment without sufficient evidence of efficacy. Using two ethical frameworks, analyze IBM's obligations to the healthcare organizations that deployed Watson and to the patients who received its recommendations.
2 Explain the difference between the access divide, the skills divide, and the quality divide. Why is each dimension necessary to close for digital equity to be achieved? Which dimension is hardest to address through policy? Why?
3 COMPAS uses historical data to predict recidivism. The creators argue it is mathematically fair. Critics argue it is racially biased. Who is correct? Is it possible for both claims to be true? What does this case reveal about the limits of mathematical definitions of fairness?
4 Evaluate the claim that automation-driven job displacement is ethically neutral because new jobs always emerge. What historical evidence supports this claim? What evidence challenges it? Under what conditions is the claim most and least persuasive?
5 Describe a specific scenario in which a healthcare organization ethically deploys an AI diagnostic tool. What validation steps are required? What disclosure is owed to patients? What human oversight processes must be in place?
Slide 34 of 35  |  Exercises
Chapter 8 Exercises
Complete before the next class session. Written responses required for exercises 3 and 4.
1 Research one algorithmic bias case not covered in this module (examples: Twitter image cropping, Apple Card credit limits, healthcare resource allocation algorithms). Identify the source of bias, the populations affected, what the organization did in response, and what you believe should have been required.
2 Find your state or county's digital inclusion plan (or note the absence of one). Assess it against the three dimensions of the digital divide (access, skills, quality). What populations are addressed? What are the largest gaps? What policy recommendations would you add?
3 Write a one-page algorithmic impact assessment template for a hypothetical AI system that ranks job applicants for a government agency. Your template should specify what information must be gathered, what analyses are required, who must review the findings, and what thresholds trigger additional action before deployment.
4 Construct an argument for and an argument against the following position: "Companies that deploy AI systems that cause documented harm to protected groups should face the same liability as companies that knowingly employ discriminatory human processes." What existing legal frameworks support each argument?
5 Identify one sector where you believe AI augmentation is most likely (meaning AI raises worker productivity without displacing workers) and one where displacement is most likely. Justify your assessment with reference to specific task characteristics, wage levels, and existing AI capability.
Slide 35 of 35  |  Summary
Module Summary
IT does not operate neutrally on society. Its effects on productivity, employment, equity, and decision-making are shaped by choices made by people -- and those choices carry ethical weight.
The Watson case, the COMPAS case, the Amazon bias case, and the digital divide are not separate ethical failures by different organizations. They are instances of a common pattern: systems built without adequate attention to who is harmed, deployed without adequate validation, maintained without adequate transparency, and governed without adequate accountability. The pattern can be broken -- by practitioners who choose to break it.
9 Facts to Carry Out of This Lecture
1 The productivity paradox: IT investment does not automatically produce proportional productivity gains. Benefits are contingent on complementary organizational investments in training and process redesign.
2 AI automation is distinct from previous waves because it can displace cognitive as well as physical tasks. The "safe from automation" category of knowledge work has shrunk significantly.
3 IBM Watson for Oncology: trained on hypothetical cases at one hospital, deployed globally in cancer treatment, produced "unsafe and incorrect" recommendations. The core failure was inadequate training data and overstated performance claims in a life-critical domain.
4 The digital divide has three dimensions: access (connectivity), skills (digital literacy), and quality (bandwidth and device capability). Closing all three is necessary for genuine digital equity.
5 Algorithmic bias arises from historical bias in training data, proxy discrimination through correlated variables, and automation bias (human over-deference to algorithmic outputs). All three are present in production systems today.
6 COMPAS: proprietary criminal risk scoring with documented racial disparities in false positive rates. Defendants cannot challenge the algorithm. Courts have allowed its use despite due process concerns.
7 Surveillance capitalism: the business model of collecting behavioral data, building predictive models, and selling predictions to advertisers. User "consent" via terms of service does not meet meaningful informed-consent standards.
8 Responsible AI governance requires: validation on representative populations, bias auditing before deployment, human oversight for consequential decisions, disclosure to affected individuals, and ongoing post-deployment monitoring.
9 Technological change is not inevitable -- it is chosen. Which tasks are automated, who receives transition support, and whether systems are audited for bias are all decisions made by people. The framing of technology as inevitable is itself an ethical choice about who bears responsibility for outcomes.