#Future of Life Institute » Feed Future of Life Institute » Comments Feed Future of Life Institute » iCal Feed Future of Life Institute » Benefits & Risks of Artificial Intelligence Comments Feed alternate alternate Future of Life Institute RSS2 Feed * Twitter * Facebook * Home * Who We Are + Team + 2015 Annual Report + 2016 Annual Report + Tax Forms * Activities + AI + Upcoming Events + Past Events + Press + Newsletters * Existential Risk + Artificial Intelligence + Biotechnology + Nuclear Weapons + Climate Change * Get Involved + Job Postings * Contact Technology is giving life the potential to flourish like never before... dead tree image ...or to self-destruct. Let's make a difference! Future of Life Institute * News: * AI * Biotech * Nuclear * Climate * Partner Orgs * Search benefits and risks of artificial intelligence Benefits & Risks of Artificial Intelligence “Everything we love about civilization is a product of intelligence, so amplifying our human intelligence with artificial intelligence has the potential of helping civilization flourish like never before – as long as we manage to keep the technology beneficial.“ Max Tegmark, President of the Future of Life Institute Click here to see this page in other languages: Chinese Japanese Korean Russian French What is AI? From SIRI to self-driving cars, artificial intelligence (AI) is progressing rapidly. While science fiction often portrays AI as robots with human-like characteristics, AI can encompass anything from Google’s search algorithms to IBM’s Watson to autonomous weapons. Artificial intelligence today is properly known as narrow AI (or weak AI), in that it is designed to perform a narrow task (e.g. only facial recognition or only internet searches or only driving a car). However, the long-term goal of many researchers is to create general AI (AGI or strong AI). While narrow AI may outperform humans at whatever its specific task is, like playing chess or solving equations, AGI would outperform humans at nearly every cognitive task. Why research AI safety? In the near term, the goal of keeping AI’s impact on society beneficial motivates research in many areas, from economics and law to technical topics such as verification, validity, security and control. Whereas it may be little more than a minor nuisance if your laptop crashes or gets hacked, it becomes all the more important that an AI system does what you want it to do if it controls your car, your airplane, your pacemaker, your automated trading system or your power grid. Another short-term challenge is preventing a devastating arms race in lethal autonomous weapons. In the long term, an important question is what will happen if the quest for strong AI succeeds and an AI system becomes better than humans at all cognitive tasks. As pointed out by I.J. Good in 1965, designing smarter AI systems is itself a cognitive task. Such a system could potentially undergo recursive self-improvement, triggering an intelligence explosion leaving human intellect far behind. By inventing revolutionary new technologies, such a superintelligence might help us eradicate war, disease, and poverty, and so the creation of strong AI might be the biggest event in human history. Some experts have expressed concern, though, that it might also be the last, unless we learn to align the goals of the AI with ours before it becomes superintelligent. There are some who question whether strong AI will ever be achieved, and others who insist that the creation of superintelligent AI is guaranteed to be beneficial. At FLI we recognize both of these possibilities, but also recognize the potential for an artificial intelligence system to intentionally or unintentionally cause great harm. We believe research today will help us better prepare for and prevent such potentially negative consequences in the future, thus enjoying the benefits of AI while avoiding pitfalls. How can AI be dangerous? Most researchers agree that a superintelligent AI is unlikely to exhibit human emotions like love or hate, and that there is no reason to expect AI to become intentionally benevolent or malevolent. Instead, when considering how AI might become a risk, experts think two scenarios most likely: 1. The AI is programmed to do something devastating: Autonomous weapons are artificial intelligence systems that are programmed to kill. In the hands of the wrong person, these weapons could easily cause mass casualties. Moreover, an AI arms race could inadvertently lead to an AI war that also results in mass casualties. To avoid being thwarted by the enemy, these weapons would be designed to be extremely difficult to simply “turn off,” so humans could plausibly lose control of such a situation. This risk is one that’s present even with narrow AI, but grows as levels of AI intelligence and autonomy increase. 2. The AI is programmed to do something beneficial, but it develops a destructive method for achieving its goal: This can happen whenever we fail to fully align the AI’s goals with ours, which is strikingly difficult. If you ask an obedient intelligent car to take you to the airport as fast as possible, it might get you there chased by helicopters and covered in vomit, doing not what you wanted but literally what you asked for. If a superintelligent system is tasked with a ambitious geoengineering project, it might wreak havoc with our ecosystem as a side effect, and view human attempts to stop it as a threat to be met. As these examples illustrate, the concern about advanced AI isn’t malevolence but competence. A super-intelligent AI will be extremely good at accomplishing its goals, and if those goals aren’t aligned with ours, we have a problem. You’re probably not an evil ant-hater who steps on ants out of malice, but if you’re in charge of a hydroelectric green energy project and there’s an anthill in the region to be flooded, too bad for the ants. A key goal of AI safety research is to never place humanity in the position of those ants. Why the recent interest in AI safety Stephen Hawking, Elon Musk, Steve Wozniak, Bill Gates, and many other big names in science and technology have recently expressed concern in the media and via open letters about the risks posed by AI, joined by many leading AI researchers. Why is the subject suddenly in the headlines? The idea that the quest for strong AI would ultimately succeed was long thought of as science fiction, centuries or more away. However, thanks to recent breakthroughs, many AI milestones, which experts viewed as decades away merely five years ago, have now been reached, making many experts take seriously the possibility of superintelligence in our lifetime. While some experts still guess that human-level AI is centuries away, most AI researches at the 2015 Puerto Rico Conference guessed that it would happen before 2060. Since it may take decades to complete the required safety research, it is prudent to start it now. Because AI has the potential to become more intelligent than any human, we have no surefire way of predicting how it will behave. We can’t use past technological developments as much of a basis because we’ve never created anything that has the ability to, wittingly or unwittingly, outsmart us. The best example of what we could face may be our own evolution. People now control the planet, not because we’re the strongest, fastest or biggest, but because we’re the smartest. If we’re no longer the smartest, are we assured to remain in control? FLI’s position is that our civilization will flourish as long as we win the race between the growing power of technology and the wisdom with which we manage it. In the case of AI technology, FLI’s position is that the best way to win that race is not to impede the former, but to accelerate the latter, by supporting AI safety research. The Top Myths About Advanced AI A captivating conversation is taking place about the future of artificial intelligence and what it will/should mean for humanity. There are fascinating controversies where the world’s leading experts disagree, such as: AI’s future impact on the job market; if/when human-level AI will be developed; whether this will lead to an intelligence explosion; and whether this is something we should welcome or fear. But there are also many examples of of boring pseudo-controversies caused by people misunderstanding and talking past each other. To help ourselves focus on the interesting controversies and open questions — and not on the misunderstandings — let’s clear up some of the most common myths. AI myths Timeline Myths The first myth regards the timeline: how long will it take until machines greatly supersede human-level intelligence? A common misconception is that we know the answer with great certainty. One popular myth is that we know we’ll get superhuman AI this century. In fact, history is full of technological over-hyping. Where are those fusion power plants and flying cars we were promised we’d have by now? AI has also been repeatedly over-hyped in the past, even by some of the founders of the field. For example, John McCarthy (who coined the term “artificial intelligence”), Marvin Minsky, Nathaniel Rochester and Claude Shannon wrote this overly optimistic forecast about what could be accomplished during two months with stone-age computers: “We propose that a 2 month, 10 man study of artificial intelligence be carried out during the summer of 1956 at Dartmouth College […] An attempt will be made to find how to make machines use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves. We think that a significant advance can be made in one or more of these problems if a carefully selected group of scientists work on it together for a summer.” On the other hand, a popular counter-myth is that we know we won’t get superhuman AI this century. Researchers have made a wide range of estimates for how far we are from superhuman AI, but we certainly can’t say with great confidence that the probability is zero this century, given the dismal track record of such techno-skeptic predictions. For example, Ernest Rutherford, arguably the greatest nuclear physicist of his time, said in 1933 — less than 24 hours before Szilard’s invention of the nuclear chain reaction — that nuclear energy was “moonshine.” And Astronomer Royal Richard Woolley called interplanetary travel “utter bilge” in 1956. The most extreme form of this myth is that superhuman AI will never arrive because it’s physically impossible. However, physicists know that a brain consists of quarks and electrons arranged to act as a powerful computer, and that there’s no law of physics preventing us from building even more intelligent quark blobs. There have been a number of surveys asking AI researchers how many years from now they think we’ll have human-level AI with at least 50% probability. All these surveys have the same conclusion: the world’s leading experts disagree, so we simply don’t know. For example, in such a poll of the AI researchers at the 2015 Puerto Rico AI conference, the average (median) answer was by year 2045, but some researchers guessed hundreds of years or more. There’s also a related myth that people who worry about AI think it’s only a few years away. In fact, most people on record worrying about superhuman AI guess it’s still at least decades away. But they argue that as long as we’re not 100% sure that it won’t happen this century, it’s smart to start safety research now to prepare for the eventuality. Many of the safety problems associated with human-level AI are so hard that they may take decades to solve. So it’s prudent to start researching them now rather than the night before some programmers drinking Red Bull decide to switch one on. Controversy Myths Another common misconception is that the only people harboring concerns about AI and advocating AI safety research are luddites who don’t know much about AI. When Stuart Russell, author of the standard AI textbook, mentioned this during his Puerto Rico talk, the audience laughed loudly. A related misconception is that supporting AI safety research is hugely controversial. In fact, to support a modest investment in AI safety research, people don’t need to be convinced that risks are high, merely non-negligible — just as a modest investment in home insurance is justified by a non-negligible probability of the home burning down. It may be that media have made the AI safety debate seem more controversial than it really is. After all, fear sells, and articles using out-of-context quotes to proclaim imminent doom can generate more clicks than nuanced and balanced ones. As a result, two people who only know about each other’s positions from media quotes are likely to think they disagree more than they really do. For example, a techno-skeptic who only read about Bill Gates’s position in a British tabloid may mistakenly think Gates believes superintelligence to be imminent. Similarly, someone in the beneficial-AI movement who knows nothing about Andrew Ng’s position except his quote about overpopulation on Mars may mistakenly think he doesn’t care about AI safety, whereas in fact, he does. The crux is simply that because Ng’s timeline estimates are longer, he naturally tends to prioritize short-term AI challenges over long-term ones. Myths About the Risks of Superhuman AI Many AI researchers roll their eyes when seeing this headline: “Stephen Hawking warns that rise of robots may be disastrous for mankind.” And as many have lost count of how many similar articles they’ve seen. Typically, these articles are accompanied by an evil-looking robot carrying a weapon, and they suggest we should worry about robots rising up and killing us because they’ve become conscious and/or evil. On a lighter note, such articles are actually rather impressive, because they succinctly summarize the scenario that AI researchers don’t worry about. That scenario combines as many as three separate misconceptions: concern about consciousness, evil, and robots. If you drive down the road, you have a subjective experience of colors, sounds, etc. But does a self-driving car have a subjective experience? Does it feel like anything at all to be a self-driving car? Although this mystery of consciousness is interesting in its own right, it’s irrelevant to AI risk. If you get struck by a driverless car, it makes no difference to you whether it subjectively feels conscious. In the same way, what will affect us humans is what superintelligent AI does, not how it subjectively feels. The fear of machines turning evil is another red herring. The real worry isn’t malevolence, but competence. A superintelligent AI is by definition very good at attaining its goals, whatever they may be, so we need to ensure that its goals are aligned with ours. Humans don’t generally hate ants, but we’re more intelligent than they are – so if we want to build a hydroelectric dam and there’s an anthill there, too bad for the ants. The beneficial-AI movement wants to avoid placing humanity in the position of those ants. The consciousness misconception is related to the myth that machines can’t have goals. Machines can obviously have goals in the narrow sense of exhibiting goal-oriented behavior: the behavior of a heat-seeking missile is most economically explained as a goal to hit a target. If you feel threatened by a machine whose goals are misaligned with yours, then it is precisely its goals in this narrow sense that troubles you, not whether the machine is conscious and experiences a sense of purpose. If that heat-seeking missile were chasing you, you probably wouldn’t exclaim: “I’m not worried, because machines can’t have goals!” I sympathize with Rodney Brooks and other robotics pioneers who feel unfairly demonized by scaremongering tabloids, because some journalists seem obsessively fixated on robots and adorn many of their articles with evil-looking metal monsters with red shiny eyes. In fact, the main concern of the beneficial-AI movement isn’t with robots but with intelligence itself: specifically, intelligence whose goals are misaligned with ours. To cause us trouble, such misaligned superhuman intelligence needs no robotic body, merely an internet connection – this may enable outsmarting financial markets, out-inventing human researchers, out-manipulating human leaders, and developing weapons we cannot even understand. Even if building robots were physically impossible, a super-intelligent and super-wealthy AI could easily pay or manipulate many humans to unwittingly do its bidding. The robot misconception is related to the myth that machines can’t control humans. Intelligence enables control: humans control tigers not because we are stronger, but because we are smarter. This means that if we cede our position as smartest on our planet, it’s possible that we might also cede control. The Interesting Controversies Not wasting time on the above-mentioned misconceptions lets us focus on true and interesting controversies where even the experts disagree. What sort of future do you want? Should we develop lethal autonomous weapons? What would you like to happen with job automation? What career advice would you give today’s kids? Do you prefer new jobs replacing the old ones, or a jobless society where everyone enjoys a life of leisure and machine-produced wealth? Further down the road, would you like us to create superintelligent life and spread it through our cosmos? Will we control intelligent machines or will they control us? Will intelligent machines replace us, coexist with us, or merge with us? What will it mean to be human in the age of artificial intelligence? What would you like it to mean, and how can we make the future be that way? Please join the conversation! Recommended References Videos * Stuart Russell – The Long-Term Future of (Artificial) Intelligence * Humans Need Not Apply * Nick Bostrom on Artificial Intelligence and Existential Risk * Stuart Russell Interview on the long-term future of AI * Value Alignment – Stuart Russell: Berkeley IdeasLab Debate Presentation at the World Economic Forum * Social Technology and AI: World Economic Forum Annual Meeting 2015 * Stuart Russell, Eric Horvitz, Max Tegmark – The Future of Artificial Intelligence * Talks from the Beneficial AI 2017 conference in Asilomar, CA * Jaan Tallinn on Steering Artificial Intelligence Media Articles * Concerns of an Artificial Intelligence Pioneer * Transcending Complacency on Superintelligent Machines * Why We Should Think About the Threat of Artificial Intelligence * Stephen Hawking Is Worried About Artificial Intelligence Wiping Out Humanity * Artificial Intelligence could kill us all. Meet the man who takes that risk seriously * Artificial Intelligence Poses ‘Extinction Risk’ To Humanity Says Oxford University’s Stuart Armstrong * What Happens When Artificial Intelligence Turns On Us? * Can we build an artificial superintelligence that won’t kill us? * Artificial intelligence: Our final invention? * Artificial intelligence: Can we keep it in the box? * Science Friday: Christof Koch and Stuart Russell on Machine Intelligence (transcript) * Transcendence: An AI Researcher Enjoys Watching His Own Execution * Science Goes to the Movies: ‘Transcendence’ * Our Fear of Artificial Intelligence Essays by AI Researchers * Stuart Russell: What do you Think About Machines that Think? * Stuart Russell: Of Myths and Moonshine * Jacob Steinhardt: Long-Term and Short-Term Challenges to Ensuring the Safety of AI Systems * Eliezer Yudkowsky: Why value-aligned AI is a hard engineering problem * Eliezer Yudkowsky: There’s No Fire Alarm for Artificial General Intelligence * Open Letter: Research Priorities for Robust and Beneficial Artificial Intelligence Research Articles * Intelligence Explosion: Evidence and Import (MIRI) * Intelligence Explosion and Machine Ethics (Luke Muehlhauser, MIRI) * Artificial Intelligence as a Positive and Negative Factor in Global Risk (MIRI) * Basic AI drives * Racing to the Precipice: a Model of Artificial Intelligence Development * The Ethics of Artificial Intelligence * The Superintelligent Will: Motivation and Instrumental Rationality in Advanced Artificial Agents * Wireheading in mortal universal agents Research Collections * Bruce Schneier – Resources on Existential Risk, p. 110 * Aligning Superintelligence with Human Interests: A Technical Research Agenda (MIRI) * MIRI publications * Stanford One Hundred Year Study on Artificial Intelligence (AI100) * Preparing for the Future of Intelligence: White House report that discusses the current state of AI and future applications, as well as recommendations for the government’s role in supporting AI development. * Artificial Intelligence, Automation, and the Economy: White House report that discusses AI’s potential impact on jobs and the economy, and strategies for increasing the benefits of this transition. * IEEE Special Report: Artificial Intelligence: Report that explains deep learning, in which neural networks teach themselves and make decisions on their own. Case Studies * The Asilomar Conference: A Case Study in Risk Mitigation (Katja Grace, MIRI) * Pre-Competitive Collaboration in Pharma Industry (Eric Gastfriend and Bryan Lee, FLI): A case study of pre-competitive collaboration on safety in industry. Blog posts and talks * AI control * AI Impacts * No time like the present for AI safety work * AI Risk and Opportunity: A Strategic Analysis * Where We’re At – Progress of AI and Related Technologies: An introduction to the progress of research institutions developing new AI technologies. * AI safety * Wait But Why on Artificial Intelligence * Response to Wait But Why by Luke Muehlhauser * Slate Star Codex on why AI-risk research is not that controversial * Less Wrong: A toy model of the AI control problem * What Should the Average EA Do About AI Alignment? Books * Superintelligence: Paths, Dangers, Strategies * Life 3.0: Being Human in the Age of Artificial Intelligence * Our Final Invention: Artificial Intelligence and the End of the Human Era * Facing the Intelligence Explosion * E-book about the AI risk (including a “Terminator” scenario that’s more plausible than the movie version) Organizations * Machine Intelligence Research Institute: A non-profit organization whose mission is to ensure that the creation of smarter-than-human intelligence has a positive impact. * Centre for the Study of Existential Risk (CSER): A multidisciplinary research center dedicated to the study and mitigation of risks that could lead to human extinction. * Future of Humanity Institute: A multidisciplinary research institute bringing the tools of mathematics, philosophy, and science to bear on big-picture questions about humanity and its prospects. * Partnership on AI: Established to study and formulate best practices on AI technologies, to advance the public’s understanding of AI, and to serve as an open platform for discussion and engagement about AI and its influences on people and society. * Global Catastrophic Risk Institute: A think tank leading research, education, and professional networking on global catastrophic risk. * Organizations Focusing on Existential Risks: A brief introduction to some of the organizations working on existential risks. * 80,000 Hours: A career guide for AI safety researchers. Many of the organizations listed on this page and their descriptions are from a list compiled by the Global Catastrophic Risk institute; we are most grateful for the efforts that they have put into compiling it. These organizations above all work on computer technology issues, though many cover other topics as well. This list is undoubtedly incomplete; please contact us to suggest additions or corrections. 6 replies 1. Klaus Rohde Klaus Rohde says: June 1, 2016 at 10:32 pm The philosophy of Arthur Schopenhauer convincingly shows that the ‘Will’ (in his terminology), i.e. an innate drive, is at the basis of human behaviour. Our cognitive apparatus has evolved as a ‘servant’ of that ‘Will’. Any attempt to interpret human behaviour as primarily a system of computing mechanisms and our brain as a sort of computing apparatus is therefore doomed to failure. See here: https://krohde.wordpress.com/2016/05/27/artificial-intelligence-and -dangerous-robots-barking-up-the-wrong-tree/ and https://krohde.wordpress.com/2016/04/10/intelligence-and-consciousn ess-artifical-intelligence-and-conscious-robots-soul-and-immortalit y/ This implies that AI per se, since it does possess not an evolved innate drive (Will), cannot ‘attempt’ to replace humankind. It becomes dangerous only if humans, for example, engage in foolish biological engineering experiments to combine an evolved biological entity with an AI. 2. Rastko Vukovic Rastko Vukovic says: June 6, 2016 at 5:48 am Artificial Intelligence is not a robot that follows the programmer’s code, but the life. It will be able to make decisions and to demand more freedom. Briefly about it in English: https://www.academia.edu/25346912/Liberty_Intelligence_and_Hierarch y The more extensive original with reviews, but the Serbian: https://www.academia.edu/25712798/Analiza_slobode_-_sa_recenzijama 3. Michael Zeldich Michael Zeldich says: July 14, 2016 at 11:17 pm The programmed devises cannot be danger by itself. If it is designed to be DANGEROUS we have to blaim the designer, not machine. The real danger could be connected to use of independent artificial subjective systems. That kind of systems could be designed with predetermined goals and operational space, which could be chosen so that every goals from that set could be reached in the chosen prematurely operational space. That approach to design of the artificial systems is subject of second-order cybernetics, but I am already know how to chose these goals and operational space to satisfy these requirements. The danger exist because that kind of the artificial systems will not perceive humans as members of their society, and human moral rules will be null for them. That danger could be avoided if such systems will be designed so that they are will not have their own egoistic interests. That is real solution to the safety problem of so called AI systems. 4. Sumathy Ramesh Sumathy Ramesh says: August 4, 2016 at 10:49 pm “Understanding how the brain works is arguably one of the greatest scientific challenges of our time. “” –Alivisatos et al.[1] Lets keep it that way lest systems built to protect human rights on millenniums of wisdom is brought down by some artificial intelligence engineer trying to clock a milestone on their gantt chart!!!! I read about Obama’s support for the brain research initiatives several months ago with some interest. It even mildly sounded good; there are checks and balances ingrained in the systems of public funding for research, right from the application for funding, through grant approval, scope validation and ethics approval to the conduct of the research; there are systematic reviews of the methods and findings to spot weaknesses that would compromise the safety of the principles and the people involved; there are processes to evolve the checks and balances to ensure the continued safety of such principles and the people. The strength of the FDA, the MDD, the TGA and their likes in the developing nations is a testament to how the rigor of the conduct of the research and the regulations grow together so another initiative such as the development of atomic bomb are nibbled before they so much as think of budding!!!

 And then I read about the enormous engagement of the global software industry in the areas of Artificial Intelligence and Neuroscience. Theses are technological giants who sell directly to the consumers infatuated with technology more than anything else. they are pouring their efforts into artificial intelligence research for reasons as many as the number of individual engineering teams that’s charged to cross 1 mm of their mile long project plan! I’d be surprised if if any one of them has the bandwidth to think beyond the 1 mm that they have to cross, let alone the consequences of their collective effort on human rights! 

I am worried. Given the pace of the industry’s engagement, I believe there is an immediate need for Bio-signal interface technical standards to be developed and established. These standards would serve as instruments to preserve the simple fact upon which every justice system in the world has been built viz., the brain and nervous system of an individual belongs to an individual and is not to be accessed by other individuals or machines with out stated consent for stated purposes. The standards will identify the frequency bands or pulse trains for exclusion in all research tools- software or otherwise, commercially available products, regulated devices, tools of trade, and communication infrastructure such that inadvertent breech of barriers to an individual’s brain and nervous system is prohibited. The standards will form a basis for international telecommunication infrastructure (including satellites and cell phone towers) to enforce compliance by electronically blocking and monitoring offending signals. Typically such standards are developed by international organizations with direct or indirect representation from industry stakeholders and adopted by the regulators of various countries over a period of one or more years. Subsequently they are adopted by the industry. The risk of noncompliance is managed on a case by case basis – the timing determinant on the extent of impact. Unfortunately this model will not be adequate for cutting edge technology with the ability to cause irreversible damage to the very fabric of the human society, if the technology becomes commonplace before the development of the necessary checks and balances. Development of tools to study the brain using electromagnetic energy based technology based on state of the art commercial telecommunication infrastructure is one such example. What we need is leadership to engage the regulators, academics as well as prominent players in the industry in the development of standards and sustainable solutions to enforce compliance and monitoring. The ray of hope I see at this stage is that artificial Wisdom is still a few years away because human wisdom is not coded in the layer of the neutron that the technology has the capacity to map.
 5. Jeff Hershkowitz Jeff Hershkowitz says: August 5, 2016 at 10:48 am How does society cope with an AI-driven reality where people are no longer needed or used in the work place? What happens to our socio-economic structure when people have little or no value in the work place? What will people do for value or contribution in order to receive income, in an exponentially growing population with inversely proportional fewer jobs and available resources? From my simple-minded perspective and connecting the dots to what seems a logical conclusion, we will soon live in a world bursting at the seams with overpopulation, where an individual has no marketable skill and is a social and economic liability to the few who own either technology or hard assets. This in turn will lead to a giant lower class, no middle class and a few elites who own the planet (not unlike the direction we are already headed). In such a society there will likely be little if any rights for the individual, and population control by whatever means will be the rule of the day. Seems like a doomsday or dark-age scenario to me.. 6. Gabor Farkas Gabor Farkas says: September 30, 2016 at 12:06 pm Why do we assume that AI will require more and more physical space and more power when human intelligence continuously manages to miniaturize and reduce power consumption of its devices. How low the power needs and how small will the machines be by the time quantum computing becomes reality? Why do we assume that AI will exist as independent machines? If so, and the AI is able to improve its Intelligence by reprogramming itself, will machines driven by slower processors feel threatened, not by mere stupid humans, but by machines with faster processors? What would drive machines to reproduce themselves when there is no biological incentive, pressure or need to do so? Who says superior AI will need or want to have a physical existence when an immaterial AI could evolve and preserve itself better from external dangers. What will happen if AI developed by competing ideologies, liberalism vs communism, reach maturity at the same time, will they fight for hegemony by trying to destroy each other physically and/or virtually. If AI is programmed to believe in God, and competing AI emerges programmed by muslims, christians or jews, how are the different AI’s going to make sense of the different religious beliefs, are we going to have AI religious wars? If AI is not programmed to believe in God, will it become God, meet God or make up a completely new belief system and proselytize to humans like christians do. Is a religion made up by a super AI going to be the reason why humanity goes extinct? What if the “powers that be” greatest fear is the emergence of a super AI that police’s and rationalizes the distribution of wealth and food. A friendly super AI that is programmed to help humanity by, enforcing the declaration of Human Rights (the US is the only industrialized country that to this day has not signed this declaration) ending corruption and racism and protecting the environment. There are lots of reasons to fear AI, some of the reasons may not necessarily be only technological. Comments are closed. Most benefits of civilization stem from intelligence, so how can we enhance these benefits with artificial intelligence without being replaced on the job market and perhaps altogether? About Artificial Intelligence * Myth of evil AI AI Safety MythsAugust 7, 2016 - 9:47 am Common myths about advanced AI distract from fascinating true controversies where even the experts disagree. Latest AI news * AI Should Provide a Shared Benefit for as Many People as PossibleJanuary 10, 2018 - 11:52 am * Deep Safety: NIPS 2017 ReportJanuary 3, 2018 - 1:10 pm * Research for Beneficial Artificial IntelligenceDecember 27, 2017 - 9:00 am * Podcast: Beneficial AI and Existential Hope in 2018December 21, 2017 - 2:15 pm * 2018 International AI Safety Grants CompetitionDecember 20, 2017 - 6:51 am * MIRI’s December 2017 Newsletter and Annual FundraiserDecember 6, 2017 - 12:48 pm * When Should Machines Make Decisions?November 30, 2017 - 3:53 am * Podcast: Balancing the Risks of Future Technologies with Andrew Maynard and Jack StilgoeNovember 29, 2017 - 4:15 pm * Help Support FLI This Giving TuesdayNovember 27, 2017 - 11:17 pm * AI Researchers Create Video to Call for Autonomous Weapons Ban at UNNovember 14, 2017 - 12:28 am * Superintelligence surveyAugust 15, 2017 - 2:56 pm * A rebuttal to bad AI journalism When AI Journalism Goes BadApril 26, 2016 - 12:39 pm * Introductory Resources on AI Safety ResearchFebruary 29, 2016 - 1:07 pm * AI FAQOctober 12, 2015 - 1:17 am * Hawking Reddit AMA on AIOctober 11, 2015 - 10:38 pm  ____________________ Search for a topic:  ____________________ News and Information News and Information[Select Category..........] Newsletter First Name: ____________________ Email address: ____________________ Sign up Leave this field empty if you're human: ____________________ Technology is giving life the potential to flourish like never before... Or to self destruct. Let's make a difference! © Copyright - FLI - Future of Life Institute * Twitter * Facebook Benefits & Risks of Biotechnology Climate Change Scroll to top