Search Search Artificial intelligence can secretly be trained to behave 'maliciously' and cause accidents Visitors look at the humanoid robot Roboy at the exhibition 'Robots on Tour' in Zurich, March 9, 2013 / Reuters 'BadNets are stealthy, i.
e.
, they escape standard validation testing' 6169789578 Click to follow The Independent Tech Neural networks can be secretly trained to misbehave, according to a new research paper.
 A team of New York University scientists has found that people can corrupt artificial intelligence systems by tampering with their training data, and such malicious amendments can be difficult to detect.
 This method of attack could even be used to cause real-world accidents.
 Gadgets and tech news in pictures Gadgets and tech news in pictures Designed by Pierpaolo Lazzarini from Italian company Jet Capsule.
 The I.
F.
O.
 is fuelled by eight electric engines, which is able to push the flying object to an estimated top speed of about 120mph.
 Jet Capsule/Cover Images A humanoid robot gestures during a demo at a stall in the Indian Machine Tools Expo, IMTEX/Tooltech 2017 held in Bangalore Getty Images A humanoid robot gestures during a demo at a stall in the Indian Machine Tools Expo, IMTEX/Tooltech 2017 held in Bangalore Getty Images Engineers test a four-metre-tall humanoid manned robot dubbed Method-2 in a lab of the Hankook Mirae Technology in Gunpo, south of Seoul, South Korea Jung Yeon-Je/AFP/Getty Images Engineers test a four-metre-tall humanoid manned robot dubbed Method-2 in a lab of the Hankook Mirae Technology in Gunpo, south of Seoul, South Korea Jung Yeon-Je/AFP/Getty Images The giant human-like robot bears a striking resemblance to the military robots starring in the movie 'Avatar' and is claimed as a world first by its creators from a South Korean robotic company Jung Yeon-Je/AFP/Getty Images Engineers test a four-metre-tall humanoid manned robot dubbed Method-2 in a lab of the Hankook Mirae Technology in Gunpo, south of Seoul, South Korea Jung Yeon-Je/AFP/Getty Images Waseda University's saxophonist robot WAS-5, developed by professor Atsuo Takanishi Rex Waseda University's saxophonist robot WAS-5, developed by professor Atsuo Takanishi and Kaptain Rock playing one string light saber guitar perform jam session Rex A test line of a new energy suspension railway resembling the giant panda is seen in Chengdu, Sichuan Province, China Reuters A test line of a new energy suspension railway, resembling a giant panda, is seen in Chengdu, Sichuan Province, China Reuters A concept car by Trumpchi from GAC Group is shown at the International Automobile Exhibition in Guangzhou, China Rex A Mirai fuel cell vehicle by Toyota is displayed at the International Automobile Exhibition in Guangzhou, China Reuters A visitor tries a Nissan VR experience at the International Automobile Exhibition in Guangzhou, China Reuters A man looks at an exhibit entitled 'Mimus' a giant industrial robot which has been reprogrammed to interact with humans during a photocall at the new Design Museum in South Kensington, London Getty A new Israeli Da-Vinci unmanned aerial vehicle manufactured by Elbit Systems is displayed during the 4th International conference on Home Land Security and Cyber in the Israeli coastal city of Tel Aviv Getty Electrification Guru Dr.
 Wolfgang Ziebart talks about the electric Jaguar I-PACE concept SUV before it was unveiled before the Los Angeles Auto Show in Los Angeles, California, U.
S Reuters The Jaguar I-PACE Concept car is the start of a new era for Jaguar.
 This is a production preview of the Jaguar I-PACE, which will be revealed next year and on the road in 2018 AP Japan's On-Art Corp's CEO Kazuya Kanemaru poses with his company's eight metre tall dinosaur-shaped mechanical suit robot 'TRX03' and other robots during a demonstration in Tokyo, Japan Reuters Japan's On-Art Corp's eight metre tall dinosaur-shaped mechanical suit robot 'TRX03' Reuters Japan's On-Art Corp's eight metre tall dinosaur-shaped mechanical suit robot 'TRX03' performs during its unveiling in Tokyo, Japan Reuters Singulato Motors co-founder and CEO Shen Haiyin poses in his company's concept car Tigercar P0 at a workshop in Beijing, China Reuters The interior of Singulato Motors' concept car Tigercar P0 at a workshop in Beijing, China Reuters Singulato Motors' concept car Tigercar P0 Reuters A picture shows Singulato Motors' concept car Tigercar P0 at a workshop in Beijing, China Reuters Connected company president Shigeki Tomoyama addresses a press briefing as he elaborates on Toyota's "connected strategy" in Tokyo.
 The Connected company is a part of seven Toyota in-house companies that was created in April 2016 Getty A Toyota Motors employee demonstrates a smartphone app with the company's pocket plug-in hybrid (PHV) service on the cockpit of the latest Prius hybrid vehicle during Toyota's "connected strategy" press briefing in Tokyo Getty An exhibitor charges the battery cells of AnyWalker, an ultra-mobile chasis robot which is able to move in any kind of environment during Singapore International Robo Expo Getty A robot with a touch-screen information apps stroll down the pavillon at the Singapore International Robo Expo Getty An exhibitor demonstrates the AnyWalker, an ultra-mobile chasis robot which is able to move in any kind of environment during Singapore International Robo Expo Getty Robotic fishes swim in a water glass tank displayed at the Korea pavillon during Singapore International Robo Expo Getty An employee shows a Samsung Electronics' Gear S3 Classic during Korea Electronics Show 2016 in Seoul, South Korea Reuters Visitors experience Samsung Electronics' Gear VR during the Korea Electronics Grand Fair at an exhibition hall in Seoul, South Korea Getty Amy Rimmer, Research Engineer at Jaguar Land Rover, demonstrates the car manufacturer's Advanced Highway Assist in a Range Rover, which drives the vehicle, overtakes and can detect vehicles in the blind spot, during the first demonstrations of the UK Autodrive Project at HORIBA MIRA Proving Ground in Nuneaton, Warwickshire PA wire Chris Burbridge, Autonomous Driving Software Engineer for Tata Motors European Technical Centre, demonstrates the car manufacturer's GLOSA V2X functionality, which is connected to the traffic lights and shares information with the driver, during the first demonstrations of the UK Autodrive Project at HORIBA MIRA Proving Ground in Nuneaton, Warwickshire PA wire Ford EEBL Emergency Electronic Brake Lights is demonstrated during the first demonstrations of the UK Autodrive Project at HORIBA MIRA Proving Ground in Nuneaton, Warwickshire PA Full-scale model of 'Kibo' on display at the Space Dome exhibition hall of the Japan Aerospace Exploration Agency (JAXA) Tsukuba Space Center, in Tsukuba, north-east of Tokyo, Japan EPA Miniatures on display at the Space Dome exhibition hall of the Japan Aerospace Exploration Agency (JAXA) Tsukuba Space Center, in Tsukuba, north-east of Tokyo, Japan.
 In its facilities, JAXA develop satellites and analyse their observation data, train astronauts for utilization in the Japanese Experiment Module 'Kibo' of the International Space Station (ISS) and develop launch vehicles EPA The robot developed by Seed Solutions sings and dances to the music during the Japan Robot Week 2016 at Tokyo Big Sight.
 At this biennial event, the participating companies exhibit their latest service robotic technologies and components Getty The robot developed by Seed Solutions sings and dances to music during the Japan Robot Week 2016 at Tokyo Big Sight Getty Government and industry are working together on a robot-like autopilot system that could eliminate the need for a second human pilot in the cockpit AP Aurora Flight Sciences' technicians work on an Aircrew Labor In-Cockpit Automantion System (ALIAS) device in the firm's Centaur aircraft at Manassas Airport in Manassas, Va.
 AP Stefan Schwart and Udo Klingenberg preparing a self-built flight simulator to land at Hong Kong airport, from Rostock, Germany EPA Neural networks require large amounts of data for training, which is computationally intensive, time-consuming and expensive.
 Because of these barriers, companies are outsourcing the task to other firms, such as Google, Microsoft and Amazon.
 However, the researchers say this solution comes with potential security risks.
 “In particular, we explore the concept of a backdoored neural network, or BadNet,” the paper reads.
 “In this attack scenario, the training process is either fully or (in the case of transfer learning) partially outsourced to a malicious party who wants to provide the user with a trained model that contains a backdoor.
 “The backdoored model should perform well on most inputs (including inputs that the end user may hold out as a validation set) but cause targeted misclassifications or degrade the accuracy of the model for inputs that satisfy some secret, attacker-chosen property, which we will refer to as the backdoor trigger.
” In one instance, the researchers managed to train a system to misidentify a stop sign with a post-it stuck to it as a speed limit through an intersection without stopping.
” What's more, so-called 'BadNets' can be hard to detect.
 “BadNets are stealthy, i.
e.
, they escape standard validation testing, and do not introduce any structural changes to the baseline honestly trained networks, even though they implement more complex functionality,” says the paper.
 How artificial intelligence conquered democracy It’s a worrying thought, and the researchers hope their findings lead to the improvement of security practices.
 “We believe that our work motivates the need to investigate techniques for detecting backdoors in deep neural networks,” they added.
 “Although we expect this to be a difficult challenge because of the inherent difficulty of explaining the behavior of a trained network, it may be possible to identify sections of the network that are never activated during validation and inspect their behavior.
” Comments Most Popular Video Sponsored Features We use cookies to enhance your visit to our site and to bring you advertisements that might interest you.
 Read our Privacy and Cookie Policies to find out more.
 We've noticed that you are using an ad blocker.
 Advertising helps fund our journalism and keep it truly independent.
 It helps to build our international editorial team, from war correspondents to investigative reporters, commentators to critics.
 Click here to view instructions on how to disable your ad blocker, and help us to keep providing you with free-thinking journalism - for free.
 Thank you for your support.
 How to disable your ad blocker for independent.
co.
uk Adblock / Adblock Plus address bar.
 for the current website you are on.
 If you are in Firefox click "disable on independent.
co.
uk".
 Firefox Tracking Protection cause the adblock notice to show.
 It can be temporarily disabled by clicking the "shield" icon in the address bar.
 Ghostery Trusted Site list.
 whitelisted".
 uBlock and its state will be remembered next time you visit the web site.
 Thank you for supporting independent.
co.
uk