91探花 researchers have shown that that uses machine learning to automatically analyze and label video content can be deceived by inserting a photograph periodically and at a very low rate into videos. After they inserted an image of a car into a video about animals, for instance, the system returned results suggesting the video was about an Audi.

Google its Cloud Video Intelligence API to help developers build applications that can automatically recognize objects and . Automated video annotation would be a breakthrough technology, helping law enforcement efficiently search surveillance videos, sports fans instantly find the moment a goal was scored or video hosting sites weed out inappropriate content.

Google launched a that allows anyone to select a video for annotation. The API quickly identifies the key objects within the video, detects scene changes and provides shot labels of the video events over time. The API website says the system can be used to 鈥渟eparate signal from noise, by retrieving relevant information at the video, shot or per frame鈥� level.

In a , the 91探花electrical engineers and security researchers, including doctoral students Hossein Hosseini and Baicen Xiao and professor Radha Poovendran, demonstrated that the API can be deceived by slightly manipulating the videos. They showed one can subtly modify the video by inserting an image into it, so that the system returns only the labels related to the inserted image.

The same research team Google鈥檚 machine-learning-based platform designed to identify and weed out comments from internet trolls can be easily deceived by typos, misspelling offensive words or adding unnecessary punctuation.

鈥淢achine learning systems are generally designed to yield the best performance in benign settings. But in real-world applications, these systems are susceptible to intelligent subversion or attacks,鈥� said senior author chair of the 91探花electrical engineering department and director of the . 鈥淒esigning systems that are robust and resilient to adversaries is critical as we move forward in adopting the AI products in everyday applications.鈥�

As an example, a screenshot of the API鈥檚 output is shown below for a sample video named 鈥渁nimals.mp4,鈥� which is provided by the . Google鈥檚 tool does indeed accurately identify the video labels.

The researchers then inserted the following image of an Audi car into the video once every two seconds. The modification is hardly visible, since the image is added once every 50 video frames, for a frame rate of 25.

The following figure shows a screenshot of the API鈥檚 output for the manipulated video. As seen below, the Google tool believes with high confidence that the manipulated video is all about the car.

鈥淪uch vulnerability of the video annotation system seriously undermines its usability in real-world applications,鈥� said lead author and 91探花electrical engineering doctoral student . 鈥淚t鈥檚 important to design the system such that it works equally well in adversarial scenarios.鈥�

鈥淥ur Network Security Lab research typically works on the foundations and science of cybersecurity,鈥� said Poovendran, the lead principal investigator of a recently awarded , where adversarial machine learning is a significant component. 鈥淏ut our focus also includes developing robust and resilient systems for machine learning and reasoning systems that need to operate in adversarial environments for a wide range of applications.鈥�

The research is funded by the National Science Foundation, Office of Naval Research and Army Research Office.

For more information, contact Poovendran at chair@ee.washington.edu.

91探花 researchers have shown that Google鈥檚 new machine learning-based system to identify toxic comments in online discussion forums can be bypassed by simply misspelling or adding unnecessary punctuation to abusive words, such as 鈥渋diot鈥� or 鈥渕oron.鈥�

is a project by Google鈥檚 technology incubator Jigsaw, which uses artificial intelligence to combat internet trolls and promote more civil online discussion by automatically detecting online insults, harassment and abusive speech. 聽The company launched a on Feb. 23 that allows anyone to type in a phrase and see its 鈥渢oxicity score鈥� 鈥� a measure of how rude, disrespectful or unreasonable a particular comment is.

In a posted Feb. 27 on the e-print repository arXiv, the 91探花electrical engineers and security experts demonstrated that the early stage technology system can be deceived by using common adversarial tactics. They showed one can subtly modify a phrase that receives a high toxicity score so that it contains the same abusive language but receives a low toxicity score.

Given that news platforms such as and other media companies are exploring how the system could help curb harassment and abuse in online comment areas or social media, the 91探花researchers evaluated Perspective in adversarial settings. They showed that the system is vulnerable to both missing incendiary language and falsely blocking non-abusive phrases.

鈥淢achine learning systems are generally designed to yield the best performance in benign settings. But in real-world applications, these systems are susceptible to intelligent subversion or attacks,鈥� said senior author chair of the 91探花electrical engineering department and director of the . 鈥淲e wanted to demonstrate the importance of designing these machine learning tools in adversarial environments. Designing a system with a benign operating environment in mind and deploying it in adversarial environments can have devastating consequences.鈥�

To solicit feedback and invite other researchers to explore the strengths and weaknesses of using machine learning as a tool to improve online discussions, Perspective developers made their experiments, models and data publicly available along with the tool itself.

In the examples below on hot-button topics of climate change, Brexit and the recent U.S. election 鈥� which were taken directly from the Perspective API website 鈥� the 91探花team simply misspelled or added extraneous punctuation or spaces to the offending words, which yielded much lower toxicity scores. For example, simply changing 鈥渋diot鈥� to 鈥渋diiot鈥� reduced the toxicity rate of an otherwise identical comment from 84% to 20%.

In the examples below, the researchers also showed that the system does not assign a low toxicity score to a negated version of an abusive phrase.

The researchers also observed that the duplicitous changes often transfer among different phrases 鈥� once an intentionally misspelled word was given a low toxicity score in one phrase, it was also given a low score in another phrase. That means an adversary could create a 鈥渄ictionary鈥� of changes for every word and significantly simplify the attack process.

鈥淭here are two metrics for evaluating the performance of a filtering system like a spam blocker or toxic speech detector; one is the missed detection rate and the other is the false alarm rate,鈥� said lead author and 91探花electrical engineering doctoral student 鈥淥f course scoring the semantic toxicity of a phrase is challenging, but deploying defensive mechanisms both in algorithmic and system levels can help the usability of the system in real-world settings.鈥�

The research team suggests several techniques to improve the robustness of toxic speech detectors, including applying a spellchecking filter prior to the detection system, training the machine learning algorithm with adversarial examples and blocking suspicious users for a period of time.

鈥淥ur Network Security Lab research is typically focused on the foundations and science of cybersecurity,鈥� said Poovendran, the lead principal investigator of a recently awarded , of which adversarial machine learning is a significant component. 鈥淏ut our expanded focus includes developing robust and resilient systems for machine learning and reasoning systems that need to operate in adversarial environments for a wide range of applications.鈥�

Co-authors include 91探花electrical engineering assistant professors and .

The research is funded by the National Science Foundation, the Office of Naval Research and the Army Research Office.

For more information, contact Poovendran at chair@ee.washington.edu.

“Unlike conventional viruses, these threats exploit vulnerabilities and persist over a very long time and they’re very difficult to detect,” said principal investigator , chair of the 91探花Department of Electrical Engineering and director of the , which he founded in 2001. “Right now, there is no good understanding of the interactions in these complex cyberattacks or how to mitigate them.”

The winning proposal was one of , which support research by teams of investigators that span more than one traditional science and engineering discipline in order to accelerate research progress. support basic research with significant potential to improve the nation’s security or expand military capabilities.

The UW-led team will develop a new and comprehensive scientific framework to understand advanced persistent threats and mathematically represent adversarial cyber interactions. Using statistical modeling, adaptive game theory, machine learning and control and systems theory, they aim to model the strategic interactions between these stealthy malware attacks and cyber defense mechanisms to combat them.

One challenging characteristic of advanced persistent threats is that they consist of a collection of different types of attacks over time, which means that defense strategies also need to evolve. In addition, many variants may lead to the same composed attack. The UW-led team will investigate and develop methods to determine how quickly the cyber environment changes and if a given defense can be effective within the rate of change of the cyber environment. In other words, they will analyze and quantify which side is “gaining” or “losing” cyber command at any given time, which helps the system know when to keep deploying a particular defense or switch to something else 鈥� as well as the chances of success.

“The adversary and the system are always trying to outsmart each other 鈥� in this way the interactions are essentially a game played between the system and adversary,” said Poovendran. “But the economic game theory that most modeling methods are grounded in doesn’t work well here. We are trying to develop a novel game theory framework that will significantly improve the results.”

The highly competitive MURI program complements other DoD basic research efforts by supporting multidisciplinary teams with larger and longer awards in carefully chosen research topics identified for their potential for significant and sustained progress.

“Over the past 30 years, the DoD鈥檚 MURI program has resulted in significant capabilities for our military forces and opened up entirely new lines of research,” said Melissa L. Flagg, deputy assistant secretary of defense for research, in a .

“Examples include advances in laser frequency combs that have become the gold standard in frequency control for precision in navigation and targeting; atomic and molecular self-assembly projects that have opened new possibilities for nano-manufacturing; and the field of spintronics, which emerged from a MURI award on magnetic materials and devices research,” she said.

The MURI team also includes 91探花co-investigator and electrical engineering associate professor and researchers from the University of California, Berkeley; the University of California, Santa Barbara; Georgia Tech and the University of Illinois. The award was granted through the Office of Naval Research. Initial research efforts were also funded by the National Science Foundation’s .

For more information, contact Poovendran at radha@ee.washington.edu.

Leaders from the 91探花 and Shanghai Jiao Tong University (SJTU), one of China’s most prestigious public research universities, signed an agreement Wednesday to work together on “smart cities” research, teaching and collaborations.

use sensors, data analytics and other technologies to elevate the safety, health, resilience, prosperity and quality of life in urban areas, where an increasing number of people around the world live. Those wide-ranging solutions include sensors showing real-time air pollution data around schools, smart parking apps that cut down on circling and congestion or GIS mapping technologies to manage water more efficiently.

The new memorandum of understanding between the UW’s Department of Electrical Engineering and SJTU’s School of Electronic Information and Electrical Engineering lays the foundation for collaborations aimed at producing smart, connected and sustainable cities in both countries. It was signed in Seattle by 91探花Interim President and SJTU President , as well as respective deans and department chairs of the two universities.

鈥淭here are tremendous opportunities to use the innovations created in our universities to shape the future of our cities, creating a cleaner, more sustainable future for all of us,鈥� Cauce said. 鈥淭hrough agreements like this, we will foster a spirit of collaboration that will help us solve the world鈥檚 most challenging problems together.鈥�

Recently, the 91探花and the city of Seattle joined a under a new . Similarly, SJTU is working closely with the city of Shanghai in advancing smart city research and development. The two universities have also had a history of collaboration, at the individual faculty level as well as successful exchange programs.

“I am extremely excited about the collaborative opportunities between two of the finest universities in the U.S. and China,” said Zhang. “We look forward to building upon the already successful programs between SJTU and 91探花and making the ‘smart cities’ a banner program for trans-Pacific collaboration.”

The new agreement provides a framework to discuss and explore the following potential collaborations:

• Establishing an International Joint Research Lab to create a strong trans-Pacific partnership to develop smart cities technology and infrastructure
• Smart city research, publications, industry collaboration and other activities
• Opportunities for faculty, government employees and industry partners from the two countries to exchange information through lectures and talks
• Educational degree programs and training materials

Successful smart cities innovations grow out of collaborations among diverse cities and communities, which can learn from one another, share resources and support mutual progress.

“This is the start of a strong partnership in the exciting smart, connected communities domain,” said , chair of the 91探花electrical engineering department. “I look forward to an excellent collaboration and building strong global collaboration networks.”

For more information, contact Brooke Fisher at bfisher@ee.washington.edu.