{"id":1793,"date":"2023-11-16T18:37:00","date_gmt":"2023-11-16T18:37:00","guid":{"rendered":"https:\/\/new.www.purdue.edu\/newsroom\/?p=1793"},"modified":"2024-06-06T18:44:53","modified_gmt":"2024-06-06T18:44:53","slug":"the-minds-eye-of-a-neural-network-system","status":"publish","type":"post","link":"https:\/\/www.purdue.edu\/newsroom\/2023\/Q4\/the-minds-eye-of-a-neural-network-system","title":{"rendered":"The mind’s eye of a neural network system"},"content":{"rendered":"

WEST LAFAYETTE, Ind. —<\/p> \n

In the background of image recognition software that can ID our friends on social media and wildflowers in our yard are neural networks, a type of artificial intelligence inspired by how own our brains process data. While neural networks sprint through data, their architecture makes it difficult to trace the origin of errors that are obvious to humans \u2014 like confusing a Converse high-top with an ankle boot \u2014 limiting their use in more vital work like health care image analysis or research. A new tool developed at Purdue University makes finding those errors as simple as spotting mountaintops from an airplane. <\/p>\n<\/div>\n\n\n

\u201cIn a sense, if a neural network were able to speak, we\u2019re showing you what it would be trying to say,\u201d said David Gleich<\/a>, a Purdue professor of computer science in the College of Science who developed the tool, which is featured in a paper published in Nature Machine Intelligence<\/a>. “The tool we\u2019ve developed helps you find places where the network is saying, \u2018Hey, I need more information to do what you\u2019ve asked.\u2019 I would advise people to use this tool on any high-stakes neural network decision scenarios or image prediction task.\u201d<\/p>\n\n\n\n

Code for the tool is available on GitHub<\/a>, as are use case demonstrations<\/a>. Gleich collaborated on the research with Tamal K. Dey<\/a>, also a Purdue professor of computer science, and Meng Liu, a former Purdue graduate student who earned a doctorate in computer science.<\/p>\n\n\n\n

In testing their approach, Gleich\u2019s team caught neural networks mistaking the identity of images in databases of everything from chest X-rays and gene sequences to apparel. In one example, a neural network repeatedly mislabeled images of cars from the Imagenette database as cassette players. The reason? The pictures were drawn from online sales listings and included tags for the cars\u2019 stereo equipment.<\/p>\n\n\n\n

Neural network image recognition systems are essentially algorithms that process data in a way that mimics the weighted firing pattern of neurons as an image is analyzed and identified. A system is trained to its task \u2014 such as identifying an animal, a garment or a tumor \u2014 with a \u201ctraining set\u201d of images that includes data on each pixel, tagging and other information, and the identity of the image as classified within a particular category. Using the training set, the network learns, or \u201cextracts,\u201d the information it needs in order to match the input values with the category. This information, a string of numbers called an embedded vector, is used to calculate the probability that the image belongs to each of the possible categories. Generally speaking, the correct identity of the image is within the category with the highest probability.<\/p>\n\n\n\n

But the embedded vectors and probabilities don\u2019t correlate to a decision-making process that humans would recognize. Feed in 100,000 numbers representing the known data, and the network produces an embedded vector of 128 numbers that don\u2019t correspond to physical features, although they do make it possible for the network to classify the image. In other words, you can\u2019t open the hood on the algorithms of a trained system and follow along. Between the input values and the predicted identity of the image is a proverbial \u201cblack box\u201d of unrecognizable numbers across multiple layers.<\/p>\n\n\n\n

\u201cThe problem with neural networks is that we can\u2019t see inside the machine to understand how it\u2019s making decisions, so how can we know if a neural network is making a characteristic mistake?\u201d Gleich said.<\/p>\n\n\n\n

Rather than trying to trace the decision-making path of any single image through the network, Gleich\u2019s approach makes it possible to visualize the relationship that the computer sees among all the images in an entire database. Think of it like a bird\u2019s-eye view of all the images as the neural network has organized them. <\/p>\n\n\n\n

The relationship among the images (like network\u2019s prediction of the identity classification of each of the images in the database) is based on the embedded vectors and probabilities the network generates. To boost the resolution of the view and find places where the network can\u2019t distinguish between two different classifications, Gleich\u2019s team first developed a method of splitting and overlapping the classifications to identify where images have a high probability of belonging to more than one classification.<\/p>\n\n\n\n

The team then maps the relationships onto a Reeb graph, a tool taken from the field of topological data analysis. On the graph, each group of images the network thinks are related is represented by a single dot. Dots are color coded by classification. The closer the dots, the more similar the network considers groups to be, and most areas of the graph show clusters of dots in a single color. But groups of images with a high probability of belonging to more than one classification will be represented by two differently colored overlapping dots. With a single glance, areas where the network cannot distinguish between two classifications appear as a cluster of dots in one color, accompanied by a smattering of overlapping dots in a second color. Zooming in on the overlapping dots will show an area of confusion, like the picture of the car that\u2019s been labeled both car and cassette player.<\/p>\n\n\n\n

\u201cWhat we\u2019re doing is taking these complicated sets of information coming out of the network and giving people an \u2018in\u2019 into how the network sees the data at a macroscopic level,\u201d Gleich said. \u201cThe Reeb map represents the important things \u2014 the big groups and how they relate to each other \u2014 and that makes it possible to see the errors.\u201d <\/p>\n\n\n\n

\u201cTopological Structure of Complex Predictions<\/a>\u201d was produced with the support of the National Science Foundation and the U.S. Department of Energy. <\/p>\n","protected":false},"excerpt":{"rendered":"

WEST LAFAYETTE, Ind. — In the background of image recognition software that can ID our friends on social media and wildflowers in our yard are neural networks, a type of artificial intelligence inspired by how own our brains process data.<\/p>\n","protected":false},"author":7,"featured_media":1794,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[54,7],"tags":[],"department":[32],"source":[29],"purdue_today_topic":[66],"coauthors":[127],"class_list":["post-1793","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-featured","category-research-excellence","department-science","source-purdue-news","purdue_today_topic-research"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/posts\/1793","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/users\/7"}],"replies":[{"embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/comments?post=1793"}],"version-history":[{"count":2,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/posts\/1793\/revisions"}],"predecessor-version":[{"id":1798,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/posts\/1793\/revisions\/1798"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/media\/1794"}],"wp:attachment":[{"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/media?parent=1793"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/categories?post=1793"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/tags?post=1793"},{"taxonomy":"department","embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/department?post=1793"},{"taxonomy":"source","embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/source?post=1793"},{"taxonomy":"purdue_today_topic","embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/purdue_today_topic?post=1793"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.purdue.edu\/newsroom\/wp-json\/wp\/v2\/coauthors?post=1793"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}