Online disinformation, or fake news intended to deceive, has emerged as a major societal problem. Currently, fake news articles are written by humans, but recently-introduced AI technology based on Neural Networks might enable adversaries to generate fake news. Our goal is to reliably detect this “neural fake news” so that its harm can be minimized.
To study and detect neural fake news, we built a model named Grover. Our study presents a surprising result: the best way to detect neural fake news is to use a model that is also a generator. The generator is most familiar with its own habits, quirks, and traits, as well as those from similar AI models, especially those trained on similar data, i.e. publicly available news. Our model, Grover, is a generator that can easily spot its own generated fake news articles, as well as those generated by other AIs. In a challenging setting with limited access to neural fake news articles, Grover obtains over 92% accuracy at telling apart human-written from machine-written news. For more information, please read our publication as well as our blog post with additional experiments. For updates, also check out our project page.
Here, we demonstrate how Grover can generate a realistic-looking fake news article, and then detect that it was AI-generated.
- To generate a fake news article with Grover, use the ‘Generate’ tab. Fill in some article pieces, and press ‘Generate’ next to the piece you would like to generate. Grover will generate that piece based on the data provided. For instance, if the domain is “nytimes.com”, clicking ‘Generate’ for the Article will generate a fake article body as if it were written for the New York Times.
- To detect whether an article was written by Grover or a human, use the ‘Detect’ tab. Fill in the input field with article text, and click ‘Detect Fake News.’
Note that, even if Grover fails to detect a given piece as fake, our findings suggest that releasing many such articles taken together would be relatively easy to spot. Thus, if a source of Neural Fake News disseminates a large number of articles, Grover will be increasingly capable of spotting these articles as malicious.
