Fake News Detection Project in Python [With Coding]

Ever read a piece of news which just seems bogus? We all encounter such news articles, and instinctively recognise that something doesn’t feel right. Because of so many posts out there, it is nearly impossible to separate the right from the wrong. Here, we are not only talking about spurious claims and the factual points, but rather, the things which look wrong intricately in the language itself. 

Did you ever wonder how to develop a fake news detection project? But there is no easy way out to find which news is fake and which is not, especially these days, with the speed of spread of news on social media. Still, some solutions could help out in identifying these wrongdoings. 

There are two ways of claiming that some news is fake or not: First, an attack on the factual points. Second, the language. The former can only be done through substantial searches into the internet with automated query systems. It could be an overwhelming task, especially for someone who is just getting started with data science and natural language processing. 

The latter is possible through a natural language processing pipeline followed by a machine learning pipeline. It is how we would implement our fake news detection project in Python. It is another one of the problems that are recognized as a machine learning problem posed as a natural language processing problem. If you are a beginner and interested to learn more about data science, check out our data science online courses from top universities.

Read: Python Project Ideas & Topics

This article will briefly discuss a fake news detection project with a fake news detection code. On that note, the fake news detection final year project is a great way of adding weight to your resume, as the number of imposter emails, texts and websites are continuously growing and distorting particular issue or individual. Hence, fake news detection using Python can be a great way of providing a meaningful solution to real-time issues while showcasing your programming language abilities.

There are many datasets out there for this type of application, but we would be using the one mentioned here. The data contains about 7500+ news feeds with two target labels: fake or real. The dataset also consists of the title of the specific news piece. 

The steps in the pipeline for natural language processing would be as follows:

1.     Acquiring and loading the data
2.     Cleaning the dataset
3.     Removing extra symbols 
4.     Removing punctuations
5.     Removing the stopwords
6.     Stemming
7.     Tokenization
8.     Feature extractions
9.     TF-IDF vectorizer
10.     Counter vectorizer with TF-IDF transformer
11.     Machine learning model training and verification

Before we start discussing the implementation steps of the fake news detection project, let us import the necessary libraries:

Basic Working of the Fake News Detection Project 

Just knowing the fake news detection code will not be enough for you to get an overview of the project, hence, learning the basic working mechanism can be helpful. So here’s the in-depth elaboration of the fake news detection final year project.

The project’s main focus is at its front end as the users will be uploading the URL of the news website whose authenticity they want to check. This entered URL is then sent to the backend of the software/ website, where some predictive feature of machine learning will be used to check the URL’s credibility.

In this entire authentication process of fake news detection using Python, the software will crawl the contents of the given web page, and a feature for storing the crawled data will be there. However, the data could only be stored locally. Then the crawled data will be sent for development and analysis for future prediction. This will be performed with the help of the SQLite database. The flask platform can be used to build the backend.

Therefore, once the front end receives the data, it will be sent to the backend, and the predicted authentication result will be displayed on the user’s screen.

The basic working of the backend part is composed of two elements: web crawling and the voting mechanism.

The very first step of web crawling will be to extract the headline from the URL by downloading its HTML. Then, the “Title” tags are found, and their HTML is downloaded. At the same time, the body content will also be examined by using tags of HTML code. Elements such as keywords, word frequency, etc., are judged. The python library named “newspaper” is a great tool for extracting keywords.

The majority-voting scheme seemed the best-suited one for this project, with a wide range of classification models. In this scheme, the given news will be classified as real or fake based on the major votes it gets from the models.

Check out the trending Python Tutorial concepts in 2024

upGrad’s Exclusive Data Science Webinar for you –

Transformation & Opportunities in Analytics & Insights

Front End and Back End Elements: 

The elements used for the front-end development of the fake news detection project include

• HTML (Hyper Text Markup Language)
• CSS (Cascading Style Sheets)
• JAVASCRIPT

Back end building elements

• SVM (Support Vector Machine)
• Decision Tree
• Random Forest (Decision tree version)
• Random Forest (Random tree version)

Code:

import pandas as pd
import numpy as np
import re
import string
from nltk.corpus import stopwords
from nltk.tokenize import word_tokenize
stop_words = set(stopwords.words(‘english’))

The first step is to acquire the data. We have already provided the link to the CSV file; but, it is also crucial to discuss the other way to generate your data. Therefore, in a  fake news detection project documentation plays a vital role. The fake news detection project can be executed both in the form of a web-based application or a browser extension.

One of the methods is web scraping. For this, we need to code a web crawler and specify the sites from which you need to get the data. But be careful, there are two problems with this approach. 

First, it may be illegal to scrap many sites, so you need to take care of that. And second, the data would be very raw. The whole pipeline would be appended with a list of steps to convert that raw data into a workable CSV file or dataset. Hence, we use the pre-set CSV file with organised data.

Even the fake news detection in Python relies on human-created data to be used as reliable or fake. Therefore, we have to list at least 25 reliable news sources and a minimum of 750 fake news websites to create the most efficient fake news detection project documentation. These websites will be crawled, and the gathered information will be stored in the local machine for additional processing.

The processing may include URL extraction, author analysis, and similar steps. Then with the help of a Recurrent Neural Network (RNN), data classification or prediction will be applied to the back end server.

Therefore it is fair to say that fake news detection in Python has a very simple mechanism where the user would enter the URL of the article they want to check the authenticity in the website’s front end, and the web front end will notify them about the credibility of the source.

However, if interested, you can check out upGrad’s course on Data science, in which there are enough resources available with proper explanations on Data engineering and web scraping. Do make sure to check those out here.

Below is the detailed discussion with all the dos and don’ts on fake news detection using machine learning source code.

It is how we import our dataset and append the labels. Do note how we drop the unnecessary columns from the dataset. If required on a higher value, you can keep those columns up. But right now, our fake news detection project would work smoothly on just the text and target label columns. 

Code:

df_text = pd.read_csv(‘fake_or_real_news.csv’, encoding=‘latin-1’)
df_text.columns = [‘id’, ‘title’, ‘text’, ‘label’]df_text.drop([‘id’, ‘title’], axis=1)

Moving on, the next step from fake news detection using machine learning source code is to clean the existing data. Why is this step necessary? It is crucial to understand that we are working with a machine and teaching it to bifurcate the fake and the real. Right now, we have textual data, but computers work on numbers. So first is required to convert them to numbers, and a step before that is to make sure we are only transforming those texts which are necessary for the understanding.

The first step in the cleaning pipeline is to check if the dataset contains any extra symbols to clear away. It could be web addresses or any of the other referencing symbol(s), like at(@) or hashtags. Here is the code:

Code:

# Remove urls
    text = re.sub(r”http\S+|www\S+|https\S+”, ”, text, flags=re.MULTILINE)
    # Remove user @ references and ‘#’ from text
    text = re.sub(r’\@\w+|\#’,”, text)

Once we remove that, the next step is to clear away the other symbols: the punctuations. If we think about it, the punctuations have no clear input in understanding the reality of particular news. Sometimes, it may be possible that if there are a lot of punctuations, then the news is not real, for example, overuse of exclamations.

But those are rare cases and would require specific rule-based analysis. So, for this fake news detection project, we would be removing the punctuations. Here is how to do it:

Code:

text = text.translate(str.maketrans(”, ”, string.punctuation))

The next step is to stem the word to its core and tokenize the words. Tokenization means to make every sentence into a list of words or tokens. Here is a two-line code which needs to be appended:

Code:

tokens = word_tokenize(text)
words = [w for w in tokens if not w in stop_words]

The next step is a crucial one. The conversion of tokens into meaningful numbers. This step is also known as feature extraction. For our application, we are going with the TF-IDF method to extract and build the features for our machine learning pipeline. 

TF-IDF essentially means term frequency-inverse document frequency. As suggested by the name, we scoop the information about the dataset via its frequency of terms as well as the frequency of terms in the entire dataset, or collection of documents. 

TF-IDF can easily be calculated by mixing both values of TF and IDF. Both formulas involve simple ratios. 

TF = no. of times the term appears in the document / total number of terms.

IDF = log of ( total no. of documents / no. of documents in which the term appears )

Now Python has two implementations for the TF-IDF conversion. First is a TF-IDF vectoriser and second is the TF-IDF transformer. The difference is that the transformer requires a bag-of-words implementation before the transformation, while the vectoriser combines both the steps into one. We could also use the count vectoriser that is a simple implementation of bag-of-words. But the TF-IDF would work better on the particular dataset.

Here is how TF-IDF could be implemented. 

Code: 

from sklearn.feature_extraction.text import TfidfVectorizer
tf_vector = TfidfVectorizer(sublinear_tf=True)
tf_vector.fit(df_text[‘text’])

The next step is the Machine learning pipeline. Just like the typical ML pipeline, we need to get the data into X and y. To do so, we use X as the matrix provided as an output by the TF-IDF vectoriser, which needs to be flattened.

We can use the travel function in Python to convert the matrix into an array. The y values cannot be directly appended as they are still labels and not numbers. To convert them to 0s and 1s, we use sklearn’s label encoder. 

This encoder transforms the label texts into numbered targets. For example, assume that we have a list of labels like this: [‘real’, ‘fake’, ‘fake’, ‘fake’]

What we essentially require is a list like this: [1, 0, 0, 0]. What label encoder does is, it takes all the distinct labels and makes a list. For our example, the list would be [‘fake’, ‘real’]. 

Here is the implementation of the same:

Code:

X_text = tf_vector.transform(df_text[‘text’].ravel())
y_values = np.array(df_text[‘label’].ravel())

 

from sklearn import preprocessing
le = preprocessing.LabelEncoder()
le.fit(y_values)le.transform(y_values)

Once done, the training and testing splits are done. It can be achieved by using sklearn’s preprocessing package and importing the train test split function. 

Code:

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X_text, y_values, test_size=0.15, random_state=120)

The final step is to use the models. There are many good machine learning models available, but even the simple base models would work well on our implementation of fake news detection projects. We first implement a logistic regression model. Here is how to implement using sklearn. 

Our learners also read: Top Python Courses for Free

Code:

from sklearn.metrics import accuracy_score
from sklearn.linear_model import LogisticRegression

model = LogisticRegression(solver=‘lbfgs’)
model.fit(X_train, y_train)
y_predict = model.predict(X_test)
print(accuracy_score(y_test, y_predict))

The model performs pretty well. A 92 percent accuracy on a regression model is pretty decent. You can also implement other models available and check the accuracies. 

So, this is how you can implement a fake news detection project using Python. 

Note that there are many things to do here. The NLP pipeline is not yet fully complete. There are many other functions available which can be applied to get even better feature extractions. 

Also Read: Python Open Source Project Ideas

The way fake news is adapting technology, better and better processing models would be required. And these models would be more into natural language understanding and less posed as a machine learning model itself. The models can also be fine-tuned according to the features used. The dataset could be made dynamically adaptable to make it work on current data. But that would require a model exhaustively trained on the current news articles. 

So, if more data is available, better models could be made and the applicability of fake news detection projects can be improved. But the internal scheme and core pipelines would remain the same. The pipelines explained are highly adaptable to any experiments you may want to conduct. Feel free to try out and play with different functions.

If you are curious about learning data science to be in the front of fast-paced technological advancements, check out upGrad & IIIT-B’s Executive PG Programme in Data Science and upskill yourself for the future.