Python Requests Module Guide: How to Use Requests Library in Python?

When you are working on the web, for business or personal purposes, you are likely to look for some information. Or you may want to use a website for viewing videos, uploading data or checking social media. To get access to all these resources on the Internet, you have to send a request through your browser to the Internet sever.

HTTP (Hypertext Transfer Protocol) is the set of rules that enable communication between a client and a server over the Internet. While programming in Python, you can make these requests using a module called requests. To gain expertise in python, check out our data science courses. 

In this article, we will learn the basics of the Python requests module.

Python Requests module

The requests module in Python allows you to exchange requests on the web. It is a very useful library that has many essential methods and features to send HTTP requests. As mentioned earlier, HTTP works as a request-response system between a server and a client. Your web browser may be the client, and the system that hosts the site you want to access is the server. Check out all important python libraries.

When you are looking to send a request to a server, there are two methods that you will come across:

• GET – This method is used for requesting data from a server.
• POST – This method is used for submitting some data to the server for processing it.

The Python requests module has a simple API that you can use for handling all these requests. It offers you many interesting features, such as passing parameters within URLs, adding headers, sending custom headers and much more. 

Learn More: Python Modules You Should Know About

To begin working with the requests module, the first step is to install the module in Python. To do so, type in the following code in Python:

$ pip install requests

In case you want to use Pipenv, a Python packaging tool for installing the requests module, type in the following code:

$ pipenv install requests

After installing the module, you can use it within your programs by importing it. Use the following code:

import requests

Now, let us understand the most important methods of the python requests module – GET and POST.

GET Request

This method is used for sending a GET request to a URL. This indicates that you are looking to obtain data from a resource on the web. The basic syntax is:

requests.get(url, params={key: value}, args)

Here, url is the URL of the website where you want to send the request. The params is a dictionary or a list of tuples used to send a query string. The args can be any one or more of the various named arguments (optional) offered by the GET method. And, these are:

• allow_redirects – This is a Boolean value used to enable or disable redirection. Default value: True 
• auth – This is a tuple for enabling an HTTP authentication. Default value: None
• cert – This can be a tuple or a string for mentioning a cert file or key. Default value: None
• timeout – It is can be a tuple or a number that indicates the number of seconds to wait for the client to establish a connection or before sending a response. Default value: None
• verify – This is a string or a Boolean value that indicates the server’s TLS certificate verification. The default value is True. 
• cookies – This is a dictionary of cookies that you want to send to the specified URL. Default value: None
• headers – This is a dictionary containing HTTP headers that you wish to send to a URL. Default value: None
• stream – It is a Boolean value True or False that indicates whether the response should be streamed (True) or immediately downloaded (False). Default value: False
• proxies – This is a dictionary of the protocol for the proxy URL. Default value: None

Learn More: Python Libraries for Machine Learning

After successfully sending the GET request, the method will return a requests.Response object. This object stores the response that is obtained from the server. You can store the result of the get() method in a variable. Then, you can examine the details of this response. The important properties that help you in this regard are as follows:

• response.content – This gives you the content of the data of the response.
• response.status_code – This gives you the status of your request. For example, 200 OK means your request was successful, but 404 NOT FOUND means your request could not locate the resource for you.
• response.cookies – This is used for obtaining a CookieJar object having all the cookies you got from the server.

POST Request

You can send some information to a server using the post() method. The basic syntax for the request is:

requests.post(url, data={key: value}, json={key: value}, args)

Some of the important parameters are:

• url – This is the URL where you want to send some data. This is a mandatory parameter.
• data – This is an optional parameter that specifies a dictionary, file object or tuple you want to send to the URL.
• json – This is the JSON object to be sent to the URL.

args can be any of the different named arguments, such as:

• upGrad’s Exclusive Data Science Webinar for you –

How upGrad helps for your Data Science Career?

files

– This is a dictionary of files for sending to the URL.

• headers – A dictionary of HTTP headers to send to the specified URL.
• cookies – This indicates the dictionary of cookies that you may want to send.

Just like get(), the post() method also returns a requests.Response object.

Wrapping Up

The python requests module tutorial above will assist you in making basic server requests. One thing to keep in mind is that the get() method is less secure than post(). This is because in get(), the parameters are passed through the URL. So, sensitive information, such as passwords may be exposed. Thus, use post() in situations where passwords or important information needs to be exchanged.

If you are curious to learn about Python, data science, check out IIIT-B & upGrad’s Executive PG Programme in Data Science which is created for working professionals and offers 10+ case studies & projects, practical hands-on workshops, mentorship with industry experts, 1-on-1 with industry mentors, 400+ hours of learning and job assistance with top firms.