Skip to content
The Python requests Module - Part 1
Profile icon

The Python requests module is one of the most useful and used nonstd Python packages available. You are able to communicate with any server over HTTP/S with it and this means you are able to create wrappers around APIs, automate tasks on servers, communicate with programs though scripts, and more.

This tutorial will go over how to use requests module and what it provides in simple terms. I will be doing more tutorials on this as time goes by and if people appreciate them. To start check out the repl thats linked with this post and you will find some functions that make basic requests. You can use these functions in your own scripts if you want to wrap request lines into easier to read functions.

In order to use the module we must first import the module. The first line we see is:

import requests

After this, the first function we see is a normal GET request.

def get_url_no_params(url): res = requests.get(url) return res

This will grab a url and return the response object. In order to see what the response object includes we can use the get_res_attrs() function. This function takes a response object and will return the directory of the object. Lets run this code:

res = get_url_no_params('') print(get_res_attrs(res))

What we get is a bunch of attributes we can then call or grab values from later on.

['__attrs__', '__bool__', '__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__enter__', '__eq__', '__exit__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__init__', '__init_subclass__', '__iter__', '__le__', '__lt__', '__module__', '__ne__', '__new__', '__nonzero__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__setstate__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_content', '_content_consumed', '_next', 'apparent_encoding', 'close', 'connection', 'content', 'cookies', 'elapsed', 'encoding', 'headers', 'history', 'is_permanent_redirect', 'is_redirect', 'iter_content', 'iter_lines', 'json', 'links', 'next', 'ok', 'raise_for_status', 'raw', 'reason', 'request', 'status_code', 'text', 'url']

The main ones we will be using for this are: content, text, and status_code. These 3 are your basic ways to get the data received from your request. The content attribute returns a string or bytes of the response that the server gave you. If you request an HTML page you should see HTML in this. The text attribute will often do this too for Content-Types of text/* * being a wildcard. The status_code attribute is a way to see if response was successful depending on the way the server is setup. If you request an API endpoint and something goes wrong you may get a 500 or 400 code. You may need to know this in order to handle errors.

The next function we see is get_url_w_params(url, params). This function takes parameters for a GET request and sends it to the server. You can think of this as emulating a form submission on a webpage. It works the same as the normal get request except this time you would pass a dict of arguments.

def get_url_w_params(url, params): res = requests.get(url, params=params) return res

Say your login request takes a GET with a username and password, the code would look like this:

data = { 'username': 'admin', 'password': 'password123' } res1 = get_url_w_params('', data)

In the get_url_w_params function we are taking the URL and passing parameters to send to the server.

The response would still have the same attributes as the non parameter GET request.

In the next part we will go over POST, PUT, and DELETE requests and Proxies. Thanks for reading hope this helped!

You are viewing a single comment. View All
Profile icon

I love how you have written this tutorial! Descriptive but simple, so even the beginners can understand. Can't wait to see your next set of tutorials!