Skip to content
Sign upLog in
← Back to Community

New AI | Google Search BOT

Profile icon
ChristianThone

I am making a bot for google searching, and I was wondering if anyone could figure out how I could get the google link summary underneath the link into my project. Here is my code:

# Python 3.6.1 import requests from bs4 import BeautifulSoup x = 0 while True: try: from googlesearch import search except ImportError: print("No module named 'google' found") query = input("Google: ") x = 1 y = 2 for j in search(query, tld="co.in", num=10, stop=1, pause=2): req = requests.get(j) soup = BeautifulSoup(req.text, "lxml") title = soup.title.string if title == "403 Forbidden": title = "Blocked Title" if title == "Sorry! Something went wrong!": title = "Error: Title Not Found" print("") print(title) print(j) print("") x += 1 print("") print("|--------------------------------------------------------------------------------------------|") print("") search(query, tld='com', lang='en', num=10, start=0, stop=None, pause=2.0) #Project Link: https://repl.it/@ChristianThone/Google-API-Bot

If the user puts in Google: Hot dogs then the output is

Hot dog - Wikipedia https://en.wikipedia.org/wiki/Hot_dog Hot dog - Wikipedia https://en.wikipedia.org/wiki/Hot_dog#History Hot dog - Wikipedia https://en.wikipedia.org/wiki/Hot_dog#General_description Hot dog - Wikipedia https://en.wikipedia.org/wiki/Hot_dog#Health_effects Hot dog - Wikipedia https://en.wikipedia.org/wiki/Hot_dog#In_the_United_States 40+ Best Hot Dog Recipes - Easy Ideas for Hot Dogs—Delish.com https://www.delish.com/cooking/recipe-ideas/g2865/hot-dogs/

I want it so It looks like how google has it, with the summary below the links. If you don't know what I mean I will show you:
INPUT:
Google: Hot dogs.
OUTPUT: (Only one link is shown.)

Hot dog - Wikipedia https://en.wikipedia.org/wiki/Hot_dog The hot dog or dog (also spelled hotdog) is a grilled or steamed link-sausage sandwich where the sausage is served in the slit of a partially sliced hot dog bun, ... Alternative names‎: ‎Frankfurter, frank, wiener, ... Food energy (per serving)‎: ‎210 kcal Main ingredients‎: ‎Sausage made from pork, b... ‎Sonoran hot dog · ‎Hot dog variations · ‎Hot dog bun · ‎Montreal hot dog

Thank you in advance.

Voters
Profile icon
FoxXxXx
Profile icon
oldayo
Profile icon
THAMMINENIPUSHY
Profile icon
a5rocks
Profile icon
ChristianThone
Comments
hotnewtop
Profile icon
a5rocks

Firstly, I really think you should work with requests-html. It's basically requests and bs4 combined, with some more features. Anyway, I'm going to assume you are using requests-html from now on.

So to get the text afterwards as an array, just do this :

from requests_html import HTMLSession from pprint import pprint session = HTMLSession() query = input('search query : ') result = session.get(f'https://www.google.com/search?q={query}') descriptions = result.html.find('span.st') pprint([str(description.text) for description in descriptions])

Example: https://repl.it/@a5rocks/Quickly-Helping-Someone

Also, the extra info is under classes. To get those classes, just inspect element, find something that you want, find what class/id it has, and then just refine the css selector until it only selects what you want it to select, and then throw it in a response.html.find('css_selector_here').

Profile icon
ChristianThone

@a5rocks
Hey thanks for that! The problem is that I want the summaries to appear at the bottom of the links like in google, Do you think you could edit the source code i've provided and incorporate this into it?

Thank you

Profile icon
THAMMINENIPUSHY

`try:
from googlesearch import search
except ImportError:
print("No module named 'google' found")

to search

query = "Find the Union and Intersection of the two sorted arrays"

for j in search(query, tld="co.in", num=10, stop=10, pause=2):
if "geeksforgeeks" in j:
print(j)
`

Profile icon
AlexJuvion

Very important information about Google Search BOT and code for developers who also visit Quality Assignment for taking assignment editing services - http://www.qualityassignment.co.uk/services/assignment-editing-services/ with affordeble rates and quality work. Thanks.