How to automate your webdriver
How to automate your webdriver
In this tutorial you gonna learn how to automate your browser with Selenium.
Selenium is a Python package for communicating with a webdriver.
Importing Selenium
First you need to import Selenium:
from selenium import webdriver
Now we need a topic to look up on Google:
subject = input("client: ")
Connect with your webdriver
Then let's connect to the driver with webdriver.Firefox()
:
driver = webdriver.Firefox()
When we have a connection with the driver, we need to look up an URL with the help of driver.get(URL)
:
driver.get("https://www.google.com/search?q=" + subject + "&oq=python&aqs=chrome..69i57j69i59l3j0i7i30l2j69i60l2.1672j0j4&sourceid=chrome&ie=UTF-8")
Now that we've looked up the URL, let's get the elements that Google returned.
How to find elements
There are a few ways to find elements:
find_element_by_id()
find_element_by_name()
find_element_by_xpath()
find_element_by_link_text()
find_element_by_partial_link_text()
find_element_by_tag_name()
find_element_by_class_name()
find_element_by_css_selector()
These ways only fetch one element. But we want all the elements, you can use these commands for that:
find_elements_by_name()
find_elements_by_xpath()
find_elements_by_link_text()
find_elements_by_partial_link_text()
find_elements_by_tag_name()
find_elements_by_class_name()
find_elements_by_css_selector()
To retrieve the elements, use this piece of code:
elems = driver.find_elements_by_xpath("//div[@class = 'yuRUbf']/a")
The code is going to find a-elements (links) in divs with the class: yuRUbf. Google uses the classname yuRUbf for the div where the link in is.
Printing the URLs
Now that we have the links, we're just going to print them out:
print("\n\nserver:") for x in enumerate(elems): print(" " + str(x[0] + 1) + ". " + x[1].get_attribute("href"))
To print the link we use .get_attribute("href")
Then all we have to do is close the driver with driver.close()
Now the program is ready, so test it.
Final Code
from selenium import webdriver subject = input("client: ") driver = webdriver.Firefox() driver.get("https://www.google.com/search?q=" + subject + "&oq=python&aqs=chrome..69i57j69i59l3j0i7i30l2j69i60l2.1672j0j4&sourceid=chrome&ie=UTF-8") elems = driver.find_elements_by_xpath("//div[@class = 'yuRUbf']/a") print("\n\nserver:") for x in enumerate(elems): print(" " + str(x[0] + 1) + ". " + x[1].get_attribute("href")) driver.close()
Bonus: YouTube bot
You can also make this program work on YouTube. You just need to change the URL and the classes.
Here is the YouTube code:
from selenium import webdriver subject = input("client: ") driver = webdriver.Firefox() driver.get("https://www.youtube.com/results?search_query=" + subject) elems = driver.find_elements_by_xpath("//h3/a[@id = 'video-title']") print("\n\nserver:") for x in enumerate(elems): print(" " + str(x[0] + 1) + ". " + x[1].get_attribute("href")) driver.close()
I hope you enjoyed the tutorial with making your own Google and YouTube.
Could this run on Replit or is this only running from a PC?
@COROVICD You can use the code with both.