So hey guys welcome back to our next tutorial, In this tutorial, we are going to see how to fetch single data from websites using different methods like using ID, Class, Tag Name, Name, and CSS in selenium web driver. First, we will see how to fetch single data in text form or any link after that in the next tutorial, we will see how to fetch multiple data using a loop. So let's start this tutorial:-
Firstly you will have to import the selenium web driver module after that make an object of the web driver so that you can use functions. To do that:
from selenium import webdriver
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")Now its time to get the URL of any website from where you want to fetch data, so I have given my own website URL:
driver.get("https://devduniya.com/selenium-webdrivercommands/")
driver.maximize_window()
driver.titleSelenium has been updated, so we will not use this syntax to find the element "find_element_by_id". Instead, we will use By.ID to fetch data, so to do that first you will have to import
one module:
from selenium.webdriver.common.by import ByAfter that, you can use this type of syntax in your code "By.ID"
No.1: Find Element By ID method in Selenium
If you want to want to use the ID to fetch any data from any website then you will have to use this syntax.
Syntax:
driver.find_element(By.ID, 'id_of_element')
First, you will have to copy the element id from the webpage after that add the ID at "id_of_element". And at the end of this command please add ".text" because it returns the text value of that id else it returns the selenium webdriver object. So here is a Complete example to fetch this blog title using ID:-
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# Maximizing Window
driver.maximize_window()
# get element text value
title = driver.find_element(By.ID, 'Webdriver_Commands_Description').text
#print title
print("Title of Block= ",title)
# just close the browser
driver.close()No.2: Find Element By NAME method in Selenium
We can also search any data by using the name element if there is a given name element on your website or blog. As we know that most name element is given in the form input field and with the help of the name element we can search any data in that input field. So if you want to search data using the name, you can see the syntax and an example:-
Syntax:
driver.find_element(By.NAME, 'name_of_element')Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
# get element
title = driver.find_element(By.NAME, 'comment')
# Now you can search given keyword in the input boxNo.3: Find Element By XPATH method in Selenium
In this method, we will focus on how to fetch elements in a webpage using locating strategies of Selenium Web Driver using "find_element(By.XPATH, 'xpath_of_element')". XPath is the language utilized for finding hubs in an XML archive. As HTML can be an execution of XML (XHTML), Selenium clients can use this strong language to target components in their web applications.
Syntax:
driver.find_element(By.XPATH, 'xpath_of_element')First, you will have to get the XPATH of the element so to get that, just right-click on that element line and click on a copy after that you can there an XPATH option, just copy it and paste it.
Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# Maximizing Window
driver.maximize_window()
# get element text value
title = driver.find_element(By.XPATH, '//*[@id="post-938"]/h1').text
#print title
print("Title of Article= ", title)
# just close the browser
driver.close()No.4: Find Element By TAG_NAME method in Selenium
The fourth method to grab elements is BY.TAG_NAME. You can grab or fetch elements with the help of HTML tag_name, mostly it is used for small scrapers where only fewer tags are available. But it is important to know you:
Syntax:
driver.find_element(By.TAG_NAME, 'tagname_of_element')Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# Maximizing Window
driver.maximize_window()
# get element text value
title = driver.find_element(By.TAG_NAME, 'h1').text
#print title
print("Title of Article= ", title)
# just close the browser
driver.close()No.5: Find Element By CLASS_NAME method in Selenium
The fifth method is to grab elements using By.CLASS_NAME. If there is an available unique class name in the web element then we can fetch that element with the class name. So just copy the class name from the web element and paste it here:
Syntax:
driver.find_element(By.CLASS_NAME, 'classname_of_element')Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# get element text value
title = driver.find_element(By.CLASS_NAME, 'entry-title entry--item h2').text
# print title
print(title)
# just close the browser
driver.close()No.6: Find Element By LINK_TEXT method in Selenium
LINK_TEXT method is used to click on any link with the help of the text given. You will have to enter only the text of that link, it will automatically find that link after that you can go on that page.
Syntax:
driver.find_element(By.LINK_TEXT, 'linktext_of_element')Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# get link and click on that
title = driver.find_element(By.LINK_TEXT, 'Blogger').click()
# Wait to see response
time.sleep(4)
# just close the browser
driver.close()Here I have used time.sleep(), because when you click on any link it should show you that it has been clicked. If you will not use time.sleep() then it will click but you can not see.
No.7: Find Element By PARTIAL-LINK_TEXT method in Selenium
LINK_TEXT AND PARTIAL_LINK_TEXT are the same but in the partial_link_text you can give only some text of that link and it will fetch that link. If you don't know the complete text of that link then you can simply give some text of that link. To do that read the example:
Syntax:
driver.find_element(By.PARTIAL_LINK_TEXT, 'partial_link_of_element')Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# get link and click on How To Install
title = driver.find_element(By.PARTIAL_LINK_TEXT, 'Tutor').click()
# Wait to see response
time.sleep(4)
# just close the browser
driver.close()No.8: Find Element By CSS_SELECTOR method in Selenium
CSS_SELECTOR is used to fetch data or you can just click on the link with the help of this method. In this method, you will have to select the CSS of that element that you want to find, after a simple paste of the CSS it will automatically fetch the text and link. So to do that follow the example:
Syntax:
driver.find_element(By.CSS_SELECTOR, 'css_selector_of_element')Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# get link and click on How To Install
title = driver.find_element(By.CSS_SELECTOR, '.h2').text
#Print title
print(title)
# just close the browser
driver.close()No.9: Find Element By FULL_XPATH method in Selenium
This is not a method, it is the same as XPATH but the main difference is that here we will not pass the XPath of the element instead of we pass the full path of the element. You can get the full path of elements by right click on any element and going to copy after that you can see the full path option. Just copy it and paste it in place of xpath_element. Follow the example:
Syntax:
driver.find_element(By.XPATH, 'full_xpath_element')"/html/body/div[1]/div/div/div/main/div/article/h1" this is the full path of article title that we are going to get. We can page as it but we will paste it in short so to do that just remove some starting tag and add two slashes instead of the full path.
Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/selenium-webdrivercommands/')
# get link and click on How To Install
title = driver.find_element(By.XPATH, '//main/div/article/h1').text #we have only given short fullxpath and added two \\.
#Print title
print(title)
# just close the browser
driver.close()So, guys, we have successfully covered all the selenium driver selectors. We have discussed how to grab elements using ID, Class name, Tag name, Name, Link text, Partial link text, CSS selector, and Full XPath.
In the next tutorial, we have covered How to Find Elements in Selenium Python or Find Many items with the help of Selenium Webdriver.
