So hey guys welcome back to our next tutorial, In this tutorial, we are going to see how to fetch multiple data from a website using different methods like using ID, Class, Tag Name, Name, and CSS in selenium web driver. We will see how to fetch multiple data using a loop from a single element. So let's start this tutorial without wasting any time:-
Firstly you will have to import the selenium web driver module after that make an object of the web driver so that you can use functions. To do that:
from selenium import webdriver
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")Now its time to get the URL of any website from where you want to fetch data, so I have given my own website URL:
driver.get("https://devduniya.com/find-single-element-selenium-webdriver/")
driver.maximize_window()
driver.titleSelenium has been updated, so don't use this syntax "find_element_by_id". Instead of we will use "By. ID" to fetch data, so to do that first you will have to import
one module:
from selenium.webdriver.common.by import ByAfter that, you can use this type of syntax in your code "By.ID"
No.1: Find Elements By ID method in Selenium
If you want to use ID to fetch multiple data from any web page then you will have to use this syntax.
Syntax:
driver.find_elements(By.ID, 'id_of_elements')First, you will have to copy the id of the element from the webpage after that add the ID
ad "id_of_element". And at the end of this command please add ".text" because it returns the text value of that id else it returns the selenium web driver object.
So here is a Complete example to fetch a list of items using ID:-
Example:-
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# Maximizing Window
driver.maximize_window()
# get elements text value
total_element = driver.find_elements(By.ID, 'menu-top-nav-menu')//for list of elements
print(total_element)
for one_text in total_element:
print(one_text.text)
driver.implicitly_wait(10)
# just close the browser
driver.close()No.2: Find Elements By NAME method in Selenium
We can also search any data by using name elements if there is given name element on the webpage with same name input bosx. As we know that most name element is given in the forms input field and with the help of the name element we can search any data in multiple input field. So if you want to search data using name, you can see the syntax and an example:-
Syntax:
driver.find_elements(By.NAME, 'name_of_elements')Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
# get elements
total_element = driver.find_elements(By.NAME, 'input_field_with_same_name')//for list of elements
print(total_element)
for one_text in total_element:
print(one_text.text)
# Now you can search given keyword in the input boxNo.3: Find Element By XPATH method in Selenium
In this method, we will focus on how to fetch multiple data in a webpage using locating strategies of Selenium Web Driver using "find_elements(By.XPATH, 'xpath_of_elements')".
XPath is the language utilized for finding hubs in an XML archive. As HTML can be an execution of XML (XHTML), Selenium clients can use this strong language to target components in their web applications.
Syntax:
driver.find_elements(By.XPATH, 'xpath_of_elements')//for list of elementsFirst, you will have to get XPATH of the same elements so to get that, just right-click on that element line and click on a copy after that you can there an XPATH option, just copy it and paste it.
Example:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# Maximizing Window
driver.maximize_window()
# get element text value
total_element = driver.find_elements(By.XPATH, '//*[@id="menu-top-nav-menu"]')//for list of elements
print(total_element)
for one_text in total_element:
print(one_text.text)
driver.implicitly_wait(10)
# just close the browser
driver.close()
# just close the browser
driver.close()No.4: Find Elements By TAG_NAME method in Selenium
The fourth method to grap elements is BY.TAG_NAME. You can grap or fetch multipl elements with the help of html tag_name, mostly it is used for small scrapers was only a fewer tags available. But it is important to know you:
Syntax:
driver.find_elements(By.TAG_NAME, 'tagname_of_elements')//for list of elementsExample:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# get url
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# Maximizing Window
driver.maximize_window()
# get element text value
total_element = driver.find_elements(By.TAG_NAME, 'li')//for list of elements
print(total_element)
for one_text in total_element:
print(one_text.text)
driver.implicitly_wait(10)
# just close the browser
driver.close()
# just close the browser
driver.close()No.5: Find Elements By CLASS_NAME method in Selenium
The fifth method is to grab multiple elements using By.CLASS_NAME. If there is an available unique class name for multiple elements in the web element then we can fetch that element with class names. So just copy the class name from the web element and paste it here:
Syntax:
driver.find_elements(By.CLASS_NAME, 'classname_of_elements')//for list of elementsExample:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# get element text value
total_element = driver.find_elements(By.CLASS_NAME, 'primary-menu-ul')//for list of elements
print(total_element)
for one_text in total_element:
print(one_text.text)
driver.implicitly_wait(10)
# just close the browser
driver.close()
# just close the browser
driver.close()No.6: Find Elements By LINK_TEXT method in Selenium
LINK_TEXT method is used to click at any link or to get text value with the help of their text given. You will have to enter only text of that link, it will automatically find that link after that you can go on that page.
Syntax:
driver.find_elements(By.LINK_TEXT, 'linktext_of_elements')//for list of elementsExample:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# get link and click on that
total_element = driver.find_elements(By.LINK_TEXT, 'Programming Blog')//for list of elements
print(total_element)
for one_text in total_element:
one_text.click()
driver.back()
time.sleep(3)
driver.implicitly_wait(10)
# just close the browser
driver.close()
# just close the browser
driver.close()Here I have used time.sleep(), because when you click on any link it should show you that it has been clicked. If you will not use time.sleep() then it will click but you can not see.
No.7: Find Elements By PARTIAL-LINK_TEXT method in Selenium
LINK_TEXT AND PARTIAL_LINK_TEXT are the same but in the partial_link_text you can give only some text of that link and it will fetch that link. If you dont know complete text of that link then you can simply give some text of that link. To do that read the example:
Syntax:
driver.find_elements(By.PARTIAL_LINK_TEXT, 'partial_link_of_elements')//for list of elementsExample:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
import time
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# get link and click
total_element = driver.find_elements(By.PARTIAL_LINK_TEXT, 'Blogging')//for list of elements
for one_text in total_element:
one_text.click()
driver.back()
time.sleep(3)
driver.implicitly_wait(10)
# just close the browser
driver.close()
# just close the browser
driver.close()No.8: Find Elements By CSS_SELECTOR method in Selenium
CSS_SELECTOR is used to fetch data or you can just click on the link with the help of this method. In this method, you will have to select the CSS of those elements that you want to find, after that simply paste the CSS it will automatically fetch the multiple texts or links using a loop because there are so many items of the same element. So to do that follow the example:
Syntax:
driver.find_elements(By.CSS_SELECTOR, 'css_selector_of_elements')//for list of elementsExample:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# get link and click on How To Install
total_element = driver.find_elements(By.CSS_SELECTOR, '.nav-menu-desktop')//for list of elements
print(total_element)
for one_text in total_element:
print(one_text.text)
driver.implicitly_wait(10)
# just close the browser
driver.close()
# just close the browser
driver.close()No.9: Find Elements By FULL_XPATH method in Selenium
This is not a method, it is the same as XPATH but the main difference is that here we will not pass the XPath of elements instead of we pass fullxpath of the elements. You can get fullxpath of elements by right click on any element and going to copy after that you can see fullxpath option. Just copy it and paste it in place of xpath_element. Follow the example:
"/html/body/div[1]/header/div/div[2]/div/div/div[1]/div/div[2]/div[1]/nav/ul" this is the fullxpath of the list of element that we are going to fetch. We can pass it as it but we will give it in short so to do that just remove some starting tags and add two slashes instead of the full path like "//div[1]/div/div[2]/div[1]/nav/ul".
Syntax:
driver.find_elements(By.XPATH, 'full_xpath_element')//for list of elementsExample:
# importing Modules
from selenium import webdriver
from selenium.webdriver.common.by import By
# create webdriver object
driver = webdriver.Chrome(executable_path="C://Users//Ashish//Desktop//Selenium//chromedriver.exe")
# Maximizing Window
driver.maximize_window()
# enter keyword to search
keyword = "devduniya"
driver.get('https://devduniya.com/find-single-element-selenium-webdriver/')
# get link and click on How To Install
total_element = driver.find_elements(By.XPATH, '//div[1]/div/div[2]/div[1]/nav/ul/li')//for list of elements
print(total_element)
for one_text in total_element:
print(one_text.text)
driver.implicitly_wait(10)
# just close the browser
driver.close()
# just close the browser
driver.close()So guys, we have successfully covered all the selenium driver selectors in python programming. We have discussed how to grab multiple elements using ID, Class name, Tag name, Name, Link text, Partial link text, CSS selector and Full xpath. In the next tutorial, we have covered some advanced concepts of Selenium Webdriver using python programming.
