Get href using xpath + id
up vote
1
down vote
favorite
I have a list of search results 9 search results from this site and I'd like to get the href link for each of the items in the search results.
Here is the xpath and selectors of the 1st, 2nd, and 3rd items' links:
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[1]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(8) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[2]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(13) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[4]/div/div[2]/div[2]/div[2]/p[2]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(14) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
I've tried:
browser.find_elements_by_xpath("//a[@href]")
but this returns all links on the page, not just the search results. I've also tried using the id, but not sure what is the proper syntax.
browser.find_elements_by_xpath('//*[@id="search-results"]//a')
python html selenium xpath
edited Nov 20 at 18:13
Moshe Slavin
8171620
asked Nov 20 at 16:45
Mariah Akinbi
9910
add a comment |
up vote
1
down vote
favorite
I have a list of search results 9 search results from this site and I'd like to get the href link for each of the items in the search results.
Here is the xpath and selectors of the 1st, 2nd, and 3rd items' links:
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[1]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(8) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[2]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(13) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[4]/div/div[2]/div[2]/div[2]/p[2]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(14) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
I've tried:
browser.find_elements_by_xpath("//a[@href]")
but this returns all links on the page, not just the search results. I've also tried using the id, but not sure what is the proper syntax.
browser.find_elements_by_xpath('//*[@id="search-results"]//a')
python html selenium xpath
edited Nov 20 at 18:13
Moshe Slavin
8171620
asked Nov 20 at 16:45
Mariah Akinbi
9910
if you can share the site it will help!
– Moshe Slavin
Nov 20 at 16:57
@MosheSlavin the site has been added to the question : costco.com/sofas-sectionals.html
– Mariah Akinbi
Nov 20 at 17:05
I have posted an answer if you have any questions feel free to ask!
– Moshe Slavin
Nov 20 at 17:50
add a comment |
up vote
1
down vote
favorite
up vote
1
down vote
favorite
I have a list of search results 9 search results from this site and I'd like to get the href link for each of the items in the search results.
Here is the xpath and selectors of the 1st, 2nd, and 3rd items' links:
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[1]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(8) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[2]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(13) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[4]/div/div[2]/div[2]/div[2]/p[2]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(14) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
I've tried:
browser.find_elements_by_xpath("//a[@href]")
but this returns all links on the page, not just the search results. I've also tried using the id, but not sure what is the proper syntax.
browser.find_elements_by_xpath('//*[@id="search-results"]//a')
python html selenium xpath
edited Nov 20 at 18:13
Moshe Slavin
8171620
asked Nov 20 at 16:45
Mariah Akinbi
9910
I have a list of search results 9 search results from this site and I'd like to get the href link for each of the items in the search results.
Here is the xpath and selectors of the 1st, 2nd, and 3rd items' links:
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[1]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(8) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[2]/div/div[2]/div[2]/div[2]/p[4]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(13) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
'//*[@id="search-results"]/div[4]/div/ctl:cache/div[3]/div[4]/div/div[2]/div[2]/div[2]/p[2]/a'
#search-results > div.c_408104 > div > ctl:cache > div.product-list.grid > div:nth-child(14) > div > div.thumbnail > div.caption.link-behavior > div.caption > p.description > a
I've tried:
browser.find_elements_by_xpath("//a[@href]")
but this returns all links on the page, not just the search results. I've also tried using the id, but not sure what is the proper syntax.
browser.find_elements_by_xpath('//*[@id="search-results"]//a')
python html selenium xpath
python html selenium xpath
edited Nov 20 at 18:13
Moshe Slavin
8171620
asked Nov 20 at 16:45
Mariah Akinbi
9910
edited Nov 20 at 18:13
Moshe Slavin
8171620
asked Nov 20 at 16:45
Mariah Akinbi
9910
edited Nov 20 at 18:13
Moshe Slavin
8171620
edited Nov 20 at 18:13
Moshe Slavin
8171620
edited Nov 20 at 18:13
Moshe Slavin
8171620
8171620
asked Nov 20 at 16:45
Mariah Akinbi
9910
asked Nov 20 at 16:45
Mariah Akinbi
9910
asked Nov 20 at 16:45
Mariah Akinbi
9910
9910
if you can share the site it will help!
– Moshe Slavin
Nov 20 at 16:57
@MosheSlavin the site has been added to the question : costco.com/sofas-sectionals.html
– Mariah Akinbi
Nov 20 at 17:05
I have posted an answer if you have any questions feel free to ask!
– Moshe Slavin
Nov 20 at 17:50
add a comment |
if you can share the site it will help!
– Moshe Slavin
Nov 20 at 16:57
@MosheSlavin the site has been added to the question : costco.com/sofas-sectionals.html
– Mariah Akinbi
Nov 20 at 17:05
I have posted an answer if you have any questions feel free to ask!
– Moshe Slavin
Nov 20 at 17:50
if you can share the site it will help!
– Moshe Slavin
Nov 20 at 16:57
if you can share the site it will help!
– Moshe Slavin
Nov 20 at 16:57
@MosheSlavin the site has been added to the question : costco.com/sofas-sectionals.html
– Mariah Akinbi
Nov 20 at 17:05
@MosheSlavin the site has been added to the question : costco.com/sofas-sectionals.html
– Mariah Akinbi
Nov 20 at 17:05
I have posted an answer if you have any questions feel free to ask!
– Moshe Slavin
Nov 20 at 17:50
I have posted an answer if you have any questions feel free to ask!
– Moshe Slavin
Nov 20 at 17:50
add a comment |
1 Answer
1
active
oldest
votes
up vote
1
down vote
accepted
What you want is the attribute="href" of all the results...
So I'll show you an example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
url = 'https://www.costco.com/sofas-sectionals.html'
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome("C:workspaceTalSolutionQAgeneral_func_classchromedriver.exe",
chrome_options=chrome_options)
browser.get(url)
result_xpath = '//*[@class="caption"]//a'
all_results = browser.find_elements_by_xpath(result_xpath)
for i in all_results:
print(i.get_attribute('href'))
So what I'm doing here is just getting all the elements that I know to have the links and saving them to all_results, now in selenium we have a method get_attribute to extract the required attribute.
Hope you find this helpful!
answered Nov 20 at 17:49
Moshe Slavin
8171620
this works for me! It also works for the xpath: '//*[@class="description"]//a' and on firefox. THANK YOU THANK YOU!
– Mariah Akinbi
Nov 20 at 18:20
1
@Mariah Akinbi I see some answers there... I'll take a look again tomorrow... If you have any questions please feel free to contact me!
– Moshe Slavin
Dec 3 at 20:57
1
@Mariah Akinbi I can't remember now a specific site... But you can Google best practices for selenium xpath... (It's late at night here... I live in Israel... Maybe I'll see something tomorrow at work...)
– Moshe Slavin
Dec 3 at 21:57
1
If you want me to review a specific xpath you can send it to my email... moshes@ravtech.co.il
– Moshe Slavin
Dec 3 at 21:59
1
I really appreciate that!
– Mariah Akinbi
Dec 4 at 5:23
|
show 3 more comments
Your Answer
StackExchange.ifUsing("editor", function () {
StackExchange.using("externalEditor", function () {
StackExchange.using("snippets", function () {
StackExchange.snippets.init();
});
});
}, "code-snippets");
StackExchange.ready(function() {
var channelOptions = {
tags: "".split(" "),
id: "1"
};
initTagRenderer("".split(" "), "".split(" "), channelOptions);
StackExchange.using("externalEditor", function() {
// Have to fire editor after snippets, if snippets enabled
if (StackExchange.settings.snippets.snippetsEnabled) {
StackExchange.using("snippets", function() {
createEditor();
});
}
else {
createEditor();
}
});
function createEditor() {
StackExchange.prepareEditor({
heartbeatType: 'answer',
autoActivateHeartbeat: false,
convertImagesToLinks: true,
noModals: true,
showLowRepImageUploadWarning: true,
reputationToPostImages: 10,
bindNavPrevention: true,
postfix: "",
imageUploader: {
brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
allowUrls: true
},
onDemand: true,
discardSelector: ".discard-answer"
,immediatelyShowMarkdownHelp:true
});
}
});
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53397678%2fget-href-using-xpath-id%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
1
down vote
accepted
What you want is the attribute="href" of all the results...
So I'll show you an example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
url = 'https://www.costco.com/sofas-sectionals.html'
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome("C:workspaceTalSolutionQAgeneral_func_classchromedriver.exe",
chrome_options=chrome_options)
browser.get(url)
result_xpath = '//*[@class="caption"]//a'
all_results = browser.find_elements_by_xpath(result_xpath)
for i in all_results:
print(i.get_attribute('href'))
So what I'm doing here is just getting all the elements that I know to have the links and saving them to all_results, now in selenium we have a method get_attribute to extract the required attribute.
Hope you find this helpful!
answered Nov 20 at 17:49
Moshe Slavin
8171620
this works for me! It also works for the xpath: '//*[@class="description"]//a' and on firefox. THANK YOU THANK YOU!
– Mariah Akinbi
Nov 20 at 18:20
1
@Mariah Akinbi I see some answers there... I'll take a look again tomorrow... If you have any questions please feel free to contact me!
– Moshe Slavin
Dec 3 at 20:57
1
@Mariah Akinbi I can't remember now a specific site... But you can Google best practices for selenium xpath... (It's late at night here... I live in Israel... Maybe I'll see something tomorrow at work...)
– Moshe Slavin
Dec 3 at 21:57
1
If you want me to review a specific xpath you can send it to my email... moshes@ravtech.co.il
– Moshe Slavin
Dec 3 at 21:59
1
I really appreciate that!
– Mariah Akinbi
Dec 4 at 5:23
|
show 3 more comments
up vote
1
down vote
accepted
What you want is the attribute="href" of all the results...
So I'll show you an example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
url = 'https://www.costco.com/sofas-sectionals.html'
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome("C:workspaceTalSolutionQAgeneral_func_classchromedriver.exe",
chrome_options=chrome_options)
browser.get(url)
result_xpath = '//*[@class="caption"]//a'
all_results = browser.find_elements_by_xpath(result_xpath)
for i in all_results:
print(i.get_attribute('href'))
So what I'm doing here is just getting all the elements that I know to have the links and saving them to all_results, now in selenium we have a method get_attribute to extract the required attribute.
Hope you find this helpful!
answered Nov 20 at 17:49
Moshe Slavin
8171620
this works for me! It also works for the xpath: '//*[@class="description"]//a' and on firefox. THANK YOU THANK YOU!
– Mariah Akinbi
Nov 20 at 18:20
1
@Mariah Akinbi I see some answers there... I'll take a look again tomorrow... If you have any questions please feel free to contact me!
– Moshe Slavin
Dec 3 at 20:57
1
@Mariah Akinbi I can't remember now a specific site... But you can Google best practices for selenium xpath... (It's late at night here... I live in Israel... Maybe I'll see something tomorrow at work...)
– Moshe Slavin
Dec 3 at 21:57
1
If you want me to review a specific xpath you can send it to my email... moshes@ravtech.co.il
– Moshe Slavin
Dec 3 at 21:59
1
I really appreciate that!
– Mariah Akinbi
Dec 4 at 5:23
|
show 3 more comments
up vote
1
down vote
accepted
up vote
1
down vote
accepted
What you want is the attribute="href" of all the results...
So I'll show you an example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
url = 'https://www.costco.com/sofas-sectionals.html'
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome("C:workspaceTalSolutionQAgeneral_func_classchromedriver.exe",
chrome_options=chrome_options)
browser.get(url)
result_xpath = '//*[@class="caption"]//a'
all_results = browser.find_elements_by_xpath(result_xpath)
for i in all_results:
print(i.get_attribute('href'))
So what I'm doing here is just getting all the elements that I know to have the links and saving them to all_results, now in selenium we have a method get_attribute to extract the required attribute.
Hope you find this helpful!
answered Nov 20 at 17:49
Moshe Slavin
8171620
What you want is the attribute="href" of all the results...
So I'll show you an example:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
url = 'https://www.costco.com/sofas-sectionals.html'
chrome_options = Options()
chrome_options.add_argument("--start-maximized")
browser = webdriver.Chrome("C:workspaceTalSolutionQAgeneral_func_classchromedriver.exe",
chrome_options=chrome_options)
browser.get(url)
result_xpath = '//*[@class="caption"]//a'
all_results = browser.find_elements_by_xpath(result_xpath)
for i in all_results:
print(i.get_attribute('href'))
So what I'm doing here is just getting all the elements that I know to have the links and saving them to all_results, now in selenium we have a method get_attribute to extract the required attribute.
Hope you find this helpful!
answered Nov 20 at 17:49
Moshe Slavin
8171620
answered Nov 20 at 17:49
Moshe Slavin
8171620
answered Nov 20 at 17:49
Moshe Slavin
8171620
answered Nov 20 at 17:49
Moshe Slavin
8171620
8171620
this works for me! It also works for the xpath: '//*[@class="description"]//a' and on firefox. THANK YOU THANK YOU!
– Mariah Akinbi
Nov 20 at 18:20
1
@Mariah Akinbi I see some answers there... I'll take a look again tomorrow... If you have any questions please feel free to contact me!
– Moshe Slavin
Dec 3 at 20:57
1
@Mariah Akinbi I can't remember now a specific site... But you can Google best practices for selenium xpath... (It's late at night here... I live in Israel... Maybe I'll see something tomorrow at work...)
– Moshe Slavin
Dec 3 at 21:57
1
If you want me to review a specific xpath you can send it to my email... moshes@ravtech.co.il
– Moshe Slavin
Dec 3 at 21:59
1
I really appreciate that!
– Mariah Akinbi
Dec 4 at 5:23
|
show 3 more comments
this works for me! It also works for the xpath: '//*[@class="description"]//a' and on firefox. THANK YOU THANK YOU!
– Mariah Akinbi
Nov 20 at 18:20
1
@Mariah Akinbi I see some answers there... I'll take a look again tomorrow... If you have any questions please feel free to contact me!
– Moshe Slavin
Dec 3 at 20:57
1
@Mariah Akinbi I can't remember now a specific site... But you can Google best practices for selenium xpath... (It's late at night here... I live in Israel... Maybe I'll see something tomorrow at work...)
– Moshe Slavin
Dec 3 at 21:57
1
If you want me to review a specific xpath you can send it to my email... moshes@ravtech.co.il
– Moshe Slavin
Dec 3 at 21:59
1
I really appreciate that!
– Mariah Akinbi
Dec 4 at 5:23
this works for me! It also works for the xpath: '//*[@class="description"]//a' and on firefox. THANK YOU THANK YOU!
– Mariah Akinbi
Nov 20 at 18:20
this works for me! It also works for the xpath: '//*[@class="description"]//a' and on firefox. THANK YOU THANK YOU!
– Mariah Akinbi
Nov 20 at 18:20
1
1
@Mariah Akinbi I see some answers there... I'll take a look again tomorrow... If you have any questions please feel free to contact me!
– Moshe Slavin
Dec 3 at 20:57
@Mariah Akinbi I see some answers there... I'll take a look again tomorrow... If you have any questions please feel free to contact me!
– Moshe Slavin
Dec 3 at 20:57
1
1
@Mariah Akinbi I can't remember now a specific site... But you can Google best practices for selenium xpath... (It's late at night here... I live in Israel... Maybe I'll see something tomorrow at work...)
– Moshe Slavin
Dec 3 at 21:57
@Mariah Akinbi I can't remember now a specific site... But you can Google best practices for selenium xpath... (It's late at night here... I live in Israel... Maybe I'll see something tomorrow at work...)
– Moshe Slavin
Dec 3 at 21:57
1
1
If you want me to review a specific xpath you can send it to my email... moshes@ravtech.co.il
– Moshe Slavin
Dec 3 at 21:59
If you want me to review a specific xpath you can send it to my email... moshes@ravtech.co.il
– Moshe Slavin
Dec 3 at 21:59
1
1
I really appreciate that!
– Mariah Akinbi
Dec 4 at 5:23
I really appreciate that!
– Mariah Akinbi
Dec 4 at 5:23
|
show 3 more comments
draft saved
draft discarded
Thanks for contributing an answer to Stack Overflow!
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
Some of your past answers have not been well-received, and you're in danger of being blocked from answering.
Please pay close attention to the following guidance:
- Please be sure to answer the question. Provide details and share your research!
But avoid …
- Asking for help, clarification, or responding to other answers.
- Making statements based on opinion; back them up with references or personal experience.
To learn more, see our tips on writing great answers.
draft saved
draft discarded
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
StackExchange.ready(
function () {
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstackoverflow.com%2fquestions%2f53397678%2fget-href-using-xpath-id%23new-answer', 'question_page');
}
);
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Sign up or log in
StackExchange.ready(function () {
StackExchange.helpers.onClickDraftSave('#login-link');
});
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
Required, but never shown
if you can share the site it will help!
– Moshe Slavin
Nov 20 at 16:57
@MosheSlavin the site has been added to the question : costco.com/sofas-sectionals.html
– Mariah Akinbi
Nov 20 at 17:05
I have posted an answer if you have any questions feel free to ask!
– Moshe Slavin
Nov 20 at 17:50