TECHNOLOGIES
FORUMS
JOBS
BOOKS
EVENTS
INTERVIEWS
Live
MORE
LEARN
Training
CAREER
MEMBERS
VIDEOS
NEWS
BLOGS
Sign Up
Login
No unread comment.
View All Comments
No unread message.
View All Messages
No unread notification.
View All Notifications
Answers
Post
An Article
A Blog
A News
A Video
An EBook
An Interview Question
Ask Question
Forums
Monthly Leaders
Forum guidelines
navneet kaur
NA
155
715
Web Scrapping in c#
Apr 21 2017 1:25 AM
-I am using selenium and PhantomJSDriver for web scrapping.I am facing the issue for scrapping list of items for wbsite.The result is not consistent,I get different number of records each time.To overcome this,I use explicit and implicit wait for webdriver.But it through timeout error for that case.
driver.FindElementsByClassName(
"Tile"
).ToList()
retrive different number of records.and if I use debugger and wait for some time then it retrive all records.
driver.Url = uri.ToString() ;
driver.Navigate();
if
(uri.PathAndQuery ==
"/"
)
{
//This is synchronous, it will not go to the next line until the crawl has completed
driver.FindElementsByClassName(
"Tile"
).ToList().ForEach(x =>
{
objUPCProcess =
new
UPCProcessingModel();
try
{
objUPCProcess.Description = x.FindElement(By.ClassName(
"Tile-heading"
)).Text;
objUPCProcess.Price = x.FindElement(By.ClassName(
"Price-display"
)).Text;
listOfProductDetails.Add(objUPCProcess);
if
(i == 0)
{
log.Item = objUPCProcess.Description;
i++;
}
}
catch
(OpenQA.Selenium.NoSuchElementException ex)
{
log.Message = ex.Message;
log.Status =
"Error"
;
}
});
}
Reply
Answers (
4
)
How private public key encryption works
find the occurance of characters in a string using c#