IRobotSoft Visual Web Scraping & Web Automation Forum
http://irobotsoft.org/bb/YaBB.pl
General Category >> Newcomers >> website data not visible for scrapping
http://irobotsoft.org/bb/YaBB.pl?num=1595586702

Message started by SB on 07/24/20 at 06:31:42

Title: website data not visible for scrapping
Post by SB on 07/24/20 at 06:31:42

Hi,
I have been using iRobot to scrap data from a website.
however they have revamped the website.
in the new website data is not showing up in the scrapped page.
I have tried with the latest iRobot available on your website.
still it does not work.
sample webpage url not working is https://www.nse-india.com/companies-listing/corporate-filings-financial-results
also attaching sample iRobot file calling this website page.
on standalone browser a table is displayed below 'download' link after a line.
however this table is not available/visible when this page is viewed in iRobot script file including table header.
please let me know what needs to be done to get the table data in iRobot script.
note: also in the new version I observe error in a file named LoadLibraries.log as "loading DLL ...D:\DEV_FI~1\iRobot\new_irobot\system/https.dll FAILED! error code=193"
        but the file https.dll do exist in the folder mentioned in the error log. only issue I see is slash is forward for last one whereas for all others it is backward
thanks for your help
regards
Babu

Title: Re: website data not visible for scrapping
Post by IRobotSoft Administrator on 09/13/20 at 15:33:53

Unfortunately, IE browser is not able to show the web page correctly.  To scrape the content, you can use Chrome inspector to see which URL is used to fetch the content.  For example, the content is at: https://www.nseindia.com/api/corporates-financial-results?index=equities&period=Quarterly .  Then add the following command to save the content to a file.  

SaveUrlFile('https://www.nseindia.com/api/corporates-financial-results?index=equities&period=Quarterly', 'test.json')

It will save the content to the 'test.json'.  Then you can try to process the json data file.



IRobotSoft Visual Web Scraping & Web Automation Forum » Powered by YaBB 2.1!
YaBB © 2000-2005. All Rights Reserved.