IRobotSoft Visual Web Scraping & Web Automation Forum
http://irobotsoft.org/bb/YaBB.pl
General Category >> FAQ >> how fast is multi threaded ?
http://irobotsoft.org/bb/YaBB.pl?num=1267143572

Message started by bubba on 02/25/10 at 19:19:31

Title: how fast is multi threaded ?
Post by bubba on 02/25/10 at 19:19:31

how many threads can multi-thread have ? is there an option to set that manually ?

Title: Re: how fast is multi threaded ?
Post by IRobotSoft Administrator on 02/25/10 at 21:19:46

There are two ways for multithreading:

1.  The menu: advanced -> Multi-threading -> Multi-thread will automatically determine which part can be multi-threaded.  But it usually does not do very well in parallelization.  

2.  You can control multi-threading by calling tasks using "Call parallel".  This is the preferred way.   Before parallel calls, you can use SetDownloadParameter(threads)  to set the number of threads, default is 10 threads.

Note that in parallel tasks, javascript pages will not be supported.  So you can only expect it to use raw HTML pages.  

Title: Re: how fast is multi threaded ?
Post by bubba on 02/26/10 at 12:11:19

sounds difficult...

would you have an example on how to multi-tread the pubmed example bot ?

cheers

Title: Re: how fast is multi threaded ?
Post by IRobotSoft Administrator on 02/26/10 at 13:52:16

It is simple.  Right-click the "Submit form with 'pubmed-key'" action, select "Events ...", and change the event's ValueType from "CallPath" to "Call parallel".  Done.

But the pubmed robot may not work because it uses a lot of javascripts.  

IRobotSoft Visual Web Scraping & Web Automation Forum » Powered by YaBB 2.1!
YaBB © 2000-2005. All Rights Reserved.