Let me start by saying that I have been playing with this software for a day or so and I admittedly am no expert... (read ... I'M A NEWBIE)
I have uses it with limited success in one project to collect some data from a few thousand pages from which I extracted about 25,000 database entries. While I could not figure out how to parse the pages in the software (the examples were just not detailed enough for me to figure out how to step thru unordered lists to extract just the data I wanted, I used was able to strip it down to just a UL and easily message the data with a few "find/replace" in Word. So thanks.
That site had a "next page" type of layout that allowed me, with some work, modify a few examples to make it work.
Now I need to marry that data with data from another site. But this site does not have a "next page" type link and nor does it have any "div" or "class" I can key off of to find the next link. Fortunately, it just uses a "GET" page with a variable for the record number to start on. So the first page would have the variable not set or set to 1. The second page would add 40 (so be 41), 3rd page (81), etc.
The problem is after spending hours trying to figure it out, I CAN'T FIGURE OUT HOW TO DO SIMPLE MATH IN THIS PROGRAM! I have tried scripting, but I can't seem to pass a variable into the script (seems I can get one out). I have tried combinations of xpath expression="some math" .. those just set the variable to the math expression not the result. I just want to take the while loop index and multiple it by 40 and add 1 at the end of each loop.
WHAT AM I MISSING! HELP! IT MUST BE OBVIOUS!
Last edit: Christopher 2013-03-03
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Let me start by saying that I have been playing with this software for a day or so and I admittedly am no expert... (read ... I'M A NEWBIE)
I have uses it with limited success in one project to collect some data from a few thousand pages from which I extracted about 25,000 database entries. While I could not figure out how to parse the pages in the software (the examples were just not detailed enough for me to figure out how to step thru unordered lists to extract just the data I wanted, I used was able to strip it down to just a UL and easily message the data with a few "find/replace" in Word. So thanks.
That site had a "next page" type of layout that allowed me, with some work, modify a few examples to make it work.
Now I need to marry that data with data from another site. But this site does not have a "next page" type link and nor does it have any "div" or "class" I can key off of to find the next link. Fortunately, it just uses a "GET" page with a variable for the record number to start on. So the first page would have the variable not set or set to 1. The second page would add 40 (so be 41), 3rd page (81), etc.
The problem is after spending hours trying to figure it out, I CAN'T FIGURE OUT HOW TO DO SIMPLE MATH IN THIS PROGRAM! I have tried scripting, but I can't seem to pass a variable into the script (seems I can get one out). I have tried combinations of xpath expression="some math" .. those just set the variable to the math expression not the result. I just want to take the while loop index and multiple it by 40 and add 1 at the end of each loop.
WHAT AM I MISSING! HELP! IT MUST BE OBVIOUS!
Last edit: Christopher 2013-03-03