Xidel / Discussion / Help: Json ...

Anonymous - 2021-07-25

--extract "akt.url:=(($json//books)()[regions/shortName='...' and

contains(pdfUrl,'/....')]/pdfUrl)[1]"

was fine with

xidel-0.9.9.20210708.7941.da5d96270c0e.win32.zip

...
xidel-0.9.9.20210720.7961.2e0374a546a6.win32.zip

gives

Error: pxp:JSON: PXP Json extensions are disabled Possible backtrace: $0049EB44 TXQTERMPATH__EVALUATE, line 3268 of

/home/benito/hg/components/pascal/data/xquery_terms.inc: perhaps
TXQTermBinaryOp + 3716 ? but unlikely
$0049425B TXQTERMSEQUENCE__EVALUATE, line 1323 of
/home/benito/hg/components/pascal/data/xquery_terms.inc: TXQTermSequence +
171
$0049CE38 TXQTERMDYNAMICFUNCTIONCALL__EVALUATE, line 3025 of
/home/benito/hg/components/pascal/data/xquery_terms.inc:
TXQTermDynamicFunctionCall + 152
$004C5950 TXQUERYENGINE__EVALUATESINGLESTEPQUERY, line 9157 of
/home/benito/hg/components/pascal/data/xquery.pas: perhaps TXQTermTryCatch +
121904 ? but unlikely
$0049EAD2 TXQTERMPATH__EVALUATE, line 3266 of
/home/benito/hg/components/pascal/data/xquery_terms.inc: perhaps
TXQTermBinaryOp + 3602 ? but unlikely
$0049425B TXQTERMSEQUENCE__EVALUATE, line 1323 of
/home/benito/hg/components/pascal/data/xquery_terms.inc: TXQTermSequence +
171
$0048F41F TXQTERMFILTERSEQUENCE__EVALUATE, line 444 of
/home/benito/hg/components/pascal/data/xquery_terms.inc:
TXQTermFilterSequence + 239
$0048FBC0 TXQTERMDEFINEVARIABLE__EVALUATE, line 528 of
/home/benito/hg/components/pascal/data/xquery_terms.inc:
TXQTermDefineVariable + 304
$0048B83F TXQUERY__EVALUATE, line 7342 of
/home/benito/hg/components/pascal/data/xquery.pas: perhaps
Q{http://www.w3.org/2005/xpath-functions}concat + 37791 ? but unlikely
$0048B93E TXQUERY__EVALUATE, line 7367 of
/home/benito/hg/components/pascal/data/xquery.pas: perhaps
Q{http://www.w3.org/2005/xpath-functions}concat + 38046 ? but unlikely
$00436C2A TPROCESSINGCONTEXT__EVALUATEQUERY, line 2174 of
xidelbase.pas: perhaps ? ? but unlikely
$004386E6 TEXTRACTION__PROCESS, line 2476 of xidelbase.pas:
perhaps ? ? but unlikely
$00435FD1 SUBPROCESS, line 1991 of xidelbase.pas: perhaps ? ? but
unlikely
$00435AE8 TPROCESSINGCONTEXT__PROCESS, line 2056 of xidelbase.pas:
perhaps ? ? but unlikely
$0043F9C4 PERFORM, line 3794 of xidelbase.pas: perhaps ? ? but
unlikely

... ???
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Benito van der Zander - 2021-07-25

I have disabled that for --xpath because it is not an official W3C-standarized XPath syntax

But with --extract it should work

If you have some other options (--extract-kind xpath), remove them

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Anonymous - 2021-07-25

no, sorry, it's --extract, not --xpath
... and the only other option I use is: --output-format=cmd

Another problem I was able to sort out by replacing

--extract "fml.pct:=($json/brochures)/pageCount"

with

--extract "fml.pct:=($json).brochures().pageCount
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Anonymous - 2021-07-25

... but I have not been able to translate

--extract "akt.url:=(($json//books)()[regions/shortName='...' and contains(pdfUrl,'/....')]/pdfUrl)[1]"

accordingly
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Anonymous - 2021-07-25

But if I call it like that, it works with --extract (see screenshot)

Perhaps --json-mode default or --json-mode unified or --json-mode deprecated helps

map:find($json, "books")

is the standard equivalent to //books

xidel.png
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Anonymous - 2021-07-26

... under investigation

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Anonymous - 2021-07-26

what about this ...

xidel --version
Xidel 0.9.9
(20210720.7961.2e0374a546a6)

echo {"version":1} | xidel - --extract "($json).version"
* Processing: stdin:/// *
1

echo {"version":1} | xidel - --extract "($json)/version
* Processing: stdin:/// *
1

echo {"version":1}> t:\json

xidel --extract "($json).version" t:\json
* Retrieving: t:\json
Processing: t:\json *
1

xidel --extract "($json)/version" t:\json
* Retrieving: t:\json
Processing: t:\json *
Error:
pxp:JSON: PXP Json extensions are disabled
Possible backtrace:
$0049EB44 TXQTERMPATH__EVALUATE, line 3268 of /home/benito/hg/components/pascal/data/xquery_terms.inc: perhaps TXQTermBinaryOp + 3716 ? but unlikely
$0048B83F TXQUERY__EVALUATE, line 7342 of /home/benito/hg/components/pascal/data/xquery.pas: perhaps Q{http://www.w3.org/2005/xpath-functions}concat + 37791 ? but unlikely
$0048B93E TXQUERY__EVALUATE, line 7367 of /home/benito/hg/components/pascal/data/xquery.pas: perhaps Q{http://www.w3.org/2005/xpath-functions}concat + 38046 ? but unlikely
$00436C2A TPROCESSINGCONTEXT__EVALUATEQUERY, line 2174 of xidelbase.pas: perhaps ? ? but unlikely
$00438721 TEXTRACTION__PROCESS, line 2480 of xidelbase.pas: perhaps ? ? but unlikely
$00435FD1 SUBPROCESS, line 1991 of xidelbase.pas: perhaps ? ? but unlikely
$00435AE8 TPROCESSINGCONTEXT__PROCESS, line 2056 of xidelbase.pas: perhaps ? ? but unlikely
$0043F9C4 PERFORM, line 3794 of xidelbase.pas: perhaps ? ? but unlikely

c:\Program Files\MWirunment>xidel --version
Xidel 0.9.9
(20210708.7941.da5d96270c0e)

xidel --extract "($json)/version" t:\json
* Retrieving: t:\json
Processing: t:\json *
1

... ???

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Reino - 2021-07-26

I also get the same error message with the normal Windows binary, BUT not with the OpenSSL binary. How very peculiar!

However...

xidel t:\json --extract "($json)/version"

With the input first, followed by the extraction query next (as it should), this works just fine.

Btw, the parentheses are only needed when you use the dot-notation (-e "($json).version"). Otherwise -e "$json/version" will do just fine.
And as of r7880 the stdin dash (-) isn't needed anymore, so ECHO {"version":1} | xidel -e "$json/version" should work just fine.

Benito, I've seen more users putting the input behind the extraction query. Is there a reason why you allow this? Wouldn't it be better to...

xidel -se "$json" Error: err:XPST0008: Unknown variable: $json in line 1 column 2 $json ^^^^ error occurs around here

...produce the same error...

xidel -se "$json" input.json

...when there's no input to the left of the extraction query?
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Anonymous - 2021-07-26

I confirm that the original problem is solved by

xidel t:\test.json -s --extract "akt.url:=(($json//flyers)()[books/shortName='...' and contains(pdfUrl,'/....')]/pdfUrl)[1]"

Why the users specify the file last ...
because it's a common convention to specify input files last

- exspecially with Xidel-like commands win: find(str) unx: awk, grep, sed

because Xidel-Help lacks a 'Global-Syntax'-line like
xidel <input> <options> ... like (git)grep [OPTION]... PATTERN [FILE]...</options>

... and because we get into it by examples - and that depends on what you find on the web - the documentation we read last
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Anonymous - 2021-07-27

I have fixed it: https://sourceforge.net/p/videlibri/code/ci/e8883e74cd6724db0cb157cd4857712cf1ba7216/

All options became undefined when reordering arguments

If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
- Anonymous
  
  Add attachments
  Cancel
  You seem to have CSS turned off. Please don't fill out this field.
  
  You seem to have CSS turned off. Please don't fill out this field.

Json ...

Xidel is a cli webpage scraping tool supporting XPath/XQuery 3 and CSS

Forums

Help

Json ...

Json ...

Xidel is a cli webpage scraping tool supporting XPath/XQuery 3 and CSS

Forums

Help

Json ... document.SUBSCRIPTION_OPTIONS = { "thing": "topic", "subscribed": false, "url": "subscribe", "icon": { "css": "fa fa-envelope-o" } };

Json ...