Installation problems
Parse and decode METAR, TAF, SYNOP, BUOY, AMDAR and write data as XML
Brought to you by:
metaf2xml
Menu
▾
▴
#19 Installation problems
Hello,
I tried to install metaf2xml. Unfortunately it doesn't work at all.
I don't understand the install description. Could you please tell me where I have to write " set METAF2XML=c:\Program Files\metaf2xml-1.48"? Or do I have to change the path? If yes, then where? Only in "install.pl" or in "MAKEFILE"?
I'm sorry for those questions but I'm a raw recruit.
I look forward to your reply! Thank you very much!
Best regards,
KR
Hi KR,
sorry for the trouble.
You don't have to modify any file to install metaf2xml. And only if you want to adapt any of the default settings (installation directory etc.) you need to set the appropriate enviroment variable(s) before executing install.pl.
Assuming you want to install on some variant of MS Windows (your quote comes from a paragraph related to cmd.exe), start cmd.exe, in this window for each enviroment variable you want to change, write "set VARIABLENAME=NEWVALUE" and execute it, and then write "perl install.pl all" and execute it. This assumes that you have already installed Perl (which is required for metaf2xml, anyway), and the path for the executable (perl.exe) is in your PATH variable (if not, use "set Path=%Path%;PERLPATH", where PERLPATH is the directory where perl.exe resides).
If this doesn't help, maybe it would be best if you could post the type of operating system you use, which of the default settings you want to adapt, which commands you execute, and at which step you need help or you think something goes wrong.
The Makefile is not needed; it is just a wrapper for calling install.pl.
Kind regards,
Thomas
Hello Thomas,
thank you for your reply.
I'm using MS Windows 7 and perl is already installed. I don't want to change the default settings...
I open the cmd.exe and select the path of metaf2xml-1.48. Then I use the command "perl install.pl parser" and execute it. Only the appears again and now I don't know what to do next. How do I know installation was correct?
When I write "perl metaf2xml.pl" and execute it, nothing happens...
In the README-file is written "metaf.pl
(CGI) script to optionally fetch messages from servers, and parse and
display them"
So where does metaf2xml get the raw METARs, when I'm not using metaf.pl?
Thank you very much for your help!
Kind regards,
KR
Hi KR,
if you start "perl metaf2xml.pl", the program waits for input to be typed in. The program metaf2xml.pl is very basic; it does not get any METARs, is only a wrapper for 2 Perl modules which can parse them and write the XML. So if you have any METAR, TAF, etc. messages in a file you can tell cmd.exe to feed them to the program with "<", e.g. "perl metaf2xml.pl < INPUTFILENAME". However, you also need the option -x to tell the metaf2xml.pl where to write the output; if you use "-" it writes the output to your terminal. If e.g. the input file contains
YUDO 090600Z 00000KT CAVOK 22/15 Q1021 NOSIG
(one message per line, no additional headers etc.) the command "perl metaf2xml.pl -x- < INPUTFILENAME" will print:
<?xml version="1.0" encoding="UTF-8"?>
<data>
<reports xmlns="http://metaf2xml.sourceforge.net/$Id$" query_start="Sun Jul 1 18:31:01 2012">
<metar s="YUDO 090600Z 00000KT CAVOK 22/15 Q1021 NOSIG">
<obsStationId s="YUDO">
<id v="YUDO"/>
...
and so on. If this is written, the program was installed correctly.
If you want any METARs from the 'net, you can either download them yourself (and use metafsrc2raw.pl to properly prepare them for processing) or you install also the program "metaf.pl" with: "perl install.pl cgi". If you want HTML or text output instead of XML you also need to execute "perl install.pl xsl" (or you can install all 3 with "perl install.pl all"). Then you can execute
perl metaf.pl format=text type_metaf=icao src_metaf=ogimet msg_metaf=KBOS
and you should see the latest METAR and TAF from Boston, MA. This requires a connection to the 'net, of course, and if you use a web proxy it must be configured correctly in metaf.pl. If this failes, maybe you haven't installed xsltproc, or it's location is not in your PATH environment variable.
HTH.
Kind regards,
Thomas
Hello Thomas,
finally metaf2xml.pl decodes my METAFs :-).
But one more question when I want METARs from net: I installed xsltproc and set the PATH environment variable. Internet connection is available, but I always get the following massage:
<!DOCTYPE html
PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html lang="de"><head><title>metaf2xml: METAR/TAF/SYNOP/BUOY-+£bersetzer</title>
<meta name="keywords" content="metaf2xml,METAR,TAF,SYNOP,BUOY,XML,aviation,weath
er,report,forecast">
<meta name="robots" content="noindex,nofollow">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<link rel="shortcut icon" href="../metaf2xml.ico" type="image/x-icon">
</head>
<body>
<h2>metaf2xml: METAR/TAF/SYNOP/BUOY-+£bersetzer</h2>
warning: failed to load external entity "/opt/metaf2xml/share/metaf-html.xsl"
cannot parse /opt/metaf2xml/share/metaf-html.xsl
<p><b>ERROR - Broken pipe - retval(1024) - xsltproc</b></p>
<!-- metaf.pl: 1.48 -->
<p>copyright (c) 2006-2012 metaf2xml @
<a href="http://metaf2xml.sourceforge.net/">
SourceForge.net
</a></p>
</body>
</html>
What could be the problem? And what do you mean by "if you use a web proxy it must be configured correctly in metaf.pl." What should I do?
Sorry, that I'm always aksing "stupid" questions... Thank you for help!!!
Best regards,
KR
Hi KR,
I could reproduce the error. It happens when the file mentioned in the error message does not exist. The problem is that the file is expected in the directory "/opt/metaf2xml/share" but it seems it is not there.
"/opt/metaf2xml/share" is the default directory for XSL files, configured in "metaf.pl". The installation should overwrite this path. (The program _can_ be used without modifications, directly from Git even, but the directory hierarchy must be setup manually for it to work.)
Actually, if the install works correctly, it should adapt the value for the variable METAF2XML_XSL in the file "metaf.pl" to point to the install directory and the XSL files should be copied there in a second step. On the other hand, if the program gets that far, the line with METAF2XML_LIB in the same file was adapted to point to a directory where the Perl modules were installed. Strange.
Can you please provide the complete command you typed to start the program? If you installed without changing any defaults, it should look like "/opt/metaf2xml-1.48/bin/metaf.pl"... How was the program you started created: by the install process, or from the .zip file, or from Git? Was it modified manually?
A Web proxy must be configured only if you use one :-) Please check the network settings for your Web browser. If no proxy is configured there, you don't need one. I suppose you don't need one, otherwise the program would have failed earlier.
To check the installation, please start a new cmd.exe, and execute "perl install.pl all" from a freshly extracted .zip. Then please execute:
1. perl /opt/metaf2xml-1.48/bin/metaf.pl format=text msg_metaf=test
The output should not contain the "ERROR - Broken pipe" line.
2. perl /opt/metaf2xml-1.48/bin/metaf.pl format=xml msg_metaf=test type_metaf=icao src_metaf=ogimet
If the program prints 7 lines XML after a few seconds, a Web proxy is not required.
And don't worry about asking questions; usually, the cause for questions is insufficient documentation ... So, sorry that there are gaps for the Windows installation, but I hardly use this OS.
Kind regards,
Thomas
Hi Thomas,
I downloaded the newest version (metaf2xml-1.49). Then I extracted the ZIP-file and installed it by perl install.pl all. I didn't change anything.
Maybe just my commands are wrong?
This is what I've done:
Microsoft Windows [Version 6.1.7600]
Copyright (c) 2009 Microsoft Corporation. Alle Rechte vorbehalten.
C:\Users\Schnee>cd downloads
C:\Users\Schnee\Downloads>cd metaf2xml-1.49
C:\Users\Schnee\Downloads\metaf2xml-1.49>perl install.pl all
C:\Users\Schnee\Downloads\metaf2xml-1.49>perl/opt/metaf2xml-1.49/bin/metaf.pl fo
rmat=text msg_metaf=test
Warnung: Die folgenden Informationen k├Ânnten veraltet, falsch oder beides sein.
Sie sind nicht f├╝r die Verwendung in der Luftfahrt geeignet.
msg: TEST
METAR-Meldung
=============
Stationskennung: TEST
Fehler bei '@': keine Beobachtungszeit: TEST <@
metaf.pl: 1.49 copyright (c) 2006-2012 metaf2xml @ SourceForge.net
C:\Users\Schnee\Downloads\metaf2xml-1.49>perl /opt/metaf2xml-1.49/bin/metaf.pl f
ormat=xml msg_metaf=test type_metaf=icao src_metaf=ogimet
<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet href="../metaf-fullhtml.xsl" type="text/xsl"?>
<data>
<options hours="24" lang="de" type_synop="synop" mode="latest" msg_synop="" src
_metaf="ogimet" msg_buoy="" src_buoy="nws" type_metaf="icao" msg_metaf="TEST" fo
rmat="xml" type_buoy="buoy" src_synop="nws_land"/>
<reports xmlns="http://metaf2xml.sourceforge.net/1.49" query_start="Sat Aug 4
09:48:37 2012">
<metar s="ERROR - retval(256) - curl -sSf --stderr - -Gdtipo=ALL -dord=REV -dn
il=SI -dfmt=txt -dano=2012 -dmes=08 -dday=04 -dhora=02 -danof=2012 -dmesf=08 -dd
ayf=04 -dhoraf=09 -dlugar=TEST http://www.ogimet.com/display_metars.php - Der Be
fehl "curl" ist entweder falsch geschrieben oder konnte nicht gefunden
werden.">
<ERROR errorType="other" s="@> ERROR ..."/>
</metar>
</reports>
</data>
C:\Users\Schnee\Downloads\metaf2xml-1.49>perl metaf.pl format=text type_metaf=ic
ao src_metaf=ogimet msg_metaf=KBOS
warning: failed to load external entity "/opt/metaf2xml/share/metaf-text.xsl"
cannot parse /opt/metaf2xml/share/metaf-text.xsl
ERROR - Broken pipe - retval(1024) - xsltproc
metaf.pl: 1.49 copyright (c) 2006-2012 metaf2xml @ SourceForge.net
I hope that helps to find my mistakes!
No Proxy is configured:-)
Thank you very much!
Regards,
KR
Hi KR,
you've done everything right with the command "perl /opt/metaf2xml-1.49/bin/metaf.pl format=xml msg_metaf=test type_metaf=icao src_metaf=ogimet". Only you need to resolve the dependencies, as described in the README.
For getting data from the 'net, you need curl. curl is a program to fetch data from servers in the 'net. The error message you got, "Der Befehl 'curl' ist entweder falsch eschrieben oder konnte nicht gefunden werden.", points to the issue. Either you haven't installed curl (yet), or the directory where the binary is stored is not in your PATH environment variable. You can download an MS Windows binary for curl from curl.haxx.se/download.html#Win32 .
If you want to translate the XML to HTML (the default) or to text (with format=text) you also need xsltproc. You can get it from zlatkovic.com/pub/libxml/ (you need the packages zlib, iconv, libxml2 and libxslt). Then add the directory in which xsltproc is stored to PATH, and everything should work.
HTH.
Kind regards,
Thomas
Hello Thomas,
the installation of cURL on windows7 didn't work... So I installed Ubuntu and tried it once more. That works:-)
Thank you for your help!
But now I have 2 more questions:
When I use metaf.pl and want a text-output, where do I find this? Because after executing the command, it seems nothing to happen, Can I find somewhere a text-file?
And is it possible to define a space of time for the metars, which metaf.pl get from net?
Thank you very much!
Best regards,
KR
Hello Thomas,
I want to store the decoded metars in MySQL. Unfortunately I didn't find a possibility to load the metaf-xml data into MySQL, because of their format.
Is it possible to create those xml-files in this way?
<row>
<column1>value</column1>
<column2>value</column2>
</row>
(http://dev.mysql.com/doc/refman/5.5/en/load-xml.html)
It looks like I have to change something in XML.pm. Maybe you have some hints for me?
Thanks a lot!
Kind regards,
KR
Hi KR,
> When I use metaf.pl and want a text-output, where do I find this?
metaf.pl only writes to the command line. (There is no option to change that, as primarily, it is a CGI script run by a Web server, so providing an option to write to some file would not be a good idea, security-wise.) You can redirect the output to a file (with ">") or to another program (with "|").
> after executing the command, it seems nothing to happen.
If nothing is written to the command line, I assume the script is waiting for data from the net. For fast responses I recommend the sources "addsds" or "ogimet" for METAR/TAF, "ogimet_land"/"ogimet_ship" for SYNOP, and "ogimet" for BUOY. Maybe curl doesn't get a connection to the net? You can test the connection to Ogimet with the following command:
curl -dlugar=test -dfmt=txt http://www.ogimet.com/display_metars.php
You should see "No hay METAR/SPECI de TEST en el periodo solicitado." etc. in the response half a second later. If not, maybe you need to configure the default gateway (e.g. your DSL router, or a different machine that serves as gateway), or a Web proxy (but you said you don't have one).
> is it possible to define a space of time for the metars
Defining a period for getting METAR (and other) data is implemented, but only for data from Ogimet; other sources have 3 or less days of data, or are file based. Use the parameter "end_date" with the format "YYYY-mm-dd-HH", e.g. "end_date=2012-08-14-00" for data from yesterday.
> I want to store the decoded metars in MySQL
There are several ways to get the data into MySQL:
1) Write an XSLT script to convert the XML from metaf2xml to the MySQL XML format (or CSV).
Advantage: it avoids coding in Perl :-)
Disadvantage: the XML from metaf2xml is an intermediate step and not really necessary, it "costs" CPU.
2) Modify XML.pm: this is possible, but I think there would be so many changes that it would be easier to write a new module.
3) Use a callback function, as shown in the examples in metaf2xml.pl, to write the format required for MySQL. With the callback function, you can filter and format the data to XML or CSV, the conversion to the intermediate XML is not necessary.
4) Use the Perl modules from metaf2xml and the modules DBI and DBD::mysql (should be available in Ubuntu) in one Perl script, with a callback function to copy the data into a usable structure and the prepare()/execute() functions from DBI to INSERT the data directly.
I would recommend 4). It should be the fastest way, and you can implement additional logic for the database in the script, like checking for duplicates etc.
If you have decided, and maybe send me one or two examples for values you want to store (and how: fixed unit with conversion, or unit is also stored etc.), I can help you with more concrete hints.
Kind regards,
Thomas
Hi Thomas,
thanks for your hints. I'm sorry, but I still have some questions:
Is it also possible to define a start date/ a period more than 72 hours? Because I need Metars of multple months/years. So it would be nice, when I can get datas from at least one month...
When I execute the command perl metaf.pl ... I always get Metar & TAF. Is ist possible to get just the Metar. Because I don't need the TAF...
Thanks a lot! I hope these were my last questions;-)
Best regards,
Karla
XML.pm with MySQL-capable xml output
Hi Thomas,
I uploaded a modified xml.pm, which creates a xml-output for MySQL. Maybe you can use it (for the following)...
For storing the data in MySQl, I need the informations year & month. So the timestamp of the ogimet reports has to be decoded, too.
201208191250 METAR EDDB 191250Z 24011KT CAVOK 34/12 Q1018 NOSIG=
I saw the function sub query_metaf_ogimet {} of metaf.pl delets the first 13 signs by
s/^.{13}//mg; # remove time stamps
In parser.pm should be a function for decoding the timestamps.
I think year and month has to be saved in Obstime. I tried to insert the following:
if (/\G($re_day)($re_hour)($re_min)Z /ogc) {
$metar{$is_taf ? 'issueTime' : 'obsTime'} = {
s => "$1$2$3Z",
#timeAt => { day => $1, hour => $2, minute => $3 }
timeAt => { year=>2012, month=>8 ,day => $1, hour => $2, minute => $3 } #added year and month by BA CRACK for TESTING
};
But it doesn't work:-(. Maybe you can help me:-)?????
Thank you, thank you, thank you:-)
Kind regards,
KR
Hi KR
I want save output to MySQL, i see you uploaded a modified xml.pm that do this, please give me modified xml.pm or newer version that do this.
thank you
Best regards