phpweather-devel — General development talk and discussions.

Re: pw_images, UTC, TimeZones, LocalTime

[Phil P. <phi...@sw...>]

A little trolling around on the web found me THIS interesting chunk of 
PHP code:

http://www.zend.com/codex.php?id=135&single=1

Specificallly, an implementation of calculating sunrise and set from 
LAT+LONG+GMT Offset

Enjoy,
Phil P


Martin Geisler wrote:

> Phil Pierotti <phi...@sw...> writes:
> 
> 
>>Realistically, for this to be complete and flexible, each ICAO will
>>need the following information:
>>	- TimeZone specified as offset from GMT
>>	- uses Daylight Savings?
>>	- DST start date
>>	- DST end date
>>	- latitude & longitude
>>
>>The sunrise and sunset times can be calculated from the information
>>from NOAA http://www.srrb.noaa.gov/highlights/sunrise/calcdetails.html
> 
> 
> Thanks a lot for that link --- we've made an attempt to implement this
> kind of functionality before, but when I searched the net I only found
> some messy algorithms that I didn't have much luck with, see
> 
>   http://makeashorterlink.com/?T162215A2
> 
> The information on the above page looks much clearer, and it should be
> easy to port the JavaScript code to PHP --- those two languages are
> very much alike.
> 
> 
>>Lat and Long can be had for ICAOs from
>>http://weather.noaa.gov/tg/site.shtml directly ->
>>http://weather.noaa.gov/data/nsd_cccc.txt
> 
> 
> Thanks again --- I've found those datafiles before, but the
> information is still not part of PHP Weather.  Given the latitude and
> longitude it should be possible to calculate an approximate timezone.
> 
> And with that timezone we should be able to give a local time for the
> reports, but I guess it will still be off by +/- 1 hours depending on
> daylight saving time, so I don't know how good that idea is.
> 
> It's just that the real solution: attaching detailed information about
> the daylight saving time to each station, sounds like a difficult
> task.  But then again, there must be some tables with this information
> somewhere on every system, so perhaps it's doable?
> 
> 
> In any case --- I'm having my exams right now!  The first one will be
> on Wednesday, so I wont start playing with this stuff now, but if
> someone else feels like it, then by all mean do!  I can add people to
> the project as developers if they believe they need CVS access.
> 

Refreshing data

[aguevara <agu...@cf...>]

I started using phpweather on my site, the installation went very well;
I am not using a db at this point in time.
 
The reason I am writing is because my data is not being refreshed and I
would like to understand how to make sure it is displaying the latest
and greatest data. I have two(2) questions, 
 
(1)     I have isolated the phpweather into its own file (i.e. temp.php)
and am using an <include> tag. Will this refresh OK ? 
(2)     I am only displaying the icons/images on my main page, I also
have a link which takes you to another page (i.e. temp2.php) where I
allow the user to select the country, city, etc. which is the default
behavior. How can I make this second page always display the latest and
greatest and not cached ?
 
Please advise.
 
Have a great day, 
Angelo A. Guevara
an...@ca... <mailto:an...@ca...ev...@ed...> 
 

Re: Refreshin the data

[Phil P. <phi...@sw...>]

Given that you're not using any database, as far as I know it'll 
download the latest METAR every time.

HUGE impact on page generation time (on the order of a second or two, 
most likely).

Even using the database (MySQL), I wasn't happy with the performance 
(though most likely that was MySQL blocking issues with the millions of 
other users at my ISP), so I wrote a small scripthack to cache the HTML 
I produced.

I've since stumbled across the PEAR CacheLite module, and will (one day) 
re-code using that. (http://pear.php.net/package-info.php?pacid=99)

Check it out :-)

http://home.swiftdsl.com.au/~Phil.Pierotti/

If you read the page generation times at the bottom, it's not insanely 
fast. Then again my ISP doesn't use any PHP OpCode cache technology, so 
every page (half-a-dozen scripts each, mostly) is reloaded from the 
filesystem+parsed_for_includes+compiled *for every page-view*. Given 
that *and* the 4 MySQL queries on every page (in the traffic summary 
box) I'm not terribly unhappy so far.

Enjoy,
Phil P

aguevara wrote:
> I have downloaded php weather and have started using it, I am not using 
> the DB (I am not concerned about caching the information quite yet). My 
> question is, does the weather data get updated every time I load my web 
> page. If not, how do I make/force that to happen ?
> 
>  
> 
> Very complete app, I really like it !. I am using it to display my 
> city’s weather in Maturin, Venezuela on my main page while also allowing 
> visitors to go to another page to select and look at their country’s, 
> city’s weather.
> 
>  
> 
> Please advise.
> 
>  
> 
> Have a great day,
> *Angelo A. Guevara*
> 
> an...@ca... <mailto:an...@ca...ev...@ed...>
> 
>  
> 

Refreshin the data

[aguevara <agu...@cf...>]

I have downloaded php weather and have started using it, I am not using
the DB (I am not concerned about caching the information quite yet). My
question is, does the weather data get updated every time I load my web
page. If not, how do I make/force that to happen ? 
 
Very complete app, I really like it !. I am using it to display my
city's weather in Maturin, Venezuela on my main page while also allowing
visitors to go to another page to select and look at their country's,
city's weather.
 
Please advise.
 
Have a great day, 
Angelo A. Guevara
an...@ca... <mailto:an...@ca...ev...@ed...> 
 

Status of TAF

[<sr...@ma...>]

Hi

What is the status of TAF decoding ?
Thanks in advance.
Steen

Re: Using DB to cache METARs ?

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

> Martin Geisler wrote:
>> Ahh... MSIE 6... you're using Windows!  That made a bell ring! :-)
>> The problem is that your client translates the newlines into
>> carrige-return newline --- Unix uses a plain newline, but Windows
>> likes to use carrige-return newline.  And of course one cannot just go
>> around changing binary files like that...
>
> Don't ya just *hate* that?

Yeah, those binary formats are no fun at all... :-)

> Ya know, I have to say:
>
> I saw an option in WinCVS saying
> 	"Checkout text files with the Unix LF (0xa)"
> And thought to myself, "nah, can't be that , these are BINARY files".
>
> Glad we've managed to hunt that one down :-)

So am I!  It's one of those things that I've read about, and when I
did, I thought to myself: yes, I'll remember that, but guess what... I
still forgot :-)

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Re: Using DB to cache METARs ?

[Phil P. <phi...@sw...>]

Martin Geisler wrote:
> Ahh... MSIE 6... you're using Windows!  That made a bell ring! :-)
> 
> The problem is that your client translates the newlines into
> carrige-return newline --- Unix uses a plain newline, but Windows
> likes to use carrige-return newline.  And of course one cannot just go
> around changing binary files like that...

Don't ya just *hate* that?

Ya know, I have to say:

I saw an option in WinCVS saying
	"Checkout text files with the Unix LF (0xa)"
And thought to myself, "nah, can't be that , these are BINARY files".

Glad we've managed to hunt that one down :-)

Phil P

Re: Using DB to cache METARs ?

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

> Actually that was my first thought: bad URLs.
>
> So I checked the HTML source.
>   - All Good
> So I checked the path/subdirs on the web server.
>   - All Good

Okay, I just wanted to make that it wasn't some trivial mistake like
that...

> My preferred graphic viewer (irfanview 3.85) replies with
> 	{filename}: PNG Decode error !
> 	This is not a valid PNG File !
>
> Also neither FireBird 0.7+ nor MSIE 6 are able to display that local file.

Ahh... MSIE 6... you're using Windows!  That made a bell ring! :-)

The problem is that your client translates the newlines into
carrige-return newline --- Unix uses a plain newline, but Windows
likes to use carrige-return newline.  And of course one cannot just go
around changing binary files like that...

But it was really my mistake, I had forgot to use a -kb flag when I
commited the files.  That flag tells CVS that the file is binary, and
so it shouldn't mess with the newlines.

If anybody finds other binary files lacking this option, then let me
know.

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Re: phpweather: stations.csv parsing script

[Martin G. <gim...@gi...>]

Eric Vaughan <eva...@pu...> writes:

> -Gimpster and phpWeather-
>
> I downloaded your script for phpweather a few days ago and really
> saw potential for its use.  Especially with its convenient METAR
> data parsing.

Yes, that's really the core functionality of PHP Weather.

> I have been looking at the stations.csv file and have written a php
> script that completely parses out the data and creates a .sql file
> to be used to import all of the ICAO codes, countries, stations into
> a SQL database (MySQL, of course).  Let me know if you are
> interested in such a script.

Sure, you can upload it as a patch to SourceForge:

  http://sourceforge.net/tracker/?atid=377954&group_id=23245

but the functionality is already there in PHP Weather, it's just
burried in config/make_db.php and the insert_stations functions found
in the MySQL, PostgreSQL, etc. backends.

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

METAR cacheing / performance / code methods

[Phil P. <phi...@sw...>]

While I was trolling through the code, I often saw what looked like many 
and various chunks of code which essentially re-processed the RAW METAR 
data from scratch. (eg interpreting the timestamp).

[
Maybe I'm not being clear here... it looks to me like different bits of 
PHPWeather that need (for example) the timestamp re-interpret it from 
the RAW data themselves, rather than a chunk of code munging that into 
an internal datastructure (ie once) which all the code chunks read as 
they need.

Let me also just say that I've only done a little trolling around 
PHPWeather sources, and I'm barely able to 'crawl' when it comes to 
writing PHP. I'm not trying to criticize the code, merely wanting to 
better understand how it works (perhaps I might try my hand at 
contributing some tweaks/features in the not-too-distant future)
]

As this makes much and heavy use of regexes, I was wondering if this 
could have a material impact on performance.

So, I did some quick hacking around with my PHP scripts and came up with 
the following stats:

- full site page, including process (ensure this has been cached) METAR 
for TEXT as well as evaluate for ICONS)
	- 0.089 seconds or thereabouts
- same page, remove the code chunk/module for the METAR processing and 
output
	- 0.018 seconds

ugh! even when it's cached, processing the METAR and generating the 
html-ified output makes the page generation take 3 times longer than 
previously.

(of course, I'm using MySQL as the datastore for METARs, so this delay 
could be mostly due to MySQL query queueing and processing delays)

So I hacked around, and wrote some additions to my scripts to cache my 
formatted HTML output into a file. Now (as long as the cached results 
are within the timeout) showing the processed weather data takes negible 
additional time over normal page generation.

That's nice as a workaround. But obviously it only helps me and nobody 
else (well, not much).  Here's the code-fragment I used to 'cache the 
output HTML' if anyone else wants to use it.


/* LockFile must exist, cacheFile must be writeable by the webserver */

	if((time() - filectime($cacheFile)) < $metarCacheTimeout) {
		readfile($cacheFile);	
	} else {
		/* here we regenerate the formatted METAR results */

		$output = {munged html-ified output you want on your site};
	
		/* grab a lock on LOCKFILE, rewrite cachefile, release lock */
		$fp = fopen($cacheFile . '-LOCK', 'r');
		if( flock($fp, LOCK_EX + LOCK_NB) ) {
			$fp2 = fopen($cacheFile, 'w');
			if($fp2) {
				fwrite($fp2, $output);
				fflush($fp2);
				fclose($fp2);
			}
			flock ($fp, LOCK_UN);
		}
		echo $output;
	}

At some point I'll have a closer look and see if I can find where the 
bulk of the time is going, but for now I just thought I'd throw my 
findings out there and listen to any comments/feedback others might have.

Thanks,
Phil P

Re: Using DB to cache METARs ?

[Phil P. <phi...@sw...>]

Actually that was my first thought: bad URLs.

So I checked the HTML source.
  - All Good
So I checked the path/subdirs on the web server.
  - All Good

So I went to the subdir where CVS had dumped phpweather on my local hard 
drive, and tried to open an icon file.... Specifically icons/temphigh.png

My preferred graphic viewer (irfanview 3.85) replies with
	{filename}: PNG Decode error !
	This is not a valid PNG File !

Also neither FireBird 0.7+ nor MSIE 6 are able to display that local file.

Loading up the icons/ subdir from the previous release (2.1.2) works 
perfectly fine without changing any code.

Also: downloading the 2.2.0 release (ZIP file) also gives me complete 
working PNGs under icons/

Basically (and I've CVS'd them down several times now) the icons/* files 
I'm receiving from CVS (I'm using WinCVS 1.3.13.2) are arriving 
corrupted and thereby not valid PNG images. All of them. Every time.

I've tried with and without TCP Compression, and WinCVS is always 
dumping into a new empty directory, so there's no mystery file-collision 
weirdness going on either.

So, either
	(a) the PNG icons/ in CVS are corrupted
	(b) WinCVS is corrupting them icons/ as they download

Anyone else out there want to CVS down the latest PHPWeather (preferably 
using Windows and not WinCVS) and let us know if the icons arrive safely?

Thanks,
Phil P



Martin Geisler wrote:
> Phil Pierotti <phi...@sw...> writes:
> 
> 
>>Just for the sake of completeness, I downloaded the icons again
>>through CVS, still corrupted.
> 
> 
> The icons themselves look fine to me --- normal PNG images.  Are you
> sure it's not just the generated URL that's wrong, and therefore the
> icons look broken in your browser?
> 
> Use icons_path to indicate the path that will be inserted infront of
> the image names, the standard setting of
> 
>   $this->properties['icons_path']    = 'icons/';
> 
> only works for index.php in the root PHP Weather folder because the
> icons are put in the icons subdirectory.
> 

Re: Using DB to cache METARs ?

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

> Just for the sake of completeness, I downloaded the icons again
> through CVS, still corrupted.

The icons themselves look fine to me --- normal PNG images.  Are you
sure it's not just the generated URL that's wrong, and therefore the
icons look broken in your browser?

Use icons_path to indicate the path that will be inserted infront of
the image names, the standard setting of

  $this->properties['icons_path']    = 'icons/';

only works for index.php in the root PHP Weather folder because the
icons are put in the icons subdirectory.

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Re: pw_images, UTC, TimeZones, LocalTime

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

> Realistically, for this to be complete and flexible, each ICAO will
> need the following information:
> 	- TimeZone specified as offset from GMT
> 	- uses Daylight Savings?
> 	- DST start date
> 	- DST end date
> 	- latitude & longitude
>
> The sunrise and sunset times can be calculated from the information
> from NOAA http://www.srrb.noaa.gov/highlights/sunrise/calcdetails.html

Thanks a lot for that link --- we've made an attempt to implement this
kind of functionality before, but when I searched the net I only found
some messy algorithms that I didn't have much luck with, see

  http://makeashorterlink.com/?T162215A2

The information on the above page looks much clearer, and it should be
easy to port the JavaScript code to PHP --- those two languages are
very much alike.

> Lat and Long can be had for ICAOs from
> http://weather.noaa.gov/tg/site.shtml directly ->
> http://weather.noaa.gov/data/nsd_cccc.txt

Thanks again --- I've found those datafiles before, but the
information is still not part of PHP Weather.  Given the latitude and
longitude it should be possible to calculate an approximate timezone.

And with that timezone we should be able to give a local time for the
reports, but I guess it will still be off by +/- 1 hours depending on
daylight saving time, so I don't know how good that idea is.

It's just that the real solution: attaching detailed information about
the daylight saving time to each station, sounds like a difficult
task.  But then again, there must be some tables with this information
somewhere on every system, so perhaps it's doable?


In any case --- I'm having my exams right now!  The first one will be
on Wednesday, so I wont start playing with this stuff now, but if
someone else feels like it, then by all mean do!  I can add people to
the project as developers if they believe they need CVS access.

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

pw_images, UTC, TimeZones, LocalTime

[Phil P. <phi...@sw...>]

To make pw_images display the correct (day or night) icon, for the 
moment I've hacked up get_sky_image() as follows

/* hacked for Sydney, Summer Time, Phil Pierotti, 04-01-03 */

  $metarDST_start = '10310200'; // MMddhhmm
  $metarDST_end = '03280300';
  $metarGMTOffset = 10 ;
  $metarUsesDST = 1;
  $metarSunRise = 6;
  $metarSunSet = 20;

  if($metarUsesDST) {
  	$todayCheck = date('mdHi');
  	if(($todayCheck >= $metarDST_start)  && ($todayCheck < $metarDST_end)) {
  		$metarDSTOffset = 1;
  	}
  } else {
  	$metarDSTOffset = 0;
  }

  $metarTime = ($regs[2] + $metarGMTOffset + $metarDSTOffset) % 24;

  if (($metarTime > $metarSunSet) || ($metarTime < $metarSunRise)) {

/* ------- */         	


Realistically, for this to be complete and flexible, each ICAO will need 
the following information:
	- TimeZone specified as offset from GMT
	- uses Daylight Savings?
	- DST start date
	- DST end date
	- latitude & longitude

The sunrise and sunset times can be calculated from the information from 
NOAA http://www.srrb.noaa.gov/highlights/sunrise/calcdetails.html

Lat and Long can be had for ICAOs from http://weather.noaa.gov/tg/site.shtml
directly -> http://weather.noaa.gov/data/nsd_cccc.txt


Enjoy,
Phil P

Re: Using DB to cache METARs ?

[Phil P. <phi...@sw...>]

Just for the sake of completeness, I downloaded the icons again through 
CVS, still corrupted.

Enjoy,
Phil P

Phil Pierotti wrote:
> Downloaded the updates via CVS:
> 
>     MySQL access is now working :-)
>     MySQL access is now caching :-) :-)
> 
> For some reason, I'm no longer getting the ICONS displayed, though this 
> could easily be some random stupidity in how I've scripted it into my 
> site. (sigh)
> 
> Thanks Very Muchly!
> 
> Phil P
> 
> Phil Pierotti wrote:
> 
>> Just to be clear, by this you mean you've fixed the timestamping/cache 
>> problem for MySQL and the NULL database?
>>
>> In that case, did you see my other post about issues getting MySQL 
>> access to work?
>>
>> [ after much actually reading the configuration page, I realized I 
>> needed to physically (ie manually) copy the defaults file to my 
>> phpweather (eg in order to enable debug) -sigh- obviously not enough 
>> coffee yesterday]
>>
>> DEBUG=everything shows me:
>>
>> Fatal error: SQL-statement failed: CREATE TABLE pw_metars( icao 
>> char(4) NOT NULL, time timestamp(14), metar varchar(255) NOT NULL, 
>> timestamp timestamp(14), PRIMARY KEY (icao)
>> MySQL said You have an error in your SQL syntax near '' at line 6 (1064).
>>
>> When I configure for MySQL database access, and then use the Create 
>> Tables page.
>>
>> Thanks,
>> Phil P
>>
>> Martin Geisler wrote:
>>
>>> Phil Pierotti <phi...@sw...> writes:
>>>
>>>
>>>> Hopefully someone out there can gimme a hand to work this one out.
>>>
>>>
>>>
>>>
>>> I've just resend my earlier responce to the list, I really don't know
>>> why, but it never made it through. :-/
>>>
>>> And to follow up on that, I've updated CVS so that things should work
>>> again for the MySQL and null databases.
>>>
>>> The rest is a mess --- we have a half-implemented archiving feature
>>> that someone needs to rewrite.  Any chance you could do that, Etienne?
>>>
>>
>>
>>
>> -------------------------------------------------------
>> This SF.net email is sponsored by: IBM Linux Tutorials.
>> Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
>> Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
>> Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
>> _______________________________________________
>> PHPWeather-devel mailing list
>> PHP...@li...
>> https://lists.sourceforge.net/lists/listinfo/phpweather-devel
>>
>>
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: IBM Linux Tutorials.
> Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
> Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
> Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
> _______________________________________________
> PHPWeather-devel mailing list
> PHP...@li...
> https://lists.sourceforge.net/lists/listinfo/phpweather-devel
> 
> 

Re: Using DB to cache METARs ?

[Phil P. <phi...@sw...>]

For some reason at least some (most likely all) of the ICONS were 
currupted when I downloaded them via CVS, copying back the set I 
received from the 2.2.0 release works fine.

Now all I need to do is FSCK with the {select an icon} code so that it 
correctly shows DAYTIME icons for my location and I'm set.

Thanks,
Phil P

Phil Pierotti wrote:

> Downloaded the updates via CVS:
> 
>     MySQL access is now working :-)
>     MySQL access is now caching :-) :-)
> 
> For some reason, I'm no longer getting the ICONS displayed, though this 
> could easily be some random stupidity in how I've scripted it into my 
> site. (sigh)
> 
> Thanks Very Muchly!
> 
> Phil P
> 
> Phil Pierotti wrote:
> 
>> Just to be clear, by this you mean you've fixed the timestamping/cache 
>> problem for MySQL and the NULL database?
>>
>> In that case, did you see my other post about issues getting MySQL 
>> access to work?
>>
>> [ after much actually reading the configuration page, I realized I 
>> needed to physically (ie manually) copy the defaults file to my 
>> phpweather (eg in order to enable debug) -sigh- obviously not enough 
>> coffee yesterday]
>>
>> DEBUG=everything shows me:
>>
>> Fatal error: SQL-statement failed: CREATE TABLE pw_metars( icao 
>> char(4) NOT NULL, time timestamp(14), metar varchar(255) NOT NULL, 
>> timestamp timestamp(14), PRIMARY KEY (icao)
>> MySQL said You have an error in your SQL syntax near '' at line 6 (1064).
>>
>> When I configure for MySQL database access, and then use the Create 
>> Tables page.
>>
>> Thanks,
>> Phil P
>>
>> Martin Geisler wrote:
>>
>>> Phil Pierotti <phi...@sw...> writes:
>>>
>>>
>>>> Hopefully someone out there can gimme a hand to work this one out.
>>>
>>>
>>>
>>>
>>> I've just resend my earlier responce to the list, I really don't know
>>> why, but it never made it through. :-/
>>>
>>> And to follow up on that, I've updated CVS so that things should work
>>> again for the MySQL and null databases.
>>>
>>> The rest is a mess --- we have a half-implemented archiving feature
>>> that someone needs to rewrite.  Any chance you could do that, Etienne?
>>>
>>
>>
>>
>> -------------------------------------------------------
>> This SF.net email is sponsored by: IBM Linux Tutorials.
>> Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
>> Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
>> Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
>> _______________________________________________
>> PHPWeather-devel mailing list
>> PHP...@li...
>> https://lists.sourceforge.net/lists/listinfo/phpweather-devel
>>
>>
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: IBM Linux Tutorials.
> Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
> Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
> Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
> _______________________________________________
> PHPWeather-devel mailing list
> PHP...@li...
> https://lists.sourceforge.net/lists/listinfo/phpweather-devel
> 
> 

Re: Using DB to cache METARs ?

[Phil P. <phi...@sw...>]

Downloaded the updates via CVS:

	MySQL access is now working :-)
	MySQL access is now caching :-) :-)

For some reason, I'm no longer getting the ICONS displayed, though this 
could easily be some random stupidity in how I've scripted it into my 
site. (sigh)

Thanks Very Muchly!

Phil P

Phil Pierotti wrote:

> Just to be clear, by this you mean you've fixed the timestamping/cache 
> problem for MySQL and the NULL database?
> 
> In that case, did you see my other post about issues getting MySQL 
> access to work?
> 
> [ after much actually reading the configuration page, I realized I 
> needed to physically (ie manually) copy the defaults file to my 
> phpweather (eg in order to enable debug) -sigh- obviously not enough 
> coffee yesterday]
> 
> DEBUG=everything shows me:
> 
> Fatal error: SQL-statement failed: CREATE TABLE pw_metars( icao char(4) 
> NOT NULL, time timestamp(14), metar varchar(255) NOT NULL, timestamp 
> timestamp(14), PRIMARY KEY (icao)
> MySQL said You have an error in your SQL syntax near '' at line 6 (1064).
> 
> When I configure for MySQL database access, and then use the Create 
> Tables page.
> 
> Thanks,
> Phil P
> 
> Martin Geisler wrote:
> 
>> Phil Pierotti <phi...@sw...> writes:
>>
>>
>>> Hopefully someone out there can gimme a hand to work this one out.
>>
>>
>>
>> I've just resend my earlier responce to the list, I really don't know
>> why, but it never made it through. :-/
>>
>> And to follow up on that, I've updated CVS so that things should work
>> again for the MySQL and null databases.
>>
>> The rest is a mess --- we have a half-implemented archiving feature
>> that someone needs to rewrite.  Any chance you could do that, Etienne?
>>
> 
> 
> 
> -------------------------------------------------------
> This SF.net email is sponsored by: IBM Linux Tutorials.
> Become an expert in LINUX or just sharpen your skills.  Sign up for IBM's
> Free Linux Tutorials.  Learn everything from the bash shell to sys admin.
> Click now! http://ads.osdn.com/?ad_id=1278&alloc_id=3371&op=click
> _______________________________________________
> PHPWeather-devel mailing list
> PHP...@li...
> https://lists.sourceforge.net/lists/listinfo/phpweather-devel
> 
> 

Re: Using DB to cache METARs ?

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

> Just to be clear, by this you mean you've fixed the timestamping/cache
> problem for MySQL and the NULL database?

Yes, I basically disabeled the code that played with the timestamps.

The problem is that Etienne wanted to add support for caching METARs
and also TAF reports --- which is a fine idea!  But a METAR report has
a timefield which only contains the current day of month together with
the time of day.

The present code always uses the current month as a basis, but with
old archived METARs this wont work anymore.  So the code was changed
to try and work around this lack of a full timestamp, but
unfortunately the result was a bit messy with some timestamps being of
the form YYYYMMDDhhmmss and other timestamps being a standard Unix
timestamp counting the number of seconds since Jan 1st 1970.


The code now works as it used to --- that is, the archiving of METAR
reports is probably broken, I don't know for I've never used it.  All
I know is that it has to be reimplemented some other way.

Could anybody do a CVS update and confirm that things work at least
using MySQL and the null db backend?  If somebody uses PostgreSQL then
it would be cool to get a report about that too!

> In that case, did you see my other post about issues getting MySQL
> access to work?
>
> [ after much actually reading the configuration page, I realized I
> needed to physically (ie manually) copy the defaults file to my
> phpweather (eg in order to enable debug) -sigh- obviously not enough
> coffee yesterday]

:-) But you're right, it might not be exactly what people expect from
the configuration builder --- but then again, making a configuration
generator that writes to the files would be dangerous, and also
require people to setup proper permissions for the files.

But they still have to do this if they want to use the DBA db backend,
so perhaps I'm wrong...

> DEBUG=everything shows me:
>
> Fatal error: SQL-statement failed: CREATE TABLE pw_metars( icao
> char(4) NOT NULL, time timestamp(14), metar varchar(255) NOT NULL,
> timestamp timestamp(14), PRIMARY KEY (icao) MySQL said You have an
> error in your SQL syntax near '' at line 6 (1064).

Yes, I've fixed those errors too --- the SQL statements lacks the
final right parenthesis...

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Re: Using DB to cache METARs ?

[Phil P. <phi...@sw...>]

Just to be clear, by this you mean you've fixed the timestamping/cache 
problem for MySQL and the NULL database?

In that case, did you see my other post about issues getting MySQL 
access to work?

[ after much actually reading the configuration page, I realized I 
needed to physically (ie manually) copy the defaults file to my 
phpweather (eg in order to enable debug) -sigh- obviously not enough 
coffee yesterday]

DEBUG=everything shows me:

Fatal error: SQL-statement failed: CREATE TABLE pw_metars( icao char(4) 
NOT NULL, time timestamp(14), metar varchar(255) NOT NULL, timestamp 
timestamp(14), PRIMARY KEY (icao)
MySQL said You have an error in your SQL syntax near '' at line 6 (1064).

When I configure for MySQL database access, and then use the Create 
Tables page.

Thanks,
Phil P

Martin Geisler wrote:
> Phil Pierotti <phi...@sw...> writes:
> 
> 
>>Hopefully someone out there can gimme a hand to work this one out.
> 
> 
> I've just resend my earlier responce to the list, I really don't know
> why, but it never made it through. :-/
> 
> And to follow up on that, I've updated CVS so that things should work
> again for the MySQL and null databases.
> 
> The rest is a mess --- we have a half-implemented archiving feature
> that someone needs to rewrite.  Any chance you could do that, Etienne?
> 

Re: Using DB to cache METARs ?

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

Hi Phil.

I'm sorry about all your trobles with this release... :-(  But you're
right, something definitely wrong, at least with the timestamps.

Some further investitagion shows that all timestamps are set to 2^31-1
which is 03:14:07 on Jan 19th 2038, the time when the 32 bit
timestamps used in Unix wraps around, bringing us all chaos.  It just
seems that the chaos have hit us a little earlier... :-)

> Supposedly using a DB of some kind to cache METARs makes things
> faster.  Is there any to DEBUG this?
>
> Because I'm finding it's simply not any faster at all.

Hmm... I think this would have something to do with the problem with
the timestamps --- the METARs are put in the database using the wrong
timestamp, and then later when they are retrieved they are deemed to
be old.  PHP Weather then goes to the NWS and refetches the METAR, and
in doing so you get poor performance.

> (and FYI I don't have shell access to the machine, so any debug will
> have to be via PHP scripting or something)

That's the common situation, I don't have shell access to gimpster.com
either...

> Page generation takes anywhere from 1.4 to 1.9 seconds.
> When I use NO DB, *and* when I use Sleepycat DB3.
>
> The METAR was generated less than 1 hour ago, so after I've refreshed
> the page a couple of times, in theory the data should be in the
> database, right?

Yes, exactly.  But when the data is inserted with the wrong timestamp,
things doesn't work as adverticed :-)

I'll see about finding out what wrong when I get back tonight or
perhaps tomorrow --- now I really must to and study for my exams...

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Does the list still work?

[Martin G. <gim...@gi...>]

Hello everybody...

I seem to be having trouble posting to the list?!  Let's see if this
one get's through...

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Re: Does the list still work?

[Martin G. <gim...@gi...>]

Martin Geisler <gim...@gi...> writes:

> Martin Geisler <gim...@gi...> writes:
>
>> Hello everybody...
>>
>> I seem to be having trouble posting to the list?!  Let's see if this
>> one get's through...
>
> Sorry about the spam, the list obviously works.  But SourceForge has
> made some kind of new check that meant that I couldn't post via my
> normal SMTP server, mail.bolignet-aarhus.dk, so I'm now posting
> through my own :-/

And for those interested... the error given in my Exim log files is:

  2004-01-04 18:09:18 1AdBjp-0000Tz-OV SMTP error from remote mailer
  after RCPT TO:<php...@li...>: host
  mail.sourceforge.net [66.35.250.206]:

    451-could not connect to gimpster.sg.bnaa.dk [62.199.203.211]:
    Connection timed out
    451-Could not complete sender verify callout for
    451-<gim...@gi...>.
    451-The mail server(s) for the domain may be temporarily
    unreachable, or
    451-they may be permanently unreachable from this server. In the
    latter case,
    451-you need to change the address or create an MX record for its
    domain
    451-if it is supposed to be generally accessible from the
    Internet.
    451 Talk to your mail administrator for details.

  2004-01-04 18:11:06 1AdBjp-0000Tz-OV ==
  php...@li... R=dnslookup T=remote_smtp
  defer (0): SMTP error from remote mailer after RCPT
  TO:<php...@li...>: host
  externalmx.valinux.com [198.186.202.147]: 451 Could not complete
  sender verify callout

Here the log refers to gimpster.sg.bnaa.dk instead of
mail.bolignet-aarhus.dk for this was before I opened my firewall.

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Re: Using DB to cache METARs ?

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

> Hopefully someone out there can gimme a hand to work this one out.

I've just resend my earlier responce to the list, I really don't know
why, but it never made it through. :-/

And to follow up on that, I've updated CVS so that things should work
again for the MySQL and null databases.

The rest is a mess --- we have a half-implemented archiving feature
that someone needs to rewrite.  Any chance you could do that, Etienne?

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

[Kari Salovaara] Bug report on version 2.2.0

[Martin G. <gim...@gi...>]

Forwarded message follows:


Hi Martin,

could You forward this bug report into the dev-list due I'm not able
to send there anything. Reason; Sourceforge checks that MTA has
postmaster email address and my ISP provider (biggest in Finland
Sonera) doesn't have it, so all my emails to any Sourceforge address
are bouncing back !!!

Luck for Your exams,
Kari




Bug report on version 2.2.0

Report (First problem):
Parsing of YYGGggZ group and translating it to some language causes=20
always following translation (tested by 'distro' index.php):
The report was made a moment ago, at 03:14 UTC.

Same parsing done with test.php (same metar) result is:
The report was made 298080 hours and 27 minutes ago, at 00:00 UTC.

Data used: EFHF 022250Z AUTO 32002KT CAVOK M16/M17 Q1029 and local time=20
was 02:27 (about)



(Second) problem is also that the time is not UTC but local in Metar.


(Third) problem, some station have groups in wrong order, eg.
Finland, Sodankyl=E4 gives following Metar :
121250Z EFSO 36004KT 9999 FEW012 SCT018 BKN030 04/02 Q1038
and output is (index.php):
This is a report for Sodankyla, Finland. The report was made a moment=20
ago, at 03:14 UTC.

output of test.php using same data:
This is a report for 1212. The report was made 298081 hours and 33=20
minutes ago, at 00:00 UTC.


Regards,
Kari Salovaara
--=20
Project Manager
Hanko
Finland

--=20
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.

Re: Using DB to cache METARs ?

[Martin G. <gim...@gi...>]

Phil Pierotti <phi...@sw...> writes:

Hi Phil.

I'm sorry about all your trobles with this release... :-(  But you're
right, something definitely wrong, at least with the timestamps.

Some further investitagion shows that all timestamps are set to 2^31-1
which is 03:14:07 on Jan 19th 2038, the time when the 32 bit
timestamps used in Unix wraps around, bringing us all chaos.  It just
seems that the chaos have hit us a little earlier... :-)

> Supposedly using a DB of some kind to cache METARs makes things
> faster.  Is there any to DEBUG this?
>
> Because I'm finding it's simply not any faster at all.

Hmm... I think this would have something to do with the problem with
the timestamps --- the METARs are put in the database using the wrong
timestamp, and then later when they are retrieved they are deemed to
be old.  PHP Weather then goes to the NWS and refetches the METAR, and
in doing so you get poor performance.

> (and FYI I don't have shell access to the machine, so any debug will
> have to be via PHP scripting or something)

That's the common situation, I don't have shell access to gimpster.com
either...

> Page generation takes anywhere from 1.4 to 1.9 seconds.
> When I use NO DB, *and* when I use Sleepycat DB3.
>
> The METAR was generated less than 1 hour ago, so after I've refreshed
> the page a couple of times, in theory the data should be in the
> database, right?

Yes, exactly.  But when the data is inserted with the wrong timestamp,
things doesn't work as adverticed :-)

I'll see about finding out what wrong when I get back tonight or
perhaps tomorrow --- now I really must to and study for my exams...

-- 
Martin Geisler                       My GnuPG Key: 0xF7F6B57B

See http://gimpster.com/ and http://phpweather.net/ for:
PHP Weather:   Shows the current weather on your webpage and
PHP Shell:     A telnet-connection (almost :-) in a PHP page.