Background: Had a working phpgedview installation with a one-man-band hosting
site, really just defraying his own costs. He gave up hosting so I moved to another host but didn't use it much. Upgraded to 4.3.0 Copied GEDCOM to FindMyPast and made a lot of entries. Ran up phpgedview on a Mac Mini at home installed at
<documentroot>/phpgedview
</documentroot> Tried to load exported FindMyPast GEDCOM into local phpgedview and
found FindMyPast had made a real pigs earole on my GEDCOM. Did manual Unix edits on the GEDCOM to remove loads of junk. Renamed <documentroot>/phpgedview to <documentroot>/phpgedview430,
created new <documentroot>/phpgedview431, installed 4.3.1 there and
copied good index files across leaving those from the previous four
GEDCOM versions behind. Did NOT start with an empty database. Soft linked <documentroot>/phpgedview431 to <documentroot>/phpgedview
for ease of use.
* Installed LetsEncrypt certificate handling so I could use https and my
browser wouldn't argue with me.</documentroot></documentroot></documentroot></documentroot></documentroot>
When using the https://host.mydomain/phpgedview, sometimes when moving
to a different individual I get 'file not found' errors. They are not
there because it has 'jumped' to an old GEDCOM. The file exists, but in
the ...430 directory. This is most likely to happen when using the
'Favorites' dropdown under the Search box. Re-selecting the correct
GEDCOM fixes this, but I don't see where the connection would come from
intermittently.
Twice now I have had the "You are not welcome here" state triggered.
This goes away after a short time. After this I get a file not found
message even though the heading indicates the correct GEDCOM.
Re-selecting the correct GEDCOM fixes this.
Ideas please ...
David
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Did you import the GEDCOM into the database, letting PhpGedView erase existing data in the database? This step is ESSENTIAL, since it's the only way you can be sure that the database indexes are built properly. When you upgrade or install a new version of PhpGedView, the database tables are upgraded as required, but the database indexes are NOT built automatically.
To be sure that the database and the GEDCOM match, launch PhpGedView and export the database to a GEDCOM, and then immediately import that just-exported GEDCOM. The Export process does not rely on database indexes, and uses ONLY the GEDCOM data in the database. That's why it works.
If you have media files, don't let PhpGedView keep existing media links. The exported GEDCOM contains media link information, so if you let PhpGedView keep existing media links, you'll end up with duplicate links. This option is intended to be used with GEDCOMs that don't contain media links. Older versions of Family Tree Maker produce such GEDCOMs.
You can figure out why you're getting the "not welcome" message. PhpGedView records a log file, and it will contain a record of the access denials and the reason for them. These access denials last for an hour.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Did you import the GEDCOM into the database, letting PhpGedView erase
existing data in the database? This step is ESSENTIAL, since it's the
only way you can be sure that the database indexes are built properly.
When you upgrade or install a new version of PhpGedView, the database
tables are upgraded as required, but the database indexes are NOT
built automatically.
To be sure that the database and the GEDCOM match, launch PhpGedView
and export the database to a GEDCOM, and then immediately import that
just-exported GEDCOM. The Export process does not rely on database
indexes, and uses ONLY the GEDCOM data in the database. That's why it
works.
If you have media files, don't let PhpGedView keep existing media
links. The exported GEDCOM contains media link information, so if you
let PhpGedView keep existing media links, you'll end up with duplicate
links. This option is intended to be used with GEDCOMs that don't
contain media links. Older versions of Family Tree Maker produce such
GEDCOMs.
You can figure out why you're getting the "not welcome" message.
PhpGedView records a log file, and it will contain a record of the
access denials and the reason for them. These access denials last for
an hour.
I didn’t tell it to erase the existing database because I have
multiple GEDCOMs there. I thought one instance was supposed to support
that.
I have now created a new sub-directory to document_root, installed 4.3.1
in there; created a new empty database and loaded an export of the other
set-up into there. The new database uses a different username and
password so they can’t get confused. I shouldn’t get the
‘jumping’ now.
Because of the FindMyPast episode I need to maintain contact with my
pre-FMP GEDCOM to check when I find things like siblings with the same
name where the original source is a relative who knew. Did the first die
and the second re-use the name, or did FMP fabricate one of them. I can
now get at pre-FMP and current at different locations on the server and
find out.
I’m still getting the “You are not welcome here” message in the
same situation as before. I’m trying to add a new Source. I enter
“R1” as the Repository because I know that’s what it is. If I hit
‘Create a new Source’ I get “You are not welcome here”. The URL
of that message is the new, correct one. The log has:
Before that in the log are several lines of the form:
15.01.2020 16:05:42 - 212.159.19.184 - DavidLedger - ERROR 8: Undefined
index: print_gedcom_favorites; 0 Error occurred on line 295 of file
index_edit.php
15.01.2020 16:05:42 - <my ip=""> - DavidLedger - ERROR 8: Undefined index:
print_upcoming_events; 0 Error occurred on line 295 of file
index_edit.php</my>
With different index values like those two.
The client and server both have the same IP as I’ve set the site URL
to the domain my ISP gives me on a fixed address. This allows me
external access, but all my browsing then goes to my ISP and back unless
my router is clever.
I’ve dropped all Media and media links. I plan on using the Media
Firewall and share them.
What does it detect as the “SQL injection”?
David
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
My choice would be to stop using Autocomplete. I find that, on my site, it really slows things down and is more of a nuisance than a help.
About erase database contents: The "erase" applies only to the GEDCOM you're trying to import. The records for other GEDCOMs are not affected. However, if you're replacing PhpGedView with a newer version, the database tables will all be updated to the schema required by the new version when you first launch that new version. This means that you MUST export and re-import ALL of your GEDCOMs that share the same database tables.
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
It doesn't like the word "and" in the query string: "leicestershire
wills and probate records".
Here's the code, lines 279 to 283 of /includes/session_spider.php:
// check for SQL injection
if (preg_match('~\b(and|sleep|unhex|join|select|insert|set|declare|drop|md5|benchmark)\b~is', $requestURI)) {
$quitReason = 'SQL injection detected';
break;
}
You have two choices: (a) don't use Autocomplete -- a GEDCOM
configuration option (b) change the code at line 280 of
/includes/session_spider.php to :
if (preg_match('~\b(sleep|unhex|join|select|insert|set|declare|drop|md5|benchmark)\b~is', $requestURI)) {
My choice would be to stop using Autocomplete. I find that, on my site,
it really slows things down and is more of a nuisance than a help.
Or (c) don't use "and" in Source (and presumably other) entries :-)
Which may be ok. There are two entries in the Create Source pane. Title
and Abbreviation. If it's only the Title that runs this check I could
use '&' there and 'and' in the abbreviation.
Presumably this applies to all Autocomplete fields. I'm surprised I
haven't come across this elsewhere. I do use Autocomplete (if that's
what I think it is) because it helps give consistency in place names.
I there any way to cancel the hour delay? I would imaging offending IPs
are held in the database with a timestamp. An SQL statement to remove my
entry would be handy. Even better would be a filter to stop it's own IP
from being caught.
Instructions to export and re-import all Gedcoms on upgrade is noted.
Thanks so much for your responses and continued work on this.
David
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
PhpGedView can't determine whether the requesting IP address is your own. With my ISP, the IP address of my computer can change from one session to the next, so hard-coding of IP addresses that should not be banned is not a viable option. Obviously, if I take my laptop to another location, I'll have a different IP address too.
You can use phpmyadmin or some similar tool to inspect the ip_address table. The last row in this table would be the most recent timed ban. Just delete that row if it points to your own IP address.
Lines 159 to 166 of session_spider.php is where the expired timed bans are removed:
$currentTime=date('Y.m.d@H:i');$sql="SELECT ip_address FROM {$TBLPREFIX}ip_address WHERE category='timedban' AND comment < '{$currentTime}'";$expiredBan=PGV_DB::prepare($sql)->fetchOneColumn();foreach($expiredBanas$address){PGV_DB::prepare("DELETE FROM {$TBLPREFIX}ip_address WHERE category='timedban' AND ip_address=?")->execute(array($address));}
If you would like to refer to this comment somewhere else in this project, copy and paste the following link:
Background:
Had a working phpgedview installation with a one-man-band hosting
site, really just defraying his own costs.
He gave up hosting so I moved to another host but didn't use it much.
Upgraded to 4.3.0
Copied GEDCOM to FindMyPast and made a lot of entries.
Ran up phpgedview on a Mac Mini at home installed at
<documentroot>/phpgedview
</documentroot> Tried to load exported FindMyPast GEDCOM into local phpgedview and
found FindMyPast had made a real pigs earole on my GEDCOM.
Did manual Unix edits on the GEDCOM to remove loads of junk.
Renamed <documentroot>/phpgedview to <documentroot>/phpgedview430,
created new <documentroot>/phpgedview431, installed 4.3.1 there and
copied good index files across leaving those from the previous four
GEDCOM versions behind.
Did NOT start with an empty database.
Soft linked <documentroot>/phpgedview431 to <documentroot>/phpgedview
for ease of use.
* Installed LetsEncrypt certificate handling so I could use https and my
browser wouldn't argue with me.</documentroot></documentroot></documentroot></documentroot></documentroot>
When using the https://host.mydomain/phpgedview, sometimes when moving
to a different individual I get 'file not found' errors. They are not
there because it has 'jumped' to an old GEDCOM. The file exists, but in
the ...430 directory. This is most likely to happen when using the
'Favorites' dropdown under the Search box. Re-selecting the correct
GEDCOM fixes this, but I don't see where the connection would come from
intermittently.
Twice now I have had the "You are not welcome here" state triggered.
This goes away after a short time. After this I get a file not found
message even though the heading indicates the correct GEDCOM.
Re-selecting the correct GEDCOM fixes this.
Ideas please ...
David
Did you import the GEDCOM into the database, letting PhpGedView erase existing data in the database? This step is ESSENTIAL, since it's the only way you can be sure that the database indexes are built properly. When you upgrade or install a new version of PhpGedView, the database tables are upgraded as required, but the database indexes are NOT built automatically.
To be sure that the database and the GEDCOM match, launch PhpGedView and export the database to a GEDCOM, and then immediately import that just-exported GEDCOM. The Export process does not rely on database indexes, and uses ONLY the GEDCOM data in the database. That's why it works.
If you have media files, don't let PhpGedView keep existing media links. The exported GEDCOM contains media link information, so if you let PhpGedView keep existing media links, you'll end up with duplicate links. This option is intended to be used with GEDCOMs that don't contain media links. Older versions of Family Tree Maker produce such GEDCOMs.
You can figure out why you're getting the "not welcome" message. PhpGedView records a log file, and it will contain a record of the access denials and the reason for them. These access denials last for an hour.
On 8 Jan 2020, at 16:53, Gerry Kroll wrote:
I didn’t tell it to erase the existing database because I have
multiple GEDCOMs there. I thought one instance was supposed to support
that.
I have now created a new sub-directory to document_root, installed 4.3.1
in there; created a new empty database and loaded an export of the other
set-up into there. The new database uses a different username and
password so they can’t get confused. I shouldn’t get the
‘jumping’ now.
Because of the FindMyPast episode I need to maintain contact with my
pre-FMP GEDCOM to check when I find things like siblings with the same
name where the original source is a relative who knew. Did the first die
and the second re-use the name, or did FMP fabricate one of them. I can
now get at pre-FMP and current at different locations on the server and
find out.
I’m still getting the “You are not welcome here” message in the
same situation as before. I’m trying to add a new Source. I enter
“R1” as the Repository because I know that’s what it is. If I hit
‘Create a new Source’ I get “You are not welcome here”. The URL
of that message is the new, correct one. The log has:
15.01.2020 16:23:07 - <my ip=""> - Anonymous - MSG>SQL injection detected;
script terminated; IP address blocked for 1 hour.
15.01.2020 16:23:07 - <my ip=""> - Anonymous - UA>Mozilla/5.0 (X11; Ubuntu;
Linux x86_64; rv:72.0) Gecko/20100101 Firefox/72.0<
15.01.2020 16:23:07 - <my ip=""> - Anonymous -
URI>/pgv431/autocomplete.php?q=leicestershire wills and probate records,
1500-1939&limit=100×tamp=1579105387716&field=SOUR_TITL<
15.01.2020 16:23:28 - <my ip=""> - Anonymous - session_spider.php blocked
IP Address: <my ip=""> by regex: <my ip=""> (2020.01.15@17:23)
15.01.2020 16:23:29 - 2<my ip=""> - Anonymous - session_spider.php blocked
IP Address: <my ip=""> by regex: <my ip=""> (2020.01.15@17:23)
15.01.2020 16:23:44 - <my ip=""> - Anonymous - session_spider.php blocked
IP Address: 2<my ip=""> by regex: <my ip=""> (2020.01.15@17:23)</my></my></my></my></my></my></my></my></my></my></my></my>
Before that in the log are several lines of the form:
15.01.2020 16:05:42 - 212.159.19.184 - DavidLedger - ERROR 8: Undefined
index: print_gedcom_favorites; 0 Error occurred on line 295 of file
index_edit.php
15.01.2020 16:05:42 - <my ip=""> - DavidLedger - ERROR 8: Undefined index:
print_upcoming_events; 0 Error occurred on line 295 of file
index_edit.php</my>
With different index values like those two.
The client and server both have the same IP as I’ve set the site URL
to the domain my ISP gives me on a fixed address. This allows me
external access, but all my browsing then goes to my ISP and back unless
my router is clever.
I’ve dropped all Media and media links. I plan on using the Media
Firewall and share them.
What does it detect as the “SQL injection”?
David
It doesn't like the word "and" in the query string: "leicestershire wills and probate records".
Here's the code, lines 279 to 283 of /includes/session_spider.php:
You have two choices: (a) don't use Autocomplete -- a GEDCOM configuration option (b) change the code at line 280 of /includes/session_spider.php to :
My choice would be to stop using Autocomplete. I find that, on my site, it really slows things down and is more of a nuisance than a help.
About erase database contents: The "erase" applies only to the GEDCOM you're trying to import. The records for other GEDCOMs are not affected. However, if you're replacing PhpGedView with a newer version, the database tables will all be updated to the schema required by the new version when you first launch that new version. This means that you MUST export and re-import ALL of your GEDCOMs that share the same database tables.
On 2020-01-15 21:33, Gerry Kroll wrote:
Or (c) don't use "and" in Source (and presumably other) entries :-)
Which may be ok. There are two entries in the Create Source pane. Title
and Abbreviation. If it's only the Title that runs this check I could
use '&' there and 'and' in the abbreviation.
Presumably this applies to all Autocomplete fields. I'm surprised I
haven't come across this elsewhere. I do use Autocomplete (if that's
what I think it is) because it helps give consistency in place names.
I there any way to cancel the hour delay? I would imaging offending IPs
are held in the database with a timestamp. An SQL statement to remove my
entry would be handy. Even better would be a filter to stop it's own IP
from being caught.
Instructions to export and re-import all Gedcoms on upgrade is noted.
Thanks so much for your responses and continued work on this.
David
PhpGedView can't determine whether the requesting IP address is your own. With my ISP, the IP address of my computer can change from one session to the next, so hard-coding of IP addresses that should not be banned is not a viable option. Obviously, if I take my laptop to another location, I'll have a different IP address too.
You can use phpmyadmin or some similar tool to inspect the ip_address table. The last row in this table would be the most recent timed ban. Just delete that row if it points to your own IP address.
Lines 159 to 166 of session_spider.php is where the expired timed bans are removed: