I create the database and run the ddl to create a table then try to do inserts and the insert does not work. I can load the table from csv files using the copy to xxxx from xxx.csv But Inserts do not work.
[postgres@cen-6-pgxl-coord ~]$ psql -U certi CRISTAL_SDP
psql (PGXL 9.2.0, based on PG 9.2.4 (Postgres-XL 9.2.0))
Type "help" for help.
CRISTAL_SDP=# INSERT INTO CRISTAL.CDSEX (CDEPIN,RORDIN,CTYENR,XTYPEN,NUMFIC)
CRISTAL_SDP-# VALUES('17',
CRISTAL_SDP(# '1',
CRISTAL_SDP(# 'CABECI',
CRISTAL_SDP(# 'O12',
CRISTAL_SDP(# '0') ;
INSERT 0 0
INSERT 0 Records. Why?? I can do a copy from to load the tables but can't insert records???
I have 4 servers, 1 gtm, 1 coordinator and 2 datanodes.
I set the following kernel values on each server:
sysctl -w kernel.shmmax=17179869184
sysctl -w kernel.shmall=4194304
pgxl definitions:
psql -c "CREATE NODE datanode1 WITH (TYPE = 'datanode', HOST = '192.168.1.43', PORT = 15432)" postgres
psql -c "CREATE NODE datanode2 WITH (TYPE = 'datanode', HOST = '192.168.1.44', PORT = 15433)" postgres
psql -c "CREATE NODE GROUP data_cluster WITH (datanode1, datanode2)" postgres
psql -c "SELECT pgxc_pool_reload()" postgres
192.168.1.41
mkdir /usr/local/pgsql/gtm
chown postgres /usr/local/pgsql/gtm
su - postgres
initgtm -D /usr/local/pgsql/gtm -Z gtm
gtm -D /usr/local/pgsql/gtm >gtm.log 2>&1 &
1:139778787112768:2014-07-09 16:09:09.475 EDT -LOG: Saving transaction restoration info, backed-up gxid: 12731
LOCATION: GTM_WriteRestorePointXid, gtm_txn.c:2649
1:139778787112768:2014-07-09 16:09:09.476 EDT -LOG: Started to run as GTM-Active.
LOCATION: main, main.c:641
1:139778787112768:2014-07-09 16:09:35.744 EDT -LOG: Any GTM standby node not found in registered node(s).
LOCATION: gtm_standby_connect_to_standby_int, gtm_standby.c:381
1:139778787112768:2014-07-09 16:09:48.058 EDT -LOG: Any GTM standby node not found in registered node(s).
LOCATION: gtm_standby_connect_to_standby_int, gtm_standby.c:381
1:139778787112768:2014-07-09 16:09:48.081 EDT -LOG: Any GTM standby node not found in registered node(s).
LOCATION: gtm_standby_connect_to_standby_int, gtm_standby.c:381
.
.
.
192.168.1.43
mkdir /usr/local/pgsql/datanode1
chown postgres /usr/local/pgsql/datanode1
su - postgres
initdb -D /usr/local/pgsql/datanode1 --nodename datanode1
postgres --datanode -p 15432 -D /usr/local/pgsql/datanode1 > datanode1.log 2>&1 &
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
.
.
.
192.168.1.44
mkdir /usr/local/pgsql/datanode2
chown postgres /usr/local/pgsql/datanode2
su - postgres
initdb -D /usr/local/pgsql/datanode2 --nodename datanode2
postgres --datanode datanode2 -p 15433 -D /usr/local/pgsql/datanode2 >datanode2.log 2>&1 &
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
.
.
.
192.168.1.42
mkdir /usr/local/pgsql/coord1
chown postgres /usr/local/pgsql/coord1
su - postgres
initdb -D /usr/local/pgsql/coord1 --nodename coord1
postgres --coordinator -D /usr/local/pgsql/coord1 >coord1.log 2>&1 &
LOG: database system is ready to accept connections
LOG: autovacuum launcher started
.
.
.
Any status on this issue?
I'd like to use postgres-xl in cloudfoundry as a distributed database but this is holding it back.
Hi Mike,
I have seen this issue whenever I miss the step to do pgxc_pool_reload() on
the datanodes after creating/altering nodes. If the step is not documented,
I think we need to improve that. In meanwhile, can you try calling
pgxc_pool_reload() on each datanode? Based on your config, you may need
something like this: You can skip the first two commands if you have
already created the nodes.
psql -c "EXECUTE DIRECT ON (datanode1) 'ALTER NODE datanode1 WITH (TYPE =
''datanode'', HOST = ''192.168.1.43'', PORT = 15432)" postgres
psql -c "EXECUTE DIRECT ON (datanode1) 'CREATE NODE datanode2 WITH (TYPE =
''datanode'', HOST = ''192.168.1.43'', PORT = 15433)" postgres
psql -c "EXECUTE DIRECT ON (datanode1) 'SELECT pgxc_pool_reload()" postgres
psql -c "EXECUTE DIRECT ON (datanode2) 'CREATE NODE datanode1 WITH (TYPE =
''datanode'', HOST = ''192.168.1.43'', PORT = 15432)" postgres
psql -c "EXECUTE DIRECT ON (datanode2) 'ALTER NODE datanode2 WITH (TYPE =
''datanode'', HOST = ''192.168.1.43'', PORT = 15433)" postgres
psql -c "EXECUTE DIRECT ON (datanode2) 'SELECT pgxc_pool_reload()" postgres
(Please note there are two single quotes around TYPE and HOST values). You
may also want to add coordinator information on the datanodes, but I am not
sure if they are required.
The current pgxc_ctl utility is also missing these steps and I will soon
make those changes to the utility as well.
Please let me know if this fixes the INSERT problem.
Thanks,
Pavan
On Wed, Jul 16, 2014 at 8:08 PM, Mike swampfoxmr@users.sf.net wrote:
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Related
Tickets: #18
Hi Pavan,
I have the same issue ( not inserting data ) and I tried what you have recommanded in your reply, but still the issue is there. Is this going to be fixed next release ?
Hi Viraj,
I've fixed a bug or two in the current master branch which fixes the
problem for me. This will get included in the GA release. If you feel
comfortable building from sources, can you please get the latest source
code and test with the current master branch?
Thanks,
Pavan
On Thu, Aug 21, 2014 at 10:18 AM, Viraj mayuraviraj@users.sf.net wrote:
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Related
Tickets: #18
Hi Pavan,
I tried the latest trunk code but it seems isssue is still there.
Let me outline the what i did
Step 1 :
Took the trunk
git clone http://git.code.sf.net/p/postgres-xl/postgres-xl
Step 2 :
Built and installed it
My test enviroment like following
Machine-1 -- Run coordinator, gtm , datanode1
Machine-2 -- Run datanode2
Step 3 :
Initiated the data directories
On Machine 1
initdb -D /home/postgresxl/installed/postgre-xl-9.2/cord_data --nodename coord1
initdb -D /home/postgresxl/installed/postgre-xl-9.2/node_data --nodename datanode1
initgtm -D /home/postgresxl/installed/postgre-xl-9.2/gtm_data -Z gtm
On Machine 2
initdb -D /home/postgresxl/installed/postgre-xl-9.2/node_data --nodename datanode2
Step 4 :
Configuration for Machine 2 datanode
postgresql.conf
listen_addresses = '*'
port = 15432
pooler_port = 6666
gtm_host = 'Machine-1-ip-address'
gtm_port = 6666
pg_hba.conf
host all all Machine-1-ip-address/mask trust
Step 5 :
Configuraion for Machine 1 datanode
postgresql.conf
pooler_port = 6666
port = 15432
Step 6 :
Configuraion for Machine 1 coordinator
postgresql.conf
pooler_port = 6667
Step 7 :
Started all
Machine - 1
gtm -D /home/postgresxl/installed/postgre-xl-9.2/gtm_data > gtmlogfile 2>&1 &
pg_ctl -Z coordinator start -D /home/postgresxl/installed/postgre-xl-9.2/cord_data
pg_ctl -Z datanode start -D /home/postgresxl/installed/postgre-xl-9.2/node_data
Machine - 2
pg_ctl -Z datanode start -D /home/postgresxl/installed/postgre-xl-9.2/node_data
Step 8 :
Login to coordinator(Machine-1) using psql and postgres database. Added datanodes
CREATE NODE datanode1 WITH (TYPE = 'datanode', PORT = 15432, HOST='localhost');
CREATE NODE datanode2 WITH (TYPE = 'datanode', PORT = 15432, HOST='machine-2-ip');
SELECT pgxc_pool_reload(); # Returns t
select * from pgxc_node;
node_name | node_type | node_port | node_host | nodeis_primary | nodeis_preferred | node_id
-----------+-----------+-----------+---------------+----------------+------------------+------------
coord1 | C | 5432 | localhost | f | f | 1885696643
datanode1 | D | 15432 | localhost | f | f | 888802358
datanode2 | D | 15432 | ip-address-machine-2 | f | f | -905831925
Step 9 :
Login to Machine-2 datanode and did SELECT pgxc_pool_reload(); ( success, returns t )
Step 10 :
created database and table t1 using coordinator. Success . Replicated to datanode1 and datanode2.
Step 11 :
Insert into table t1 using coordinator : FAILED
Returns INSERT 0 0
Log message
LOG: statement: insert into t2 values(1,'name');
ERROR: node "coord1_4383" does not exist
STATEMENT: SET global_session TO coord1_4383;
Simple select : log message
LOG: statement: select * from t2;
ERROR: node "coord1_4383" does not exist
STATEMENT: SET global_session TO coord1_4383;
Last edit: Viraj 2014-08-21
On Thu, Aug 21, 2014 at 6:53 PM, Viraj mayuraviraj@users.sf.net wrote:
I would also recommend using pgxc_ctl utility to set up the cluster since
that takes care of all of these steps internally.
Please let us know if it helps.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Related
Tickets: #18
Hi Pavan,
I tried execute direct on each node. Now table created with distribute by replication works fine. Inserts are working. But table created with distribute by hash(id) having problmes with insert. For some inserts it returns true ( i.e data is inserted) , but for some there is no insert happening. I belive insert are failing on one datanode and that data node has negagive node id. See below result,
node_name | node_type | node_port | node_host | nodeis_primary | nodeis_preferred | node_id
-----------+-----------+-----------+---------------+----------------+------------------+------------
coord1 | C | 5432 | localhost | f | f | 1885696643
datanode1 | D | 15432 | ip-1 | f | f | 888802358
datanode2 | D | 15432 | ip-2 | f | f | -905831925
Is it ok to have negative id ?
Hi Viraj, I have same problem with insertion: http://dba.stackexchange.com/questions/97514/postgres-xl-does-not-insert-rows
I have downloaded postgres-xl from git https://github.com/snaga/postgres-xl
Hi guys! I faced with the same kind of issue. Has anybody succeeded in fixing this?
Thank you
Dmitry
On Thursday, August 28, 2014, dsavinkov dsavinkov@users.sf.net wrote:
Regards,
Mason
On Fri, Aug 29, 2014 at 8:05 AM, mason_s mason_s@users.sf.net wrote:
Hope this helps.
Thanks,
Pavan
--
Pavan Deolasee
http://www.linkedin.com/in/pavandeolasee
Hi Pavan
I am using release version, it is not from repo.
The problem is reproduced. I have weird negative node_id indicated above. Is it normal? :
EXECUTE DIRECT command is not working in my env. My log also contains:
I guess,my config for coordinator/nodes is not handled properly. Instead of 'coord1_PID' value I expect to see my coordinator node name 'coord1'. Could you please clarify it for me?
Thank you
Dmitry
Last edit: dsavinkov 2014-09-02
Hi All
Everything works fine for me now. Somehow my pgxc_node table on datanodes conained only 1 record with datanode config. The root cause of the issue was taht 'node_type' column contained 'C' which is Coordinator, but not 'D' - Datanode. After I changed this value and run EXECUTE DIRECT on both datanodes, I was able to INSERT data.
Thanks for assistance
Great impression so far!
Dmitry Savinkov
Hi, I'm having the same problem attempting to insert records. Please post details of how you fixed this error:
ERROR: node "coord1_15197" does not exist
Hi Dmitry,
First of all guys thanks for all the comments , it made my job a lot easier as I am a newbie and trying to install it for the first time. But I am having the problem as described below:
My setup is as follows :
1st Server has : GTM ,1 Co-ordinator,1 Datanode
2nd Server has : 1 Datanode
Using code from the postgres XL download link :"http://www.postgres-xl.org/download/"
I have created a new database using the co-ordinator and it got reflected on both servers. Next i created a table ans it also got reflected.But I am stuck here ,as i am not able to enter data. The problem is same as it shows a -ve value for datanode 2 when you do a select * from pgxc_node;
You had the same issue and have solved it.Could you please let me know how you had changed the value from co-ordinator to datanode.
I solved this problem, here's how...
The most basic installation is 1 gtm, coordinator and datanode on the same machine.
By default a coordinator is created with port = 5432.
After installation...
Note I've abbreviated column headings for better readability
In the catalog pgxc_node we need a coordinator and a datanode. Every datanode has this catalog and must be updated accordingly.
Like the documentation says we need to create a datanode...
The step missing from the documentation is these "EXECUTE DIRECT" commands mentioned by Pavan Deolasee.
These commands are specific to datanode1.
By default we have:
This is incorrect, we have datanode1 looking like coord1.
Create coord1 on datanode1
Fix datanode1
Hi all,
I am facing the exact same problems with insert. CSV import works fine. Updates and Delete operations also work fine. Create table ,drop table works perfectly too. The only problem is that INSERT is not working. This is blocking me from setting up ETL from SQL Server OLTP system to PostgresXL OLAP.
I enabled logging levels to debug5. But there is no clue what is the problem with INSERTs only. I did some correlation analysis between coordinator and data node logs. I could not find any clues in them too.
Any advice regarding this would be very helpful.
Thanking all of you for your help in advance!
Pradyumna
P.S. I have huge logs, which are not suitable for posting here. If anyone is initiating troubleshooting, I would be happy to share the logs as well.
Hi,
Can you please attach a self-contained test case? How do you set up the
cluster? If you just send logs from the coordinator and datanodes (please
include only that portion when the problem occurs) that will be helpful too.
I've seen multiple people reporting this issue in the past and then somehow
gets fixed after a proper setup. Its time to get to the bottom of it and
fix the bug or at least add appropriate error message/hint or even some
utility to check the status of the cluster.
Thanks,
Pavan
On Thu, Sep 3, 2015 at 7:08 AM, Pradyumna Muppirala <pradyumnam@users.sf.net
--
Pavan Deolasee http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
Related
Tickets: #18
Hello Pavan,
Thank you very much for response. Please find the log files on dropbox.com . URL is below.
URL: https://www.dropbox.com/sh/fc5bbpnpxggjudo/AAByLiKOYjGChC9KppkAyo5sa?dl=0
I provided almost all the config information also attached below.
Please review and let me know if more information is required.
thanks
Pradyumna
Please find the related logs and CSV sample file attached.
Please find the SQL command shell dump.
I followed the setup guide , described in this short version of the setup.
URL: http://files.postgres-xl.org/documentation/install-short.html
Hi,
Please let me know, if there is any update related to this INSERT failure issue.
Thanks in advance for your help!
Pradyumna
Hi Pradyumna,
Can you please post the hardware/configuration of each machine in the
cluster? Can you please build with debug flags and also set "ulimit -c
unlimited" to see if any core is generated on the machine. I assume you are
connecting to a coordinator using psql. If not, please let us know.
Does this happen even on a cluster which has been restarted? I am asking to
see if some previous error on a connection causes this. Can you please also
post data from pgxc_node catalog table on each node?
Thanks,
Pavan
On Tue, Sep 8, 2015 at 8:28 PM, Pradyumna Muppirala <pradyumnam@users.sf.net
Related
Tickets: #18
Hi Pavan,
Thanks for quick response. Here are answers to some of your questions.
I was trying to test the setup process on local machine. I was successful in installation. The only problem I faced is that Inserts are not happening.
If I notice any core files, I will pass them on. Could you please review the detailed attachments , attached above?
thanks
Pradyumna