Menu
▾
▴
Re: [Postgres-xc-general] "invalid memory alloc request size ..." error
|
From: Eiji U. <uwa...@as...> - 2013-08-16 02:55:12
|
Hi Nikhil, At the request of Hemmi-san, By applying the patch to the Postgres-XC v1.1, I have conducted a DBT-1. However, the results did not change unfortunately. [results] $ psql -p 5422 -c "copy address from '/tmp/address.data' delimiter '>';" ERROR: invalid memory alloc request size 1073741824 CONTEXT: COPY address, line 30305382: "30305382>TXnEREEwqOLV11zEFGDXV4AXEDEeqLn9JDDwGxHw>TCyFEgXDNVOCihBdmdExEBAAvnXdvEEBDEE>TGEgxF5RdiuAEz..." Regards, uwaizumi (2013/08/15 18:06), Hitoshi HEMMI wrote: > Hi Nikhil, > > Thank you for the patch. > I will be off from tomorrow, but I ask our team to test it > and post the result. > > Best, > -hemmi > > > Nikhil Sontakke さんは書きました: >> Hi Hitoshi-san, >> >> PFA, patch which should help to use less memory while running COPY >> commands. This is against REL 1.1. Please do let me know if this helps >> in your case. >> >> Regards, >> Nikhils >> >> >> On Wed, Aug 14, 2013 at 3:38 PM, Hitoshi HEMMI >> <hem...@la... <mailto:hem...@la...>> wrote: >> >> Thank you, Nikhil. >> We will wait until a patch is available. >> >> Best, >> -hemmi >> >> >> Nikhil Sontakke さんは書きました: >> > Hi Hitoshi-san, >> > >> > I am working on a similar issue coincidentally. Basically what's >> > happening is that the COPY command is creating tuples which will get >> > freed up only at the end when the command will finish, so the RAM >> > memory ends up being consumed and can lead to the error that you >> have >> > provided. >> > >> > See the discussion around RemoteQueryNext on another thread. >> > >> > Regards, >> > Nikhils >> > >> > On Wed, Aug 14, 2013 at 1:56 PM, Hitoshi HEMMI >> > <hem...@la... >> <mailto:hem...@la...> >> <mailto:hem...@la... >> <mailto:hem...@la...>>> wrote: >> > >> > Hi, >> > >> > We encountered "invalid memory alloc request size ..." error of >> > Postgres-XC, when preparing DBT-1 benchmark, and need >> > someones help. >> > (First of all, what does the error message mean?) >> > >> > ------------------ >> > Version of XC: v1.1.beta >> > >> > [process to cause an error] >> > - Data generation >> > cd <DBT-1 dir>/datagen/ >> > ./datagen -i 10000 -u 15000 -p <data dir> >> > >> > - Prepare XC cluster >> > We are using following HA configuration. >> > We are not sure this configuration is essential for the error, >> > but we have never encountered the error without XC slave nodes. >> > >> > ServerA: GTM (master) >> > ServerB: GTM (slave) >> > ServerC: GTM-Proxy, Coordinator1 (master), Datanode1 (master), >> > Coordinator2 (slave), Datanode2 (slave) >> > ServerD: GTM-Proxy, Coordinator2 (master), Datanode2 (master), >> > Coordinator1 (slave), Datanode1 (slave) >> > >> > - Data loading >> > $ createdb -E SQL_ASCII -T template0 -p 5422 DBT1 >> > $ psql -a -p 5422 -f <DBT-1 dir>/scripts/pgsql/create_tables.sql >> > $ psql -p 5422 -c "copy address from '/tmp/address.data' delimiter >> > '>';" >> > >> > [the error] >> > ERROR: invalid memory alloc request size 1073741824 >> > CONTEXT: COPY address, line 33191219: >> > "33191219>EDpcFnEEAe954E5Tg6XbkiBVGLAcuHBhFAEElEE7>zxPzRj >> > >> > [Other information] >> > It seems small sized data can be copied: >> > >> > - DBT-1 tables with size (item 10000 ,user 15000) >> > address 4.7G ERROR >> > author 894K OK >> > cc_xacts 4.4G ERROR >> > customer 20G ERROR >> > item 5.0M OK >> > order_line 11G ERROR >> > orders 4.1G ERROR >> > stock 78K OK >> > -------------------- >> > >> > Best regards, >> > >> > -hemmi >> > >> > -- >> > Hitoshi HEMMI >> > NTT Open Source Software Center >> > hem...@la... <mailto:hem...@la...> >> <mailto:hem...@la... >> <mailto:hem...@la...>> >> > Tel:(03)5860-5115 >> > Fax:(03)5463-5490 >> > >> > >> > >> ------------------------------------------------------------------------------ >> > Get 100% visibility into Java/.NET code with AppDynamics Lite! >> > It's a free troubleshooting tool designed for production. >> > Get down to code-level detail for bottlenecks, with <2% overhead. >> > Download for free and get started troubleshooting in minutes. >> > >> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk >> <http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk> >> > >> <http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk >> <http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk>> >> > _______________________________________________ >> > Postgres-xc-general mailing list >> > Pos...@li... >> <mailto:Pos...@li...> >> > <mailto:Pos...@li... >> <mailto:Pos...@li...>> >> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> > >> > >> > >> > >> > -- >> > StormDB - http://www.stormdb.com >> > The Database Cloud >> >> >> -- >> Hitoshi HEMMI >> NTT Open Source Software Center >> hem...@la... <mailto:hem...@la...> >> Tel:(03)5860-5115 >> Fax:(03)5463-5490 >> >> >> ------------------------------------------------------------------------------ >> Get 100% visibility into Java/.NET code with AppDynamics Lite! >> It's a free troubleshooting tool designed for production. >> Get down to code-level detail for bottlenecks, with <2% overhead. >> Download for free and get started troubleshooting in minutes. >> http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk >> <http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk> >> _______________________________________________ >> Postgres-xc-general mailing list >> Pos...@li... >> <mailto:Pos...@li...> >> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general >> >> >> >> >> -- >> StormDB - http://www.stormdb.com >> The Database Cloud > > -- ----------------------------------- 上泉 英二 uwa...@as... ----------------------------------- |
