Re: [Postgres-xc-general] pgxc seems to be running into a deadlock

SourceForge Headquarters 1320 Columbia Street Suite 310 San Diego, CA 92101 +1 (858) 422-6466

Hi Koichi,

 We are not invoking the statements concurrently. On second thoughts my
question on how  "how to know if a command failed...etc"
doesn't make sense. What is happening right now is that index creation
fails with datanode shutting down.

We did try a workaround by turning off the autovacuum  but then the memory
usage will hit 100% and would result in tuple not found error.
We don't have to create index on the fly. We would just like to get the
index build somehow.

-Sandeep

On Mon, Feb 3, 2014 at 4:46 PM, Koichi Suzuki <koi...@gm...> wrote:

> Unless you invoke "CONCURRENTLY" statement, having psql to request
> next command means the command completed.   At present, XC does not
> support "CONCURRENTLY" command.
>
> Please let us find a time to test "CREATE INDEX" in parallel with
> autovacuum.
>
> Do you need to create index on the fly, I mean as a part of usual
> database operation?   If not, there could be some more workaround on
> this.
>
> Regards;
> ---
> Koichi Suzuki
>
>
> 2014-02-04 Sandeep Gupta <gup...@gm...>:
> > Hi Ashutosh,
> >
> >  For us the app+ autovaccum is quite harmful. We are not able to run the
> > application because the index creation gets aborted in middle. The
> datanodes
> > crashes.
> >
> >  We could somehow restart the datanodes and start the index creation but
> my
> > feeling it will happen quite often.
> >
> >
> > I have a related question: is there anyway to know if a command has
> failed.
> > Usually we fire a command using psql.
> > And move to the next command. Is there any way to know if the previous
> > command failed or was a success?
> >
> > Thanks.
> > Sandeep
> >
> >
> >
> >
> > On Mon, Feb 3, 2014 at 1:13 PM, Sandeep Gupta <gup...@gm...>
> > wrote:
> >>
> >> Hi Ashutosh, Koichi,
> >>
> >>  Initially my feeling was that this was a postgres bug. That is why I
> >> posted it in the postgres community. However, I now feel that it is due
> to
> >> the changes made in XC.
> >>
> >> I have started the same test on standalone postgres. So far it hasn't
> >> crashed. My feeling is that it won't. If in case it does I will report
> >> accordingly.
> >>
> >> As requested, I started the test with verbose log on. Attached are the
> log
> >> files for the coordinator and the datanodes.
> >> There are several redundant messages that gets printed such as
> "checkpoint
> >> too often". Please use some filters etc. to view the log file. I
> thought it
> >> was best to send across the whole file.
> >>
> >> To clarify, I create a very large table (using copy) and then repeatedly
> >> create and drop index. I understand this is not the actual workload but
> that
> >> was the only way to reproduce the error.
> >>
> >> The other complication is that  in real system we get two kinds of
> errors
> >> "tuple on found" and this deadlock. I feel that they connected.
> >>
> >> Let me know if the log file help or is any other suggestions that you
> have
> >> may.
> >>
> >> -Sandeep
> >>
> >>
> >>
> >>
> >>
> >>
> >> On Sun, Feb 2, 2014 at 11:49 PM, Ashutosh Bapat
> >> <ash...@en...> wrote:
> >>>
> >>> Hi Sandeep,
> >>> Can you please check if similar error happens on vanilla PG. It may be
> an
> >>> application + auto-vacuum error, which can happen in PG as well and
> might be
> >>> harmless. It's auto-vacuum being cancelled. Auto-vacuum will run again
> >>> during the next iteration.
> >>>
> >>>
> >>> On Fri, Jan 31, 2014 at 11:21 PM, Sandeep Gupta <
> gup...@gm...>
> >>> wrote:
> >>>>
> >>>> Hi,
> >>>>
> >>>>  I was debugging an outstanding issue with pgxc
> >>>> (
> http://sourceforge.net/mailarchive/forum.php?thread_name=CABEZHFtr_YoWb22UAnPGQz8M5KqpwzbviYiAgq_%3DY...@ma...&forum_name=postgres-xc-general
> ).
> >>>>
> >>>> I couldn't reproduce that error. But I do get this error.
> >>>>
> >>>>
> >>>> LOG:  database system is ready to accept connections
> >>>> LOG:  autovacuum launcher started
> >>>> LOG:  sending cancel to blocking autovacuum PID 17222
> >>>> DETAIL:  Process 13896 waits for AccessExclusiveLock on relation 16388
> >>>> of database 12626.
> >>>> STATEMENT:  drop index mdn
> >>>> ERROR:  canceling autovacuum task
> >>>> CONTEXT:  automatic analyze of table
> >>>> "postgres.public.la_directednetwork"
> >>>> PreAbort Remote
> >>>>
> >>>>
> >>>> It seems to be a deadlock issue and may be related to the earlier
> >>>> problem as well.
> >>>> Please let me know your comments.
> >>>>
> >>>> -Sandeep
> >>>>
> >>>>
> >>>>
> >>>>
> ------------------------------------------------------------------------------
> >>>> WatchGuard Dimension instantly turns raw network data into actionable
> >>>> security intelligence. It gives you real-time visual feedback on key
> >>>> security issues and trends.  Skip the complicated setup - simply
> import
> >>>> a virtual appliance and go from zero to informed in seconds.
> >>>>
> >>>>
> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk
> >>>> _______________________________________________
> >>>> Postgres-xc-general mailing list
> >>>> Pos...@li...
> >>>> https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >>>>
> >>>
> >>>
> >>>
> >>> --
> >>> Best Wishes,
> >>> Ashutosh Bapat
> >>> EnterpriseDB Corporation
> >>> The Postgres Database Company
> >>
> >>
> >
> >
> >
> ------------------------------------------------------------------------------
> > Managing the Performance of Cloud-Based Applications
> > Take advantage of what the Cloud has to offer - Avoid Common Pitfalls.
> > Read the Whitepaper.
> >
> http://pubads.g.doubleclick.net/gampad/clk?id=121051231&iu=/4140/ostg.clktrk
> > _______________________________________________
> > Postgres-xc-general mailing list
> > Pos...@li...
> > https://lists.sourceforge.net/lists/listinfo/postgres-xc-general
> >
>