Re: [Postgres-xc-developers] Commit be024c02d1cb1ba308f53a3d74d22f2f11171a84

[Amit K. <ami...@en...>]

I will try to reply to this as soon as I get a chance, but meanwhile,
including this in dev list.


On 16 November 2013 01:50, Mason Sharp <mas...@gm...> wrote:

> Hi Amit,
>
> Can I ask you about this commit?
>
>
>> 0. In initdb.c vacuum-freeze all the databases, so that there will not be
>> any data loss even if it's local xid happens to be in the future w.r.t.
>> GXID.
>>
>
> What case does this handle? A GTM that has a high XID and one decides to
> re-initdb?
>
>
>
>> After the server is brought up, the pg_class.relfrozenxid,
>> pg_database.relfrozenxid, and the xid-wraparound limits should now be
>> adjusted according to the new GXID.
>>
>
> Here, you are just talking about a brand new node/database?
>
>
>> 1. If gxid exceeds "Stop" limit (ShmemVariableCache->xidStopLimit), allow
>> auto-vacuum to adjust the oldest frozen xids, and the various xid wrap
>> limits w.r.t. the new gxid.
>>
>
>
>
>> 2. If gxid is greater than the current xid-wrap limit (i.e. oldest xid +
>> 2^31), don't even allow auto-vacuum to run. DBA would run vacuum-freeze on
>> all the databases using standalone backend.
>>
>
> You suppress autovacuum? Are operations on the database blocked until a
> manual vacuum freeze?
>
>
>
>>  3. Allow standalone backend to run using GXIDs, rather than using local
>> xids. This way, if gxid exceeds wrap limit, the user would use standalone
>> backend to run vacuum-freeze on all databases. Currently, because
>> standalone backend uses local xids, there is no way vacuum-freeze can
>> update the oldest frozen xids. Hence, the need for using global xids.
>> Anyway, it makes sense to start using global xids.
>>
>
> Kind of confused what you mean by standalone backends.
>
>
>> 5. In GetNewTransactionID(), while printing error message about "commands
>> can't run due to xid-wraparound", get_database_name() is called to print
>> the database name. But if the context is auto-vacuum launcher,
>> get_database_name() fails because MyDatabaseId is not set in auto-vacuuum
>> launcher. For this, call get_database_name() only if MyDatabaseId is set.
>>
>> 6. Try to keep ShmemVariableCache->nextXid in sync with gxid. nextXid is
>> used at a lot of places, so if it remains out-of-sync, we get unexpected
>> error/warning messages when running vacuum-freeze on a standalone backend.
>> I have not changed the current behaviour by keeping this sync up only in
>> standalone backend. There must be some reason why we increment nextXid only
>> in certain scenarios (see increment_xid), so did not totally remove that
>> logic.
>>
>
> I am not sure. Seems like we should always use global XIDs and global
> snapshots.
>
>
>>  7. If the server is brought up with xids far in the future w.r.t. GXID,
>> even the relfrozenxids can be in the future w.r.t. to the new GXID. In PG,
>> while updating pg_class.relfrozenxid, it is never set backwards because
>> that is never possible.
>>
>> In XC, that's possible now, so allow it to go backwards. But this will
>> happen only in standalone backend because we disallow commands if gxid is
>> already wrapped around. Similarly, allow pg_database.datfrozenxid to be set
>> in the "past". (vac_update_relstats() and vac_update_datfrozenxid()).
>>
>
> What do you mean by standalone backends here?
>
>
>
>> 8. Keep the current behaviour of *not* truncating the clog if wrap limits
>> are to be set backwards (See vac_truncate_clog()). But once they are set,
>> and all xids are in sync, it will continue normal pg_clog truncation in the
>> next vacuum-freeze cycle.
>>
>> 9. initdb will continue to use local xids. Since a standlalone postgres
>> backend now would use gxid by default, we need a way for initdb to spawn
>> postgres that would use local xids. For this, we have added a new postgres
>> cmd-line argument --localxid. This is to be used only by initdb.
>>
>
> So.... when would --localxid be used? Only by pgxc_ctl when adding new
> nodes? When would a DBA manually use it?
>
> BTW, we have an XC support customer who appeared to run into some data
> errors after running a vacuum freeze, hence I was looking into these
> commits.
>
> Thanks,
>
> Mason
>
>
>
>