safekeep-devel — Development and support discussion list for SafeKeep.

Re: [Safekeep-devel] Usage with independently distributed ssh keys

[Jan Š. <sk...@js...>]

On 6.5.2015 13:11, Frank Crawford wrote:
> In particular, if you are mainly talking about distributing root's keys,
> then it will work fine, with standard protection.

I finally got it to work by copying the backup id_rsa key over
safekeep-server-*-key and deleting safekeep-server-*-key.pub (dont know 
why the deletition was necessary). It's kind of dirty though, but easy 
to automate.

> Alternatively, do you have more details about ploop, so I can look at
> what it would take to add it.

Standard way to backup openvz vm is:
https://openvz.org/Ploop/Backup#File-based_backup

There are also lower-level commands for manipulating ploop:
https://openvz.org/Ploop/readme

I could add the above code as pre and post-backup hook, but as there are 
no backup tools able to incrementally and consistently backup openvz vm, 
I think it would be great if safekeep could do that :).

--
Jan Škoda

Re: [Safekeep-devel] Usage with independently distributed ssh keys

[Frank C. <fr...@cr...>]

Jan,

On Wed, 2015-05-06 at 10:25 +0200, Jan Škoda wrote:

> Hello,
> 
> is it possible to use safekeep with ssh keys, that are distributed 
> otherwise? For example manually or using puppet/ansible/salt/chef?



Yes, you can distribute ssh keys by any method you wish, but you do have
to consider the security implications.  If you are running with the
normal configuration you need to protect the fact that you have a normal
user account that can be promoted to root.

In particular, if you are mainly talking about distributing root's keys,
then it will work fine, with standard protection.


> Also, is it possible to sponsor development or pay someone for 
> implementing new features for safekeep? I would love to see ploop 
> snapshot support, because it would allow safekeep to backup openvz vms.



I don't know the plans of any others involved, but if you know someone
who is willing to write the appropriate code, then we will certainly
look at including it.

Alternatively, do you have more details about ploop, so I can look at
what it would take to add it.


> Thank you!
> --
> Jan Škoda
> 

Regards

Frank


> ------------------------------------------------------------------------------
> One dashboard for servers and applications across Physical-Virtual-Cloud 
> Widest out-of-the-box monitoring support with 50+ applications
> Performance metrics, stats and reports that give you Actionable Insights
> Deep dive visibility with transaction tracing using APM Insight.
> http://ad.doubleclick.net/ddm/clk/290420510;117567292;y
> _______________________________________________ Safekeep-devel mailing list Saf...@li... https://lists.sourceforge.net/lists/listinfo/safekeep-devel

[Safekeep-devel] Usage with independently distributed ssh keys

[Jan Š. <sk...@js...>]

Hello,

is it possible to use safekeep with ssh keys, that are distributed 
otherwise? For example manually or using puppet/ansible/salt/chef?

Also, is it possible to sponsor development or pay someone for 
implementing new features for safekeep? I would love to see ploop 
snapshot support, because it would allow safekeep to backup openvz vms.

Thank you!
--
Jan Škoda

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Dimi P. <di...@la...>]

Hi guys,

Sorry for the late response -- my issue with changing the semantics of the
current switch is that it will quietly break any restore script out there.

That's a very high cost to pay.

This is why I suggested a version that doesn't create the breakage, at the
cost of a bit more work.

Frank had a complaint about my proposal, but it wasn't clear to me what
that was.

Frank, mind if you re-read my past proposal and comment on it?

Dimi Paun

Sent from my phone.
On Feb 22, 2015 2:45 AM, "Frank Crawford" <fr...@cr...> wrote:

> Marco, Dimi,
>
> What was the final decision here?  While we talked about an option, I'm
> not sure if we agreed to it in the end, and if it was only limited to
> postgres?
>
> Regards
> Frank
>
> On Tue, 2014-10-21 at 15:45 +0200, Marco Bozzolan wrote:
> > * Dimi Paun <di...@la...> [2014 10 21, 08:33]:
> > > As an aside: if no database is specified, it means "dump all" and
> unless
> > > you have exactly one, it just doesn't make sense not to have a CREATE
> > > DATABASE.
> >
> >   This has been my first thought as well, but actually the dump could
> > just contain a  "\connect <dbname>" or a "USE <dbname>" for each
> > database. This could be useful in a situation where you have the
> > databases created by an external script which can perform additional
> > operations (i.e. setting default privileges).
> >
> > > For the rest, you've already worked out the right switches, no?
> >
> >   It would seem so, but I think that what was really noticeable was the
> > complexity required in order to apply this attribute to every
> > (present, maybe future?) database-dumping tool. In my opinion
> > correcting this behaviour only for PostgreSQL and letting the user
> > customize it for every other tool would just be simpler and more easily
> > extended in future (i.e. there will be no need to find out other
> > switches or to add other warnings when adding support for the next
> > tool).
> >
> > Regards,
> >
>
>
>
>
> ------------------------------------------------------------------------------
> Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
> from Actuate! Instantly Supercharge Your Business Reports and Dashboards
> with Interactivity, Sharing, Native Excel Exports, App Integration & more
> Get technology previously reserved for billion-dollar corporations, FREE
>
> http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Frank C. <fr...@cr...>]

On Mon, 2015-02-23 at 16:57 +0100, Marco Bozzolan wrote:
> * Frank Crawford <fr...@cr...> [2015 02 22, 18:44]:
> > Marco, Dimi,
> > What was the final decision here?  While we talked about an option, I'm
> > not sure if we agreed to it in the end, and if it was only limited to
> > postgres?
> 
> Hello,
> 
>   I'm sorry I read your last mail about the new release but I didn't 
> manage to post my answer until now.
> 
>   While I believe the final decision is up to you because - as I wrote in 
> my last mail - making the option available to all databases would 
> require a much bigger effort to implement support for other dumping tools in 
> the future, I offer to provide a patch (via pull request?) to implement 
> a new attribute for PostgreSQL only to handle the problematic case.
> 
>   If you agree, I think I'll be able to provide the patch by the end of 
> this week.
> 
> 
> Best regards,
> 
Marco,

How is the patch going?

Frank

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[F. C. <fr...@cr...>]

Marco,

Please provide the patch, as for now let's fix the specific case and maybe we will generalise it later.

Thanks
Frank

Sent from my HTC

----- Reply message -----
From: "Marco Bozzolan" <ma...@s1...>
To: <saf...@li...>
Subject: [Safekeep-devel] Proposal: removal of '-C' pg_dump option
Date: Tue, Feb 24, 2015 2:57 AM

* Frank Crawford <fr...@cr...> [2015 02 22, 18:44]:
> Marco, Dimi,
> What was the final decision here?  While we talked about an option, I'm
> not sure if we agreed to it in the end, and if it was only limited to
> postgres?

Hello,

I'm sorry I read your last mail about the new release but I didn't 
manage to post my answer until now.

While I believe the final decision is up to you because - as I wrote in 
my last mail - making the option available to all databases would 
require a much bigger effort to implement support for other dumping tools in 
the future, I offer to provide a patch (via pull request?) to implement 
a new attribute for PostgreSQL only to handle the problematic case.

If you agree, I think I'll be able to provide the patch by the end of 
this week.


Best regards,

-- 
https://s19n.net

------------------------------------------------------------------------------
Download BIRT iHub F-Type - The Free Enterprise-Grade BIRT Server
from Actuate! Instantly Supercharge Your Business Reports and Dashboards
with Interactivity, Sharing, Native Excel Exports, App Integration & more
Get technology previously reserved for billion-dollar corporations, FREE
http://pubads.g.doubleclick.net/gampad/clk?id=190641631&iu=/4140/ostg.clktrk
_______________________________________________
Safekeep-devel mailing list
Saf...@li...
https://lists.sourceforge.net/lists/listinfo/safekeep-devel

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Marco B. <ma...@s1...>]

* Frank Crawford <fr...@cr...> [2015 02 22, 18:44]:
> Marco, Dimi,
> What was the final decision here?  While we talked about an option, I'm
> not sure if we agreed to it in the end, and if it was only limited to
> postgres?

Hello,

  I'm sorry I read your last mail about the new release but I didn't 
manage to post my answer until now.

  While I believe the final decision is up to you because - as I wrote in 
my last mail - making the option available to all databases would 
require a much bigger effort to implement support for other dumping tools in 
the future, I offer to provide a patch (via pull request?) to implement 
a new attribute for PostgreSQL only to handle the problematic case.

  If you agree, I think I'll be able to provide the patch by the end of 
this week.


Best regards,

-- 
https://s19n.net

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Frank C. <fr...@cr...>]

Marco, Dimi,

What was the final decision here?  While we talked about an option, I'm
not sure if we agreed to it in the end, and if it was only limited to
postgres?

Regards
Frank

On Tue, 2014-10-21 at 15:45 +0200, Marco Bozzolan wrote:
> * Dimi Paun <di...@la...> [2014 10 21, 08:33]:
> > As an aside: if no database is specified, it means "dump all" and unless 
> > you have exactly one, it just doesn't make sense not to have a CREATE 
> > DATABASE.
> 
>   This has been my first thought as well, but actually the dump could 
> just contain a  "\connect <dbname>" or a "USE <dbname>" for each 
> database. This could be useful in a situation where you have the 
> databases created by an external script which can perform additional 
> operations (i.e. setting default privileges).
> 
> > For the rest, you've already worked out the right switches, no?
> 
>   It would seem so, but I think that what was really noticeable was the 
> complexity required in order to apply this attribute to every 
> (present, maybe future?) database-dumping tool. In my opinion 
> correcting this behaviour only for PostgreSQL and letting the user 
> customize it for every other tool would just be simpler and more easily 
> extended in future (i.e. there will be no need to find out other 
> switches or to add other warnings when adding support for the next 
> tool).
> 
> Regards,
> 

[Safekeep-devel] Some SafeKeep Updates Coming

[Frank C. <fr...@cr...>]

Folks,

Sorry for the long silence, as with most people other things have been
going on.

Anyway, while I suspect SafeKeep is dying, I will shortly apply a number
of outstanding patches and updates.

The three things I have are:

- HTML fixups from Stanislav Blokhin (patches supplied)
- Client sort order from Carl Franks (minor change), and
- pg_dump options as proposed by Marco Bozzolan (no patch yet).

In addition, remember that the source is now at GitHub
( https://github.com/dimipaun/safekeep/ ), and that is where these
patches will be applied.

Any comments would be appreciated.

Regards
Frank

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Marco B. <ma...@s1...>]

* Dimi Paun <di...@la...> [2014 10 21, 08:33]:
> As an aside: if no database is specified, it means "dump all" and unless 
> you have exactly one, it just doesn't make sense not to have a CREATE 
> DATABASE.

  This has been my first thought as well, but actually the dump could 
just contain a  "\connect <dbname>" or a "USE <dbname>" for each 
database. This could be useful in a situation where you have the 
databases created by an external script which can perform additional 
operations (i.e. setting default privileges).

> For the rest, you've already worked out the right switches, no?

  It would seem so, but I think that what was really noticeable was the 
complexity required in order to apply this attribute to every 
(present, maybe future?) database-dumping tool. In my opinion 
correcting this behaviour only for PostgreSQL and letting the user 
customize it for every other tool would just be simpler and more easily 
extended in future (i.e. there will be no need to find out other 
switches or to add other warnings when adding support for the next 
tool).

Regards,

-- 
https://s19n.net

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Dimi P. <di...@la...>]

As an aside: if no database is specified, it means "dump all" and unless
you have exactly one,
it just doesn't make sense not to have a CREATE DATABASE.

What we can do is simply allow the attribute in all cases, and maybe issue
a warning
for the "PG with no db name" case which we cannot honour. For the rest,
you've already
worked out the right switches, no?


On Mon, Oct 20, 2014 at 8:03 AM, Marco Bozzolan <ma...@s1...> wrote:

> * Dimi Paun <di...@la...> [2014 10 17, 21:15]:
> > I'd suggest a slightly different approach:
> > - add a switch like save-db-name="true|false" (help with naming much
> > appreciated)
> > - if present, we do the obvious; if absent, we maintain current
> > semantics.
> > Messier, but safer. Thoughts?
>
>   While trying to list all the possibilities I'm afraid I found a case
> which could not be handled: using postgresql, with no database
> specified and "save-db-name=false" I wasn't able to find an option for
> pg_dumpall not to generate the CREATE instruction.
>
> The complete list follows:
>
> - no save-db-name attribute:
>    - mysql/no database specified: CREATE DATABASE is generated;
>    - mysql/with database specified: CREATE DATABASE not generated;
>    - pgsql/no database specified: CREATE DATABASE is generated;
>    - pgsql/with database specified: CREATE DATABASE is generated ("-C");
>
> - save-db-name="true": CREATE DATABASE generated in all cases:
>    - mysql/no database specified: by default in "--all-databases";
>    - mysql/with database specified: using "--databases";
>    - pgsql/no database specified: by default with pg_dumpall;
>    - pgsql/with database specified: using "-C";
>
> - save-db-name="false": CREATE DATABASE not generated:
>    - mysql/no database specified: using "--no-create-db";
>    - mysql/with database specified: not using "--databases";
>    - pgsql/no database specified: could not find an option for
>      pg_dumpall;
>    - pgsql/with database specified: by default.
>
>   Since the issue relates only to PostgreSQL, could we allow the new
> attribute only when both of the following are true:
> - /backup/setup/dump/@type = "postgres";
> - /backup/setup/dump/@db is specified?
>
> This way we would fix this "special case" and let the user customize
> other tools via the available "options" attribute.
>
> Regards,
>
> --
> https://s19n.net
>
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>



-- 
Dimi Paun <di...@la...>
Lattica, Inc.

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Marco B. <ma...@s1...>]

* Dimi Paun <di...@la...> [2014 10 17, 21:15]:
> I'd suggest a slightly different approach:
> - add a switch like save-db-name="true|false" (help with naming much
> appreciated)
> - if present, we do the obvious; if absent, we maintain current
> semantics.
> Messier, but safer. Thoughts?

  While trying to list all the possibilities I'm afraid I found a case 
which could not be handled: using postgresql, with no database 
specified and "save-db-name=false" I wasn't able to find an option for 
pg_dumpall not to generate the CREATE instruction.

The complete list follows:

- no save-db-name attribute:
   - mysql/no database specified: CREATE DATABASE is generated;
   - mysql/with database specified: CREATE DATABASE not generated;
   - pgsql/no database specified: CREATE DATABASE is generated;
   - pgsql/with database specified: CREATE DATABASE is generated ("-C");

- save-db-name="true": CREATE DATABASE generated in all cases:
   - mysql/no database specified: by default in "--all-databases";
   - mysql/with database specified: using "--databases";
   - pgsql/no database specified: by default with pg_dumpall;
   - pgsql/with database specified: using "-C";

- save-db-name="false": CREATE DATABASE not generated:
   - mysql/no database specified: using "--no-create-db";
   - mysql/with database specified: not using "--databases";
   - pgsql/no database specified: could not find an option for 
     pg_dumpall;
   - pgsql/with database specified: by default.

  Since the issue relates only to PostgreSQL, could we allow the new 
attribute only when both of the following are true:
- /backup/setup/dump/@type = "postgres";
- /backup/setup/dump/@db is specified?

This way we would fix this "special case" and let the user customize 
other tools via the available "options" attribute.

Regards,

-- 
https://s19n.net

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Frank C. <fr...@cr...>]

Dimi,

Okay, not an issue.  The issue I see is that to implement it for
different databases it is very different configuration, even if it is
supported.

Regards
Frank

On Mon, 2014-10-20 at 01:14 -0400, Dimi Paun wrote: 

> 
> On Sat, Oct 18, 2014 at 12:41 AM, Frank Crawford
> <fr...@cr...> wrote:
> 
>         No, I don't think the new switch is a good idea, since the
>         issue is just with the Postgres dump, not MySQL or possibly
>         any other ones we add.
>         
> 
> 
> I'm not sure I understand. What I'm proposing is that if the switch is
> not present, we keep 100% the current situation as is, across all
> database types.
> It's only when it's present that we'll do what we're asked to do.
> 
> 
> What am I missing?
> 
> 
> 
> 
> -- 
> Dimi Paun <di...@la...>
> Lattica, Inc.
> 
> 
> 
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Dimi P. <di...@la...>]

On Sat, Oct 18, 2014 at 12:41 AM, Frank Crawford <fr...@cr...>
wrote:

> No, I don't think the new switch is a good idea, since the issue is just
> with the Postgres dump, not MySQL or possibly any other ones we add.
>

I'm not sure I understand. What I'm proposing is that if the switch is not
present, we keep 100% the current situation as is, across all database
types.
It's only when it's present that we'll do what we're asked to do.

What am I missing?



-- 
Dimi Paun <di...@la...>
Lattica, Inc.

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Frank C. <fr...@cr...>]

Dimi,

No, I don't think the new switch is a good idea, since the issue is just
with the Postgres dump, not MySQL or possibly any other ones we add.

Maybe we need to extend the syntax of options to allow
overwriting/replacing the default, possibly with an NULL value.

What do you think about that?

Frank

On Fri, 2014-10-17 at 21:15 -0400, Dimi Paun wrote: 

> Sorry for the delay, I needed some time to think about this one.
> 
> 
> Marco's point is well taken, and it's tempting to just go ahead.
> 
> 
> However, doing so has the potential to quietly break peoples restore
> scripts,
> and they will find out at the worst possible time.
> To add insult to injury, these things are usually done once and then
> people
> 
> forget about them until they are really needed.
> 
> 
> I'd suggest a slightly different approach:
>   - add a switch like save-db-name="true|false" (help with naming much
> appreciated)
>   - if present, we do the obvious; if absent, we maintain current
> semantics.
> 
> 
> Messier, but safer. Thoughts?
> 
> 
> 
> 
> 
> 
> 
> On Fri, Oct 17, 2014 at 7:30 PM, Frank Crawford
> <fr...@cr...> wrote:
> 
>         Folks,
>         
>         What do others think?  As Marco says, if we remove the option
>         as the
>         default, others can add it in, if required, through the
>         configuration
>         file.  As it stands at present, you cannot remove the default
>         options,
>         only extend them.
>         
>         Frank
>         
>         On Fri, 2014-10-17 at 15:07 +0200, Marco Bozzolan wrote:
>         > Hello,
>         >
>         >   I am writing to propose the removal of the '-C' option
>         passed to
>         > 'pg_dump' (which results in the generation of SQL
>         instructions to
>         > create the database), because:
>         >
>         > * the Safekeep behaviour seems different for mysqldump,
>         where the
>         > CREATE DATABASE instruction is generated only if the
>         --databases or
>         > --all-databases option is given;
>         >
>         > * when using the plain-text output format, the resulting
>         database dump
>         > can be reimported with a different name on the same (or a
>         different)
>         > server with no need to edit the - sometimes very large -
>         dump itself.
>         >
>         >   I see this could break compatibility with previous
>         versions of Safekeep,
>         > but on the other hand the '-C' option could be easily
>         re-added by using
>         > the 'options' XML attribute of the <dump> element.
>         >
>         >   In case my proposal is accepted I'll be happy to provide a
>         pull
>         > request to https://github.com/dimipaun/safekeep/ .
>         >
>         > Best regards,
>         >
>         
>         
>         
>         ------------------------------------------------------------------------------
>         Comprehensive Server Monitoring with Site24x7.
>         Monitor 10 servers for $9/Month.
>         Get alerted through email, SMS, voice calls or mobile push
>         notifications.
>         Take corrective actions from your mobile device.
>         http://p.sf.net/sfu/Zoho
>         _______________________________________________
>         Safekeep-devel mailing list
>         Saf...@li...
>         https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>         
> 
> 
> 
> 
> 
> -- 
> Dimi Paun <di...@la...>
> Lattica, Inc.
> 
> 
> 
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Dimi P. <di...@la...>]

Sorry for the delay, I needed some time to think about this one.

Marco's point is well taken, and it's tempting to just go ahead.

However, doing so has the potential to quietly break peoples restore
scripts,
and they will find out at the worst possible time.
To add insult to injury, these things are usually done once and then people
forget about them until they are really needed.

I'd suggest a slightly different approach:
  - add a switch like save-db-name="true|false" (help with naming much
appreciated)
  - if present, we do the obvious; if absent, we maintain current semantics.

Messier, but safer. Thoughts?




On Fri, Oct 17, 2014 at 7:30 PM, Frank Crawford <fr...@cr...>
wrote:

> Folks,
>
> What do others think?  As Marco says, if we remove the option as the
> default, others can add it in, if required, through the configuration
> file.  As it stands at present, you cannot remove the default options,
> only extend them.
>
> Frank
>
> On Fri, 2014-10-17 at 15:07 +0200, Marco Bozzolan wrote:
> > Hello,
> >
> >   I am writing to propose the removal of the '-C' option passed to
> > 'pg_dump' (which results in the generation of SQL instructions to
> > create the database), because:
> >
> > * the Safekeep behaviour seems different for mysqldump, where the
> > CREATE DATABASE instruction is generated only if the --databases or
> > --all-databases option is given;
> >
> > * when using the plain-text output format, the resulting database dump
> > can be reimported with a different name on the same (or a different)
> > server with no need to edit the - sometimes very large - dump itself.
> >
> >   I see this could break compatibility with previous versions of
> Safekeep,
> > but on the other hand the '-C' option could be easily re-added by using
> > the 'options' XML attribute of the <dump> element.
> >
> >   In case my proposal is accepted I'll be happy to provide a pull
> > request to https://github.com/dimipaun/safekeep/ .
> >
> > Best regards,
> >
>
>
>
>
> ------------------------------------------------------------------------------
> Comprehensive Server Monitoring with Site24x7.
> Monitor 10 servers for $9/Month.
> Get alerted through email, SMS, voice calls or mobile push notifications.
> Take corrective actions from your mobile device.
> http://p.sf.net/sfu/Zoho
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>



-- 
Dimi Paun <di...@la...>
Lattica, Inc.

Re: [Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Frank C. <fr...@cr...>]

Folks,

What do others think?  As Marco says, if we remove the option as the
default, others can add it in, if required, through the configuration
file.  As it stands at present, you cannot remove the default options,
only extend them.

Frank

On Fri, 2014-10-17 at 15:07 +0200, Marco Bozzolan wrote: 
> Hello,
> 
>   I am writing to propose the removal of the '-C' option passed to 
> 'pg_dump' (which results in the generation of SQL instructions to 
> create the database), because:
> 
> * the Safekeep behaviour seems different for mysqldump, where the 
> CREATE DATABASE instruction is generated only if the --databases or 
> --all-databases option is given;
> 
> * when using the plain-text output format, the resulting database dump 
> can be reimported with a different name on the same (or a different) 
> server with no need to edit the - sometimes very large - dump itself.
> 
>   I see this could break compatibility with previous versions of Safekeep, 
> but on the other hand the '-C' option could be easily re-added by using 
> the 'options' XML attribute of the <dump> element.
> 
>   In case my proposal is accepted I'll be happy to provide a pull 
> request to https://github.com/dimipaun/safekeep/ .
> 
> Best regards,
> 

[Safekeep-devel] Proposal: removal of '-C' pg_dump option

[Marco B. <ma...@s1...>]

Hello,

  I am writing to propose the removal of the '-C' option passed to 
'pg_dump' (which results in the generation of SQL instructions to 
create the database), because:

* the Safekeep behaviour seems different for mysqldump, where the 
CREATE DATABASE instruction is generated only if the --databases or 
--all-databases option is given;

* when using the plain-text output format, the resulting database dump 
can be reimported with a different name on the same (or a different) 
server with no need to edit the - sometimes very large - dump itself.

  I see this could break compatibility with previous versions of Safekeep, 
but on the other hand the '-C' option could be easily re-added by using 
the 'options' XML attribute of the <dump> element.

  In case my proposal is accepted I'll be happy to provide a pull 
request to https://github.com/dimipaun/safekeep/ .

Best regards,

-- 
https://s19n.net

Re: [Safekeep-devel] sort order of client IDs

[Frank C. <fr...@cr...>]

Carl,

Thanks for your comments and plan to assist.  I've often considered
sorting the list.

I'd be interested in what others feel, but my personal opinion would be
to NOT make it configurable, as it is just an extra option for no reason
and secondly,  just sort within each directory.  That would allow users
to still partly control the order through `cfglocs' (which I don't think
often has more than one entry anyway).

BTW, with our recent change to github, I seem to be a bit behind with
applying "recent" patches.  I will get them on the new repository soon.

Regards
Frank

On Mon, 2014-07-07 at 11:17 +0100, Carl Franks wrote: 

> Hi,
> I have a requirement that some hosts are backed up in a known order.
> 
> Currently, hosts are backed up based on the order returned by 
> `os.listdir` which is documented as returning an "arbitrary order".
> I'd like the hosts to be sorted by the client.config filename.
> 
> If I submit a patch for this, should I simply change the current 
> behaviour, or should I create a new safekeep.conf option to 
> enable/disable sorting?
> If we want a config option, should the default behaviour be to retain 
> the current unsorted order, or should the default change to sorted?
> 
> There are 2 significantly different ways of sorting the filenames.
> Currently, there is an outer loop iterating the `cfglocs` list of 
> directories, and an inner loop iterating the *.config files in that 
> directory.
> 
> Option 1 would be to add a .sort() to the inner loop.
> This would retain the order of directories, and only sort the files in 
> each directory.
> 
> Option 2 would be to use a custom sort routine after the outer loop has 
> ended, sorting by the basename of each config filepath.
> This would effectively ignore the directory of each file in the order.
> 
> It looks like the --conf opt is deprecated, so `cfglocs` will generally 
> only contain a single directory anyway, so option 2 might be an 
> unnecessary complication.
> 
> Background:
> My reason for this requirement is an application whose hot-backup 
> process is:
> 1) Backup lucene-backup directory
> 2) Backup database
> 3) Backup data directory
> 
> The simplest way of doing this with safekeep would appear to be using 2 
> client config files:
> host1
>    data - lucene-backup
> host2
>    setup - db
>    data - data-directory
> 
> Cheers,
> Carl
> 
> 
> 
> ------------------------------------------------------------------------------
> Open source business process management suite built on Java and Eclipse
> Turn processes into business applications with Bonita BPM Community Edition
> Quickly connect people, data, and systems into organized workflows
> Winner of BOSSIE, CODIE, OW2 and Gartner awards
> http://p.sf.net/sfu/Bonitasoft
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
> 

[Safekeep-devel] sort order of client IDs

[Carl F. <ca...@fi...>]

Hi,
I have a requirement that some hosts are backed up in a known order.

Currently, hosts are backed up based on the order returned by 
`os.listdir` which is documented as returning an "arbitrary order".
I'd like the hosts to be sorted by the client.config filename.

If I submit a patch for this, should I simply change the current 
behaviour, or should I create a new safekeep.conf option to 
enable/disable sorting?
If we want a config option, should the default behaviour be to retain 
the current unsorted order, or should the default change to sorted?

There are 2 significantly different ways of sorting the filenames.
Currently, there is an outer loop iterating the `cfglocs` list of 
directories, and an inner loop iterating the *.config files in that 
directory.

Option 1 would be to add a .sort() to the inner loop.
This would retain the order of directories, and only sort the files in 
each directory.

Option 2 would be to use a custom sort routine after the outer loop has 
ended, sorting by the basename of each config filepath.
This would effectively ignore the directory of each file in the order.

It looks like the --conf opt is deprecated, so `cfglocs` will generally 
only contain a single directory anyway, so option 2 might be an 
unnecessary complication.

Background:
My reason for this requirement is an application whose hot-backup 
process is:
1) Backup lucene-backup directory
2) Backup database
3) Backup data directory

The simplest way of doing this with safekeep would appear to be using 2 
client config files:
host1
   data - lucene-backup
host2
   setup - db
   data - data-directory

Cheers,
Carl

Re: [Safekeep-devel] safekeep-server script hooks (for server scripting - not client scripting)?

[Andres T. <an...@op...>]

Hi folks,

We are planning to implement server side scripting support with the 
following details:
* adding the "run-on" config parameter to the script element - like 
|<script path="/path/to/script" run-on="server" />|
* "run-on" value should default to client (so the logic will stay the 
same - as backward compability should be preferred)
* "run-on" values can be client or server

Any other suggestions?

Cheers,
Andres
> Dimi Paun <mailto:di...@la...>
> 5. mai 2014 23:35
> I'm fine without the "auto" mode, I agree.
>
>
>
>
>
>
> -- 
> Dimi Paun <di...@la... <mailto:di...@la...>>
> Lattica, Inc.
>
> Andres Toomsalu <mailto:an...@op...>
> 4. mai 2014 21:46
>
> My 2 cents on this:
>> Dimi Paun <mailto:di...@la...>
>> 4. mai 2014 16:44
>> Any suggestions for the syntax? Do you see a pb with my suggestion 
>> (e.g. run-on="server")?
>>
>> My suggestion would be:
>>    - run-on can be either "client" or "server" or "auto"
> Run-on itself is ok - but I would suggest not to introduce "auto" 
> value here - as it can be confusing to safekeep users - what is "auto" 
> and what is exactly its logic (without reading the manual - that 
> nobody does these days) :-)
>>    - if not specified it is "auto"
> I would prefer that the default is "client" - that would be simple and 
> clear and how its use-to-be.
>>    - if auto, we look at the script file path
>>      - if the filepath exists on the server, we assume "server"
>>      - otherwise we assume "client"
>>
>> This scheme has a small chance of backwards incompatibility, but 
>> given it's a rarely used feature
>> I think we're fine.
>>
>> Thoughts?
>>
>> Dimi.
>>
>>
>>
>>
>>
>>
>>
>> -- 
>> Dimi Paun <di...@la... <mailto:di...@la...>>
>> Lattica, Inc.
>>
>> ------------------------------------------------------------------------------
>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>> Instantly run your Selenium tests across 300+ browser/OS combos. Get
>> unparalleled scalability from the best Selenium testing platform 
>> available.
>> Simple to use. Nothing to install. Get started now for free."
>> http://p.sf.net/sfu/SauceLabs
>> _______________________________________________
>> Safekeep-devel mailing list
>> Saf...@li...
>> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>> Frank Crawford <mailto:fr...@cr...>
>> 4. mai 2014 11:36
>> Dimi,
>>
>> Yep, although it never really got further than a concept.  I don't 
>> think it is hard to add, but the syntax needs to be better defined.
>>
>> Regards
>> Frank
>>
>> On Sun, 2014-05-04 at 02:32 -0400, Dimi Paun wrote:
>>
>> ------------------------------------------------------------------------------
>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>> Instantly run your Selenium tests across 300+ browser/OS combos. Get
>> unparalleled scalability from the best Selenium testing platform 
>> available.
>> Simple to use. Nothing to install. Get started now for free."
>> http://p.sf.net/sfu/SauceLabs
>> _______________________________________________
>> Safekeep-devel mailing list
>> Saf...@li...
>> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>> Dimi Paun <mailto:di...@la...>
>> 4. mai 2014 10:32
>> You're right, I was the one confused -- my bad Andres!
>> Your recollection is 100% correct: the server-side scripts were left 
>> as a TODO,
>> as there was no clear need for them at the time.
>>
>> IIRC, the plan was to add a flag to the <script> tag, something like 
>> run-on="server".
>>
>>
>>
>>
>>
>>
>> -- 
>> Dimi Paun <di...@la... <mailto:di...@la...>>
>> Lattica, Inc.
>>
>> ------------------------------------------------------------------------------
>> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
>> Instantly run your Selenium tests across 300+ browser/OS combos. Get
>> unparalleled scalability from the best Selenium testing platform 
>> available.
>> Simple to use. Nothing to install. Get started now for free."
>> http://p.sf.net/sfu/SauceLabs
>> _______________________________________________
>> Safekeep-devel mailing list
>> Saf...@li...
>> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>> Frank Crawford <mailto:fr...@cr...>
>> 4. mai 2014 10:26
>> However, that script still runs on the client, it is just the 
>> location for the script likes on the server and is passed across.
>>
>> I think Andres is after scripts running on the server during the backup.
>>
>> Regards
>> Frank
>>
>> On Tue, 2014-04-29 at 01:38 -0400, Dimi Paun wrote:
>>
>> Dimi Paun <mailto:di...@la...>
>> 29. aprill 2014 9:38
>> Andres,
>>
>> I think we already have that. From safekeep.backup(5) 
>> <http://safekeep.sourceforge.net/safekeep.backup.html>:
>>
>>
>> |<!-- location of a script to be executed at different stages of the 
>> run -->
>> <script path="server:/path/to/script" />
>> |
>> (note the "server:" prefix), the docs say:
>>
>> /backup/setup/script/@path
>>
>>     Execute the script specified path on the client at certain steps
>>     of the backup process. This script is executed with three arguments:
>>
>>      *
>>
>>         Backup id (/backup/@id)
>>
>>      *
>>
>>         Backup step
>>
>>      *
>>
>>         Backup root directory (valid after creation of a snapshot)
>>         The |location| optionally consists of an optional |host| and
>>         a mandatory |path|, separated by a ":", where the host part
>>         is either "client" or "server". If no host part is specified
>>         then it is first looked for on the client, and if not found,
>>         then is looked for on the server. If it not found on either,
>>         then a warning is raised. See the |CLIENT SCRIPT| section for
>>         more information. Mandatory for a |<script>| element.
>>
>> |
>> |
>>
>>
>>
>>
>>
>> -- 
>> Dimi Paun <di...@la... <mailto:di...@la...>>
>> Lattica, Inc.
>>
>
> Dimi Paun <mailto:di...@la...>
> 4. mai 2014 16:44
> Any suggestions for the syntax? Do you see a pb with my suggestion 
> (e.g. run-on="server")?
>
> My suggestion would be:
>    - run-on can be either "client" or "server" or "auto"
>    - if not specified it is "auto"
>    - if auto, we look at the script file path
>      - if the filepath exists on the server, we assume "server"
>      - otherwise we assume "client"
>
> This scheme has a small chance of backwards incompatibility, but given 
> it's a rarely used feature
> I think we're fine.
>
> Thoughts?
>
> Dimi.
>
>
>
>
>
>
>
> -- 
> Dimi Paun <di...@la... <mailto:di...@la...>>
> Lattica, Inc.
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos. Get
> unparalleled scalability from the best Selenium testing platform 
> available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
> Frank Crawford <mailto:fr...@cr...>
> 4. mai 2014 11:36
> Dimi,
>
> Yep, although it never really got further than a concept.  I don't 
> think it is hard to add, but the syntax needs to be better defined.
>
> Regards
> Frank
>
> On Sun, 2014-05-04 at 02:32 -0400, Dimi Paun wrote:
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos. Get
> unparalleled scalability from the best Selenium testing platform 
> available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
> Dimi Paun <mailto:di...@la...>
> 4. mai 2014 10:32
> You're right, I was the one confused -- my bad Andres!
> Your recollection is 100% correct: the server-side scripts were left 
> as a TODO,
> as there was no clear need for them at the time.
>
> IIRC, the plan was to add a flag to the <script> tag, something like 
> run-on="server".
>
>
>
>
>
>
> -- 
> Dimi Paun <di...@la... <mailto:di...@la...>>
> Lattica, Inc.
>
> ------------------------------------------------------------------------------
> "Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
> Instantly run your Selenium tests across 300+ browser/OS combos. Get
> unparalleled scalability from the best Selenium testing platform 
> available.
> Simple to use. Nothing to install. Get started now for free."
> http://p.sf.net/sfu/SauceLabs
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel

Re: [Safekeep-devel] Failed backups

[Dimi P. <di...@la...>]

Hmmm, I get this:

DBG: Do server main loop
INFO: ------------------------------------------------------------------
INFO: Server backup starting for client ajax
dbg: Do client main loop
dbg: Server versions:  1.2, 1.4.0
dbg: Do client_side_script: step PRE-SETUP
dbg: Do setup of ajax.lattica.com
dbg: Doing DB dumps
dbg: Run [ 'pg_dumpall'  '-U'  'postgres' >  '/var/lib/pgsql/backups/all_
dbs']
ERR: Server backup for client ajax: FAILED
Traceback (most recent call last):
  File "/usr/bin/safekeep", line 1294, in do_server
    bdir = do_server_getanswer(cout)
  File "/usr/bin/safekeep", line 1085, in do_server_getanswer
    raise Exception('client died unexpectedly')
Exception: client died unexpectedly

It's a bit confusing as I have no script for this host:

  <setup>
    <dump type="postgres" dbuser="postgres"
file="/var/lib/pgsql/backups/all_dbs" />
    <dump type="mysql" dbuser="root" dbpasswd="xxxxxx"
file="/var/lib/mysql/backup/ajax.sql" cleanup="true" />
  </setup>

Also, I thought we wanted to issue a warning if the dump fails, not fail
outright, no?




On Tue, May 6, 2014 at 10:08 AM, Dimi Paun <di...@la...> wrote:

> Good point, I'll give it a try now.
>
>
>
> On Tue, May 6, 2014 at 7:58 AM, Frank Crawford <fr...@cr...>wrote:
>
>> Dimi,
>>
>> Have you tried turning up the verbosity of the output?
>>
>> Frank
>>
>> On Mon, 2014-05-05 at 09:44 -0400, Dimi Paun wrote:
>> > Hi folks,
>> >
>> >
>> > Every now and then I see this in the logs:
>> >
>> >
>> > INFO: Server backup starting for client ajax
>> > ERR: Server backup for client ajax: FAILED
>> >
>> >
>> >
>> > A day or two later, it works OK.
>> > Mind you, "ajax" is running for sure, so it's not like the host is
>> > down/
>> >
>> >
>> > Do you guys see this at all?
>> > Any idea what can cause it?
>> >
>> >
>> > --
>> > Dimi Paun <di...@la...>
>> > Lattica, Inc.
>> >
>> >
>> >
>> ------------------------------------------------------------------------------
>> > Is your legacy SCM system holding you back? Join Perforce May 7 to find
>> out:
>> > &#149; 3 signs your SCM is hindering your productivity
>> > &#149; Requirements for releasing software faster
>> > &#149; Expert tips and advice for migrating your SCM now
>> > http://p.sf.net/sfu/perforce
>> > _______________________________________________
>> > Safekeep-devel mailing list
>> > Saf...@li...
>> > https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>>
>>
>>
>>
>> ------------------------------------------------------------------------------
>> Is your legacy SCM system holding you back? Join Perforce May 7 to find
>> out:
>> &#149; 3 signs your SCM is hindering your productivity
>> &#149; Requirements for releasing software faster
>> &#149; Expert tips and advice for migrating your SCM now
>> http://p.sf.net/sfu/perforce
>> _______________________________________________
>> Safekeep-devel mailing list
>> Saf...@li...
>> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>>
>
>
>
> --
> Dimi Paun <di...@la...>
> Lattica, Inc.
>
>


-- 
Dimi Paun <di...@la...>
Lattica, Inc.

Re: [Safekeep-devel] Failed backups

[Dimi P. <di...@la...>]

Good point, I'll give it a try now.



On Tue, May 6, 2014 at 7:58 AM, Frank Crawford <fr...@cr...>wrote:

> Dimi,
>
> Have you tried turning up the verbosity of the output?
>
> Frank
>
> On Mon, 2014-05-05 at 09:44 -0400, Dimi Paun wrote:
> > Hi folks,
> >
> >
> > Every now and then I see this in the logs:
> >
> >
> > INFO: Server backup starting for client ajax
> > ERR: Server backup for client ajax: FAILED
> >
> >
> >
> > A day or two later, it works OK.
> > Mind you, "ajax" is running for sure, so it's not like the host is
> > down/
> >
> >
> > Do you guys see this at all?
> > Any idea what can cause it?
> >
> >
> > --
> > Dimi Paun <di...@la...>
> > Lattica, Inc.
> >
> >
> >
> ------------------------------------------------------------------------------
> > Is your legacy SCM system holding you back? Join Perforce May 7 to find
> out:
> > &#149; 3 signs your SCM is hindering your productivity
> > &#149; Requirements for releasing software faster
> > &#149; Expert tips and advice for migrating your SCM now
> > http://p.sf.net/sfu/perforce
> > _______________________________________________
> > Safekeep-devel mailing list
> > Saf...@li...
> > https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>
>
>
>
> ------------------------------------------------------------------------------
> Is your legacy SCM system holding you back? Join Perforce May 7 to find
> out:
> &#149; 3 signs your SCM is hindering your productivity
> &#149; Requirements for releasing software faster
> &#149; Expert tips and advice for migrating your SCM now
> http://p.sf.net/sfu/perforce
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel
>



-- 
Dimi Paun <di...@la...>
Lattica, Inc.

Re: [Safekeep-devel] Failed backups

[Frank C. <fr...@cr...>]

Dimi,

Have you tried turning up the verbosity of the output?

Frank

On Mon, 2014-05-05 at 09:44 -0400, Dimi Paun wrote: 
> Hi folks,
> 
> 
> Every now and then I see this in the logs:
> 
> 
> INFO: Server backup starting for client ajax
> ERR: Server backup for client ajax: FAILED
> 
> 
> 
> A day or two later, it works OK.
> Mind you, "ajax" is running for sure, so it's not like the host is
> down/
> 
> 
> Do you guys see this at all? 
> Any idea what can cause it?
> 
> 
> -- 
> Dimi Paun <di...@la...>
> Lattica, Inc.
> 
> 
> ------------------------------------------------------------------------------
> Is your legacy SCM system holding you back? Join Perforce May 7 to find out:
> &#149; 3 signs your SCM is hindering your productivity
> &#149; Requirements for releasing software faster
> &#149; Expert tips and advice for migrating your SCM now
> http://p.sf.net/sfu/perforce
> _______________________________________________
> Safekeep-devel mailing list
> Saf...@li...
> https://lists.sourceforge.net/lists/listinfo/safekeep-devel

[Safekeep-devel] Failed backups

[Dimi P. <di...@la...>]

Hi folks,

Every now and then I see this in the logs:

INFO: Server backup starting for client ajax
ERR: Server backup for client ajax: FAILED

A day or two later, it works OK.
Mind you, "ajax" is running for sure, so it's not like the host is down/

Do you guys see this at all?
Any idea what can cause it?

-- 
Dimi Paun <di...@la...>
Lattica, Inc.