Thread: [Dar-libdar_api] problem compiling KDar with libdar CVS
For full, incremental, compressed and encrypted backups or archives
Brought to you by:
edrusb
Menu
▾
▴
dar-libdar_api
|
From: Johnathan B. <jk...@us...> - 2004-10-30 07:03:11
|
Hi Denis,
I've decided it is time to port KDar over to the new libdar API3. Libdar=20
installed without obvious problems. However, KDar's configure script=20
chokes, and spits out the following log messages:
configure:32503: checking dar/libdar.hpp usability
configure:32515: g++ -c -Wnon-virtual-dtor -Wno-long-long -Wundef -Wall=20
=2Dpedantic
=2DW -Wpointer-arith -Wmissing-prototypes -Wwrite-strings -ansi=20
=2DD_XOPEN_SOURCE=3D500
=2DD_BSD_SOURCE -Wcast-align -Wconversion -DNDEBUG -DNO_DEBUG -O2 -O2 -pipe=
=20
=2Dmarch=3Da
thlon-xp -mmmx -msse -m3dnow -fno-exceptions -fno-check-new -fexceptions=20
=2Dfexcepti
ons -DQT_THREAD_SUPPORT -D_REENTRANT conftest.cc >&5
In file included from /usr/local/include/dar/storage.hpp:29,
from /usr/local/include/dar/real_infinint.hpp:38,
from /usr/local/include/dar/infinint.hpp:30,
from /usr/local/include/dar/compressor.hpp:30,
from /usr/local/include/dar/libdar.hpp:32,
from conftest.cc:148:
/usr/local/include/dar/erreurs.hpp: In constructor
`libdar::Ememory::Ememory(const std::string&)':
/usr/local/include/dar/erreurs.hpp:67: `gettext' undeclared (first use this
function)
/usr/local/include/dar/erreurs.hpp:67: (Each undeclared identifier is=20
reported only once for each function it appears in.)
configure:32521: $? =3D 1
I am not familiar with gettext, but it seems that the function should be=20
declared somewhere.
Any ideas?
Cheers,
JB
=2D-=20
Johnathan K. Burchill, Ph.D.
jk...@us...
|
|
From: Johnathan B. <jk...@sh...> - 2004-11-01 01:37:02
|
On Saturday 30 October 2004 01:00, Johnathan Burchill wrote: > Hi Denis, > > I've decided it is time to port KDar over to the new libdar API3. Libdar > installed without obvious problems. However, KDar's configure script > chokes, and spits out the following log messages: > > configure:32503: checking dar/libdar.hpp usability > configure:32515: g++ -c -Wnon-virtual-dtor -Wno-long-long -Wundef -Wall > -pedantic > -W -Wpointer-arith -Wmissing-prototypes -Wwrite-strings -ansi > -D_XOPEN_SOURCE=3D500 > -D_BSD_SOURCE -Wcast-align -Wconversion -DNDEBUG -DNO_DEBUG -O2 -O2 > -pipe -march=3Da > thlon-xp -mmmx -msse -m3dnow -fno-exceptions -fno-check-new -fexceptions > -fexcepti > ons -DQT_THREAD_SUPPORT -D_REENTRANT conftest.cc >&5 > In file included from /usr/local/include/dar/storage.hpp:29, > from /usr/local/include/dar/real_infinint.hpp:38, > from /usr/local/include/dar/infinint.hpp:30, > from /usr/local/include/dar/compressor.hpp:30, > from /usr/local/include/dar/libdar.hpp:32, > from conftest.cc:148: > /usr/local/include/dar/erreurs.hpp: In constructor > `libdar::Ememory::Ememory(const std::string&)': > /usr/local/include/dar/erreurs.hpp:67: `gettext' undeclared (first use > this function) > /usr/local/include/dar/erreurs.hpp:67: (Each undeclared identifier is > reported only once for each function it appears in.) > configure:32521: $? =3D 1 > > I am not familiar with gettext, but it seems that the function should be > declared somewhere. > > Any ideas? I've just upgraded to Slackware 10, and the problem disappeared. There=20 might have been a glibc version problem or mismatch somewhere. Cheers, JB > > Cheers, > JB =2D-=20 Johnathan K. Burchill, Ph.D. Department of Physics and Astronomy University of Calgary 2500 University Drive N.W. Calgary, AB T2N 1N4 Canada (403) 217-4286 jk...@sh... |
|
From: Denis C. <dar...@fr...> - 2004-11-01 11:42:48
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello Johnathan, There is two points I see about the API: - - first, since I have added multi-language support (the gettext function comes from here), I have not yet reviewed the general installation process, maybe there is some points to review about installed header files...(this will be done before pre-release phase, which target is december 2004). - - second, there is still changes to come in the API as discussed in this forum (see details bellow). I will look at this soon, as I am about to finish the implementation about the strong encryption features. Before pre-release phase, I will send a mail to this mailing list for an API review before release. I expect it before half november. Things to review in the API are : - - op_create & other functions must become methods of the archive class (for a more correct C++ design) - - what about thread cancelation ? Do we assume kdar will fork() rather than require a more complex mechanism to cancel a thread ? Cheers, Denis. Johnathan Burchill wrote: | On Saturday 30 October 2004 01:00, Johnathan Burchill wrote: | |>Hi Denis, |> |>I've decided it is time to port KDar over to the new libdar API3. Libdar |>installed without obvious problems. However, KDar's configure script |>chokes, and spits out the following log messages: |> |>configure:32503: checking dar/libdar.hpp usability |>configure:32515: g++ -c -Wnon-virtual-dtor -Wno-long-long -Wundef -Wall |>-pedantic |>-W -Wpointer-arith -Wmissing-prototypes -Wwrite-strings -ansi |>-D_XOPEN_SOURCE=500 |>-D_BSD_SOURCE -Wcast-align -Wconversion -DNDEBUG -DNO_DEBUG -O2 -O2 |>-pipe -march=a |>thlon-xp -mmmx -msse -m3dnow -fno-exceptions -fno-check-new -fexceptions |>-fexcepti |>ons -DQT_THREAD_SUPPORT -D_REENTRANT conftest.cc >&5 |>In file included from /usr/local/include/dar/storage.hpp:29, |> from /usr/local/include/dar/real_infinint.hpp:38, |> from /usr/local/include/dar/infinint.hpp:30, |> from /usr/local/include/dar/compressor.hpp:30, |> from /usr/local/include/dar/libdar.hpp:32, |> from conftest.cc:148: |>/usr/local/include/dar/erreurs.hpp: In constructor |> `libdar::Ememory::Ememory(const std::string&)': |>/usr/local/include/dar/erreurs.hpp:67: `gettext' undeclared (first use |>this function) |>/usr/local/include/dar/erreurs.hpp:67: (Each undeclared identifier is |>reported only once for each function it appears in.) |>configure:32521: $? = 1 |> |>I am not familiar with gettext, but it seems that the function should be |>declared somewhere. |> |>Any ideas? | | | I've just upgraded to Slackware 10, and the problem disappeared. There | might have been a glibc version problem or mismatch somewhere. | | Cheers, | JB | | |>Cheers, |>JB | | -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBhiHapC5CI8gYGlIRAtL8AJsFEvg12nous1lYzEVfZWkAgpWWCgCfUyGx VMPGVO1YnQWYccsMJjwjbl8= =aU4m -----END PGP SIGNATURE----- |
|
From: Denis C. <dar...@fr...> - 2004-11-01 18:45:47
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Denis Corbin wrote: | Hello Johnathan, | |[...] | | Things to review in the API are : | - op_create & other functions must become methods of the archive class | (for a more correct C++ design) this has been done today (CVS). Still needs documentation. | - what about thread cancelation ? Do we assume kdar will fork() rather | than require a more complex mechanism to cancel a thread ? If the thread cancelation does not implies any change, then we are very near the definitive API specifications for version 3 so we could "freeze" the API. Some last feature will only add a type in enumeration (LZO compression, RSA encryption, etc.), but class, function or methods definitions should not change at all. My current actions are the following in order of priority: - - documentation (review all in HTML format, API documentation first) - - finish strong encryption (password from interactive question, or from file, some few other strong encryptions algorithms) - - LZO compression if time allows it (all development stop at end of November for 2.2.0 pre-release phase) | | Cheers, | Denis. | -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBhoT9pC5CI8gYGlIRApa6AKCJAoCIASYsnjIon7/SolGXqbR9GgCeObOm 1b4v7uWQmtWL6ll7Y3m5PwE= =FtiH -----END PGP SIGNATURE----- |
|
From: Johnathan B. <jk...@sh...> - 2004-11-02 00:57:56
|
On Monday 01 November 2004 11:48, Denis Corbin wrote: > Denis Corbin wrote: > | Hello Johnathan, > | > |[...] > | > | Things to review in the API are : > | - op_create & other functions must become methods of the archive class > | (for a more correct C++ design) > > this has been done today (CVS). Still needs documentation. > > | - what about thread cancelation ? Do we assume kdar will fork() rather > | than require a more complex mechanism to cancel a thread ? One weekend a while back I attempted to get the fork() working, with no=20 success. There was some problem with forking a qt application and I did=20 not understand how to fix it. I'd much prefer to stick with the thread=20 cancellation technique. > > If the thread cancelation does not implies any change, then we are very It depends how you implement it. You once suggested a wrapper class that=20 polls for a signal and throws an Euser_abort() if there is a cancellation=20 request. What signal would it look for? Would it be independent of any of=20 the pthread functions? How would the application set the signal? Another way is to use cancellation points suitably located througout the=20 code (see "man pthread_cancel"). As far as I can tell, what is needed is=20 #include <pthread.h> and a bunch of pthread_testcancel()'s. You need so-called cancellation cleanup handlers for unlocking mutexes and= =20 freeing memory (see "man pthread_cleanup_push" and the examples therein). It may require quite a bit of work. Also, I don't know whether it is=20 directly portable to the cygwin version of libdar. > near the definitive API specifications for version 3 so we could > "freeze" the API. Some last feature will only add a type in enumeration > (LZO compression, RSA encryption, etc.), but class, function or methods > definitions should not change at all. Making the threads cancellable should not affect the API in any significant= =20 way. Since the various libdar operations are now archive methods, you=20 should have an "archive::cancel_operation()" method that toggles a boolean= =20 which triggers the "Euser_abort()" exception between the filesystem reads=20 or writes. Now that I'm beginning to understand your wrapper class idea, I rather like= =20 it. Cheers, JB > > My current actions are the following in order of priority: > - documentation (review all in HTML format, API documentation first) > - finish strong encryption (password from interactive question, or from > file, some few other strong encryptions algorithms) > - LZO compression if time allows it (all development stop at end of > November for 2.2.0 pre-release phase) > > | Cheers, > | Denis. > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=3D5588&alloc_id=3D12065&op=3Dclick > _______________________________________________ > Dar-libdar_api mailing list > Dar...@li... > https://lists.sourceforge.net/lists/listinfo/dar-libdar_api =2D-=20 Johnathan K. Burchill, Ph.D. Department of Physics and Astronomy University of Calgary 2500 University Drive N.W. Calgary, AB T2N 1N4 Canada (403) 217-4286 jk...@sh... |
|
From: Denis C. <dar...@fr...> - 2004-11-02 23:04:08
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Johnathan Burchill wrote: | On Monday 01 November 2004 11:48, Denis Corbin wrote: | |>Denis Corbin wrote: |>| Hello Johnathan, |>| |>|[...] |>| |>| Things to review in the API are : |>| - op_create & other functions must become methods of the archive class |>| (for a more correct C++ design) |> |>this has been done today (CVS). Still needs documentation. |> |>| - what about thread cancelation ? Do we assume kdar will fork() rather |>| than require a more complex mechanism to cancel a thread ? | | | One weekend a while back I attempted to get the fork() working, with no | success. There was some problem with forking a qt application and I did | not understand how to fix it. I'd much prefer to stick with the thread | cancellation technique. | OK | |>If the thread cancelation does not implies any change, then we are very | | | It depends how you implement it. You once suggested a wrapper class that | polls for a signal and throws an Euser_abort() if there is a cancellation | request. What signal would it look for? Would it be independent of any of | the pthread functions? How would the application set the signal? the implementation I am working on does not rely on Unix signaling. | | Another way is to use cancellation points suitably located througout the | code (see "man pthread_cancel"). As far as I can tell, what is needed is | #include <pthread.h> and a bunch of pthread_testcancel()'s. | You need so-called cancellation cleanup handlers for unlocking mutexes and | freeing memory (see "man pthread_cleanup_push" and the examples therein). | | It may require quite a bit of work. Also, I don't know whether it is | directly portable to the cygwin version of libdar. where there will not be pthread mutex available there will not be thread cancelation available (and probably the system will not support multi-threading neither). All is activated from the configuration script. | | |>near the definitive API specifications for version 3 so we could |>"freeze" the API. Some last feature will only add a type in enumeration |>(LZO compression, RSA encryption, etc.), but class, function or methods |>definitions should not change at all. | | | Making the threads cancellable should not affect the API in any significant | way. Since the various libdar operations are now archive methods, you | should have an "archive::cancel_operation()" method that toggles a boolean | which triggers the "Euser_abort()" exception between the filesystem reads | or writes. This is not possible this way, I proposes the following API : ~ void cancel_thread(pthread_t tid); ~ bool cancel_current(pthread_t & tid); ~ void cancel_clear(); where cancel_thread() is used to cancel the given thread. Which will make libdar throw a Euser_abort exception, cleanly free memory and release any mutex. cancel_current() return true and the tid of the thread that will be canceled when execution will reach the next checkpoint, false if there is not pending cancelation. cancel_clear() abort cancelation process if the thread has not already terminated. Only one thread can be canceled at a time. | | Now that I'm beginning to understand your wrapper class idea, I rather like | it. ;-) The mechanisme is the following. When cancel_thread is called, it put the requested tid in memory of a thread_cancelation object. Some key classes (sar, tuyau, fichier and null_file) now inherit from the thread_cancelation class. Thus they dispose of a method to check if the thread Id (tid) they are run from has not been designed for cancelation. If so the method throws the Euser_abort execption, which has the final consequence of cleanly returning from the current libdar call. the sar, tuyau fichier and null_file are the one used as low layer objects in archive operation. The thread_cancelation checkpoint is put in their read and write functions. To not overload the checkpoint process I have limited the implementation to have a simple check. The drawback is that only one thread can be canceled at a time (need to wait for the cancelation has complete to as for a new cancelation). Implementation is finished, I am testing it, and should upgrade CVS tomorrow if all is fine. | | Cheers, | JB | Cheers, Denis. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBiBDWpC5CI8gYGlIRAsCVAJ44wNiSKbn6rq4f5btWK0oLvW0JCACggSTi m1PO5FMg5IlbLC9T4jjNPNE= =lukb -----END PGP SIGNATURE----- |
|
From: Denis C. <dar...@fr...> - 2004-11-03 20:14:40
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Denis Corbin wrote: |[...] | I proposes the following API : | | ~ void cancel_thread(pthread_t tid); | ~ bool cancel_current(pthread_t & tid); | ~ void cancel_clear(); | | where cancel_thread() is used to cancel the given thread. Which will | make libdar throw a Euser_abort exception, cleanly free memory and | release any mutex. | | cancel_current() return true and the tid of the thread that will be | canceled when execution will reach the next checkpoint, false if there | is not pending cancelation. | | cancel_clear() abort cancelation process if the thread has not already | terminated. | | Only one thread can be canceled at a time. | | |[...] Hello, This is implemented and tested. CVS is up to date. If you have no objections, The API is now frozen (version 3) for the release 2.2.0 that will come in december I hope. The points that remains to do before the pre-release phase are : - - documentation (API, user guide, etc.) - - password/key from file - - password/key from keyboard out of command-line - - some few other strong encryption algorithms - - LZO compression the two password/key features do not have any impact on the API (these are command-line related feature). The LZO and the new cypher support only add some more values in the "enum compression" and "enum crypto_algo" enumeration types. The pre-release target date is the beginning of december. Until that time, it is still time to fix API problem you could meet, so please give your feedback before pre-release phase. For short about API, waiting for an up to date documentation, all calls you need are in libdar.hpp and archive.hpp. The comments in the theses header files should help you understand the changes brought from API 2. Cheers, Denis. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBiTqjpC5CI8gYGlIRAsV2AJ9wlNk64pXl0w8lOxRcAH2bqIUYLgCfQW7O JlONqEWXWWBxFeg7hyKUnj8= =a07r -----END PGP SIGNATURE----- |
|
From: Johnathan B. <jk...@sh...> - 2004-11-03 22:22:15
|
On Wednesday 03 November 2004 13:08, Denis Corbin wrote: > Denis Corbin wrote: > |[...] > | I proposes the following API : > | > | ~ void cancel_thread(pthread_t tid); > | ~ bool cancel_current(pthread_t & tid); > | ~ void cancel_clear(); > | > | where cancel_thread() is used to cancel the given thread. Which will > | make libdar throw a Euser_abort exception, cleanly free memory and > | release any mutex. > | > | cancel_current() return true and the tid of the thread that will be > | canceled when execution will reach the next checkpoint, false if there > | is not pending cancelation. > | > | cancel_clear() abort cancelation process if the thread has not already > | terminated. > | > | Only one thread can be canceled at a time. > | > | > |[...] > > Hello, > > This is implemented and tested. CVS is up to date. If you have no > objections, The API is now frozen (version 3) for the release 2.2.0 that > will come in december I hope. The points that remains to do before the > pre-release phase are : > - documentation (API, user guide, etc.) > - password/key from file > - password/key from keyboard out of command-line > - some few other strong encryption algorithms > - LZO compression > > the two password/key features do not have any impact on the API (these > are command-line related feature). The LZO and the new cypher support > only add some more values in the "enum compression" and "enum > crypto_algo" enumeration types. As far as I can tell from the latest cvs API, when I read an existing archive with the archive "read" constructor, I need to give it a "const std::string &pass" argument, i.e. the password. So it seems that the application has to know beforehand whether the user is opening an encrypted archive or an unencrypted archive. I see that archive::archive calls "macro_tools_open_archive" which in turn calls dialog.warning(...) if the archive is encrypted and no password is given. Perhaps if "macro_tools_open_archive" could throw an Epassword() exception, that could be caught by the application, which could transparently ask the user for a password and try the read again. Or, is there a way in the API to check the archive, before opening it, to see whether it is encrypted? Cheers, JB > > The pre-release target date is the beginning of december. Until that > time, it is still time to fix API problem you could meet, so please give > your feedback before pre-release phase. > > For short about API, waiting for an up to date documentation, all calls > you need are in libdar.hpp and archive.hpp. The comments in the theses > header files should help you understand the changes brought from API 2. > > Cheers, > Denis. > > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Dar-libdar_api mailing list > Dar...@li... > https://lists.sourceforge.net/lists/listinfo/dar-libdar_api -- Johnathan K. Burchill, Ph.D. Department of Physics and Astronomy University of Calgary 2500 University Drive N.W. Calgary, AB T2N 1N4 Canada (403) 217-4286 jk...@sh... |
|
From: Denis C. <dar...@fr...> - 2004-11-04 09:33:39
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Johnathan Burchill wrote: Hello Johnathan, | On Wednesday 03 November 2004 13:08, Denis Corbin wrote: | |>Denis Corbin wrote: |>|[...] |[...] | As far as I can tell from the latest cvs API, when I read an existing | archive with the archive "read" constructor, I need to give it a "const | std::string &pass" argument, i.e. the password. So it seems that the | application has to know beforehand whether the user is opening an | encrypted archive or an unencrypted archive. In my point of view, the user has to know if the archive he tends to open is encrypted or not and if so it has to know too the password. The pass argument may be an empty string, or any other value, the given value will only be used if the archive has a flag set in the header of the archive telling that the archive has been encrypted (no information about the algorithm used or the block size used --- or the password of course --- is present in the header). Note that the pass has the structure described in the dar man page (option -K), "[algo:]pass" . where algo is bf, blowfish, scram or scrambling (more will follow I hope before the end of november) with "bf" = "blowfish", "scram" = "scrambling". If the password must contain a ':' the algo must be specified (in the form <algo>:<password_with_column>), so you can safely always specify the cypher used. A warning is issued if an encrypted archive is openned, because the diagnostic of a wrong password, is not easy to diagnostic neither for the user nor for libdar. | | I see that archive::archive calls "macro_tools_open_archive" which in turn | calls dialog.warning(...) if the archive is encrypted and no password is | given. Perhaps if "macro_tools_open_archive" could throw an Epassword() | exception, that could be caught by the application, which could | transparently ask the user for a password and try the read again. I like the idea, but throwing an exception is not a reversible operation ~ from the stack point of view (you cannot continue execution where it has been thrown). I suppose you mean that libdar should rather use a the user_interaction::pause(...) method to ask the password to the user, it the pass argument was something like "bf:" or "scram:" where just the algorithm (the cypher) is given ? This is what I was looking for, to implement the password out of command-line for dar. Libdar could interactively ask the user for password when the user gives "-K bf:" or maybe just "-K :" where no password is given and the cypher is blowfish or where nothing is given at all, so the default cypher (bf actually) is used and no password is used. Thanks for the idea ! :-) | | Or, is there a way in the API to check the archive, before opening it, to | see whether it is encrypted? well, it may have implications for the user if openning the archive once just to see if the archive is encrypted and a second time to open it for real. Implications can be extra change of disk for example. | | Cheers, | JB | | Cheers, Denis. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBifXmpC5CI8gYGlIRArVsAJ9cIMUSRMbDISUFOTKf0oFFAbrCXgCePP/4 8u5/JizTPNvTX56PZM/OrVA= =+UQ4 -----END PGP SIGNATURE----- |
|
From: Denis C. <dar...@fr...> - 2004-11-04 10:27:53
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Denis Corbin wrote: |[...] I made a confusion between the API and the command-line syntax. Sorry. The API has three different arguments : ~ crypto_algo crypto, ~ const std::string & pass, ~ U_32 crypto_size, the pass and crypto_size arguments are used only if crypto is different than crypto_none | Note that the pass has the | structure described in the dar man page (option -K), "[algo:]pass" . | where algo is bf, blowfish, scram or scrambling (more will follow I hope | before the end of november) with "bf" = "blowfish", "scram" = | "scrambling". If the password must contain a ':' the algo must be | specified (in the form <algo>:<password_with_column>), so you can safely | always specify the cypher used. | |[...] here too below, I made the same confusion. To correct the idea, if pass is an empty string and crypto is not crypto_none then libdar could ask the password to the user thanks to the user_interaction::pause(...) method. | | I like the idea, but throwing an exception is not a reversible operation | ~ from the stack point of view (you cannot continue execution where it | has been thrown). I suppose you mean that libdar should rather use a the | user_interaction::pause(...) method to ask the password to the user, it | the pass argument was something like "bf:" or "scram:" where just the | algorithm (the cypher) is given ? | |[...] | | Cheers, | Denis. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBigKVpC5CI8gYGlIRAm37AJ0YtxRGIk3npMOplMNqhQGl+hlbDACfbYga gTjklmjFWYag/+88z7FiUbM= =LEIW -----END PGP SIGNATURE----- |
|
From: Johnathan B. <jk...@sh...> - 2004-11-04 16:11:46
|
On Thursday 04 November 2004 02:27, Denis Corbin wrote: > Johnathan Burchill wrote: > > Hello Johnathan, > > | On Wednesday 03 November 2004 13:08, Denis Corbin wrote: > |>Denis Corbin wrote: > |>|[...] > | > |[...] > | As far as I can tell from the latest cvs API, when I read an existing > | archive with the archive "read" constructor, I need to give it a > | "const std::string &pass" argument, i.e. the password. So it seems > | that the application has to know beforehand whether the user is > | opening an encrypted archive or an unencrypted archive. > > In my point of view, the user has to know if the archive he tends to > open is encrypted or not and if so it has to know too the password. The > pass argument may be an empty string, or any other value, the given > value will only be used if the archive has a flag set in the header of > the archive telling that the archive has been encrypted (no information > about the algorithm used or the block size used --- or the password of > course --- is present in the header). Note that the pass has the > structure described in the dar man page (option -K), "[algo:]pass" . > where algo is bf, blowfish, scram or scrambling (more will follow I hope > before the end of november) with "bf" = "blowfish", "scram" = > "scrambling". If the password must contain a ':' the algo must be > specified (in the form <algo>:<password_with_column>), so you can safely > always specify the cypher used. > > A warning is issued if an encrypted archive is openned, because the > diagnostic of a wrong password, is not easy to diagnostic neither for > the user nor for libdar. > > | I see that archive::archive calls "macro_tools_open_archive" which in > > turn > > | calls dialog.warning(...) if the archive is encrypted and no password > | is given. Perhaps if "macro_tools_open_archive" could throw an > | Epassword() exception, that could be caught by the application, which > | could transparently ask the user for a password and try the read > | again. > > I like the idea, but throwing an exception is not a reversible operation > ~ from the stack point of view (you cannot continue execution where it > has been thrown). I suppose you mean that libdar should rather use a the > user_interaction::pause(...) method to ask the password to the user, it > the pass argument was something like "bf:" or "scram:" where just the > algorithm (the cypher) is given ? I didn't expect libdar to continue where it left off after issuing Epassword(). Just that it would be a way to inform the application to ask the user for a password and try again to read the archive from the beginning. However, your idea of using user_interaction::pause looks promising. I did not think of that one because the current implementation seems to be able to handle boolean questions, not arbitrary responses. The user_interaction::pause method is nice, because the application can use whatever external program or library it wants to interact with the user. For example, I think KDE or QT has a library call for securely requesting passwords. The application would have to know how user_interaction::pause is to be used, though, right? Libdar would have to indicate that a particular user_interaction::pause call is to be for passwords, and not a yes/no response. > > This is what I was looking for, to implement the password out of > command-line for dar. Libdar could interactively ask the user for > password when the user gives "-K bf:" or maybe just "-K :" where no > password is given and the cypher is blowfish or where nothing is given > at all, so the default cypher (bf actually) is used and no password is > used. > > Thanks for the idea ! :-) > > | Or, is there a way in the API to check the archive, before opening it, > | to see whether it is encrypted? > > well, it may have implications for the user if openning the archive once > just to see if the archive is encrypted and a second time to open it for > real. Implications can be extra change of disk for example. > > | Cheers, > | JB > > Cheers, > Denis. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Dar-libdar_api mailing list > Dar...@li... > https://lists.sourceforge.net/lists/listinfo/dar-libdar_api -- Johnathan K. Burchill, Ph.D. Department of Physics and Astronomy University of Calgary 2500 University Drive N.W. Calgary, AB T2N 1N4 Canada (403) 217-4286 jk...@sh... |
|
From: Denis C. <dar...@fr...> - 2004-11-04 18:06:58
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Johnathan Burchill wrote: | On Thursday 04 November 2004 02:27, Denis Corbin wrote: | |>Johnathan Burchill wrote: |> |[...] |> |>| calls dialog.warning(...) if the archive is encrypted and no password |>| is given. Perhaps if "macro_tools_open_archive" could throw an |>| Epassword() exception, that could be caught by the application, which |>| could transparently ask the user for a password and try the read |>| again. |> |>I like the idea, but throwing an exception is not a reversible operation |>~ from the stack point of view (you cannot continue execution where it |>has been thrown). I suppose you mean that libdar should rather use a the |>user_interaction::pause(...) method to ask the password to the user, it |>the pass argument was something like "bf:" or "scram:" where just the |>algorithm (the cypher) is given ? | | | I didn't expect libdar to continue where it left off after issuing | Epassword(). Just that it would be a way to inform the application to ask | the user for a password and try again to read the archive from the | beginning. | | However, your idea of using user_interaction::pause looks promising. I did | not think of that one because the current implementation seems to be able | to handle boolean questions, not arbitrary responses. Yes you are right. user_interaction::pause() is only able to handle boolean questions. Thus I had to add a new method to user_interaction class: ~ std::string get_string(const std::string & message, bool echo); the message argument shall be displayed to the user and the user answer be the returned value of the method. The echo argument is the "echo local" value. In brief, if libdar set it to true the user answer can be displayed, if libdar set it to false while calling get_string() all that the user types must be hidden (like when entering passwords). The inherited class like the user_interaction_callback must implement get_string(). For this later class this implies a new callback function, but nothing extraordinary there. Dar is updated and used this behavior. So if the pass is an empty string while encryption cypher is set, this mechanism is used to ask the pass to the user. Same thing when reading an archive, if the archive uses encryption while a cypher encryption is set and the pass argument is an empty string, libdar will ask the user for a password through the same mechanisme. I have added another check when reading an archive that is encrypted. If the no cypher has been set (crypto_none) libdar will abort and refuse too perform the operation as obviously this will lead to failure to read the archive. | | The user_interaction::pause method is nice, because the application can use | whatever external program or library it wants to interact with the user. | For example, I think KDE or QT has a library call for securely requesting | passwords. | | The application would have to know how user_interaction::pause is to be | used, though, right? Libdar would have to indicate that a particular | user_interaction::pause call is to be for passwords, and not a yes/no | response. Yes. I realized this just after my previous mail. | | | Cheers, Denis. P.S.: CVS is up to date. Last feature added has been tested and seems to work fine so far. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBimsxpC5CI8gYGlIRAvsRAKC0NVROXpnuXshNPFdH3ezS2K/y6wCgnQWf 8VPx0+AD3AOJA1xBMaLoCF4= =182c -----END PGP SIGNATURE----- |
|
From: Johnathan B. <jk...@us...> - 2004-11-07 17:03:41
|
Hi Denis, On Thursday 04 November 2004 10:47, Denis Corbin wrote: > Johnathan Burchill wrote: > | On Thursday 04 November 2004 02:27, Denis Corbin wrote: > |>Johnathan Burchill wrote: > | > |[...] > | > |>| calls dialog.warning(...) if the archive is encrypted and no > |>| password is given. Perhaps if "macro_tools_open_archive" could throw > |>| an Epassword() exception, that could be caught by the application, > |>| which could transparently ask the user for a password and try the > |>| read again. > |> > |>I like the idea, but throwing an exception is not a reversible > |> operation ~ from the stack point of view (you cannot continue > |> execution where it has been thrown). I suppose you mean that libdar > |> should rather use a the user_interaction::pause(...) method to ask > |> the password to the user, it the pass argument was something like > |> "bf:" or "scram:" where just the algorithm (the cypher) is given ? > | > | I didn't expect libdar to continue where it left off after issuing > | Epassword(). Just that it would be a way to inform the application to > | ask the user for a password and try again to read the archive from the > | beginning. > | > | However, your idea of using user_interaction::pause looks promising. I > > did > > | not think of that one because the current implementation seems to be > | able to handle boolean questions, not arbitrary responses. > > Yes you are right. user_interaction::pause() is only able to handle > boolean questions. Thus I had to add a new method to user_interaction > class: > > ~ std::string get_string(const std::string & message, bool echo); > I'd like to get your opinion on confirming passwords. When a user needs to enter a password to encrypt a file, I think he/she=20 should be asked to confirm what they entered. If the password is for=20 decryption, no confirmation should be required. Is this something that the application or libdar should be concerned about?= =20 If the application has to keep track of whether an archive is being=20 created or read, that is okay, but means repetitive coding for everyone=20 who uses libdar, and more complicated code at the application level. I'd=20 rather let libdar decide whether to confirm passwords, with a method like std::string get_string( const std::string & message, bool echo, bool=20 confirm ); What do you think? [...] JB =2D-=20 Johnathan K. Burchill, Ph.D. jk...@us... |
|
From: Denis C. <dar...@fr...> - 2004-11-07 18:34:33
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Johnathan Burchill wrote: | Hi Denis, Hello Johnathan, | |[...] |>Yes you are right. user_interaction::pause() is only able to handle |>boolean questions. Thus I had to add a new method to user_interaction |>class: |> |>~ std::string get_string(const std::string & message, bool echo); |> | | | I'd like to get your opinion on confirming passwords. | | When a user needs to enter a password to encrypt a file, I think he/she | should be asked to confirm what they entered. If the password is for | decryption, no confirmation should be required. Yes, that's right, but only when password is given interactively (not from the command-line of configuration file for example --- beside the security aspect they raise). | | Is this something that the application or libdar should be concerned about? | If the application has to keep track of whether an archive is being | created or read, that is okay, but means repetitive coding for everyone | who uses libdar, and more complicated code at the application level. I'd | rather let libdar decide whether to confirm passwords, with a method like | | std::string get_string( const std::string & message, bool echo, bool | confirm ); | | What do you think? The problem here is that get_string() is called by libdar only. So the application cannot tell libdar this way for a confirmation. I agree that It may be more simple to make the confirmation code in libdar once and for all other client programs. So, I would rather add a boolean argument called "pass_confirm" (for example) to the "create" and "isolate" archive constructors. This way the application decides whether password confirmation is needed or not. And libdar will call once, twice or more time the user_interaction::get_string() method to get two identical passwords or just one. | | [...] | | JB | Cheers, Denis. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBjmqupC5CI8gYGlIRAlrNAJ4jC3w6+5fwhiN99/63pQ1ozIVeaACeIGZt nthncMOA05GvE58tknElS7Y= =8Ce5 -----END PGP SIGNATURE----- |
|
From: Johnathan B. <jk...@us...> - 2004-11-06 22:53:51
|
Hi Denis, On Wednesday 03 November 2004 13:08, Denis Corbin wrote: > Denis Corbin wrote: > |[...] > | I proposes the following API : > | > | ~ void cancel_thread(pthread_t tid); > | ~ bool cancel_current(pthread_t & tid); > | ~ void cancel_clear(); > | > | where cancel_thread() is used to cancel the given thread. Which will > | make libdar throw a Euser_abort exception, cleanly free memory and > | release any mutex. > | > | cancel_current() return true and the tid of the thread that will be > | canceled when execution will reach the next checkpoint, false if there > | is not pending cancelation. > | > | cancel_clear() abort cancelation process if the thread has not already > | terminated. > | > | Only one thread can be canceled at a time. > | > | > |[...] > > Hello, > > This is implemented and tested. CVS is up to date. If you have no > objections, The API is now frozen (version 3) for the release 2.2.0 that > will come in december I hope. The points that remains to do before the > pre-release phase are : > - documentation (API, user guide, etc.) > - password/key from file > - password/key from keyboard out of command-line > - some few other strong encryption algorithms > - LZO compression > > the two password/key features do not have any impact on the API (these > are command-line related feature). The LZO and the new cypher support > only add some more values in the "enum compression" and "enum > crypto_algo" enumeration types. > > The pre-release target date is the beginning of december. Until that > time, it is still time to fix API problem you could meet, so please give > your feedback before pre-release phase. I've got KDar running on libdar API V3.0.0, CVS. Works great! The thread cancelation works perfectly, my limited testing shows. The response is very snappy, and the user does not have to wait or restart= =20 the program to stop an operation anymore. I think that you have a good=20 implementation of thread cancelation. Quite elegant. One point about libdar's "make install": the config.h header that gets put= =20 into <installdir>/include/dar/ does not contain "#define MUTEX_WORKS 1"=20 when I configure libdar with pthreads, so I have to add it manually to get= =20 KDar to compile. As an aside, here is a tip for people who want to implement thread=20 cancellation in QT: Note that to cancel a libdar operation, you need to pass the ID of the=20 thread to cancel to libdar::cancel_thread(). You have to get this ID from=20 the "run()" method of the QThread, not it's constructor or anywhere else.=20 i.e. put "pthread_self()" inside run() and store it with a protected=20 member variable. This is not well documented by Trolltech, and it had me=20 puzzled for many hours today... [...] Cheers, JB =2D-=20 Johnathan K. Burchill, Ph.D. jk...@us... |
|
From: Denis C. <dar...@fr...> - 2004-11-07 11:18:19
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Johnathan Burchill wrote: | Hi Denis, Hello Johnathan, | | On Wednesday 03 November 2004 13:08, Denis Corbin wrote: | |>Denis Corbin wrote: |>|[...] |>| I proposes the following API : |>| |>| ~ void cancel_thread(pthread_t tid); |>| ~ bool cancel_current(pthread_t & tid); |>| ~ void cancel_clear(); |>| |>| where cancel_thread() is used to cancel the given thread. Which will |>| make libdar throw a Euser_abort exception, cleanly free memory and |>| release any mutex. |>| |>| cancel_current() return true and the tid of the thread that will be |>| canceled when execution will reach the next checkpoint, false if there |>| is not pending cancelation. |>| |>| cancel_clear() abort cancelation process if the thread has not already |>| terminated. |>| |>| Only one thread can be canceled at a time. |>| |>| |>|[...] |> |>Hello, |> |>[...] | | | I've got KDar running on libdar API V3.0.0, CVS. Works great! | The thread cancelation works perfectly, my limited testing shows. | The response is very snappy, and the user does not have to wait or restart | the program to stop an operation anymore. I think that you have a good | implementation of thread cancelation. Quite elegant. Thanks for the compliment :-) However I must check that the execution speed is not too lowered by the simple but much frequent check done to detect thread cancelation requests. | | One point about libdar's "make install": the config.h header that gets put | into <installdir>/include/dar/ does not contain "#define MUTEX_WORKS 1" | when I configure libdar with pthreads, so I have to add it manually to get | KDar to compile. OK, I will fix that. Thanks for feedback. | | [...] | | Cheers, | JB | Cheers, Denis. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBjgRvpC5CI8gYGlIRAmzaAJ98wVAgXkOHSSfbIZi0jcXmi+4ikQCgr++D hCA1/6UqrHaFE8+vywcbO74= =HMOb -----END PGP SIGNATURE----- |
|
From: Johnathan B. <jk...@sh...> - 2004-11-07 16:28:47
|
On Sunday 07 November 2004 04:18, Denis Corbin wrote: > Johnathan Burchill wrote: > | Hi Denis, > > Hello Johnathan, > > | On Wednesday 03 November 2004 13:08, Denis Corbin wrote: > |>Denis Corbin wrote: > |>|[...] > |>| I proposes the following API : > |>| > |>| ~ void cancel_thread(pthread_t tid); > |>| ~ bool cancel_current(pthread_t & tid); > |>| ~ void cancel_clear(); > |>| > |>| where cancel_thread() is used to cancel the given thread. Which will > |>| make libdar throw a Euser_abort exception, cleanly free memory and > |>| release any mutex. > |>| > |>| cancel_current() return true and the tid of the thread that will be > |>| canceled when execution will reach the next checkpoint, false if > |>| there is not pending cancelation. > |>| > |>| cancel_clear() abort cancelation process if the thread has not > |>| already terminated. > |>| > |>| Only one thread can be canceled at a time. > |>| > |>| > |>|[...] > |> > |>Hello, > |> > |>[...] > | > | I've got KDar running on libdar API V3.0.0, CVS. Works great! > | The thread cancelation works perfectly, my limited testing shows. > | The response is very snappy, and the user does not have to wait or > > restart > > | the program to stop an operation anymore. I think that you have a good > | implementation of thread cancelation. Quite elegant. > > Thanks for the compliment :-) However I must check that the execution > speed is not too lowered by the simple but much frequent check done to > detect thread cancelation requests. > Here's a data point for you. On my Athlon-XP 1800+, 512 MB RAM, I created two archives of the kdar cvs source directory, using bzip2 compression. The two archives were created with the same settings, the only difference being that one was made with libdar 2.5 and the other with libdar 3.0.0 (from KDar). Both operations took 32 seconds to complete. Here's the command-line equivalent of the operation: dar -v -c /opt/user/backups/dar_backups/test-kdar-backup-20041107 -R /opt/user/src/kdarcvs/kdar/ -w -s 4613734400 -D -y -m 150 -Z "*.Z" -Z "*.avi" -Z "*.bz2" -Z "*.gif" -Z "*.gz" -Z "*.jpg" -Z "*.mov" -Z "*.mpg" -Z "*.pbm" -Z "*.pdf" -Z "*.png" -Z "*.pnm" -Z "*.zip" -X "*.dar" -X "*~" -X ".*~" When run with the statically-linked dar, here are the results: API V2.5: -------------------------------------------- 992 inode(s) saved with 0 hard link(s) recorded 0 inode(s) not saved (no file change) 0 inode(s) failed to save (filesystem error) 33 files(s) ignored (excluded by filters) 0 files(s) recorded as deleted from reference backup -------------------------------------------- -------------------------------------------- Total number of file considered: 1025 real 0m31.268s user 0m26.900s sys 0m0.980s API V3.0.0 (dev): -------------------------------------------- 992 inode(s) saved with 0 hard link(s) recorded 0 inode(s) not saved (no file change) 0 inode(s) failed to save (filesystem error) 33 files(s) ignored (excluded by filters) 0 files(s) recorded as deleted from reference backup -------------------------------------------- -------------------------------------------- Total number of file considered: 1025 real 0m31.703s user 0m27.170s sys 0m0.850s These are not rigorous benchmarks. In particular, the thread cancellation overhead might be compensated for some other speed improvement in API 3.0 compared to API 2.5. Anyway, for what it's worth... Cheers, JB > | One point about libdar's "make install": the config.h header that gets > | put > | into <installdir>/include/dar/ does not contain "#define MUTEX_WORKS > | 1" when I configure libdar with pthreads, so I have to add it manually > | to get KDar to compile. > > OK, I will fix that. Thanks for feedback. > > | [...] > | > | Cheers, > | JB > > Cheers, > Denis. > > > > ------------------------------------------------------- > This SF.Net email is sponsored by: > Sybase ASE Linux Express Edition - download now for FREE > LinuxWorld Reader's Choice Award Winner for best database on Linux. > http://ads.osdn.com/?ad_id=5588&alloc_id=12065&op=click > _______________________________________________ > Dar-libdar_api mailing list > Dar...@li... > https://lists.sourceforge.net/lists/listinfo/dar-libdar_api -- Johnathan K. Burchill, Ph.D. Department of Physics and Astronomy University of Calgary 2500 University Drive N.W. Calgary, AB T2N 1N4 Canada (403) 217-4286 jk...@sh... |
|
From: Denis C. <dar...@fr...> - 2004-11-07 18:40:29
|
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Johnathan Burchill wrote: | On Sunday 07 November 2004 04:18, Denis Corbin wrote: | |[...] |>Thanks for the compliment :-) However I must check that the execution |>speed is not too lowered by the simple but much frequent check done to |>detect thread cancelation requests. |> | | | Here's a data point for you. | | On my Athlon-XP 1800+, 512 MB RAM, I created two archives of the kdar cvs | source directory, using bzip2 compression. The two archives were created | with the same settings, the only difference being that one was made with | libdar 2.5 and the other with libdar 3.0.0 (from KDar). Both operations | took 32 seconds to complete. I did some tests too in my side, and I arrive to the same conclusion: the thread cancelation feature does not introduces significant speed reduction (I even saw a fit faster execution sometimes by 1% or less, which may be due to mesure errors margin). :-) So we keep thread cancelation this way. | Cheers, Denis. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.4 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFBjmwNpC5CI8gYGlIRAjgAAKCvG946BIcGxwxKVAn0Ak2p6TwBkQCfRgaI veSOWtUpQPxFMECB/GBydXU= =jiln -----END PGP SIGNATURE----- |
Thanks for helping keep SourceForge clean.
X