[SourceJammer-devel] Re: Fw: SJ error -- missing file

[Robert M. <rob...@ya...>]

Hi, Kevin.

Thanks for bringing this to our attention and I'm sorry to hear that you've been experiencing this
problem. 

Losing files is about as bad a thing as a source control system can do and I do not doubt that
you're having this problem and that it has been very frustrating to you. But I'm not yet ready to
accept that SourceJammer itself is actually the culprit. 

This is not a problem that anyone else has reported to me. We've been using SJ at work for about 3
years and have never experienced anything of that sort. I'm not saying that SourceJammer is not
doing this, but it strikes me as unlikely. There is only one place in the SJ code base where
source files are deleted, and this is the permanenty delete functionality. A file has to be in a
"removed" state before this will even be possible. And then, not only will the source file be
deleted--all evidence of the file will be deleted, including all source files and the file meta
data.

Does anyone out there know, if Java is in the middle of a file write operation and gets
interrupted, might it simply delete the file it's writing to? That seems unlikely, but this is the
only scenario under which I could see that SJ might be doing this. This would involve the SJ
server somehow getting interrupted while it was in the middle of a file add or file check in
operation. (Note, this would not just be an interruption of the connection between client and
server. This would mean that the client check-in request had come into the server successfully and
that the server, in the middle of processing the part of the request that involves updating the
source of the latest version of the file, gets interrupted.)

As for other possible culprits, other than the OS, I really don't know without knowing what sort
of processes you are running.

As for log files, SJ does not have it's own log file. Any logging gets spit out to the tomcat
standard logs files. You can also check the logs in the plugin-work/event-logger directory. These
are not log files exactly, but logs of each day's activities (all modifications to an archive).

A good practice is to back up the source control directories every night. Here at work we use a
little utility that I wrote called SyncJammer. This is a pretty simple app that basically "knows"
if a file's changed since it was last backed up, so it doesn't just bulk copy every single file.
Anyway, if you had a backup system like this in place you'd be able to recover the missing file
that way and could simply drop it in the property directory and SJ would pick it right up.

--Rob


--- Kev...@re... wrote:

> Hello Robert,
> 
> Where can I find log files, error messages and the like for source jammer? 
>  We are still experiencing files disappearing.
> 
> Below is a foward from a developer about another missing file on the SJ 
> server and what he did to get it back. 
> 
> Here are steps to see evidence of files disappearing:
> 
> I'm hoping that this information may serve as a means to stir up a 
> creative dialogue that would eventually yield a solution for us.
> 
> Because this has happened 2 or 3 times already, We have set up a cronjob 
> to dump all filenames in the nas01 source directory to a flat file with 
> the current days date in the name.   This is the directory from which I am 
> daily compiling a list of filenames:
> 
>                 /usr/local/sourcejammer/server/archives/nas01/source
> 
> We have been doing this since Thursday.  I have 5 days worth of source 
> filename dumps.
> 
> We have a perl script that compares two dumps of filenames to find 
> potentially missing files.
> 
> Over the past 5 days, 4 different files have disappeared from the source 
> directory.
> 
> One of the files that have disappeared is the file with the name: 35731
> 
> Here is a list of the daily filename dumps of the source directory:
> $ ls -ltr *.sj_source.log
> -rw-r--r--   1 kcloudt  staff     213050 Aug  4 07:15 
> 20050804.sj_source.log
> -rw-r--r--   1 kcloudt  staff     214226 Aug  5 16:19 
> 20050805.sj_source.log
> -rw-r--r--   1 kcloudt  staff     214334 Aug  6 07:15 
> 20050806.sj_source.log
> -rw-r--r--   1 kcloudt  staff     214340 Aug  7 07:15 
> 20050807.sj_source.log
> -rw-r--r--   1 kcloudt  staff     214586 Aug  8 07:15 
> 20050808.sj_source.log
> 
> Here we see that filename 35731 is not showing up in dumps after August 
> 5th.
> $ grep 35731 *.sj_source.log
> 20050804.sj_source.log:35731
> 20050805.sj_source.log:35731
> 
> Here we are verifying that filename 35731 is not in the source directory
> $ ls /usr/local/sourcejammer/server/archives/nas01/source/35731
> /usr/local/sourcejammer/server/archives/nas01/source/35731: No such file 
> or directory
> 
> Here we see that the source directory has 37640 files
> $ ls /usr/local/sourcejammer/server/archives/nas01/source | wc
>    37640   37640  214784
> 
> Here we see some of the files in the source directory
> $ ls /usr/local/sourcejammer/server/archives/nas01/source | more
> 0
> 1
> 10
> 100
> 1000
> 10000
> 10001
> 10002
> 10003
> 10004
> 10005
> 10006
> 10007
> 10008
> 10009
> 1001
> 10010
> 10011
> 10012
> 10013
> 10014
> 10015
> 10016
> --More--
> 
> Thanks,
> 
> Kevin Cloudt
> 770-806-2648
> kev...@re...
> ----- Forwarded by Kevin Cloudt/Regulus on 08/08/2005 11:53 AM -----
> 
> Robin Levine/Regulus 
> 08/05/2005 05:16 PM
> 
> To
> Kevin Cloudt/Regulus@Regulus
> cc
> Michael Horner/Regulus@Regulus, Chuck Patton/Regulus@Regulus
> Subject
> SJ error -- missing file
> 
> 
> 
> 
> 
> The file 
> Iview-new/iview/Migration/src/com/regulusgroup/dbms/TemplateAttributeDAO.java 
> was throwing this error when attempting to check it back in.
> 
> From previous experience, it indicates the file has gone away on the SJ 
> server ;-(
> 
> Recovered as follows:
> -- Copied my file on disk to another location
> -- Unchecked out the file from SJ
> -- Removed the file from SJ
> -- Copied the file back
> -- Added back to SJ
> 
> The latest version is back, but history for that file is now lost
> 
> Thanks;
> 
> Robin S Levine
> rob...@re...
> 770-806-2655
> 
> 


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com