#7 Manifest file has mismatched rows/columns

[Allan Cunliffe]

Testing Branch v2.0.23

I processed a series of CDs which contain 0byte files. I looked at the file name of these files in a hex editor and the last character is Hex 0A. In OpenOffice and Kwrite, this is being treated as a line feed. When I open the manifest file, everywhere the filename contains the line feed character, rows are mismatched with the columns. I've attached some samples: the 0 byte file and the manifest.

[Allan Cunliffe]

[Matthew Oliver]

Fixed in version 2.0.27 on the testing branch, tagged as bug-3051295.

[Allan Cunliffe]

Tested in Testing Branch - Manifest Maker v2.0.28

Manifest file is being created as expected. Non-printing characters are being removed from filenames.

[Allan Cunliffe]

Testing Branch - Windows Executable (v 2.0.30)

This issue is causing an error in the console when processing a file with line return character:

IOError: [Errno 22] invalid mode ('wb') or filename: u'E:\\Documents and Setting
s\\Allan\\Desktop\\test1-0byte\\media1\\records\\media1\\records\\chogm_logos\\s
mall\\icon\r'

[Allan Cunliffe]

Testing Branch Manifest Maker v2.0.31.

The following is from my latest test in Windows 7 (64-bit):

Z:\chog\media1\records\chogm_logos\compressed\min_pl_bw_solid.sea = f708e424be87
1e500321dd864ab1ed929146242dfefe79c206db855d7f62a3379168c9374d75e7a3b6fd2ce76b2c
5b23131bf4f5033ef8c2ca60ff092ec7fb3c
copying file media1\records\chogm_logos\compressed\min_pp_bw_solid.sea to media
1. Size is 21026
Z:\chog\media1\records\chogm_logos\compressed\min_pp_bw_solid.sea = 166cbcd32ec7
725d2e5a31d012175105e21a88aa46076c34cfda657fe05c7e317d86a34f9063b5c466a1eae4c4cd
e256496d770c5b5d9d0e4beb2d3b28d676a4
to media 1. Size is 0cords\chogm_logos\small\icon
Traceback (most recent call last):
File "ManifestMakerFrame.py", line 1171, in OnNext
if not self.Process():
File "ManifestMakerFrame.py", line 1024, in Process
if self.writer.writeAndCopy() == False:
File "C:\Users\allan\Desktop\manifest\ManifestWriter.py", line 204, in writeAn
dCopy
shutil.copy2(fullFilePath, destPath)
File "C:\Python26\lib\shutil.py", line 99, in copy2
copyfile(src, dst)
File "C:\Python26\lib\shutil.py", line 53, in copyfile
fdst = open(dst, 'wb')
IOError: [Errno 22] invalid mode ('wb') or filename: u'C:\\Users\\allan\\Desktop
\\test-2-0-31\\media1\\records\\media1\\records\\chogm_logos\\small\\icon\r'

[Allan Cunliffe]

Fixed in Manifest Maker v2.0.32.

Tested in Windows 7 - 32 bit, Linux Fedora 13 and Mac OS X 10.6.4

[Allan Cunliffe]

I just discovered that this fix does not work well with DPR.

I processed a set of files which include a file with a filename that includes a line return character:

* created a manifest
* in QF of DPR, I processed the files and manifest.

As manifest maker removed the end of line character, DPR says it can't find the file and that it finds a file not on the manifest.

Not sure what we can do here - if we sanitise the manifest to prevent these characters creating a new line, we make sure the manifest is OK but we then have problems with DPR processing. If we leave the filenames "as is", we affect the manifest file.

Having files with filenames containing line returns (\r and \n) would probably be rare but the issue is that we are changing a filename in the manifest while the actual file is unchanged.

[John]

Maybe the best option would be to have manifest Maker not change the filename in the manifest, but to alert the user that file "foo.doc has an illegal character in the filename - please manually fix it and start again"

[John]

Maybe the best option would be to have manifest Maker not change the filename in the manifest, but to alert the user that file "foo.doc has an illegal character in the filename - please manually fix it and start again"

[Allan Cunliffe]

I think that is a good solution. That way the user knows what's wrong, rather than us changing filenames in the manifest with no notification.

[margotc]

Create correct warnings on illegal file names.