#86 Error - Invalid character: Char 0x0 out of allowed range

[Caio Cezar Silva]

Hey there Jason,

Software was working perfectly fine, then I reloaded my notebook and tried to open Hypernomicon. After the database was loaded, I received the following message:

File: D:\Users\Caio\Documents\Hypernomicon\database\XML ParseError at [row,cow]:[94937,49]
Message: Invalid character
An invalid character was found in text content.

I openned the file mentioned in the error to see what would happen and then I visualized the following messages in the top of the page:

warning on line 3 at column 48: xmlns: URI org.hypernomicon is not absolute
error on line 94937 at column 49: Invalid character: Char 0x0 out of allowed range

I tried to update the software to 1.32.1 but the error persisted. Not even sure what could have caused it.

[Jason Winning]

Somehow a "null" character got into one of the XML files.

I assume you see a popup window similar to the attached image of a popup window. The first line should say which file is affected (in my screenshot it is Notes.xml).

Assuming you are on Windows, it would be helpful if you could open the file in Notepad++, an application that allows you to view non-printing characters. In Notepad++, go to View -> Show Symbol -> Show Control Characters & Unicode EOL (make sure that option is turned on). Then scroll to the line number (94937) and find which record has the null character. The null character will show as "NUL" with inverted colors. Copy and paste the surrounding "record" element (starts with <record, ends with </record>) into a response on this thread, specifying where the NUL character was. That will help me diagnose how the invalid character got there.

I am attaching a second screenshot showing Notepad++ with Notes.xml loaded (for you the file may be different). In that screenshot you can see how there is a NUL character on line 8333. So in a situation like that you would want to copy all the text from lines 8329 to 8341 (the surrounding "record" element) and specify that it is in this line (you should type out NUL because it won't show up when you paste it here):
<name>WCWP 10A[NUL]</name>

You would also be able to simply delete the NUL character in Notepad++, which will hopefully allow the database to load again. (In an XML file, a null (or "NUL") character doesn't serve any purpose and can safely be deleted.)

If you are not on Windows, let me know your OS and I can give you an alternative way of doing the same thing.

[Jason Winning]

I have done some investigation and can't find any way that an invalid character like that can get saved to the XML files from Hypernomicon. The application already has safeguards that are supposed to prevent that. So it would be very helpful, again, if you can let me know in which file, and where in the file, the null character was.

[Caio Cezar Silva]

I'm very sorry for the late response, Jason. Had to stay without my notebook in the last few days.

I tried to do what you've suggested in the first reply; it wasn't possible since:

1. Although the error was in the line 94937, the Notes file ends in line 2088;
2. I've tried to search for "NUL" occurrences in the file but there was none.

As the error is to be found elsewhere, I looked for larger files in the XML folder and tried the larger one, Works.xml. I did what you've suggested once again and deleted the NUL line. It disappeared, but now the error went for the penultimate line - the now last line - in the last character (94936,150) in the attached image below.
This last line does not end in the same way than the previous ones with an "LF". Could it be this the reason for the error?

[Jason Winning]

I was just using the Notes.xml file as an example; is Works.xml the one indicated in the error message you're getting when you try to load the database?

Assuming Works.xml was the file with the null character earlier, something seems to have happened to the file since you tried loading it in Hypernomicon; based on your latest screenshot, everything after line 94936 appears to be missing. I don't think Hypernomicon could have caused this because it wasn't able to load the files in the first place.

Also, based on my investigation I don't think the null character could have been saved to the file by Hypernomicon. I tested forcing the live data in Hypernomicon to have a null character in it immediately before saving, and what actually gets saved to the file in that situation is a substitute character (U+FFFD) which technically is valid in XML and wouldn't have prevented the data from loading.

So the file apparently got corrupted by something else, or (less likely) the file saving process could have been interrupted, like if your computer lost power during saving. Other possible causes are drive errors, cloud sync issues, etc.

Are you able to restore a backed up version, or are you using a cloud service like Dropbox?

[Caio Cezar Silva]

Yeah, I was able to a previous version. However, I decided to restore only the relevant file. As far as I can see it's all working now. Is there any problem on doing this instead of restoring the whole thing?

[Jason Winning]

Glad to hear you were able to restore the file. If any of the other XML files were partially missing there would be an error during loading so it sounds like only the one file was affected. If you didn't receive an error while loading then your database should be in pretty good shape.

How much older was the Works.xml version you restored?

Works.xml contains Label records and Work records. Other than losing those records that weren't present in the backed up version, the main issue is that there are separate records for the Work itself and for the individual Work Files, and the Work File records are actually in Files.xml. So if any works were present in the new version but not the old version, there will be one or more Work File entries in Files.xml that are now orphaned.

You can see a list of these if you do the following query on the Queries tab (see attached image):
Query Type: Any records
Query: Where set of records related by being
Parameter 1: Work(s) having this work file
Parameter 2: Is empty

Execute the query. If there are any results, right-click on one of them and choose "Show in system explorer". That should show the actual file (PDF etc.) in Windows Explorer if there is one. Then drag and drop that file onto Hypernomicon; you will have the option to create a new Work record, and it should automatically re-use the orphaned Work File record. Do the same with any other results from the query. This should restore any missing works as long as there was an associated file.

Other than that, I don't think there would be any other significant side effects. Let me know if you have any luck determining how the XML file got corrupted.

To be on the safe side, you can also set a "Log file path" in Tools -> Settings -> Settings applying to this computer. Check to see if there is anything in the log file after loading the database in Hypernomicon again.

[Caio Cezar Silva]

The restored version is from 10/25/2025. About your suggestion on Quries, I did as you instructed. No results were found. Also tried to look for any log files but nothing was found.
Lsatly, I still have no idea how the file got corrupted. Nevertheless, thank you very much for your help!