Concerning PDF annotation I think it would be useful to have an option to store the xoj-file and the original PDF as attachments to the exported PDF file. This option coud be called "attach sources" and can be implemented very easily using pdftk. On the other hand when opening a PDF file for annotation, Xournal should look if there is already a xoj-file (and an original PDF) attached, if yes extract both and open the xournal file instead. Additionally Xournal should in this situation default to exporting PDF with the xoj-file and the original PDF attached instead of saving.
The advantage would be, that the user much more has the feeling of really making annotations to his PDF, and not generating a new file which is somehow linked to the original. He "works" with the PDF file. Also he only has to care about one file, while now he has to deal with three files (the original, the xoj-file and the generated PDF). Lastly he is always able to recover the original PDF (and the xoj-file) from his annotated version.