Bugsy - 2018-02-10

Hi Alex, Idrassi et Al.,

First of all let me thank Idrassi & Alex for great work done on Veracrypt and around.

Before going into long and winded descryption, the problem is as simple as:
How easily restore a working system from backup without too many manual steps/re-encryption.

This in itself dictates that restore most probably would need to be at image/encrypted partition level, but stay with me and I'll try to describe probably crazy idea wondering if that would have a chance to work.

I could not find a description of how to approach the problem of incremental backup.

One of the key problems for easy backup is the lack of conciousness for an app running in encrypted OS that the file-system is encrypted. This drives me towards the approach that backup in such a case could be done from other OS booted which would have knowledge of a particular system being encrypted (including hiddesn OS - HOS?).

For a start, let's first focus on situation with following pre-requisities:

  • normal encrypted OS scenario (not full disk type of). Later we can add complications of hidden OS.
  • restore to blank HDD (for simplicity same HW to avoid Windows 10 screaming about anything).
  • new blank HDD will be of same or bigger size as source of backup.

Given above we start with following partition set:

# sfdisk -d /dev/sdb
label: gpt
label-id: 78FDB05C-8DA6-476D-B077-7ACC05606E56
device: /dev/sdb
unit: sectors
first-lba: 34
last-lba: 83886046

/dev/sdb1 : start=        2048, size=     1021952, type=DE94BBA4-06D1-4D40-A16A-BFD50179D6AC, uuid=D137E800-E122-4CB6-8EDE-724C0333078F, name="Basic data partition", attrs="RequiredPartiton"
/dev/sdb2 : start=     1024000, size=      202752, type=C12A7328-F81F-11D2-BA4B-00A0C93EC93B, uuid=89C6765B-41FA-499F-8F8A-FA731D7A2EAD, name="EFI system partition"
/dev/sdb3 : start=     1226752, size=       32768, type=E3C9E316-0B5C-4DB8-817D-F92DF00215AE, uuid=29B6D0A3-A5C8-4CCE-BA8F-B0E32B65B910, name="Microsoft reserved partition"
/dev/sdb4 : start=     1259520, size=    29460480, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=901334C1-3C6B-470E-9A18-4DEF1CFD1BCF, name="Basic data partition"
/dev/sdb5 : start=    30720000, size=    32768000, type=EBD0A0A2-B9E5-4433-87C0-68B6B72699C7, uuid=249A4594-758F-4381-8C0D-1060167CD79E, name="sec"
/dev/sdb6 : start=    63488000, size=     2097152, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=65C51FF5-5713-456D-B58C-C548B97E1D95, name="boot"
/dev/sdb7 : start=    65585152, size=    18298880, type=0FC63DAF-8483-4772-8E79-3D69D8477DE4, uuid=BA248348-4F59-4740-B630-95AB098CEFBB, name="root"

A little bit of explanation - in our interest are only partiotions 1-4 related to encrypted OS (Windows 10). Partitions 5 and above are for hidden OS and other system (Linux).

To properly backup and be able to fully recreate system on new disk we need to take following:

  1. partition table.
  2. OS volume headers + backup OS volume headers.
  3. image copy of partitions 1, 2, 3 - these are not modified often if at all. It could be equally file-level dump, assuming restore procedure would have information about file-system type and would format partition to that file-system format. The risk though would be that different file-system ID/UUID would be created and this could cause some dependency issues from Windows side. I'm unsure how sensitive to that Windows is, so let's skip this and take partition image. The biggest one is Windows recovery environment partition 499M. EFI partition of size 100M, shouldn't be an issue at all.
  4. Partition 4 (sdb4 in my case) meta-data dump. We would need to dump all characteristics of partition, just in case if Windows 10 is sensitive to i.e. partition UUID again.
  5. File-level dump or NTFSclone to i.e. dedup location (encrypted of course).

How to address following?
Ad. 1. This one should be easy:
sfdisk -d /dev/sdX > sdx_partition_dump
run from other OS.

Ad. 2. To get OS volume headers + backup OS svolume headers we could use

DEV=/dev/sdX
dd if=$DEV of=1primary_vh_NOS.dd bs=512 count=1
dd if=$DEV of=2backup_vh_NOS.dd bs=512 count=1 skip=$[$(blockdev --getsz $DEV)-256]

The first question is if second command using /dev/sda is correct? I guess it is not as it uses root device whilst it is just partition encrypted.
I'm unsure if in case of System OS encryption there is a backup volume header (based on lack of mention here: https://www.veracrypt.fr/en/VeraCrypt%20Volume%20Format%20Specification.html - hereafter VC-VFS)

UPDATE2: on below - I'm stuck - tried to re-create volume, but I have no clue how to re-create system type of a volume as it seems like it is different than standard one creation process.
THIS IS BLOCKER NOW

UPDATE: below up to "Ad. 3" is irrelevant - tested it and it does not work to dump just headers.
The question is what is required to dump to re-create VC volume? Is it required at all to dump it that way (if possible at all)? Maybe it would be just easier to re-create volume? It has negative consequences on H_OS so would prefer to dump volume instead of re-creating it. The more I think about dumping encrypted volume the more crazy it sounds, but it should be possible to dump key areas to be able to re-create it. Question is which areas should be dumped?

**Big question here is if backing up headers is enough to re-create encrypted/empty volume during restore procedure? We're effectively trying to clone the encrypted volume characteristics with file-system level clone at later stage.
If headers is not enough - what else do we need to back up?
**
The data area does not contain any specific data and is encrypted with keys located in header at offset = 256 with variable length (as per cited earlier VC-VFS).

Ad. 3. For purpose of simplicity here - let's assume backup of all other partitions outside of encrypted one is simple dd to a file which will be restored later in the same way.

Ad. 4. To get NTFS partition details we first need to open encrypted VC and expose it. Let the magic happen by either tcplay with VC support or native VC tools.

VCPART=/dev/sdXN
echo "supersecret_and_very_long_password" | veracrypt --mount-options=system --non-interactive --stdin $VCPART ""

Problem: can't find how to open volume without mounting it but create a specific fuse mount-point. Above mounts fuse under:

/tmp/.veracrypt_aux_mnt1

This allows us to check NTFS status:

ntfsresize --info  /tmp/.veracrypt_aux_mnt1/volume
ntfsresize v2015.3.14AR.1 (libntfs-3g)
Device name        : /tmp/.veracrypt_aux_mnt1/volume
NTFS volume version: 3.1
Cluster size       : 4096 bytes
Current volume size: 15083762176 bytes (15084 MB)
Current device size: 15083765760 bytes (15084 MB)
Checking filesystem consistency ...
100.00 percent completed
Accounting clusters ...
Space in use       : 12647 MB (83,8%)
Collecting resizing constraints ...
You might resize at 12646359040 bytes or 12647 MB (freeing 2437 MB).
Please make a test run using both the -n and -s options before real resizing!

Should ntfsresize complains about state of the system, make sure to run chkdsk /f and re-try. My experience with other situations and use of ntfs-3g tools shows that it is better to use native Windows tools for disk checking.

ntfsclone --save-image --output /mnt/backup_point/unencrypted_NTFS_dump /tmp/.veracrypt_aux_mnt1/volume

Having this, should at least theoretically allow us to restore system into fully working state (incremental backup to be discussed later).

Restore procedure would encompass:

  1. restore partition table
    sfdisk /dev/sdY < sdx_partition_dump
  2. Restore volume headers
DEV=/dev/sdY
dd if=1primary_vh_NOS.dd of=$DEV  bs=512 count=1
dd if=2backup_vh_NOS.dd of=$DEV bs=512 count=1 seek=$[$(blockdev --getsz $DEV)-256]

Another BIG, BIG, BIG question - I need to verify manually but think might be worth checking theory with @Idrassi & @Alex if that would be enough to open volume with specific encryption pass/keys???

This has flaw that if drive was completely filled with some recognizable data, i.e. all zeros, we will be defeating a lot as restoring NTFS at later stage would expose which sectors are occupied - let's leave random data feeling for later stage

  1. Restore of non-encrypted partitions using simple dd (for simplicity reasons here) - it is not an issue of itself.

  2. Assuming it was enough to backup headers to have the same encryption pass, etc. all what we need to do prior restoring NTFS filesystem level is to open volume to be exposed at block level:

VCPART=/dev/sdXN
echo "supersecret_and_very_long_password" | veracrypt --mount-options=system --non-interactive --stdin $VCPART ""

This should expose volume at block level as:

/tmp/.veracrypt_aux_mnt1

Should above be correct then NTFS restore would be as simple as:

ntfsclone --restore-image --overwrite /tmp/.veracrypt_aux_mnt1/volume /mnt/backup_point/unencrypted_NTFS_dump

Big question again - where is the mistake in the process?
If Windows is sensitive to UUIDs of partitions and NTFS details - then we trick it with partitions restore and NTFS clone + dd of other partitions.
If Windows is not sensitive to it, then we could even try to take file level clone or use any other tools which would run from Linux OS level to backup and do incremental backup of Windows partition - need to check if any exist.

Incremental backup - theoretically could be done at NTFScloned level using block level dedup - tool to be identified, yet.
Big step forward here is that we would not be dumping full encrypted image but only used FS space.

From other Alex's post (thanks) I've seen that WinPE with Veracrypt driver + DISM is used as backup method. This though seems to be difficult to automate for me and would require manual steps to re-create partition, etc. But maybe I'm missing something? I don't know exact details of that approach and would be happy to learn more.

On hidden OS - same approach hopefully could be taken, the trick could be opening volume and details around that - but let's not overcomplicate everything now. It is already long enough post.

Did you get to here and read through? Thank you for spending time on reading and hopefully we will find a nice receipe for backup/restore.

Thanks!!!

 

Last edit: Bugsy 2018-02-10