Note: I found this file on my hard drive. I don't remember writing it, nor do I remember what I tested it on or why I never published it. Use at your own risk, but for what it's worth I vaguely remember that this worked perfectly. gl;hf
Problem: You have a Windows PC with interesting software installed on it - say, an industrial controller or embedded system - but it has a 500GB disk, of which only a tiny portion is actually used. How do you produce a disk image from this that doesn't suck to work with?
The conventional wisdom is "just dd it", but that will produce a 500GB image. Nobody wants you to upload a 500GB image to internet archive; it's too big to easily extract onto many drives, it will take eons to download, and it'll take up tons of space for no good reason, since _most_ of the time, only a few gigs of that huge drive will actually contain data.
You can dd the disk and then zip it. This will almost certainly improve things, but zip can only go so far. Suppose the disk was filled up to capacity once upon a time; it is now full of garbage data that isn't linked in the file table, but zip won't know that. Chances are, your 500GB image will shrink to say, 32 or 40GB - a huge improvement, but still very inconvenient.
And really, this only saves time downloading. The underlying image still can't be used until you extract all 500 useless gigs, which means you need someplace to put all that before you can even inspect the file. A raw binary image is the ideal distribution format; you can open it in 7zip (et al), mount it as a loopback in linux, and so on.
"Just Use Ntfsclone!" This has problems too. ntfsclone is a cool utility - it understands the actual contents of an NTFS partition and will copy just the blocks that are in use. You will, without question, get the smallest possible image this way. But... not by default. If you don't pass the "save-image" flag, it apparently just dds the partition. I have no idea why this mode exists and nobody else seems to either.
To make ntfsclone useful, you need to add "--save-image" which will cause it to output what they call "the special image format." On its face this does exactly what you want: if the drive has 18GB of actual files on it, "ntfsclone --save-image [destination].ntfsclone /dev/[source]" will produce an 18GB image, even if the disk has tons of junk data due to fragmentation, etc. The problem is that nothing can read it.
The "special image format" is completely unique to ntfsclone and nobody has bothered to implement support for it. You can't mount it as a loopback device, 7zip won't open it, and so on. The only possible thing you can do with one of these files is write it back to a block device with ntfsclone. That makes the image useless on... you know... Windows machines.
I've been trying to figure out what to do about this. Copying the files themselves isn't an option; the kind of machines I'm talking about are usually embedded systems with pre-installed programs, often embedded serial numbers and registry entries that simply can't be extracted easily. They are basically tied to that exact instance of Windows, so you need to copy the OS in a replicable, bootable state.
My solution: Shrink the partition, dd it to an image, then dd the MBR of the drive separately. To restore, dd both images back to the destination drive, then boot into the Windows instance and use Disk Management to expand the drive.
This gets you the best of all worlds: you get a bit-perfect image, which is only the size of the used content of the drive (plus a small operating margin) and this can be used to reproduce a perfectly working, bootable system, at the cost of some tedium.
Note: I think I found that this process does not work if the existing bootable OS partition is not the last partition on the disk. I must have been really frustrated, given how I worded it, so it's possible there's a solution to this situation I didn't have the patience to discover; maybe you can just delete all following partitions and it'll be fine? but i don't know, so uh... be aware of that before you start following these steps.
now you have an image that's as small as possible, and which you can open in e.g. 7zip. to restore this to a drive:
I've tested this and it seems to work. This was on Windows 8, but should work fine with Vista or newer. XP lacks the ability to resize partitions as I recall; I was unable to get ntfsresize to play ball, for some reason it gets mad about the backup file map not matching, I don't know how to fix that and neither does anyone else it seems. So XP/2000 are still wildcards, but for anything newer this should work.