I have a camera server recording multitude of security camera feeds to a local drive. It has been running Windows 2016 with no issues but the 8TB drive started to get to about 75% full, so I figured I would test Windows 2019 with ReFS and deduplication.
Had to move the data to temporary drive, upgraded the OS to Windows Server 2019 Datacentre release version (Version 10.0.17763.134) and formatted the drive with ReFS (REFS Version: 3.4) and turned on deduplication. Moved the main data folder back
to it and started the deduplication job. The initial job ran for about 2 days and got very minimal saved space:
PS C:\Windows\system32> get-dedupstatus
FreeSpace SavedSpace OptimizedFiles InPolicyFiles Volume
--------- ---------- -------------- ------------- ------
2.38 TB 75.73 GB 4271 4329 E:
I noticed there was another folder that I had not moved, so I moved that one to the drive also. The optimization ran over that and the result was strange:
PS C:\Windows\system32> get-dedupstatus
FreeSpace SavedSpace OptimizedFiles InPolicyFiles Volume
--------- ---------- -------------- ------------- ------
2.15 TB 0 B 4565 4555
E:
So the optimized files number was larger than inPolicyFiles but the optimization kept running and getting worse. So after another 2 days I was out of space:
PS C:\Windows\system32> get-dedupstatus
FreeSpace SavedSpace OptimizedFiles InPolicyFiles Volume
--------- ---------- -------------- ------------- ------
9.98 GB 0 B 5739 4555
E:
The number of inPolicy Files did not change, and the number of files on the drive did not change as I had not yet started the camera software, just number of OptimizedFiles kept growing. Not sure how it got over 1200 files more to be optimized than
there were files on the drive. So I stopped the deduplication of the volume but the space did not free up again, so I started another job:
Start-DedupJob -Volume "E:" -Type Unoptimization
And this appears to have started to free up space now. After 12 hours I had already got back over 2 TB:
PS C:\Windows\system32> get-dedupstatus
FreeSpace SavedSpace OptimizedFiles InPolicyFiles Volume
--------- ---------- -------------- ------------- ------
2.28 TB 0 B 5739 4555
E:
So my question is, if this is a bug in the ReFS deduplication? Or is this some other issue? Shouldn't the optimization just leave the file alone if it cannot gain anything from optimizing and where do those extra number of optimized files come from
when they do not exist on the drive?