Disk compression
Disk compression increases the amount of information that can be stored on a disk by automatically compressing and decompressing data as it is written to and read from it. A disk compression utility is software that provides disk compression by overriding the system's file access layer so that all programs participate in compression. When a program writes a file, the utility compresses the data before storage. When a program reads a file, the utility decompresses the stored data before providing it to the program. Disk compression uses a single, normal file for storing the logical, user-visible files of a disk volume. This file tends to be relatively large and acts as a virtual drive. Typically, access to the compressed storage is via device driver. Similar utilities automatically compress and decompress each file instead of using a single, large file, but such a utility is not classified as disk compression. Unlike a file compression utility which is generally interactive (user specifies which files to compress and when to compress them), a disk compression utility is noninteractive (files are processed automatically as part of normal file use). This automatic behavior is sometimes described as on-the-fly, transparent, real-time and online. Disk compression was popular especially in the early 1990s, when microcomputer hard disks were relatively small (20 to 80 megabytes) and rather expensive (roughly 10 USD per megabyte). Such utilities tended to be a more economic means of having more disk space as opposed to buying a disk with more space. A good disk compression utility could, on average, double the available space with negligible speed loss. Disk compression fell into disuse by the late 1990s, as advances in hard drive technology and manufacturing led to lower cost per unit of space.
 | This article needs additional citations for verification. (April 2019) |
Disk compression increases the amount of information that can be stored on a disk by automatically compressing and decompressing data as it is written to and read from it. A disk compression utility is software that provides disk compression by overriding the system's file access layer so that all programs participate in compression. When a program writes a file, the utility compresses the data before storage. When a program reads a file, the utility decompresses the stored data before providing it to the program.
Disk compression uses a single, normal file for storing the logical, user-visible files of a disk volume. This file tends to be relatively large and acts as a virtual drive. Typically, access to the compressed storage is via device driver. Similar utilities automatically compress and decompress each file instead of using a single, large file, but such a utility is not classified as disk compression.
Unlike a file compression utility which is generally interactive (user specifies which files to compress and when to compress them), a disk compression utility is noninteractive (files are processed automatically as part of normal file use). This automatic behavior is sometimes described as on-the-fly, transparent, real-time and online.
Disk compression was popular especially in the early 1990s, when microcomputer hard disks were relatively small (20 to 80 megabytes) and rather expensive (roughly 10 USD per megabyte).[1] Such utilities tended to be a more economic means of having more disk space as opposed to buying a disk with more space. A good disk compression utility could, on average, double the available space with negligible speed loss. Disk compression fell into disuse by the late 1990s, as advances in hard drive technology and manufacturing led to lower cost per unit of space.
Use
[edit]To start using disk compression on an existing drive, the utility would follow steps like:
- Create an empty, compressed, virtual drive (a normal file on the host drive).
- Transfer files from the host drive to the compressed drive except for certain system files such as the swap file.
- Increase the size of the compressed drive to accommodate files and allow empty space when complete.
Since a device driver is required to access a compressed drive, a compressed boot drive (C:) required changes to the boot process as follows:
- The System BIOS loads sector 0 of the first physical hard drive (Master Boot Record).
- The partition sector loads sector 0 of the bootable partition (Volume Boot Record). In this case, it's the host drive.
- Host drive sector 0 loads (in the case of DOS) the DOS BIOS (IO.SYS, IBMBIO.COM etc.) Depending on the version, only the first few sectors of the file or the whole file may be loaded into memory. In the first case, the loaded stub contains another loader loading the remainder of the file from disk. In the case of DR-DOS, the BIOS is packed and will decompress itself through an embedded decompression routine.[2][3]
- The DOS BIOS then loads the DOS kernel (MSDOS.SYS, IBMDOS.COM etc.)
- If the DOS supports the preload API, the DOS BIOS attempts to load a preloadable compression driver[4][5] (DBLSPACE.BIN, DRVSPACE.BIN or STACKER.BIN) from the root of the host drive and attempts to communicate with it through the DOS preload API.[2][6] If the driver responds as expected, the DOS BIOS will keep the driver, if not, it will get discarded.[7] If kept and configured accordingly, drive letters may get swapped, so that the compressed drive becomes C:.
- The DOS BIOS starts CONFIG.SYS processing by looking for the file in the root of the boot drive. In the case of DR-DOS, DCONFIG.SYS takes precedence over CONFIG.SYS, if present.[6][2]
- Compression drivers not loaded through the preload API may be loaded through DEVICE statements.
- Depending on configuration and if this hasn't happened already, drive letters may get swapped, so that the compressed drive becomes C:.
- Processing continues from compressed drive.
Considerations
[edit]- Performance
- If a system is generally CPU bound, disk compression tends to decrease overall performance because data compression requires significant CPU time. But for a system that is more I/O bound, frequently waiting for storage accesses to complete, then compressing a drive can increase overall performance. With compressed data, there is less data to read from or write to a drive.[6]
- Data loss due to crosslinked files
- Not all compression utilities would confirm the absence of errors in the FAT file system before compressing a disk in place. Some errors, such as crosslinked files, could result in data loss during the transfer process.
- Boot drive accessibility may depend on another drive
- The compressed drive is only visible if the device driver is loaded and the compressed drive is mounted. A boot disk, for example, might not contain the driver. This was not normally an issue for solutions that came bundled with the operation systems unless booting foreign operating systems, as system utilities like SYS were modified to copy the compression drivers to boot disks by default,[6][3] and the corresponding operation systems would also fall back to look for them on drives other than the boot disk if they couldn't be found there.
- Uninstall limited by host drive size
- Uninstallation was problematic if the expanded size of data on the compressed disk was larger than the normal disk capacity.
- Accidentally deleting virtual drive file
- Users did not always realize that the large file on the host drive contained the compressed drive. While it was usually hidden by the compression utility, users who did find the large file curious or suspicious were able to delete it which resulted in the loss of all the files stored in it.
Examples
[edit]Hardware-assisted
[edit]Some of the initial disk compression solutions used a coprocessor via driver software. Examples include:
- Stacker
- From Stac Electronics.[8][9]
- Expanz!
- From InfoChip Systems.[8][9][10]
- DiskDoubler dd2000
- From Datran Corporation.
Software only
[edit]As processor power increased over time, software-only solutions began to reach or even outperform the performance of hardware-assisted solutions. These utilities were third-party components, either bundled in an OS distribution or installed separately.
- Squish
- From Sundog Software.[11][12][13]
- Stacker
- From Stac Electronics.[9]
- XtraDrive
- From Integrated Information Technology.
- SuperStor
- From AddStor.[8][9]
- DoubleDisk
- From Vertisoft Systems.[8][9]
- DoubleSpace
- Based, in part, on Vertisoft Systems' DoubleDisk.[7]
- DriveSpace
- DoubleSpace with a different compression algorithm (DRVSPACE.BIN).
- JAM
- (JAM.SYS) from JAM Software, Kiev, Ukraine.[14]
- DiskDoubler
- From Salient Software.
- Double Density
- From Data Becker.
Bundled
[edit]Bundling disk compression with a new computer appealed to both resellers and users. Resellers could claim more storage space, and users liked that they did not have to install or configure the software. Operating systems pre-installed on new computers with bundled disk compression included:
- DR DOS 6.0 (1991)
- Bundled with a custom version of SuperStor.
- PalmDOS 1.0 (1992)
- Bundled with a DPMS-enabled version of SuperStor.
- MS-DOS 6.0 (1993)
- Bundled with DoubleSpace. (MS-DOS 6.0 introduced the so-called DOS preload API, an undocumented[4][5] challenge–response type interface which allowed DoubleSpace to be loaded before CONFIG.SYS was processed.)
- PC DOS 6.1 (1993)
- While the original release did not include a disk compressor, a custom version of SuperStor/DS was included in a later release, also introducing the preload API in the PC DOS line.
- MS-DOS 6.2 (1993)
- Bundled with an improved version of DoubleSpace.
- Novell DOS 7 (1993)
- Bundled with a DPMS-enabled version of Stacker (3.12), which also supported compressed data exchange with remote NetWare and Personal NetWare servers, also running Stacker, in order to avoid unnecessary recompression when copying files.[6] It also introduced support for the DOS 6 preload API in order to maintain compatibility with Microsoft's DoubleSpace in multi-boot scenarios.[4][6] A similar mechanism was used to load the optional single- or multi-user SECURITY component.
- PTS-DOS Extended 6.4 (1994)
- Bundled with a disk compression component named Folder (FOLDER.SYS).[15]
- PC DOS 6.3 (1994)
- Bundled with SuperStor/DS.
- MS-DOS 6.22 (1994)
- Bundled with DriveSpace. (Note that MS-DOS 6.21 shipped with no disk compression software for legal reasons.)
- PC DOS 7.0 (1995)
- Bundled with Stacker 4.02 (DPMS-enabled).
- Windows 95 (1995)
- Bundled with DriveSpace 2.
- Windows 95 OSR2 (1996)
- Bundled with a stripped-down version of DriveSpace 3.
- Windows 95 Plus! (1996)
- Bundled with the full version of DriveSpace 3.
- OpenDOS 7.01 (1997)
- Bundled with Stacker 3.12 (DPMS-enabled).[6]
- DR-DOS 7.02 (1998)
- Bundled with Stacker 3.12 (DPMS-enabled).[16] DR-DOS 7.02 also added support for DriveSpace[2]
- PC DOS 2000 (1998)
- Bundled with Stacker 4.02 (DPMS-enabled).
- Windows 98 (1998)
- Bundled with DriveSpace 3.
- DR-DOS 7.03 (1999)
- Bundled with Stacker 3.12 (DPMS-enabled).
- Windows 98 SE (1999)
- Bundled with DriveSpace 3.
- PTS-DOS Pro 2000 (1999)
- Bundled with Datacomp.
Related
[edit]Notable software adjacent to disk compression includes:
- Squeeze
- Squeeze (from Turner Hall Publishing) was a resident DOS file compressor for Lotus 1-2-3 and Lotus Symphony files.[17]
- Squish Plus ROM
- This special ROM chip (from Sundog Software and distributed through Personalized Software) contained a custom version of Squish Plus with only 36 KB footprint adapted for the HP Portable Plus.[13]
- Double Tools for DoubleSpace
- Double Tools (from Addstor) enhances the functionality of the DoubleSpace utility bundled with MS-DOS 6.0.
- Multimedia Stacker
- Multimedia Stacker (from Helix Software) bundled Stac's DPMS-enabled Stacker 4.01 with Helix Cloaking and utilities, utilizing Cloaking's built-in DPMS server to relocate and run in protected mode.
- THS filesystem
- THS (from Thomas Scheuermann; 1994–1996) was a compressed file system driver for Linux which could read some compressed volume files.[18][19][20]
- DMSDOS
- DMSDOS (1996) was a Linux kernel driver to support the reading and writing of DoubleSpace, DriveSpace 2 & 3, and Stacker 3 & 4 volumes.[21][22]
- NTFS compression
- Windows includes support for compressing files and folders in an NTFS partition. As such, it has aspects of both file and disk compression.
See also
[edit]- cramfs
- Disk fragmentation
- e2compr
- Executable compression
- Microsoft Real-Time Compression Interface
- RAM compression
- Self-extracting archive
- SquashFS
References
[edit]- ^ Kozierok, Charles M. (2001-04-17). "Disk Compression". The PC Guide.
- ^ a b c d Paul, Matthias R. (1997-10-02). "Caldera OpenDOS 7.01/7.02 Update Alpha 3 IBMBIO.COM README.TXT". Archived from the original on 2003-10-04. Retrieved 2009-03-29. [1]
- ^ a b DR-DOS 7.03 WHATSNEW.TXT — Changes from DR-DOS 7.02 to DR-DOS 7.03. Caldera, Inc. 1998-12-24. Archived from the original on 2019-04-08. Retrieved 2019-04-08.
- ^ a b c Schulman, Andrew; Brown, Ralf D.; Maxey, David; Michels, Raymond J.; Kyle, Jim (1994) [November 1993]. Undocumented DOS: A programmer's guide to reserved MS-DOS functions and data structures - expanded to include MS-DOS 6, Novell DOS and Windows 3.1 (2 ed.). Addison Wesley. ISBN 0-201-63287-X. (xviii+856+vi pages, 3.5"-floppy) Errata: [2][3]
- ^ a b Chappell, Geoff (1994). DOS Internals. Addison Wesley. ISBN 978-0-201-60835-9. [4][5] Errata: [6][7][8]
- ^ a b c d e f g Paul, Matthias R. (1997-07-30). NWDOS-TIPs — Tips & Tricks rund um Novell DOS 7, mit Blick auf undokumentierte Details, Bugs und Workarounds. MPDOSTIP (in German) (3 ed.). Archived from the original on 2017-09-10. Retrieved 2015-04-05. NWDOSTIP.TXT is a comprehensive work on Novell DOS 7 and OpenDOS 7.01, including the description of many undocumented features and internals. It is part of the author's yet larger MPDOSTIP.ZIP collection maintained up to 2001 and distributed on many sites at the time. The provided link points to a HTML-converted older version of the NWDOSTIP.TXT file.
- ^ a b Slivka, Benjamin W.; Straub, Eric; Freedman, Richard (July 1993). "Hands on - Under the Hood: Inside MS-DOS 6 - The developers of MS-DOS describe the operating system's memory optimization and disk-compression technology". BYTE Magazine. 18 (8): 197–202. Retrieved 2016-11-01. [9]
- ^ a b c d Slovick, Linda (1992-02-17). "The big squeeze". InfoWorld. 14 (7): 57–63. Retrieved 2015-04-06.
- ^ a b c d e Pastrick, Greg (1992-01-28). "Double - no - triple your hard disk space with on-the-fly data compression utilities". PC Magazine. 11 (2): 261. Retrieved 2015-04-06.
- ^ Mendelson, Edward (1990-11-13). "Expanz! board quickly compresses files on all disk types but forfeits some DOS compatibilities". PC Magazine. 9 (19): 13. Retrieved 2015-04-06.
- ^ Walkenbach, John (1987-02-02). "Squish - File compressor crams data into smaller space, but slowly". InfoWorld. 9 (6): 55. Retrieved 2015-04-05.
- ^ Davis, Stephen Randy (1987-02-24). "Squish Puts a Squeeze on Files Before They Are Written to Disk". PC Magazine. 6 (4): 46. Retrieved 2015-04-06.
- ^ a b Goldstein, Hal (1989-07-08). "50% more file space for as little as $89.95" (PDF). The Portable Paper. 4 (4). Personalized Software, now Thaddeus Computing: 46. Retrieved 2015-04-05.
- ^ http://files.mpoli.fi/unpacked/software/dos/compress/jam125sw.zip/readme.doc [bare URL plain text file]
- ^ "MS-DOS-Clone made in Russia". Computerwoche. 1994-05-06. Retrieved 2015-04-12.
- ^ Caldera DR-DOS 7.02 User Guide. Caldera, Inc. 1998 [1993, 1997]. Archived from the original on 2016-11-05. Retrieved 2013-08-10.
- ^ "Squeeze 10 times as many spreadsheets on a disk - SQZ! - The data squeezer for Lotus". PC Magazine. Turner Hall Publishing: 155. 1986-10-14. Retrieved 2016-11-18.
- ^ "FTP link". sunsite.unc.edu (FTP).[dead ftp link] (To view documents see Help:FTP)
- ^ "Re: Disk Compression - Stacker - Drivespace, etc".
- ^ http://ftp.nluug.nl/ftp/ibiblio/system/filesystems/thsfs.lsm [bare URL plain text file]
- ^ "DMSDOS - Linux kernel driver". 2005-03-18. Archived from the original on 2016-11-11. Retrieved 2016-05-25.
- ^ "DMSDOS CVF module" (dmsdoc.doc). 0.9.2.0. 1998-11-19. Archived from the original on 2016-11-02. Retrieved 2016-11-01.
External links
[edit]- "MS-DOS 6 Technical Reference". Microsoft.
- "DoubleSpace CVF file layout". ff.
- "How Safe is Disk Compression?". BYTE Magazine. February 1994. Archived from the original on 2008-06-19.
- "How To... Compress Hard Drives". Smart Computing. April 1999.