ZFS

ZFS is a combined file system and logical volume manager designed by Sun Microsystems.

Sun Microsystems, Inc. (Sun for short) was an American company that sold computers, computer components, software, and information technology services and created the Java programming language, the Solaris operating system, ZFS, the Network File System (NFS), and SPARC.

Originally, ZFS was proprietary, closed-source software developed internally by Sun as part of Solaris, with a team led by the CTO of Sun's storage business unit and Sun Fellow Jeff Bonwick.

Jeff Bonwick invented and led development of the ZFS file system, which was used in Oracle Corporation's ZFS storage products as well as startups including Nexenta, Delphix, Joyent, and Datto, Inc.

ZFS is scalable, and includes extensive protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z, native NFSv4 ACLs, and can be very precisely configured.

Many of them, for example AIX, FreeBSD, Mac OS X beginning with version 10.4 ("Tiger"), or Solaris with ZFS filesystem, support NFSv4 ACLs, which are part of the NFSv4 standard.

The two main implementations, by Oracle and by the OpenZFS project, are extremely similar, making ZFS widely available within Unix-like systems.

ZFS is a combined file system and logical volume manager designed by Sun Microsystems.

On Linux, bcache or dm-cache may be used for this purpose, while Fusion Drive may be used on OS X. ZFS also implements this functionality at the file system level, by allowing administrators to configure multi-level read/write caching.

In 2005, the bulk of Solaris, including ZFS, was licensed as open-source software under the Common Development and Distribution License (CDDL), as the OpenSolaris project.

The later OpenSolaris 2008.11 release included a GUI for ZFS' snapshotting capabilities, known as Time Slider, that provides functionality similar to macOS's Time Machine.

ZFS also includes a mechanism for dataset and pool level snapshots and replication, including snapshot cloning which is described by the FreeBSD documentation as one of its "most powerful features", having features that "even other file systems with snapshot functionality lack". FreeNAS, an embedded open source network-attached storage (NAS) distribution based on FreeBSD, has the same ZFS support as FreeBSD and PC-BSD. NAS4Free, an embedded open source network-attached storage (NAS) distribution based on FreeBSD, has the same ZFS support as FreeBSD, ZFS storage pool version 5000.

From 7.0 onward, FreeBSD supports the ZFS filesystem.

In response, the illumos project was founded, to maintain and enhance the existing open source Solaris, and in 2013 OpenZFS was founded to coordinate the development of open source ZFS.

* napp-it, ZFS web interface for Illumos-based NAS or SAN appliances.

ZFS is scalable, and includes extensive protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z, native NFSv4 ACLs, and can be very precisely configured. ZFS also includes a mechanism for dataset and pool level snapshots and replication, including snapshot cloning which is described by the FreeBSD documentation as one of its "most powerful features", having features that "even other file systems with snapshot functionality lack".

Sun Microsystems ZFS has a hybrid implementation which tracks read-write snapshots at the block level, but makes branched file sets nameable to user applications as "clones".

This checksumming continues all the way up the file system's data hierarchy to the root node, which is also checksummed, thus creating a Merkle tree.

Hash trees are also used in the IPFS, Btrfs and ZFS file systems (to counter data degradation ); Dat protocol; Apache Wave protocol; Git and Mercurial distributed revision control systems; the Tahoe-LAFS backup system; Zeronet; the Bitcoin and Ethereum peer-to-peer networks; the Certificate Transparency framework; and a number of NoSQL systems such as Apache Cassandra, Riak, and Dynamo.

Other file systems such as ZFS support variable block sizes.

ZFS is scalable, and includes extensive protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z, native NFSv4 ACLs, and can be very precisely configured. ZFS is designed to ensure (subject to suitable hardware) that data stored on disks cannot be lost due to physical errors or misprocessing by the hardware or operating system, or bit rot events and data corruption which may happen over time, and its complete control of the storage system is used to ensure that every step, whether related to file management or disk management, is verified, confirmed, corrected if needed, and optimized, in a way that storage controller cards and separate volume and file managers cannot achieve.

As an example, ZFS creator Jeff Bonwick stated that the fast database at Greenplum, which is a database software company specializing in large-scale data warehousing and analytics, faces silent corruption every 15 minutes.

They can be conceived as groups of disks that each provide redundancy against failure of their physical devices.

Data redundancy can also be used as a measure against silent data corruption; for example, file systems such as Btrfs and ZFS use data and metadata checksumming in combination with copies of stored data to detect silent data corruption and repair its effects.

ZFS is scalable, and includes extensive protection against data corruption, support for high storage capacities, efficient data compression, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z, native NFSv4 ACLs, and can be very precisely configured.

CoW may also be used as the underlying mechanism for snapshots, such as those provided by logical volume management, file systems such as Btrfs and ZFS, and database servers such as Microsoft SQL Server.

Data deduplication capabilities were added to the ZFS source repository at the end of October 2009, and relevant OpenSolaris ZFS development packages have been available since December 3, 2009 (build 128).

Deduplication is implemented in some filesystems such as in ZFS or Write Anywhere File Layout and in different disk arrays models.

FreeNAS, an embedded open source network-attached storage (NAS) distribution based on FreeBSD, has the same ZFS support as FreeBSD and PC-BSD.

A 1999 study showed that neither any of the then-major and widespread filesystems (such as UFS, Ext, XFS, JFS, or NTFS), nor hardware RAID (which has some issues with data integrity) provided sufficient protection against data corruption problems.

A quota can be set to limit the amount of space a file system instance can occupy, and a reservation can be set to guarantee that space will be available to a file system instance.

In some filesystems (e.g. ext4, XFS, f2fs, ZFS, Lustre) it is also possible to also define block and inode quota limits for a particular project or directory, by adding a project ID to files that directory tree and defining quota limits for the project ID.

ZFS is a 128-bit file system, so it can address 1.84 × 10 19 times more data than 64-bit systems such as Btrfs.

In 2009, Btrfs was expected to offer a feature set comparable to ZFS, developed by Sun Microsystems.

NAS4Free, an embedded open source network-attached storage (NAS) distribution based on FreeBSD, has the same ZFS support as FreeBSD, ZFS storage pool version 5000.

It includes ZFS v5000, Software RAID (0,1,5), disk encryption, S.M.A.R.T. and email reports etc., with the following protocols: SMB, Samba Active Directory Domain Controller AD, FTP, NFS, TFTP, AFP, rsync, Unison (file synchronizer), iSCSI (initiator and target), HAST, CARP, Bridge, UPnP, and BitTorrent.

All block pointers within the filesystem contain a 256-bit checksum or 256-bit hash (currently a choice between Fletcher-2, Fletcher-4, or SHA-256) of the target block, which is verified when the block is read.

ZFS is designed to ensure (subject to suitable hardware) that data stored on disks cannot be lost due to physical errors or misprocessing by the hardware or operating system, or bit rot events and data corruption which may happen over time, and its complete control of the storage system is used to ensure that every step, whether related to file management or disk management, is verified, confirmed, corrected if needed, and optimized, in a way that storage controller cards and separate volume and file managers cannot achieve.

The ZFS file system was designed to address many of these data corruption issues.

ZFS is a 128-bit file system, so it can address 1.84 × 10 19 times more data than 64-bit systems such as Btrfs.

Historically, ZFS has not shipped with tools such as fsck to repair damaged file systems, because the file system itself was designed to self-repair, so long as it had been built with sufficient attention to the design of storage and redundancy of data.

Full copy-on-write file systems such as ZFS and Btrfs are designed to avoid most causes of corruption and have no traditional "fsck" repair tool.

Originally, ZFS was proprietary, closed-source software developed internally by Sun as part of Solaris, with a team led by the CTO of Sun's storage business unit and Sun Fellow Jeff Bonwick.

Solaris is known for its scalability, especially on SPARC systems, and for originating many innovative features such as DTrace, ZFS and Time Slider.