f2fs: support data compression
This patch tries to support compression in f2fs.
- New term named cluster is defined as basic unit of compression, file can
be divided into multiple clusters logically. One cluster includes 4 << n
(n >= 0) logical pages, compression size is also cluster size, each of
cluster can be compressed or not.
- In cluster metadata layout, one special flag is used to indicate cluster
is compressed one or normal one, for compressed cluster, following metadata
maps cluster to [1, 4 << n - 1] physical blocks, in where f2fs stores
data including compress header and compressed data.
- In order to eliminate write amplification during overwrite, F2FS only
support compression on write-once file, data can be compressed only when
all logical blocks in file are valid and cluster compress ratio is lower
than specified threshold.
- To enable compression on regular inode, there are three ways:
* chattr +c file
* chattr +c dir; touch dir/file
* mount w/ -o compress_extension=ext; touch file.ext
Compress metadata layout:
[Dnode Structure]
+-----------------------------------------------+
| cluster 1 | cluster 2 | ......... | cluster N |
+-----------------------------------------------+
. . . .
. . . .
. Compressed Cluster . . Normal Cluster .
+----------+---------+---------+---------+ +---------+---------+---------+---------+
|compr flag| block 1 | block 2 | block 3 | | block 1 | block 2 | block 3 | block 4 |
+----------+---------+---------+---------+ +---------+---------+---------+---------+
. .
. .
. .
+-------------+-------------+----------+----------------------------+
| data length | data chksum | reserved | compressed data |
+-------------+-------------+----------+----------------------------+
Changelog:
20190326:
- fix error handling of read_end_io().
- remove unneeded comments in f2fs_encrypt_one_page().
20190327:
- fix wrong use of f2fs_cluster_is_full() in f2fs_mpage_readpages().
- don't jump into loop directly to avoid uninitialized variables.
- add TODO tag in error path of f2fs_write_cache_pages().
20190328:
- fix wrong merge condition in f2fs_read_multi_pages().
- check compressed file in f2fs_post_read_required().
20190401
- allow overwrite on non-compressed cluster.
- check cluster meta before writing compressed data.
20190402
- don't preallocate blocks for compressed file.
- add lz4 compress algorithm
- process multiple post read works in one workqueue
Now f2fs supports processing post read work in multiple workqueue,
it shows low performance due to schedule overhead of multiple
workqueue executing orderly.
20190921
- compress: support buffered overwrite
C: compress cluster flag
V: valid block address
N: NEW_ADDR
One cluster contain 4 blocks
before overwrite after overwrite
- VVVV -> CVNN
- CVNN -> VVVV
- CVNN -> CVNN
- CVNN -> CVVV
- CVVV -> CVNN
- CVVV -> CVVV
20191029
- add kconfig F2FS_FS_COMPRESSION to isolate compression related
codes, add kconfig F2FS_FS_{LZO,LZ4} to cover backend algorithm.
note that: will remove lzo backend if Jaegeuk agreed that too.
- update codes according to Eric's comments.
20191101
- apply fixes from Jaegeuk
20191113
- apply fixes from Jaegeuk
- split workqueue for fsverity
20191216
- apply fixes from Jaegeuk
20200117
- fix to avoid NULL pointer dereference
[Jaegeuk Kim]
- add tracepoint for f2fs_{,de}compress_pages()
- fix many bugs and add some compression stats
- fix overwrite/mmap bugs
- address 32bit build error, reported by Geert.
- bug fixes when handling errors and i_compressed_blocks
Reported-by:
<noreply@ellerman.id.au>
Signed-off-by:
Chao Yu <yuchao0@huawei.com>
Signed-off-by:
Jaegeuk Kim <jaegeuk@kernel.org>
-
mentioned in commit f275addc
-
mentioned in commit 676a65ee
-
mentioned in commit 5c8519f6
-
mentioned in commit 02a1ce0c
-
mentioned in commit 0c3d1c94
-
mentioned in commit ba28d24c
-
mentioned in commit 785f9159
-
mentioned in commit 86d3fae0
-
mentioned in commit b5725e88
-
mentioned in commit f17d5a9c
-
mentioned in commit bcb16259
-
mentioned in commit 904b4f7b
-
mentioned in commit c3ab4aad
-
mentioned in commit 5760d4e6
-
mentioned in commit e9722236
-
mentioned in commit da07cf2f
-
mentioned in commit 9de8bb79
-
mentioned in commit 447da182
-
mentioned in commit a193d086
-
mentioned in commit 14d5521f
-
mentioned in commit 2d892a8e
-
mentioned in commit 0377c9ce
-
mentioned in commit 465473f3
-
mentioned in commit ab46becb
-
mentioned in commit 0370701e
-
mentioned in commit b533a951
-
mentioned in commit 5fbc4bbd
-
mentioned in commit d7203397
-
mentioned in commit 6912c344
-
mentioned in commit f533f16d
-
mentioned in commit 035cf9dc
-
mentioned in commit 5b13b5b5
-
mentioned in commit 7b11e715
-
mentioned in commit 0f006f89
-
mentioned in commit 7784418f
-
mentioned in commit b8719b37
-
mentioned in commit 7c6c8dac
-
mentioned in commit dae4a078
-
mentioned in commit 4c4dcb8c
-
mentioned in commit b6202381
-
mentioned in commit 390e5fb2
-
mentioned in commit ff4dd22e
-
mentioned in commit 72c3de67
-
mentioned in commit d120a17d
-
mentioned in commit 64acb100
-
mentioned in commit 530d4804