요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
Block Groups
------------
Layout
~~~~~~
The layout of a standard block group is approximately as follows (each
of these fields is discussed in a separate section below):
.. list-table::
:widths: 1 1 1 1 1 1 1 1
:header-rows: 1
* - Group 0 Padding
- ext4 Super Block
- Group Descriptors
- Reserved GDT Blocks
- Data Block Bitmap
- inode Bitmap
- inode Table
- Data Blocks
* - 1024 bytes
- 1 block
- many blocks
- many blocks
- 1 block
- 1 block
- many blocks
- many more blocks
For the special case of block group 0, the first 1024 bytes are unused,
to allow for the installation of x86 boot sectors and other oddities.
The superblock will start at offset 1024 bytes, whichever block that
happens to be (usually 0). However, if for some reason the block size =
1024, then block 0 is marked in use and the superblock goes in block 1.
For all other block groups, there is no padding.
The ext4 driver primarily works with the superblock and the group
descriptors that are found in block group 0. Redundant copies of the
superblock and group descriptors are written to some of the block groups
across the disk in case the beginning of the disk gets trashed, though
not all block groups necessarily host a redundant copy (see following
paragraph for more details). If the group does not have a redundant
copy, the block group begins with the data block bitmap. Note also that
when the filesystem is freshly formatted, mkfs will allocate “reserve
GDT block” space after the block group descriptors and before the start
of the block bitmaps to allow for future expansion of the filesystem. By
default, a filesystem is allowed to increase in size by a factor of
1024x over the original filesystem size.
The location of the inode table is given by ``grp.bg_inode_table_*``. It
is continuous range of blocks large enough to contain
``sb.s_inodes_per_group * sb.s_inode_size`` bytes.
As for the ordering of items in a block group, it is generally
established that the super block and the group descriptor table, if
present, will be at the beginning of the block group. The bitmaps and
the inode table can be anywhere, and it is quite possible for the
bitmaps to come after the inode table, or for both to be in different
groups (flex_bg). Leftover space is used for file data blocks, indirect
block maps, extent tree blocks, and extended attributes.
Flexible Block Groups
~~~~~~~~~~~~~~~~~~~~~
Starting in ext4, there is a new feature called flexible block groups
(flex_bg). In a flex_bg, several block groups are tied together as one
logical block group; the bitmap spaces and the inode table space in the
first block group of the flex_bg are expanded to include the bitmaps
and inode tables of all other block groups in the flex_bg. For example,
if the flex_bg size is 4, then group 0 will contain (in order) the
superblock, group descriptors, data block bitmaps for groups 0-3, inode
bitmaps for groups 0-3, inode tables for groups 0-3, and the remaining
space in group 0 is for file data. The effect of this is to group the
block group metadata close together for faster loading, and to enable
large files to be continuous on disk. Backup copies of the superblock
and group descriptors are always at the beginning of block groups, even
if flex_bg is enabled. The number of block groups that make up a
flex_bg is given by 2 ^ ``sb.s_log_groups_per_flex``.
Meta Block Groups
~~~~~~~~~~~~~~~~~
Without the option META_BG, for safety concerns, all block group
descriptors copies are kept in the first block group. Given the default
128MiB(2^27 bytes) block group size and 64-byte group descriptors, ext4
can have at most 2^27/64 = 2^21 block groups. This limits the entire
filesystem size to 2^21 * 2^27 = 2^48bytes or 256TiB.
The solution to this problem is to use the metablock group feature
(META_BG), which is already in ext3 for all 2.6 releases. With the
META_BG feature, ext4 filesystems are partitioned into many metablock
groups. Each metablock group is a cluster of block groups whose group
descriptor structures can be stored in a single disk block. For ext4
filesystems with 4 KB block size, a single metablock group partition
includes 64 block groups, or 8 GiB of disk space. The metablock group
feature moves the location of the group descriptors from the congested
first block group of the whole filesystem into the first group of each
metablock group itself. The backups are in the second and last group of
each metablock group. This increases the 2^21 maximum block groups limit
to the hard limit 2^32, allowing support for a 512PiB filesystem.
The change in the filesystem format replaces the current scheme where
the superblock is followed by a variable-length set of block group
descriptors. Instead, the superblock and a single block group descriptor
block is placed at the beginning of the first, second, and last block
groups in a meta-block group. A meta-block group is a collection of
block groups which can be described by a single block group descriptor
block. Since the size of the block group descriptor structure is 64
bytes, a meta-block group contains 16 block groups for filesystems with
a 1KB block size, and 64 block groups for filesystems with a 4KB
blocksize. Filesystems can either be created using this new block group
descriptor layout, or existing filesystems can be resized on-line, and
the field s_first_meta_bg in the superblock will indicate the first
block group using this new layout.
Please see an important note about ``BLOCK_UNINIT`` in the section about
block and inode bitmaps.
Lazy Block Group Initialization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A new feature for ext4 are three block group descriptor flags that
enable mkfs to skip initializing other parts of the block group
metadata. Specifically, the INODE_UNINIT and BLOCK_UNINIT flags mean
that the inode and block bitmaps for that group can be calculated and
therefore the on-disk bitmap blocks are not initialized. This is
generally the case for an empty block group or a block group containing
only fixed-location block group metadata. The INODE_ZEROED flag means
that the inode table has been initialized; mkfs will unset this flag and
rely on the kernel to initialize the inode tables in the background.
By not writing zeroes to the bitmaps and inode table, mkfs time is
reduced considerably. Note the feature flag is RO_COMPAT_GDT_CSUM,
but the dumpe2fs output prints this as “uninit_bg”. They are the same
thing.
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
표준 block group layout
1-63표준 block group은 대략 group 0 padding, ext4 super block, group descriptor, reserved GDT block, data block bitmap, inode bitmap, inode table, data block 순으로 구성됩니다.
group 0의 첫 1024 bytes는 x86 boot sector 등 특수 용도를 위해 비워 둡니다. superblock은 어떤 block에 속하든 offset 1024 bytes에서 시작하며 보통 block 0입니다. block size가 1024이면 block 0을 사용 중으로 표시하고 superblock을 block 1에 둡니다. 다른 block group에는 padding이 없습니다.
ext4 driver는 주로 block group 0의 superblock과 group descriptor를 사용합니다. disk 시작 부분이 손상될 때를 대비해 일부 block group에 redundant copy를 두지만 모든 group이 사본을 갖는 것은 아닙니다. 사본이 없는 group은 data block bitmap부터 시작합니다.
새로 format한 filesystem에서 `mkfs`는 group descriptor 뒤와 block bitmap 앞에 reserved GDT block 공간을 할당해 미래 확장을 허용합니다. 기본적으로 원래 filesystem size의 1024배까지 증가할 수 있습니다.
inode table 위치는 `grp.bg_inode_table_*`에 있습니다. 크기는 `sb.s_inodes_per_group * sb.s_inode_size` bytes를 담을 수 있는 연속 block 범위입니다.
super block과 group descriptor table이 존재한다면 group 시작에 옵니다. bitmap과 inode table은 어디에나 있을 수 있으며 bitmap이 inode table 뒤에 오거나 둘이 다른 group에 놓일 수도 있습니다(`flex_bg`). 남은 공간은 file data block, indirect block map, extent tree block, extended attribute에 사용합니다.
원문 layout 표의 순서와 대표 크기입니다.
.. SPDX-License-Identifier: GPL-2.0
Block Groups
------------
Layout
~~~~~~
The layout of a standard block group is approximately as follows (each
of these fields is discussed in a separate section below):
.. list-table::
:widths: 1 1 1 1 1 1 1 1
:header-rows: 1
* - Group 0 Padding
- ext4 Super Block
- Group Descriptors
- Reserved GDT Blocks
- Data Block Bitmap
- inode Bitmap
- inode Table
- Data Blocks
* - 1024 bytes
- 1 block
- many blocks
- many blocks
- 1 block
- 1 block
- many blocks
- many more blocks
For the special case of block group 0, the first 1024 bytes are unused,
to allow for the installation of x86 boot sectors and other oddities.
The superblock will start at offset 1024 bytes, whichever block that
happens to be (usually 0). However, if for some reason the block size =
1024, then block 0 is marked in use and the superblock goes in block 1.
For all other block groups, there is no padding.
The ext4 driver primarily works with the superblock and the group
descriptors that are found in block group 0. Redundant copies of the
superblock and group descriptors are written to some of the block groups
across the disk in case the beginning of the disk gets trashed, though
not all block groups necessarily host a redundant copy (see following
paragraph for more details). If the group does not have a redundant
copy, the block group begins with the data block bitmap. Note also that
when the filesystem is freshly formatted, mkfs will allocate “reserve
GDT block” space after the block group descriptors and before the start
of the block bitmaps to allow for future expansion of the filesystem. By
default, a filesystem is allowed to increase in size by a factor of
1024x over the original filesystem size.
The location of the inode table is given by ``grp.bg_inode_table_*``. It
is continuous range of blocks large enough to contain
``sb.s_inodes_per_group * sb.s_inode_size`` bytes.
As for the ordering of items in a block group, it is generally
established that the super block and the group descriptor table, if
present, will be at the beginning of the block group. The bitmaps and
the inode table can be anywhere, and it is quite possible for the
bitmaps to come after the inode table, or for both to be in different
groups (flex_bg). Leftover space is used for file data blocks, indirect
block maps, extent tree blocks, and extended attributes.flexible block group
64-82ext4의 `flex_bg`는 여러 block group을 하나의 logical block group으로 묶습니다. flex_bg 첫 group의 bitmap 공간과 inode table 공간을 확장해 flex_bg 안의 모든 다른 group의 bitmap과 inode table을 함께 담습니다.
flex_bg size가 4라면 group 0에 superblock, group descriptor, group 0~3의 data block bitmap, group 0~3의 inode bitmap, group 0~3의 inode table이 순서대로 옵니다. group 0의 나머지 공간은 file data에 씁니다.
이 방식은 block group metadata를 가까이 모아 빠르게 load하고 large file을 disk에서 연속 배치할 수 있게 합니다.
flex_bg가 활성화돼도 superblock과 group descriptor backup은 항상 해당 block group 시작에 있습니다. flex_bg를 이루는 block group 수는 `2 ^ sb.s_log_groups_per_flex`입니다.
group 0에 네 group의 metadata를 모으는 순서입니다.
Flexible Block Groups
~~~~~~~~~~~~~~~~~~~~~
Starting in ext4, there is a new feature called flexible block groups
(flex_bg). In a flex_bg, several block groups are tied together as one
logical block group; the bitmap spaces and the inode table space in the
first block group of the flex_bg are expanded to include the bitmaps
and inode tables of all other block groups in the flex_bg. For example,
if the flex_bg size is 4, then group 0 will contain (in order) the
superblock, group descriptors, data block bitmaps for groups 0-3, inode
bitmaps for groups 0-3, inode tables for groups 0-3, and the remaining
space in group 0 is for file data. The effect of this is to group the
block group metadata close together for faster loading, and to enable
large files to be continuous on disk. Backup copies of the superblock
and group descriptors are always at the beginning of block groups, even
if flex_bg is enabled. The number of block groups that make up a
flex_bg is given by 2 ^ ``sb.s_log_groups_per_flex``.
META_BG와 filesystem 크기 확장
83-120`META_BG`가 없으면 안전을 위해 모든 block group descriptor 사본을 첫 block group에 보관합니다. 기본 128MiB(`2^27` bytes) block group과 64-byte descriptor에서는 최대 group 수가 `2^27 / 64 = 2^21`입니다.
따라서 전체 filesystem size는 `2^21 * 2^27 = 2^48 bytes`, 즉 256TiB로 제한됩니다.
해결책인 metablock group feature(`META_BG`)는 ext3의 모든 2.6 release에도 들어 있습니다. filesystem을 여러 metablock group으로 나누며, 각 metablock group은 descriptor structure 전체가 disk block 하나에 들어가는 block group 집합입니다.
4KiB block size ext4에서는 metablock group 하나가 64 block group 또는 8GiB disk space를 포함합니다. group descriptor를 전체 filesystem의 혼잡한 첫 group에서 각 metablock group의 첫 group으로 옮기고, backup은 각 metablock group의 두 번째와 마지막 group에 둡니다.
이 방식은 최대 block group 수를 `2^21`에서 hard limit `2^32`로 늘려 512PiB filesystem을 지원합니다.
새 format은 superblock 뒤에 가변 길이 descriptor 집합을 두는 대신 meta-block group의 첫 번째, 두 번째, 마지막 block group 시작에 superblock과 group descriptor block 하나를 둡니다.
64-byte descriptor를 사용할 때 1KiB block filesystem의 meta-block group은 16 block group, 4KiB block filesystem은 64 block group을 포함합니다. 새 layout으로 filesystem을 만들거나 기존 filesystem을 online resize할 수 있으며 superblock의 `s_first_meta_bg`가 새 layout을 처음 사용하는 block group을 가리킵니다.
`BLOCK_UNINIT` 해석에는 block과 inode bitmap 절의 중요한 주의 사항을 함께 적용해야 합니다.
descriptor 배치 변경이 block group과 filesystem 상한에 미치는 영향입니다.
Meta Block Groups
~~~~~~~~~~~~~~~~~
Without the option META_BG, for safety concerns, all block group
descriptors copies are kept in the first block group. Given the default
128MiB(2^27 bytes) block group size and 64-byte group descriptors, ext4
can have at most 2^27/64 = 2^21 block groups. This limits the entire
filesystem size to 2^21 * 2^27 = 2^48bytes or 256TiB.
The solution to this problem is to use the metablock group feature
(META_BG), which is already in ext3 for all 2.6 releases. With the
META_BG feature, ext4 filesystems are partitioned into many metablock
groups. Each metablock group is a cluster of block groups whose group
descriptor structures can be stored in a single disk block. For ext4
filesystems with 4 KB block size, a single metablock group partition
includes 64 block groups, or 8 GiB of disk space. The metablock group
feature moves the location of the group descriptors from the congested
first block group of the whole filesystem into the first group of each
metablock group itself. The backups are in the second and last group of
each metablock group. This increases the 2^21 maximum block groups limit
to the hard limit 2^32, allowing support for a 512PiB filesystem.
The change in the filesystem format replaces the current scheme where
the superblock is followed by a variable-length set of block group
descriptors. Instead, the superblock and a single block group descriptor
block is placed at the beginning of the first, second, and last block
groups in a meta-block group. A meta-block group is a collection of
block groups which can be described by a single block group descriptor
block. Since the size of the block group descriptor structure is 64
bytes, a meta-block group contains 16 block groups for filesystems with
a 1KB block size, and 64 block groups for filesystems with a 4KB
blocksize. Filesystems can either be created using this new block group
descriptor layout, or existing filesystems can be resized on-line, and
the field s_first_meta_bg in the superblock will indicate the first
block group using this new layout.
Please see an important note about ``BLOCK_UNINIT`` in the section about
block and inode bitmaps.lazy block group initialization
121-138ext4는 `mkfs`가 block group metadata 일부를 즉시 초기화하지 않아도 되도록 세 가지 block group descriptor flag를 제공합니다.
`INODE_UNINIT`과 `BLOCK_UNINIT`은 해당 group의 inode bitmap과 block bitmap을 계산할 수 있으므로 on-disk bitmap block을 초기화하지 않았음을 뜻합니다. 보통 빈 block group이거나 고정 위치 block group metadata만 든 group입니다.
`INODE_ZEROED`는 inode table이 초기화됐음을 뜻합니다. `mkfs`는 이 flag를 unset하고 kernel이 background에서 inode table을 초기화하도록 맡깁니다.
bitmap과 inode table에 zero를 쓰지 않으므로 `mkfs` 시간이 크게 줄어듭니다. feature flag 이름은 `RO_COMPAT_GDT_CSUM`이지만 `dumpe2fs` 출력에는 `uninit_bg`로 표시되며 둘은 같은 기능입니다.
생성 시 생략할 metadata 초기화와 상태를 나타냅니다.
Lazy Block Group Initialization
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
A new feature for ext4 are three block group descriptor flags that
enable mkfs to skip initializing other parts of the block group
metadata. Specifically, the INODE_UNINIT and BLOCK_UNINIT flags mean
that the inode and block bitmaps for that group can be calculated and
therefore the on-disk bitmap blocks are not initialized. This is
generally the case for an empty block group or a block group containing
only fixed-location block group metadata. The INODE_ZEROED flag means
that the inode table has been initialized; mkfs will unset this flag and
rely on the kernel to initialize the inode tables in the background.
By not writing zeroes to the bitmaps and inode table, mkfs time is
reduced considerably. Note the feature flag is RO_COMPAT_GDT_CSUM,
but the dumpe2fs output prints this as “uninit_bg”. They are the same
thing.
요약·해설
blockgroup.rst:1-138ext4 block group은 superblock·descriptor·bitmap·inode table·data를 담습니다. `flex_bg`는 여러 group의 metadata를 모아 locality를 높이고, `META_BG`는 descriptor를 분산해 256TiB 한계를 512PiB까지 늘립니다. lazy initialization은 계산 가능한 bitmap과 inode table zeroing을 뒤로 미뤄 mkfs 시간을 줄입니다.
기본 layout에서 대규모 filesystem 지원으로 이어지는 구조입니다.