요약·해설과 원문, 전문 번역을 서로 분리했습니다. API 이름, symbol, source path는 원문 표기를 사용합니다.
1. 요약·해설
원문의 핵심 논리와 kernel programming 관점의 보충 설명입니다. 아래의 전문 번역과는 별도로 작성했습니다.
2. 영어 원문 전체
번역 기준이 된 Linux v6.18.37 원문입니다. 줄 번호는 이 버전의 파일 좌표입니다.
원문 전체 펼치기
.. SPDX-License-Identifier: GPL-2.0
============================
Ceph Distributed File System
============================
Ceph is a distributed network file system designed to provide good
performance, reliability, and scalability.
Basic features include:
* POSIX semantics
* Seamless scaling from 1 to many thousands of nodes
* High availability and reliability. No single point of failure.
* N-way replication of data across storage nodes
* Fast recovery from node failures
* Automatic rebalancing of data on node addition/removal
* Easy deployment: most FS components are userspace daemons
Also,
* Flexible snapshots (on any directory)
* Recursive accounting (nested files, directories, bytes)
In contrast to cluster filesystems like GFS, OCFS2, and GPFS that rely
on symmetric access by all clients to shared block devices, Ceph
separates data and metadata management into independent server
clusters, similar to Lustre. Unlike Lustre, however, metadata and
storage nodes run entirely as user space daemons. File data is striped
across storage nodes in large chunks to distribute workload and
facilitate high throughputs. When storage nodes fail, data is
re-replicated in a distributed fashion by the storage nodes themselves
(with some minimal coordination from a cluster monitor), making the
system extremely efficient and scalable.
Metadata servers effectively form a large, consistent, distributed
in-memory cache above the file namespace that is extremely scalable,
dynamically redistributes metadata in response to workload changes,
and can tolerate arbitrary (well, non-Byzantine) node failures. The
metadata server takes a somewhat unconventional approach to metadata
storage to significantly improve performance for common workloads. In
particular, inodes with only a single link are embedded in
directories, allowing entire directories of dentries and inodes to be
loaded into its cache with a single I/O operation. The contents of
extremely large directories can be fragmented and managed by
independent metadata servers, allowing scalable concurrent access.
The system offers automatic data rebalancing/migration when scaling
from a small cluster of just a few nodes to many hundreds, without
requiring an administrator carve the data set into static volumes or
go through the tedious process of migrating data between servers.
When the file system approaches full, new nodes can be easily added
and things will "just work."
Ceph includes flexible snapshot mechanism that allows a user to create
a snapshot on any subdirectory (and its nested contents) in the
system. Snapshot creation and deletion are as simple as 'mkdir
.snap/foo' and 'rmdir .snap/foo'.
Snapshot names have two limitations:
* They can not start with an underscore ('_'), as these names are reserved
for internal usage by the MDS.
* They can not exceed 240 characters in size. This is because the MDS makes
use of long snapshot names internally, which follow the format:
`_<SNAPSHOT-NAME>_<INODE-NUMBER>`. Since filenames in general can't have
more than 255 characters, and `<node-id>` takes 13 characters, the long
snapshot names can take as much as 255 - 1 - 1 - 13 = 240.
Ceph also provides some recursive accounting on directories for nested files
and bytes. You can run the commands::
getfattr -n ceph.dir.rfiles /some/dir
getfattr -n ceph.dir.rbytes /some/dir
to get the total number of nested files and their combined size, respectively.
This makes the identification of large disk space consumers relatively quick,
as no 'du' or similar recursive scan of the file system is required.
Finally, Ceph also allows quotas to be set on any directory in the system.
The quota can restrict the number of bytes or the number of files stored
beneath that point in the directory hierarchy. Quotas can be set using
extended attributes 'ceph.quota.max_files' and 'ceph.quota.max_bytes', eg::
setfattr -n ceph.quota.max_bytes -v 100000000 /some/dir
getfattr -n ceph.quota.max_bytes /some/dir
A limitation of the current quotas implementation is that it relies on the
cooperation of the client mounting the file system to stop writers when a
limit is reached. A modified or adversarial client cannot be prevented
from writing as much data as it needs.
Mount Syntax
============
The basic mount syntax is::
# mount -t ceph [email protected]_name=/[subdir] mnt -o mon_addr=monip1[:port][/monip2[:port]]
You only need to specify a single monitor, as the client will get the
full list when it connects. (However, if the monitor you specify
happens to be down, the mount won't succeed.) The port can be left
off if the monitor is using the default. So if the monitor is at
1.2.3.4::
# mount -t ceph [email protected]=/ /mnt/ceph -o mon_addr=1.2.3.4
is sufficient. If /sbin/mount.ceph is installed, a hostname can be
used instead of an IP address and the cluster FSID can be left out
(as the mount helper will fill it in by reading the ceph configuration
file)::
# mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=mon-addr
Multiple monitor addresses can be passed by separating each address with a slash (`/`)::
# mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=192.168.1.100/192.168.1.101
When using the mount helper, monitor address can be read from ceph
configuration file if available. Note that, the cluster FSID (passed as part
of the device string) is validated by checking it with the FSID reported by
the monitor.
Mount Options
=============
mon_addr=ip_address[:port][/ip_address[:port]]
Monitor address to the cluster. This is used to bootstrap the
connection to the cluster. Once connection is established, the
monitor addresses in the monitor map are followed.
fsid=cluster-id
FSID of the cluster (from `ceph fsid` command).
ip=A.B.C.D[:N]
Specify the IP and/or port the client should bind to locally.
There is normally not much reason to do this. If the IP is not
specified, the client's IP address is determined by looking at the
address its connection to the monitor originates from.
wsize=X
Specify the maximum write size in bytes. Default: 64 MB.
rsize=X
Specify the maximum read size in bytes. Default: 64 MB.
rasize=X
Specify the maximum readahead size in bytes. Default: 8 MB.
mount_timeout=X
Specify the timeout value for mount (in seconds), in the case
of a non-responsive Ceph file system. The default is 60
seconds.
caps_max=X
Specify the maximum number of caps to hold. Unused caps are released
when number of caps exceeds the limit. The default is 0 (no limit)
rbytes
When stat() is called on a directory, set st_size to 'rbytes',
the summation of file sizes over all files nested beneath that
directory. This is the default.
norbytes
When stat() is called on a directory, set st_size to the
number of entries in that directory.
nocrc
Disable CRC32C calculation for data writes. If set, the storage node
must rely on TCP's error correction to detect data corruption
in the data payload.
dcache
Use the dcache contents to perform negative lookups and
readdir when the client has the entire directory contents in
its cache. (This does not change correctness; the client uses
cached metadata only when a lease or capability ensures it is
valid.)
nodcache
Do not use the dcache as above. This avoids a significant amount of
complex code, sacrificing performance without affecting correctness,
and is useful for tracking down bugs.
noasyncreaddir
Do not use the dcache as above for readdir.
noquotadf
Report overall filesystem usage in statfs instead of using the root
directory quota.
nocopyfrom
Don't use the RADOS 'copy-from' operation to perform remote object
copies. Currently, it's only used in copy_file_range, which will revert
to the default VFS implementation if this option is used.
recover_session=<no|clean>
Set auto reconnect mode in the case where the client is blocklisted. The
available modes are "no" and "clean". The default is "no".
* no: never attempt to reconnect when client detects that it has been
blocklisted. Operations will generally fail after being blocklisted.
* clean: client reconnects to the ceph cluster automatically when it
detects that it has been blocklisted. During reconnect, client drops
dirty data/metadata, invalidates page caches and writable file handles.
After reconnect, file locks become stale because the MDS loses track
of them. If an inode contains any stale file locks, read/write on the
inode is not allowed until applications release all stale file locks.
More Information
================
For more information on Ceph, see the home page at
https://ceph.com/
The Linux kernel client source tree is available at
- https://github.com/ceph/ceph-client.git
and the source for the full system is at
https://github.com/ceph/ceph.git
3. 한국어 전문 번역
영어 원문의 문단 순서와 의미를 유지한 전체 번역입니다. 코드, 함수명, symbol과 URL은 원문 표기를 유지합니다.
Ceph 분산 파일시스템 구조와 snapshot
1-59Ceph는 높은 성능, 신뢰성, 확장성을 제공하도록 설계된 분산 네트워크 파일시스템입니다. POSIX semantics, 단일 노드부터 수천 노드까지의 연속적인 확장, single point of failure가 없는 고가용성과 신뢰성, storage node 사이의 N-way data replication을 기본 특성으로 제공합니다.
노드 장애에서 빠르게 복구하고, 노드를 추가하거나 제거하면 데이터를 자동으로 재배치합니다. 대부분의 파일시스템 구성요소가 userspace daemon이므로 배포가 쉽습니다. 어느 디렉터리에서든 유연한 snapshot을 만들 수 있고 중첩된 file, directory, byte를 재귀적으로 계산할 수도 있습니다.
GFS, OCFS2, GPFS 같은 cluster filesystem은 모든 client가 공유 block device에 대칭적으로 접근하는 데 의존합니다. Ceph는 Lustre와 비슷하게 data management와 metadata management를 독립적인 server cluster로 분리합니다. 다만 Lustre와 달리 metadata node와 storage node가 전적으로 userspace daemon으로 실행됩니다.
파일 데이터는 큰 chunk로 storage node에 striping되어 workload가 분산되고 높은 throughput을 낼 수 있습니다. storage node가 실패하면 cluster monitor의 최소한의 조정만 받아 storage node들이 분산 방식으로 data를 다시 replicate하므로 효율과 확장성이 높습니다.
metadata server는 file namespace 위에 크고 일관된 distributed in-memory cache를 형성합니다. workload 변화에 맞춰 metadata를 동적으로 재분배하고 Byzantine failure가 아닌 임의의 node failure를 견딜 수 있습니다.
일반적인 workload의 성능을 높이기 위해 link가 하나뿐인 inode는 directory 안에 embed합니다. 그러면 dentry와 inode를 포함한 디렉터리 전체를 한 번의 I/O로 cache에 올릴 수 있습니다. 매우 큰 directory의 내용은 fragment로 나누어 서로 다른 metadata server가 관리할 수 있으므로 동시 접근도 확장됩니다.
작은 cluster에서 수백 node 규모로 커질 때도 데이터를 자동으로 rebalance하고 migrate합니다. 관리자가 dataset을 정적 volume으로 쪼개거나 server 사이에서 수동으로 옮길 필요가 없습니다. 파일시스템이 가득 차기 시작하면 새 node를 추가하는 것만으로 재배치가 진행됩니다.
Ceph snapshot은 임의의 subdirectory와 그 아래 내용을 대상으로 합니다. snapshot 생성은 `mkdir .snap/foo`, 삭제는 `rmdir .snap/foo`처럼 수행합니다.
client 요청이 독립적인 metadata와 storage cluster를 거치는 구조를 정리합니다.
.. SPDX-License-Identifier: GPL-2.0
============================
Ceph Distributed File System
============================
Ceph is a distributed network file system designed to provide good
performance, reliability, and scalability.
Basic features include:
* POSIX semantics
* Seamless scaling from 1 to many thousands of nodes
* High availability and reliability. No single point of failure.
* N-way replication of data across storage nodes
* Fast recovery from node failures
* Automatic rebalancing of data on node addition/removal
* Easy deployment: most FS components are userspace daemons
Also,
* Flexible snapshots (on any directory)
* Recursive accounting (nested files, directories, bytes)
In contrast to cluster filesystems like GFS, OCFS2, and GPFS that rely
on symmetric access by all clients to shared block devices, Ceph
separates data and metadata management into independent server
clusters, similar to Lustre. Unlike Lustre, however, metadata and
storage nodes run entirely as user space daemons. File data is striped
across storage nodes in large chunks to distribute workload and
facilitate high throughputs. When storage nodes fail, data is
re-replicated in a distributed fashion by the storage nodes themselves
(with some minimal coordination from a cluster monitor), making the
system extremely efficient and scalable.
Metadata servers effectively form a large, consistent, distributed
in-memory cache above the file namespace that is extremely scalable,
dynamically redistributes metadata in response to workload changes,
and can tolerate arbitrary (well, non-Byzantine) node failures. The
metadata server takes a somewhat unconventional approach to metadata
storage to significantly improve performance for common workloads. In
particular, inodes with only a single link are embedded in
directories, allowing entire directories of dentries and inodes to be
loaded into its cache with a single I/O operation. The contents of
extremely large directories can be fragmented and managed by
independent metadata servers, allowing scalable concurrent access.
The system offers automatic data rebalancing/migration when scaling
from a small cluster of just a few nodes to many hundreds, without
requiring an administrator carve the data set into static volumes or
go through the tedious process of migrating data between servers.
When the file system approaches full, new nodes can be easily added
and things will "just work."
Ceph includes flexible snapshot mechanism that allows a user to create
a snapshot on any subdirectory (and its nested contents) in the
system. Snapshot creation and deletion are as simple as 'mkdir
.snap/foo' and 'rmdir .snap/foo'.
Snapshot 이름, 재귀 회계와 quota
60-92snapshot 이름에는 두 가지 제한이 있습니다. underscore(`_`)로 시작하는 이름은 MDS 내부 용도로 예약되어 있으므로 사용할 수 없고, 길이는 240자를 넘을 수 없습니다.
MDS는 내부적으로 `_<SNAPSHOT-NAME>_<INODE-NUMBER>` 형식의 긴 이름을 사용합니다. 일반 filename 한도 255자에서 두 underscore와 13자의 inode 식별 부분을 제외하면 snapshot name에 쓸 수 있는 길이는 `255 - 1 - 1 - 13 = 240`자입니다. 원문은 설명에서 `<node-id>`라고 적지만 형식 자체는 `<INODE-NUMBER>`를 사용합니다.
Ceph는 directory 아래에 중첩된 file 수와 byte 수를 재귀적으로 계산합니다. `getfattr -n ceph.dir.rfiles /some/dir`은 중첩된 전체 file 수를, `getfattr -n ceph.dir.rbytes /some/dir`은 그 파일들의 합산 크기를 반환합니다. `du` 같은 재귀 scan 없이 큰 disk space 소비자를 빠르게 찾을 수 있습니다.
directory마다 quota를 설정할 수도 있습니다. `ceph.quota.max_files`는 해당 지점 아래의 file 수를, `ceph.quota.max_bytes`는 byte 수를 제한합니다. 예시는 `setfattr -n ceph.quota.max_bytes -v 100000000 /some/dir`로 제한을 설정하고 `getfattr -n ceph.quota.max_bytes /some/dir`로 확인합니다.
현재 quota 구현은 mount한 client가 limit에 도달했을 때 writer를 중지한다는 협력에 의존합니다. 수정되었거나 적대적인 client가 제한을 무시하면 server가 그 client의 추가 write를 막을 수 없다는 보안상 한계가 있습니다.
snapshot 제한과 recursive accounting·quota에 사용하는 이름을 구분합니다.
Snapshot names have two limitations:
* They can not start with an underscore ('_'), as these names are reserved
for internal usage by the MDS.
* They can not exceed 240 characters in size. This is because the MDS makes
use of long snapshot names internally, which follow the format:
`_<SNAPSHOT-NAME>_<INODE-NUMBER>`. Since filenames in general can't have
more than 255 characters, and `<node-id>` takes 13 characters, the long
snapshot names can take as much as 255 - 1 - 1 - 13 = 240.
Ceph also provides some recursive accounting on directories for nested files
and bytes. You can run the commands::
getfattr -n ceph.dir.rfiles /some/dir
getfattr -n ceph.dir.rbytes /some/dir
to get the total number of nested files and their combined size, respectively.
This makes the identification of large disk space consumers relatively quick,
as no 'du' or similar recursive scan of the file system is required.
Finally, Ceph also allows quotas to be set on any directory in the system.
The quota can restrict the number of bytes or the number of files stored
beneath that point in the directory hierarchy. Quotas can be set using
extended attributes 'ceph.quota.max_files' and 'ceph.quota.max_bytes', eg::
setfattr -n ceph.quota.max_bytes -v 100000000 /some/dir
getfattr -n ceph.quota.max_bytes /some/dir
A limitation of the current quotas implementation is that it relies on the
cooperation of the client mounting the file system to stop writers when a
limit is reached. A modified or adversarial client cannot be prevented
from writing as much data as it needs.
Mount 문법과 monitor bootstrap
93-123기본 mount 문법은 `mount -t ceph [email protected]_name=/[subdir] mnt -o mon_addr=monip1[:port][/monip2[:port]]`입니다. device string에는 user, cluster FSID, filesystem name과 선택적인 subdirectory를 지정하고, mount option에는 bootstrap monitor 주소를 줍니다.
client는 첫 monitor에 연결한 뒤 전체 monitor 목록을 받으므로 보통 하나만 지정하면 됩니다. 다만 지정한 monitor가 down이면 최초 mount는 실패합니다. monitor가 기본 port를 사용한다면 port를 생략할 수 있습니다.
예를 들어 monitor가 `1.2.3.4`라면 `mount -t ceph [email protected]=/ /mnt/ceph -o mon_addr=1.2.3.4`로 충분합니다.
`/sbin/mount.ceph` helper가 설치되어 있으면 IP address 대신 hostname을 사용할 수 있고 cluster FSID도 생략할 수 있습니다. helper가 Ceph configuration file을 읽어 FSID를 채우기 때문입니다. 예시는 `mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=mon-addr`입니다.
여러 monitor address는 slash(`/`)로 구분합니다. 예를 들어 `mon_addr=192.168.1.100/192.168.1.101`로 두 주소를 전달합니다.
mount helper를 사용하면 configuration file에서 monitor address도 읽을 수 있습니다. device string에 전달한 cluster FSID는 monitor가 보고한 FSID와 대조하여 검증합니다.
하나 이상의 초기 monitor에서 실제 monitor map을 얻는 절차입니다.
Mount Syntax
============
The basic mount syntax is::
# mount -t ceph [email protected]_name=/[subdir] mnt -o mon_addr=monip1[:port][/monip2[:port]]
You only need to specify a single monitor, as the client will get the
full list when it connects. (However, if the monitor you specify
happens to be down, the mount won't succeed.) The port can be left
off if the monitor is using the default. So if the monitor is at
1.2.3.4::
# mount -t ceph [email protected]=/ /mnt/ceph -o mon_addr=1.2.3.4
is sufficient. If /sbin/mount.ceph is installed, a hostname can be
used instead of an IP address and the cluster FSID can be left out
(as the mount helper will fill it in by reading the ceph configuration
file)::
# mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=mon-addr
Multiple monitor addresses can be passed by separating each address with a slash (`/`)::
# mount -t ceph cephuser@cephfs=/ /mnt/ceph -o mon_addr=192.168.1.100/192.168.1.101
When using the mount helper, monitor address can be read from ceph
configuration file if available. Note that, the cluster FSID (passed as part
of the device string) is validated by checking it with the FSID reported by
the monitor.
Ceph mount options
124-210`mon_addr=ip_address[:port][/ip_address[:port]]`는 cluster 연결을 bootstrap할 monitor 주소입니다. 연결 뒤에는 monitor map의 주소를 따릅니다. `fsid=cluster-id`는 `ceph fsid` 명령으로 얻는 cluster FSID입니다. `ip=A.B.C.D[:N]`은 client가 로컬에서 bind할 IP와 port를 지정하며, 생략하면 monitor 연결의 출발 주소로 client IP를 판단합니다.
`wsize=X`와 `rsize=X`는 각각 최대 write·read 크기를 byte 단위로 정하며 기본값은 모두 64 MB입니다. `rasize=X`는 최대 readahead 크기이며 기본값은 8 MB입니다. `mount_timeout=X`는 응답하지 않는 Ceph filesystem을 mount할 때 기다릴 시간으로 기본값은 60초입니다.
`caps_max=X`는 보유할 capability의 최대 개수입니다. cap 수가 한도를 넘으면 사용하지 않는 cap을 반환합니다. 기본값 0은 제한이 없다는 뜻입니다.
`rbytes`는 directory에 `stat()`을 호출했을 때 `st_size`를 그 아래 모든 file size의 합인 recursive bytes로 설정하며 기본 동작입니다. `norbytes`는 대신 해당 directory의 직접 entry 수를 `st_size`로 설정합니다.
`nocrc`는 data write의 CRC32C 계산을 끕니다. 이 경우 storage node는 payload corruption 검출을 TCP error correction에 의존해야 합니다.
`dcache`는 client가 directory 전체 내용을 cache하고 있을 때 negative lookup과 `readdir`에 dcache를 사용합니다. lease 또는 capability가 cached metadata의 유효성을 보장할 때만 사용하므로 correctness는 바뀌지 않습니다. `nodcache`는 이 최적화를 끄며 성능을 희생하는 대신 복잡한 코드를 피하므로 bug 추적에 유용합니다. `noasyncreaddir`는 `readdir`에서만 이 dcache 사용을 끕니다.
`noquotadf`는 `statfs`가 root directory quota 대신 filesystem 전체 사용량을 보고하게 합니다.
`nocopyfrom`은 remote object copy에 RADOS `copy-from` operation을 사용하지 않습니다. 현재 이 operation은 `copy_file_range`에서만 사용되며, option을 켜면 기본 VFS 구현으로 돌아갑니다.
`recover_session=<no|clean>`은 client가 blocklist에 올랐을 때의 자동 reconnect 모드를 지정합니다. 기본값 `no`는 reconnect를 시도하지 않으므로 blocklist 이후 operation이 대체로 실패합니다.
`clean`은 blocklist를 감지하면 Ceph cluster에 자동으로 reconnect합니다. reconnect 중 dirty data와 metadata를 버리고 page cache와 writable file handle을 invalidation합니다. MDS가 기존 file lock 추적을 잃으므로 lock은 stale이 됩니다. inode에 stale file lock이 하나라도 있으면 application이 이를 모두 release할 때까지 해당 inode의 read/write는 허용되지 않습니다.
I/O 크기, capability, directory size와 cache 최적화 option을 정리합니다.
데이터 검증, quota 보고, remote copy와 blocklist 복구의 trade-off입니다.
Mount Options
=============
mon_addr=ip_address[:port][/ip_address[:port]]
Monitor address to the cluster. This is used to bootstrap the
connection to the cluster. Once connection is established, the
monitor addresses in the monitor map are followed.
fsid=cluster-id
FSID of the cluster (from `ceph fsid` command).
ip=A.B.C.D[:N]
Specify the IP and/or port the client should bind to locally.
There is normally not much reason to do this. If the IP is not
specified, the client's IP address is determined by looking at the
address its connection to the monitor originates from.
wsize=X
Specify the maximum write size in bytes. Default: 64 MB.
rsize=X
Specify the maximum read size in bytes. Default: 64 MB.
rasize=X
Specify the maximum readahead size in bytes. Default: 8 MB.
mount_timeout=X
Specify the timeout value for mount (in seconds), in the case
of a non-responsive Ceph file system. The default is 60
seconds.
caps_max=X
Specify the maximum number of caps to hold. Unused caps are released
when number of caps exceeds the limit. The default is 0 (no limit)
rbytes
When stat() is called on a directory, set st_size to 'rbytes',
the summation of file sizes over all files nested beneath that
directory. This is the default.
norbytes
When stat() is called on a directory, set st_size to the
number of entries in that directory.
nocrc
Disable CRC32C calculation for data writes. If set, the storage node
must rely on TCP's error correction to detect data corruption
in the data payload.
dcache
Use the dcache contents to perform negative lookups and
readdir when the client has the entire directory contents in
its cache. (This does not change correctness; the client uses
cached metadata only when a lease or capability ensures it is
valid.)
nodcache
Do not use the dcache as above. This avoids a significant amount of
complex code, sacrificing performance without affecting correctness,
and is useful for tracking down bugs.
noasyncreaddir
Do not use the dcache as above for readdir.
noquotadf
Report overall filesystem usage in statfs instead of using the root
directory quota.
nocopyfrom
Don't use the RADOS 'copy-from' operation to perform remote object
copies. Currently, it's only used in copy_file_range, which will revert
to the default VFS implementation if this option is used.
recover_session=<no|clean>
Set auto reconnect mode in the case where the client is blocklisted. The
available modes are "no" and "clean". The default is "no".
* no: never attempt to reconnect when client detects that it has been
blocklisted. Operations will generally fail after being blocklisted.
* clean: client reconnects to the ceph cluster automatically when it
detects that it has been blocklisted. During reconnect, client drops
dirty data/metadata, invalidates page caches and writable file handles.
After reconnect, file locks become stale because the MDS loses track
of them. If an inode contains any stale file locks, read/write on the
inode is not allowed until applications release all stale file locks.
추가 정보와 source tree
211-221Ceph의 추가 정보는 공식 홈페이지 `https://ceph.com/`에서 확인할 수 있습니다.
Linux kernel client source tree는 `https://github.com/ceph/ceph-client.git`에 있고, Ceph 전체 시스템 source는 `https://github.com/ceph/ceph.git`에 있습니다. 첫 URL은 kernel client만, 두 번째 URL은 userspace daemon과 전체 시스템을 포함하는 저장소를 가리킵니다.
운영 문서와 kernel client·전체 시스템 source를 구분합니다.
More Information
================
For more information on Ceph, see the home page at
https://ceph.com/
The Linux kernel client source tree is available at
- https://github.com/ceph/ceph-client.git
and the source for the full system is at
https://github.com/ceph/ceph.git
요약·해설
ceph.rst:1-221CephFS는 metadata를 MDS cluster가, file data를 storage node cluster가 독립적으로 관리하는 분산 파일시스템입니다. metadata cache와 directory fragment를 workload에 맞게 이동하고, data를 큰 chunk로 striping·replication하여 노드 추가·제거와 장애 복구 때 자동으로 rebalance합니다.
directory 단위 snapshot, recursive accounting과 quota를 제공하며 kernel client는 monitor를 통해 cluster map을 얻어 mount합니다. 성능 option 외에도 CRC, quota 표시, RADOS copy, blocklist 후 session 복구 정책을 mount 시 명시할 수 있습니다. quota enforcement와 `recover_session=clean`의 state 폐기·stale lock 조건은 운영 시 특히 주의해야 합니다.
mount에서 namespace·data 접근과 장애 복구까지의 핵심 경로입니다.