HBase is using more disk space than expected
HBase StoreFiles (also called HFiles) store HBase row data on disk.
HBase stores other information on disk, such as write-ahead logs (WALs),
snapshots, data that would otherwise be deleted but would be needed to
restore from a stored snapshot.
| Location | Purpose | Troubleshooting Notes |
|---|---|---|
/hbase/.snapshots
|
Contains one subdirectory per snapshot. | To list snapshots, use the HBase Shell command
list_snapshots. To remove a snapshot, use
delete_snapshot. |
/hbase/.archive
|
Contains data that would otherwise have been deleted (either because it was explicitly deleted or expired due to TTL or version limits on the table) but that is required to restore from an existing snapshot. | To free up space being taken up by excessive archives,
delete the snapshots that refer to them. Snapshots never expire
so data referred to by them is kept until the snapshot is
removed. Do not remove anything from
/hbase/.archive manually, or you will corrupt
your snapshots. |
/hbase/.logs
|
Contains HBase WAL files that are required to recover regions in the event of a RegionServer failure. | WALs are removed when their contents are verified to have
been written to StoreFiles. Do not remove them manually. If the
size of any subdirectory of /hbase/.logs/ is
growing, examine the HBase server logs to find the root cause
for why WALs are not being processed correctly. |
/hbase/logs/.oldWALs
|
Contains HBase WAL files that have already been written to disk. A HBase maintenance thread removes them periodically based on a TTL. | To tune the length of time a WAL stays in the
.oldWALs before it is removed, configure the
hbase.master.logcleaner.ttl property, which
defaults to 60000 milliseconds, or 1 hour. |
/hbase/.logs/.corrupt
|
Contains corrupted HBase WAL files. | Do not remove corrupt WALs manually. If the size of any
subdirectory of /hbase/.logs/ is growing,
examine the HBase server logs to find the root cause for why
WALs are not being processed correctly. |
