HDFS - EditLog (transaction log)

Yarn Hortonworks

About

The NameNode uses a transaction log called the EditLog to persistently record every change that occurs to file system metadata.

Example

The NameNode to insert a record into the EditLog when

  • a new file is created
  • the replication factor of a file is changed

Management

Offline Edits Viewer

Offline Edits Viewer is a tool to parse the Edits log file.

Location

The NameNode uses a file in its local host OS file system to store the EditLog.





Discover More
Yarn Hortonworks
HDFS - Checkpoint

During a checkpoint the changes from the transaction log (Editlog) are applied to the metadata store (FsImage) (because it's not efficient to record each change on the metadata store (FsImage) When...
Yarn Hortonworks
HDFS - File System Metadata

The file system metadata section of HDFS. The NameNode is the repository of all HDFS metadata. The metadata are stored in two files: fsimage file which is the metadata store EditLog transaction...
Hadoop Hdfs Fsimage
HDFS - FsImage File

The HDFS file system metadata are stored in a file called the FsImage. It contains: the entire file system namespace the mapping of blocks to files and file system properties The FsImage...
Yarn Hortonworks
HDFS - High Availibilty

The metadata files (FsImage and EditLog) are central data structures of HDFS. A corruption of these files can cause the HDFS instance to be non-functional. In a typical HA cluster: two separate...



Share this page:
Follow us:
Task Runner