Hadoop快照介绍

4rfdf

　　快照使用场景：

• Protection against user errors（防止用户误操作）:  Admin sets up a process to take read-only (RO) snapshots periodically in a rolling manner so that there are always x number of RO snapshots on HDFS. If a user accidentally deletes a file, the file can be restored from the latest RO snapshot that contains the file.
  
• Backup（备份）: Admin wants backup the entire file system, a subtree in the file system or just a file. Depending on the requirements, admin takes a read-only (henceforth referred to as RO) snapshot and uses this snapshot as the starting point of a full backup. Incremental backups are then taken by doing a diff between two snapshots.
  
• Experimental/Test setups（进行试验或测试）:  A user wants to test an application against the main dataset. Normally, without doing a full copy of the dataset, this is a very risky proposition because the test setup can corrupt/overwrite production data. Admin creates a read-write (henceforth referred to as RW) snapshot of the production dataset and assigns the RW snapshot to the user to be used for experiment. Changes done to the RW snapshot will not be reflected on the production dataset.
  
• Disaster Recovery（灾难恢复）:  RO Snapshots can be used to create a consistent point in time image for replication and this can be copied over to remote site for Disaster Recovery.
  

　　快照类别：

• Read-only (RO) snapshots: These are immutable copies of underlying elements of the file system.
  
• Read-write (RW) snapshots: RW snaps can be modified by a user.
  

　　快照的两种实现：
　　Option #1（namenode与datanode都做快照）: Both datanodes and namenode are aware of the snapshots and save state internally about the snapshots. Datanode is aware of the fact that some of the blocks are for the snapshot files.
　　Option #2（仅namenode做快照）: Only namenode is aware of the snapshot. Datanode is not aware of the fact that some of the blocks are owned by snapshots of the original file.
　　快照操作:
　　A key requirement is to ensure that it is very easy to create and delete snapshots. Snapshot creation and deletion is an admin-only capability. To create a snapshot, one specifies a  snapshot name, a path to the root of the subtree whose snapshot is to be taken, and whether or not the snapshot is read-only or a read-write. Deleting snapshot requires just a snapshot name. A command to list all the snaps in the filesystem will be provided.（需保证操作简捷，一键完成）
　　快照使用：
　　Examples:  Consider a directory structure of /a/b/c/foo.txt. Admin has created a snapshot hdfs1 at /a/b. To access data related to snapshot hdfs1, some examples of the commands would be:
　　hadoop dfs -ls /a/b/.snapshot_hdfs1/c/foo.txt
　　快照说明：
　　Snapshots are a very useful feature to have in a mature filesystem. This is a work in progress and we have a functional prototype implemented. The first version of this feature will support RO snapshots only. The support for RW snapshots will be added in the subsequent releases. There are several features that can be incorporated into snapshots, such as time to live for snapshots with auto deletion, schedule based creation of snapshots, marking specific directories as snapshot-worthy, quota based restriction on space used by RW snapshots and delegation of authority for creating/deleting snapshots at specific locations to users etc.（目前只有RO模式，RW模式后续开发）
　　HBASE使用快照代替EXP/CP的好处
　　Aside from the better consistency guarantees that a snapshot can provide compared to a Copy\/Export Job, the main difference between Exporting a Snapshot and Copying\/Exporting a table is that ExportSnapshot operates at HDFS level. This means that Master and Region Servers are not involved in this operations. Consequently, no unnecessary caches for data are created and there is no triggering of additional GC pauses due to the number of objects created during the scan process. The performance impact on the HBase cluster stems from the extra network and disk workload experienced by the DataNodes.（主要是操作级别的问题，快照不需要在Master与Region服务器上进行，不需要进行垃圾回收等操作）