Secure File sharing Mechanism with OTP Service in Big Data Environment
Keywords:
Hadoop, HDFS, Map Reduce, Name Node, Data Node, Task Tracker, Job TrackerAbstract
File sharing has been an essential part of this century. Using various applications, files can be shared to large number of users. For the purpose of storage, the Hadoop Distributed File System (HDFS) can be used. HDFS is mainly used for the unstructured data analysis. The HDFS handles large size of files in a single server. Common sharing methods like removable media, servers or computer network, World Wide Web based hyperlink documents. In the proposed project, the files are merged using MapReduce programming model on Hadoop. This process improves the performance of Hadoop by rejecting the files which are larger than the size of Hadoop and reduces the memory size required by the NameNode
References
An approach to solve a Small File problem in Hadoop by using Dynamic Merging and Indexing Scheme
An Improved HDFS for Small File by Liu Changtong China
Dealing with Small Files Problem in Hadoop 1Distributed File System
Dealing with Small Files Problem in Hadoop Distributed File System by Sachin Bendea, Rajashree Shedgeb
Dean, Jeffrey; Ghemawat, Sanjay. "MapReduce: Simplified Data Processing on Large Clusters" [5] "Google Research Publication: The Google File System".
Defining Hadoop". Wiki. Apache. Org
Google Research Publication: MapReduce".
NHAR: Archive and Metadata Distribution! Why Not Both? By Dipayan Dev, Ripon Patgiri
What is the Hadoop Distributed File System (HDFS)?"
Palmer, B., “Hadoop: Strengths and Limitations in National Security Missions”, SAP National Security Services
