Volume 16, No 2, 2019
Development of Intellectual System for Data De-Duplication and Distribution in Cloud Storage
Vasyl Lytvyn, Victoria Vysotska, Mykhailo Osypov, Olha Slyusarchuk and Yuriy Slyusarchuk
Abstract
The system for backing up the data is designed. Client software works on the computer of user, takes all the necessary files for backup, and turns them into Stream of bytes. Then breaks it into blocks (from 32 KB to 64KB) using a Rabin algorithm. It is based on hash ring that is absorbing every incoming byte if the current hash mask or equal to a certain reach 64KB, there is division committed. This approach helps to avoid is coping all data if the content of a file has changed in separately. For each data block client software calculates the hash. Then sends parts even 256 times or more of those hashes to the server and checks if they already know the system. The blocks are not known, refers to the server. As part of the de-duplication, data on distribution and cloud distribution occurs on hashes, hashes are SHA-1, so that 20 bytes are given in hexadecimal format. First number or letter and will serve key distribution. So essentially, you can evenly distribute the data among the workers from 2 to 16 pieces. You need to take the second number of hash, and more commit distribution.
Pages: 1-42
Keywords: Stochastic game; Clustering, Ontology; Knowledge base; Intelligent agent; Data Sharing; Data de-duplication; Data Hashing; Cloud Environment; Cloud computing; Rabin algorithm; Hybrid De-duplication