Skip to content

Commit

Permalink
update docs
Browse files Browse the repository at this point in the history
  • Loading branch information
DanielUH2019 committed Jul 12, 2023
1 parent dc0e83f commit 6692c6e
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 1 deletion.
11 changes: 11 additions & 0 deletions docs/description/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -697,6 +697,16 @@ <h1 id="system-description">System Description</h1>
<p>The system utilizes a similar approach to Kademlia, where keys are stored as 160-bit values. Each node in the network has a unique ID, and the (key, value) pairs are stored on nodes with "close" IDs using the XOR metric proposed by Kademlia.</p>
<p>The system treats nodes as leaves of a binary tree, where the position of each node is determined by the shortest unique prefix of its ID. These nodes store contact information to route query messages.</p>
<p>The Kademlia protocol is used, which includes the PING, STORE, FINDNODE, and FIND-VALUE Remote Procedure Calls (RPCs). Additionally, other RPCs are implemented, such as CONTAINS, BOOTSTRAPPABLE-NEIGHBOR, GET, GET-FILE-CHUNKS, UPLOAD-FILE, SET-KEY, CHECK-IF-NEW-VALUE-EXIST, GET-FILE-CHUNK-VALUE, GET-FILE-CHUNK-LOCATION, FIND-CHUNK-LOCATION, and FIND-NEIGHBORS.</p>
<ul>
<li><code>GET</code>: Retrieves information identified by the key of a file, returns a list with the keys of each chunk.</li>
<li><code>UPLOAD-FILE</code>: Uploads the file to the file system, divides it into chunks, and saves the file's metadata in a way that unifies all the chunks.</li>
<li><code>GET-FIND-CHUNK-VALUE</code>: Returns the value associated with the chunk.</li>
<li><code>GET-FIND-CHUNK-LOCATION</code>: Receives the key of a chunk and returns a tuple (IP, port) where it is located.</li>
<li><code>FIND-NEIGHBORS</code>: Returns the neighbors to the node according to proximity based on the XOR metric. Additionally, the servers use the following RPCs:</li>
<li><code>CONTAINS</code>: Determines if a key is in a node. This is used for both information replication and to find if a node has the desired information.</li>
<li><code>GET-FILE-CHUNKS</code>: Retrieves the list of chunk locations for the information.</li>
<li><code>SET-KEY</code>: Stores a key-value pair in the system.</li>
</ul>
<p>The system also includes a persistence module called <code>PersistentStorage</code>, which handles data read and write operations. It uses the following paths:</p>
<ul>
<li><code>static/metadata</code>: stores the filenames representing the hashes of the data divided into chunks of up to 1000kb. These files contain Python lists saved with pickle, which contain the hashes of each chunk obtained by splitting the data.</li>
Expand All @@ -709,6 +719,7 @@ <h1 id="system-description">System Description</h1>
<p>To ensure data replication, nodes must periodically republish keys. This is because some of the k nodes that initially obtain a key-value pair may leave the network, and new nodes with IDs closer to the key may join the network. Therefore, nodes that store the key-value pair must republish it to ensure it is available on the k nodes closest to the key.</p>
<p>When a client requests a certain value from the system, they are returned a list of locations of the different data chunks, which may be on different PCs. The client then establishes a connection with the PC closest to the information to retrieve the data and unify it. Once a node sends information, it marks the file as pending republishing and updates its timestamp, informing neighbors that they should also replicate the information.</p>
<p>When a server discovers a new node, for each key in storage, the system retrieves the k closest nodes. If the new node is closer than the furthest node in that list, and the node for this server is closer than the closest node in that list, then the key/value pair is stored on the new node.</p>
<p>The server spawns a thread for each connection, supporting multiple clients.</p>



Expand Down
1 change: 1 addition & 0 deletions docs/es/description/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -723,6 +723,7 @@ <h1 id="descripcion-del-sistema">Descripción del sistema</h1>
<p>Para garantizar la replicación de datos, los nodos deben republicar periódicamente las claves. Esto se debe a que algunos de los k nodos que inicialmente obtienen un par clave-valor pueden abandonar la red, y nuevos nodos con IDs más cercanos a la clave pueden unirse a la red. Por lo tanto, los nodos que almacenan el par clave-valor deben republicarlo para asegurarse de que esté disponible en los k nodos más cercanos a la clave.</p>
<p>Cuando un cliente solicita un determinado valor al sistema, se le devuelve una lista de las ubicaciones de los distintos chunks de datos, que pueden estar en diferentes PCs. Luego, el cliente establece una conexión con la PC más cercana a la información para obtener los datos y unificarlos. Una vez que un nodo envía información, marca el archivo como pendiente de republicar y actualiza su timestamp, informando a los vecinos que también deben replicar la información.</p>
<p>Cuando un servidor descubre un nuevo nodo, para cada clave almacenada, el sistema recupera los k nodos más cercanos. Si el nuevo nodo está más cerca que el nodo más alejado de esa lista, y el nodo para este servidor está más cerca que el nodo más cercano de esa lista, entonces el par clave/valor se almacena en el nuevo nodo.</p>
<p>Cuando un servidor inicia una conexión nueva con un cliente, inicia un Thread nuevo para manejar dicha conexión.</p>



Expand Down
2 changes: 1 addition & 1 deletion docs/search/search_index.json

Large diffs are not rendered by default.

Binary file modified docs/sitemap.xml.gz
Binary file not shown.

0 comments on commit 6692c6e

Please sign in to comment.