This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Note

This section contains information on how all crates in this repo work at the network level, allowing for recreation and modification without the requirement for understanding the Rust code included. This section is recommended for technologically advanced users.

Caution

Where this section says 'packet', it is NOT referring to a standard TCP packet. While this repo does use TCP, it uses a special header value unique to all DitherDude networking projects. See How to Write a Packet before blindly attempting to implement the below functionality.

Additionally, where this section mentions usage of status/error codes, this refers to a custom implementation of HTTP status codes. This means status codes used may have a slightly different implication to standard HTTP status codes, or include non-standard status codes. See Status Codes for a comprehensive list of how to map status codes to u32 byte chunks, and Sending Error Codes when a fatal error is to be reported to the client.

Server

Warning

This is not the guide for how to host a website, but rather how to construct the server framework. See Hosting a Website (will link) if you wish to build a website instead.

On Connection

The client will initiate a connection with the server with a data packet of size 14 bytes or greater. If the data is too short, the server responds with a TOO_SMALL error code. The data received is to be chunked accordingly:

Bytes start	Bytes stop	Data type	Usage
0	3	u32	Client version major (e.g. the 3 in 3.7.2)
4	7	u32	Client version minor (e.g. the 7 in 3.7.2)
8	11	u32	Client version patch ("tiny") (e.g. the 2 in 3.7.2)

This version is then compared against server's version. See Compatibility to determinine if either:

• The client's version is too new for server and is not compatible => return error code DOWNGRADE_REQUIRED.
• The client's version it too old for server and is not compatible => return error code UPGRADE_REQUIRED.

data is now set to bytes 12 and onward.

Note

As of version 0.1.1, the server checks if the data is empty here. I have no idea why I implemented this, as we confirmed earlier that the data is at least 13b, meaning that there would be at least 1b of data after trimming, and therefore this check will never run.

Bytes start	Bytes stop	Data type	Usage
0	{x ∈ Z⁺ ∣ x≡0 (mod 5)}	UTF-8	Stacks supported by client. Grouped as char sets of 5b, and terminated by a / byte.
x+1	x+1	Byte => 0x2F	Terminator of stacks field. Indicates the start of the request location.
x+2	EOF	String	The location which the user requests to access.

The following is looped until the 0x2F character is reached:

1. server checks if data length is less than 5. If so, return error TOO_SMALL
2. Five bytes are popped off the front of data, and appended to an array of [u8; 5] (array of unknown quantity of array of type Byte grouped in sets of 5).
3. If data is empty, return error UNPROCESSABLE.
4. If data[0] (the first Byte of data) is 0x2F (the / character), exit the loop.

Tip

As there are 256 UTF-8 characters, and the only limitation is that a stack name cannot start with a / character, this means that there is a total of 255*(256^4)=1,095,216,660,480 possible stacks that could be implemented!

A sample packet could look like this:

\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\x4D\x52\x4B\x44\x4E\x48\x54\x4D\x4C\x58\x2F\x70\x61\x67\x65

which translates to:

• Client is using version 0.1.1.
• Client supports protocols MRKDN and HTMLX. As MRKDN is before HTMLX, MRKDN is preferred over HTMLX.
• Client would like to access endpoint /page.

server then loops through each client protocol, first to last. If the server supports the stack specified by the client (see Creating a stacks.txt File), the stack is selected, and the loop is exited. If no stack could be selected, return error UNPROCESSABLE.

Creating a stacks.txt File

This is the file used by server to define how stacks are to be parsed. There are three different types of stack protocols: Relative, Definite, and Wildcard. The following example showcases the usage of all three types of protcols, and will be used for the explanation of each protocol below. For this example, assume your website is hosted on /path/to/website.

user@linux:/path/to/website$ cat stacks.txt
MRKDN index.md
HTMLX /index.htmx
RWDTA
user@linux:/path/to/website$

Tip

While I have a space between the stacks and the protocol, this is not needed, as the first 5b represent the stack, and the protocol is whitespace trimmed. This means that the first line could also read MRKDNindex.md or MRKDN index.md.

Relative protocol

MRKDN index.md

This is the most common protocol. This indicates that the following should be appended to the end of the request; i.e.

• Client requests endpoint / => file at /path/to/website/index.md is returned.
• Client requests endpoint /page => file at /path/to/website/page/index.md is returned.
• Client requests endpoint ////////account/login////// => file at /path/to/website/account/login/index.md is returned.

If the requested file does not exist, server returns error NOT_FOUND. If the requested file exists but server cannot access it (e.g. missing required permissions), returns error NOT_FOUND. I don't know why it does this, in a future commit I may change this to error code FORBIDDEN.

Definite protocol

HTMLX /index.htmx

This protocol will always point to the same file, regardless of the endpoint; i.e.

• Client requests endpoint / => file at /path/to/website/index.htmx is returned.
• Client requests endpoint /page => file at /path/to/website/index.htmx is returned.
• Client requests endpoint ////////account/login////// => file at /path/to/website/index.htmx is returned.

If the file at /path/to/website/index.htmx doesn't exist, error code SHAT_THE_BED is returned, as this is the result of a server-side misconfiguration.

Wildcard protocol

RWDTA

This protocol points to the file specified by the user's endpoint; i.e.

• Client requests endpoint / => return error NOT_FOUND (as no file can exist with no name).
• Client requests endpoint /page => file at /path/to/website/page is returned.
• Client requests endpoint /forbidden/logs/credit_cards.csv => file at /path/to/website/forbidden/logs/credit_cards.csv is returned.

Note

The Wildcard protocol uses the exact same method as the Define protocol; i.e. the returned file is {endpoint}/{protocol}. This means that technically requesting Wildcard /page will request /page/. However, the pathcheck() function (which determines if the path is safe to access; e.g. /../../passwords.txt should in theory point to /path/passwords.txt, but this is caught and blocked) will trim the excess / at the end when creating the path buffer, returning file /path/to/website/page. If the requested file doesn't exist, error code NOT_FOUND is returned.

Returning the File

If a file is to be returned to the client, the return packet is structured as so:

Bytes start	Bytes stop	Data type	Usage
0	3	Byte => 0xC8000000	Success status code
4	8	String	Stack server used. As server might not use the first stack the user requested, server reports to the client which stack was used
9	EOF	String	Content of file to be returned to the client.

A sample response could look like this:

\xC8\x00\x00\x00\x4D\x52\x4B\x44\x4E\x23\x20\x48\x65\x6C\x6C\x6F\x2C\x20\x57\x6F\x72\x6C\x64\x21

which translates to:

• server responds to request with staus code SUCCESS.
• server resolved to stack MRKDN.
• server sent file payload of # Hello, World! to client.

The connection to client is now dropped silently, and the connection handshake is over.

DNS Provider

Caution

The code of version 0.1.1 is completely fucked up. While this will hopefully be fixed in a future commit, for now this wiki preempt each erroneous section of the API with a CAUTION box like this.

On Connection

The client will initiate a connection with the server with a data packet of size 14 bytes or greater. If the data is too short, the server responds with a TOO_SMALL error code. The data received is to be chunked accordingly:

Bytes start	Bytes stop	Data type	Usage
0	3	u32	Client version major (e.g. the 3 in 3.7.2)
4	7	u32	Client version minor (e.g. the 7 in 3.7.2)
8	11	u32	Client version patch ("tiny") (e.g. the 2 in 3.7.2)

This version is then compared against server's version. See Compatibility to determinine if either:

• The client's version is too new for server and is not compatible => return error code DOWNGRADE_REQUIRED.
• The client's version it too old for server and is not compatible => return error code UPGRADE_REQUIRED.

Caution

is_last_block is likely supposed to be byte 12. I will confirm this when I write the Backend (will link) wiki page.

data is set to bytes 13 and onward, and is_last_block (Bool) is set to byte 13

Bytes start	Bytes stop	Data type	Usage
12	12	Garbage	{UNUSED}. Likely supposed to be the value for is_last_block.
13	13	Bool	is_last_block. Note how this is also the first byte of data.
13	EOF	String	data; The block in a DNS chain that the user is requesting.

is_last_block: Bool

Note

For this example, assume the user requests to resolve my.site.com. To prevent confusion with future websites, I will use local 192.168.0.XXX IPs as the "resolved" addresses.

When resolving a URL, dns_provider needs to know if the data you are requesting is the last block in the chain of URL blocks. Using the above example, if we query the dns_provider for all of com's children for site, we want it to return the address of where we should query the block my - and not return the location of the webpage for site.com. The is_last_block flag ensures that we return the specific record fields for our query type.

A sample packet could look like this:

\x00\x00\x00\x00\x01\x00\x00\x00\x01\x00\x00\x00\xFF\x73\x69\x74\x65

which translates to:

• Client is using version 0.1.1.
• Client has the is_last_block bool set to 0x73.
• Client is requesting the block "site".

Response

Note

There are a multitude of responses that dns_provider can return to client. These will all be displayed concisely as a table at the end of this section. From this point on, the database that holds the data related to URL blocks is assumed to have at least the following columns: name, domain_ip, domain_port, dns_ip, and dns_port. There is no reason that domain_ip+domain_port, and dns_ip+dns_port should be separate (i.e. name, domain, and dns should be sufficient). However, as this is how the system was initially set up, this is what will be followed in the below documentation.

Warning

This code uses is_some() and unwrap() syntax instead of if let Some(...). This will likely be corrected in a future commit.

The Wildcard Record

This special record is denoted by the . value, as it is a value that can be guaranteed won't collide with any valid records in the database. Though this record has a second use, only the first will be covered for now: The dns record. In the event that this dns_provider is to move locations, dns_provider needs a way to inform client that all future requests should be sent to the new dns_provider. If the . record exists AND has dns_ip and dns_port values, dns_provider notifies client with error code PERMANENT_REDIRECT that it (dns_provider) is obsolete, and that the new (specified) dns_provider should be used instead. The new location is sent in the format {dns_ip}:{dns_port}. A sample PERMANENT_REDIRECT response could look like this:

\x2D\x01\x00\x00\x31\x39\x32\x2E\x31\x36\x38\x2E\x30\x2E\x32\x3A\x36\x32\x30\x32

which translates to:

• dns_provider has moved to 192.168.0.2 port 6202. All future DNS queries intended for this dns_provider should be instead forwarded to 192.168.0.2:6202.

Requested Block Record

If the . record is non-existent, or doesn't have dns_ip and dns_port values, the database is re-queried for client's specified block's domain_ip, domain_port, dns_ip, and dns_port fields. In this example, this would be the my block of the site dns_provider, OR the site block of the com dns_provider. If the record does not exist, see Resolve Wildcard. is_last_block is now checked.

Caution

The is_last_block implementation of status codes is a literal hellhole. I will have to investigate if the below implementation of GONE is correct OR if it should align with the one specified in Universal Information, and update the code. It should be assumed, however, that the implementation specified in Universal Information is correct, and that the dns_provider code below is corrupted.

is_last_block (True)

If the record has domain_ip and domain_port, the record is returned to client in the format {domain_ip}:{domain_port} with status code SUCCESS. E.g.

\xC8\x00\x00\x00\x31\x39\x32\x2E\x31\x36\x38\x2E\x30\x2E\x33\x3A\x36\x32\x30\x34

which translates to

• Block (which was the last block of a DNS block-chain) points to the domain at 192.168.0.3 port 6204.

If the record does not have a domain_ip or domain_port, (i.e. these values are NULL), dns_ip and dns_port are read. If they exist, they are returned to client with status code FOUND in the format {dns_ip}:{dns_port}. E.g.

\x2E\x01\x00\x00\x31\x39\x32\x2E\x31\x36\x38\x2E\x30\x2E\x33\x3A\x36\x32\x30\x32

which translates to

• Status code FOUND.
• dns_provider acknowledges that client requested this record as is_last_block. However, the record held a DNS value instead of the requested domain value. Hence, while the record was found, there is no domain at the requested location. The DNS value in the record points to 192.168.0.3 port 6202.

If the record is missing all four fields (i.e. they are all NULL, but the record does exist), the status code GONE is sent to client.

Tip

Creating a record like so is usually for reserving blocks - i.e. you hint to client that while no block exists here yet, one will likely be here soon, or that this block may not be requested for personal ownership, as it is in use. However, this is not the recommended implementation for parking blocks.

A sample response could be:

\x9A\x01\x00\x00

which translates to

• dns_provider has found the requested record, but there was no data associated with it.

!is_last_block (False)

Caution

Status codes SUCCESS and FOUND below should likely be swapped.

If the record has dns_ip and dns_port, the record is returned to client in the format {dns_ip}:{dns_port} with status code FOUND. E.g.

\x2E\x01\x00\x00\x31\x39\x32\x2E\x31\x36\x38\x2E\x30\x2E\x33\x3A\x36\x32\x30\x32

which translates to

• Intermediary block points to the dns_provider at 192.168.0.3 port 6202.

If the record does not have a dns_ip or dns_port, (i.e. these values are NULL), domain_ip and domain_port are read. If they exist, they are returned to client with status code SUCCESS in the format {domain_ip}:{domain_port}. E.g.

\xC8\x00\x00\x00\x31\x39\x32\x2E\x31\x36\x38\x2E\x30\x2E\x33\x3A\x36\x32\x30\x34

which translates to

• Status code SUCCESS
• dns_provider acknowledges that client requested this record as an intermediary block - i.e. as a DNS. However, the record held a domain value instead of the requested DNS value. Hence, while the record was found, there is no dns_provider at the specified location. The domain value in the record points to 192.168.0.3 port 6204.

If the record is missing all four fields (i.e. they are all NULL, but the record does exist), see Resolve Wildcard.

Caution

This should return the status code GONE like it does for is_last_block (True).

Resolve Wildcard

If the record cannot be found, the . record is again queried - but this time for the fields domain_ip and domain_port. The location these fields point to indicate where the next sister dns_provider is located, if at all. Think of this as a kind of webring: This record ties in numerous dns_providers into one long chain. If the current dns_provider exhausts its records, it returns to the client a location where the next dns_provider is, and the query can be resumed there. Think of an example where there is provider1 which holds records starting with a-m, and provider2 which holds records starting with n-z. If client requests website from provider1, provider1 will exhaust its database, as it does not hold records starting with the letter w, and thus tells client where the sister provider2 is. client will then repeat the request for website to provider2, who has the record. provider2 will return the result as shown above. This Wildcard record is returned to client with status code NON_AUTHORITATIVE in the format {domain_ip}:{domain_port}. E.g.

\xCB\x00\x00\x00\x31\x39\x32\x2E\x31\x36\x38\x2E\x30\x2E\x34\x3A\x36\x32\x30\x32

which translates to

• dns_provider has exhausted its database, and couldn't find the specifed record. However, client should be aware that dns_provider located at 192.168.0.4 port 6202 is the next dns_provider in the chain, and should be queried for the requested record.

Warning

It is possible that the Wildcard chain is cyclic, i.e. provider1 points to provider2 which points to provider1. In the client implementation in this repo, this can cause a Stack Overflow (which requires ~130 NON_AUTHORITATIVE redirects). It is therefore the duty of client to keep a record of all prior Wildcard locations, and break the chain with a local error code of LOOP_DETECTED if a dns_provider is to appear again.

Fallthrough

If the specified record doesn't exist, and the . record does not exist or doesn't contain the domain_ip and domain_port values, dns_provider returns the error code MISDIRECTED, and drops the connection.

Warning

MISDIRECTED is likely an unsuitable error code to be returned. MISDIRCTED indicates that the request was not of a valid recognisable format that dns_provider could interpret. However, dns_provider was able to interpret the request, except that it could not resolve the request.

A sample response could look like:

\xA5\x01\x00\x00

which translates to

• dns_provider has not moved. dns_provider could not locate the requested block in the DNS chain. dns_provider did not have a pointer to a sister dns_provider to resume the request.

Ending the Connection

Once data has been sent to client, dns_provider is expected to silently drop the connection to client, as the handshake is now over.

List of Possible Responses

The following table lists all the possible responses of dns_provider to a query. It should be followed in order from top-to-bottom. It is not intended to replace the above information, but instead preview it for quick reference.

is_last_block	STATUS CODE	Data	Purpose
-	PERMANENT_REDIRECT	{dns_ip}:{port}	The location of the new dns_provider if this current one is to become obsolete.
True	SUCCESS	{domain_ip}:{domain_port}	The location of the requested domain.
True	FOUND	{dns_ip}:{port}	The location of the DNS field of the requested record.
True	GONE	-	There is no location associated with the requested record; however the record does exist.
False	FOUND	{dns_ip}:{port}	The location of the requested DNS.
False	SUCCESS`	{domain_ip}:{domain_port}	The location of the domain field of the requested record.
-	NON_AUTHORITATIVE	{domain_ip}:{domain_port}	The location of the sister dns_provider.
-	MISDIRECTED	-	Exhausted all attempts to locate the requested record.