Skip to content

Commit

Permalink
Added BigInteger and BigUnsignedInteger types (#1)
Browse files Browse the repository at this point in the history
  • Loading branch information
calebsander committed Aug 15, 2016
1 parent 3de5868 commit a1bb675
Show file tree
Hide file tree
Showing 59 changed files with 3,104 additions and 473 deletions.
11 changes: 8 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@ A NodeJS library for making more efficient data transfers by separating the stru
[![Coverage Status](https://coveralls.io/repos/github/calebsander/structure-bytes/badge.svg?branch=master)](https://coveralls.io/github/calebsander/structure-bytes?branch=master)

## Concept
Most data nowadays is stored in files or transfered over HTTP as either text files which can represent a wide variety of data structures (e.g. JSON or YAML) or in a format created to represent only one specific sort of data (e.g. MP3). The idea with this project is to get the advantages of both sorts of formats. To accomplish this, the project was designed with several principles in mind:
A lot of data, especially data designed to be used in many different languages, is stored in files or transfered over HTTP as either text files which can represent a wide variety of data structures (e.g. JSON) or in a format created to represent only one specific sort of data (e.g. MP3). The idea with this project is to get the advantages of both sorts of formats. This project is somewhat similar to Google's [Protocol Buffers](https://developers.google.com/protocol-buffers/), but was not designed to match its functionality. To accomplish this, the project was designed with several principles in mind:

- Types (user-created formats of data) are created by combining a wide variety of datatypes, both primitive and recursive. This allows for representations which accurately describe types (e.g. distinguishing between structs and mappings of strings to values, and numeric types, unlike JSON). Types are very customizable, so you can build any desired type formats.
- Types are kept separate from values (instances of a type) because types are designed to be created once and used to store many different values. This allows for communications to cache the type format after the first use and only have to send the values in subsequent requests.
- Redundancy in data storage is kept to a minimum. For example, in an array of structs, the field names are only specified once in the type spec rather than for each element in the array.
Expand All @@ -16,18 +17,22 @@ Most data nowadays is stored in files or transfered over HTTP as either text fil
- Use when you plan to have many different values for the type (either many different files storing the same sort of information, or many communications of the same sort of information). This will give you the benefit of being able to keep only a single copy of the type spec.
- Use when there is a lot of repetition in the data. If you don't have any arrays, sets, or maps, you can't really benefit from the cutdown on redundancy.

## Differences from Protocol Buffers
- Types are generated programmatically rather than by reading `.proto` files. This allows for functionality like a function which turns a type into another type that either contains an error message or an instance of the original type.
- This project is designed with downloading data of known types from servers over HTTP in mind. If the client has already received data of the same type, the server only sends the value and the client reads it using its cached type. If the client doesn't know what the type looks like, the server sends it in byte form along with the value and the client caches the type. This way, the type does not need to be specified in the client-side JavaScript and repeated requests are very efficient.

## Data types
- Primitive types
- `Byte` (1-byte signed integer)
- `Short` (2-byte signed integer)
- `Int` (4-byte signed integer)
- `Long` (8-byte signed integer)
- (planned) `BigInt` (a signed integer with up to 256 bytes of precision)
- `BigInt` (a signed integer with up to 65535 bytes of precision)
- `UnsignedByte` (1-byte unsigned integer)
- `UnsignedShort` (2-byte unsigned integer)
- `UnsignedInt` (4-byte unsigned integer)
- `UnsignedLong` (8-byte unsigned integer)
- (planned) `BigUnsignedInt` (an unsigned integer with up to 256 bytes of precision)
- `BigUnsignedInt` (an unsigned integer with up to 65535 bytes of precision)
- `Date` (8-byte unsigned integer representing number of milliseconds since Jan 1, 1970)
- `Float` (IEEE 32-bit floating-point number)
- `Double` (IEEE 64-bit floating-point number)
Expand Down
4 changes: 2 additions & 2 deletions compiled/download.js

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions compiled/upload-download.js

Large diffs are not rendered by default.

4 changes: 2 additions & 2 deletions compiled/upload.js

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion config.js
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ if (__dirname === '/') __dirname = '';
const assert = require(__dirname + '/lib/assert.js');
const base64 = require('base64-js');

const VERSION = 7;
const VERSION = 8;
assert.between(0, VERSION, 65536);
const VERSION_BYTES = 2;
const VERSION_BUFFER = new ArrayBuffer(VERSION_BYTES);
Expand Down
20 changes: 10 additions & 10 deletions doc/ArrayType.html
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ <h5>Parameters:</h5>

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line818">line 818</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line919">line 919</a>
</li></ul></dd>


Expand Down Expand Up @@ -317,7 +317,7 @@ <h5>Parameters:</h5>

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line61">line 61</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line70">line 70</a>
</li></ul></dd>


Expand Down Expand Up @@ -475,7 +475,7 @@ <h5>Parameters:</h5>

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line165">line 165</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line174">line 174</a>
</li></ul></dd>


Expand All @@ -502,7 +502,7 @@ <h5>Returns:</h5>


<div class="param-desc">
<code>true</code> iff the types have the same constructor and the same field values for fields not starting with <code>'cached'</code>
<code>true</code> iff the types have the same constructor and the same field values for fields that don't store cached results
</div>


Expand Down Expand Up @@ -584,7 +584,7 @@ <h4 class="name" id="getHash"><span class="type-signature"></span>getHash<span c

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line105">line 105</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line114">line 114</a>
</li></ul></dd>


Expand Down Expand Up @@ -694,7 +694,7 @@ <h4 class="name" id="getSignature"><span class="type-signature"></span>getSignat

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line123">line 123</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line132">line 132</a>
</li></ul></dd>


Expand Down Expand Up @@ -804,7 +804,7 @@ <h4 class="name" id="toBuffer"><span class="type-signature"></span>toBuffer<span

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line86">line 86</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line95">line 95</a>
</li></ul></dd>


Expand Down Expand Up @@ -959,7 +959,7 @@ <h5>Parameters:</h5>

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line155">line 155</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line164">line 164</a>
</li></ul></dd>


Expand Down Expand Up @@ -1147,7 +1147,7 @@ <h5>Parameters:</h5>

<dt class="tag-source">Source:</dt>
<dd class="tag-source"><ul class="dummy"><li>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line853">line 853</a>
<a href="structure-types.js.html">structure-types.js</a>, <a href="structure-types.js.html#line954">line 954</a>
</li></ul></dd>


Expand Down Expand Up @@ -1223,7 +1223,7 @@ <h5>Example</h5>
</div>

<nav>
<h2><a href="index.html">Home</a></h2><h3>Externals</h3><ul><li><a href="external-ArrayBuffer.html">ArrayBuffer</a></li><li><a href="external-http.html">http</a></li></ul><h3>Classes</h3><ul><li><a href="ArrayType.html">ArrayType</a></li><li><a href="BooleanArrayType.html">BooleanArrayType</a></li><li><a href="BooleanTupleType.html">BooleanTupleType</a></li><li><a href="BooleanType.html">BooleanType</a></li><li><a href="BufferStream.html">BufferStream</a></li><li><a href="ByteType.html">ByteType</a></li><li><a href="CharType.html">CharType</a></li><li><a href="ChoiceType.html">ChoiceType</a></li><li><a href="DateType.html">DateType</a></li><li><a href="DoubleType.html">DoubleType</a></li><li><a href="EnumType.html">EnumType</a></li><li><a href="external-http.IncomingMessage.html">IncomingMessage</a></li><li><a href="external-http.ServerResponse.html">ServerResponse</a></li><li><a href="FloatType.html">FloatType</a></li><li><a href="GrowableBuffer.html">GrowableBuffer</a></li><li><a href="IntType.html">IntType</a></li><li><a href="LongType.html">LongType</a></li><li><a href="MapType.html">MapType</a></li><li><a href="OctetsType.html">OctetsType</a></li><li><a href="OptionalType.html">OptionalType</a></li><li><a href="PointerType.html">PointerType</a></li><li><a href="SetType.html">SetType</a></li><li><a href="ShortType.html">ShortType</a></li><li><a href="StringType.html">StringType</a></li><li><a href="StructType.html">StructType</a></li><li><a href="TupleType.html">TupleType</a></li><li><a href="Type.html">Type</a></li><li><a href="UnsignedByteType.html">UnsignedByteType</a></li><li><a href="UnsignedIntType.html">UnsignedIntType</a></li><li><a href="UnsignedLongType.html">UnsignedLongType</a></li><li><a href="UnsignedShortType.html">UnsignedShortType</a></li></ul><h3>Namespaces</h3><ul><li><a href="r.html">r</a></li></ul><h3>Global</h3><ul><li><a href="global.html#httpRespond">httpRespond</a></li><li><a href="global.html#readType">readType</a></li><li><a href="global.html#readTypeAndValue">readTypeAndValue</a></li><li><a href="global.html#readValue">readValue</a></li><li><a href="global.html#writeType">writeType</a></li><li><a href="global.html#writeTypeAndValue">writeTypeAndValue</a></li><li><a href="global.html#writeValue">writeValue</a></li></ul>
<h2><a href="index.html">Home</a></h2><h3>Externals</h3><ul><li><a href="external-ArrayBuffer.html">ArrayBuffer</a></li><li><a href="external-http.html">http</a></li></ul><h3>Classes</h3><ul><li><a href="ArrayType.html">ArrayType</a></li><li><a href="BigIntType.html">BigIntType</a></li><li><a href="BigUnsignedIntType.html">BigUnsignedIntType</a></li><li><a href="BooleanArrayType.html">BooleanArrayType</a></li><li><a href="BooleanTupleType.html">BooleanTupleType</a></li><li><a href="BooleanType.html">BooleanType</a></li><li><a href="BufferStream.html">BufferStream</a></li><li><a href="ByteType.html">ByteType</a></li><li><a href="CharType.html">CharType</a></li><li><a href="ChoiceType.html">ChoiceType</a></li><li><a href="DateType.html">DateType</a></li><li><a href="DoubleType.html">DoubleType</a></li><li><a href="EnumType.html">EnumType</a></li><li><a href="external-http.IncomingMessage.html">IncomingMessage</a></li><li><a href="external-http.ServerResponse.html">ServerResponse</a></li><li><a href="FloatType.html">FloatType</a></li><li><a href="GrowableBuffer.html">GrowableBuffer</a></li><li><a href="IntType.html">IntType</a></li><li><a href="LongType.html">LongType</a></li><li><a href="MapType.html">MapType</a></li><li><a href="OctetsType.html">OctetsType</a></li><li><a href="OptionalType.html">OptionalType</a></li><li><a href="PointerType.html">PointerType</a></li><li><a href="SetType.html">SetType</a></li><li><a href="ShortType.html">ShortType</a></li><li><a href="StringType.html">StringType</a></li><li><a href="StructType.html">StructType</a></li><li><a href="TupleType.html">TupleType</a></li><li><a href="Type.html">Type</a></li><li><a href="UnsignedByteType.html">UnsignedByteType</a></li><li><a href="UnsignedIntType.html">UnsignedIntType</a></li><li><a href="UnsignedLongType.html">UnsignedLongType</a></li><li><a href="UnsignedShortType.html">UnsignedShortType</a></li></ul><h3>Namespaces</h3><ul><li><a href="r.html">r</a></li></ul><h3>Global</h3><ul><li><a href="global.html#httpRespond">httpRespond</a></li><li><a href="global.html#readType">readType</a></li><li><a href="global.html#readTypeAndValue">readTypeAndValue</a></li><li><a href="global.html#readValue">readValue</a></li><li><a href="global.html#writeType">writeType</a></li><li><a href="global.html#writeTypeAndValue">writeTypeAndValue</a></li><li><a href="global.html#writeValue">writeValue</a></li></ul>
</nav>

<br class="clear">
Expand Down
Loading

0 comments on commit a1bb675

Please sign in to comment.