Skip to content

Commit

Permalink
Merge pull request #13 from dojyorin/dev
Browse files Browse the repository at this point in the history
add more feat.
  • Loading branch information
dojyorin authored Nov 21, 2022
2 parents cb3f38d + 98f1a6e commit 3abcedf
Show file tree
Hide file tree
Showing 21 changed files with 220 additions and 80 deletions.
66 changes: 51 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,38 +5,44 @@
A handy utility collection.

# Example

<p>
<details>
<summary>Show more details...</summary>
<p>

**BASE64 Binary**

```ts
const file = await Deno.readFile("/path/to/binary.bin");

const encoded = base64Encode(file); // BASE64 encoded string.
const decoded = base64Decode(encoded); // Restored byte.
const encoded = base64Encode(file); // base64 code.
const decoded = base64Decode(encoded); // Restored.
```

**Date UnixTime**

```ts
const date = new Date();

const encoded = await dateEncode(date); // UTC unix time.
const decoded = await dateDecode(encoded); // Restored Date object.
const encoded = await dateEncode(date); // unixtime in seconds.
const decoded = await dateDecode(encoded); // Restored.
```

**DEFLATE Compress**

```ts
const file = await Deno.readFile("/path/to/binary.bin");

const encoded = await deflateEncode(file); // DEFLATE compressed byte.
const decoded = await deflateDecode(encoded); // Restored byte.
const encoded = await deflateEncode(file); // "deflate" compressed byte array.
const decoded = await deflateDecode(encoded); // Restored.
```

**Extended Fetch API**

```ts
const json = await fetchExtend("https://path/to/get", "json"); // Response as JSON.
const bytes = await fetchExtend("https://path/to/get", "byte"); // Response as Uint8Array.
const json = await fetchExtend("https://path/to/get", "json"); // response as JSON.
const bytes = await fetchExtend("https://path/to/get", "byte"); // response as Uint8Array.
```

**Minipack Archive**
Expand All @@ -46,30 +52,43 @@ const files = [
["binary.bin", Deno.readFileSync("/path/to/binary.bin")]
];

const encoded = await minipackEncode(files); // Minipack archived byte.
const decoded = await minipackDecode(encoded); // Restored array of name and byte.
const encoded = await minipackEncode(files); // byte array in "minipack" format.
const decoded = await minipackDecode(encoded); // Restored.
```

**Platform Specific**

```ts
const win = isWin(); // "true" if running on Windows.
const tmp = tmpPath(); // "C:/Windows/Temp" if running on Windows, or "/tmp" if running on Linux or Mac.
```

**Text Convert**

```ts
const text = "Lorem ipsum dolor sit amet.";
const text = " Lorem ipsum \t dolor \r sit amet.";

const encoded = await textEncode(text); // UTF-8 byte.
const decoded = await textDecode(encoded); // Restored string.
const encoded = await ucEncode(text); // byte array in UTF-8 format.
const decoded = await ucDecode(encoded); // Restored.
const hexadecimal = hexEncode(encoded); // hexadecimal string.
const formatted = trimExtend(decoded); // formatted string.
```

</p>
</details>
</p>

# Details
It's basically a thin wrapper around Deno's functions to improve usability, but some features are original to this module.

This section describes the original features of this module.

## Minipack
Minipack is a file archive format original to this module.
"Minipack" is a file archive format original to this module.

It's structure is inspired by the famous "tar" and is minimal as an archive.

Originally developed for browsers, the purpose was to aggregate multiple files input with the HTML File API into a single file.
Originally developed for web browser, the purpose was to aggregate multiple files input with the HTML File API into a single binary.

Therefore, there is no concept of directory or filesystem, and it's feature by simple structure that stores only the file body, file name, and hash value for verification.

Expand All @@ -85,5 +104,22 @@ The actual binary structure looks like this:

This is for one file and repeats for the number of files.

# Tips
This section is not directly related to this module, but provides a few line snippets to help you implement your application.

<p>
<details>
<summary>Show more details...</summary>
<p>

**JSON Import**
```ts
const {default: config} = await import("./config.json", {assert: {type: "json"}});
```

</p>
</details>
</p>

# API
See [Deno Document](https://deno.land/x/simple_utility/mod.ts) for details.
3 changes: 3 additions & 0 deletions deps.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
/**
* no dependencies.
*/
1 change: 1 addition & 0 deletions mod.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,5 @@ import "./test/date.test.ts";
import "./test/deflate.test.ts";
import "./test/fetch.test.ts";
import "./test/minipack.test.ts";
import "./test/platform.test.ts";
import "./test/text.test.ts";
6 changes: 5 additions & 1 deletion mod.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,4 +3,8 @@ export * from "./src/date.ts";
export * from "./src/deflate.ts";
export * from "./src/fetch.ts";
export * from "./src/minipack.ts";
export * from "./src/text.ts";
export * from "./src/platform.ts";
export * from "./src/text.ts";

export * from "./src/core.d.ts";
export * from "./src/web.d.ts";
7 changes: 7 additions & 0 deletions src/_utility.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
export async function streamConvert(data:Uint8Array, ts:TransformStream<Uint8Array, Uint8Array>){
return new Uint8Array(await new Response(new Blob([data]).stream().pipeThrough(ts)).arrayBuffer());
}

export async function deriveHash(data:Uint8Array){
return new Uint8Array(await crypto.subtle.digest("SHA-256", data));
}
8 changes: 4 additions & 4 deletions src/base64.ts
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
/**
* Convert from BASE64 code to byte.
* @param data The byte.
* @param data byte array.
* @return base64 code.
*/
export function base64Encode(data:Uint8Array){
return btoa([...data].map(n => String.fromCharCode(n)).join(""));
}

/**
* Convert from byte to BASE64 code.
* @param data The base64 code.
* @param data base64 code.
* @return byte array.
*/
export function base64Decode(data:string){
return new Uint8Array([...atob(data)].map(s => s.charCodeAt(0)));
Expand Down
9 changes: 9 additions & 0 deletions src/core.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
/**
* function returning arbitrary types.
*/
export type SyncFunction<T extends unknown = void> = () => T;

/**
* function returning arbitrary async types.
*/
export type AsyncFunction<T extends unknown = void> = SyncFunction<Promise<T>>;
12 changes: 6 additions & 6 deletions src/date.ts
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
/**
* Convert from Date object to unix time.
* Since the UnixTime that can be handled by the Date object is in milliseconds, this method output downscaled to 1/1000.
* @param date The date object. If blank output the current time.
* Since the unixtime that can be handled by the `Date` is in milliseconds, this method output downscaled to 1/1000.
* @param date date object for any datetime, if blank output the current datetime.
* @return unixtime in seconds.
*/
export function dateEncode(date?:Date){
return Math.floor((date ?? new Date()).getTime() / 1000);
}

/**
* Convert from unix time to Date object.
* Since the UnixTime that can be handled by the Date object is in milliseconds, the argument of this method is internally multiplied by x1000.
* @param time The unix time.
* Since the unixtime that can be handled by the `Date` is in milliseconds, the argument of this method is internally multiplied by x1000.
* @param time unixtime in seconds.
* @return date object specified by `time`.
*/
export function dateDecode(time:number){
return new Date(time * 1000);
Expand Down
14 changes: 7 additions & 7 deletions src/deflate.ts
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
async function streamConvert(data:Uint8Array, ts:TransformStream<Uint8Array, Uint8Array>){
return new Uint8Array(await new Response(new Blob([data]).stream().pipeThrough(ts)).arrayBuffer());
}
import {streamConvert} from "./_utility.ts";

/**
* Compresses raw binary in "deflate" format (RFC1951 compliant).
* Compress binary in "deflate" format (RFC1951).
* It does not include header information like "gzip" (RFC1952) or "zlib" (RFC1950) as it does purely "compression only".
* @param data The byte.
* @param data byte array.
* @return "deflate" compressed byte array.
*/
export async function deflateEncode(data:Uint8Array){
return await streamConvert(data, new CompressionStream("deflate-raw"));
}

/**
* Decompress "deflate" format (RFC1951 compliant) binary.
* Decompress "deflate" format (RFC1951) binary.
* Binaries containing header information like "gzip" (RFC1952) or "zlib" (RFC1950) cannot be decompressed.
* @param data The byte.
* @param data "deflate" compressed byte array.
* @return decompressed byte array.
*/
export async function deflateDecode(data:Uint8Array){
return await streamConvert(data, new DecompressionStream("deflate-raw"));
Expand Down
15 changes: 8 additions & 7 deletions src/fetch.ts
Original file line number Diff line number Diff line change
@@ -1,8 +1,4 @@
export type JsonStruct = string | number | boolean | null | JsonStruct[] | {
[key: string]: JsonStruct;
};

export type QueryInit = Exclude<HeadersInit, Headers> | URLSearchParams;
import {JsonStruct, QueryInit} from "./web.d.ts";

export interface FetchInit extends Omit<RequestInit, "window">{
query?: QueryInit;
Expand All @@ -16,13 +12,16 @@ export interface FetchResponseType{
"buffer": ArrayBuffer;
"blob": Blob;
"ok": boolean;
"code": number;
"header": Headers;
"response": Response;
}

/**
* @param path Target URL. Since the query string is ignored, please specify it in the `option.query` property instead of writing it directly in the URL.
* @param path Since the query string is ignored, please specify it in the `option.query` property instead of writing it directly in the URL.
* @param type The type you want to receive in the response.
* @param option Fetch option. `window` is removed from `RequestInit` and `query` is added to write the query string.
* @param option `window` is removed from `RequestInit` and `query` is added to write the query string.
* @return response from the server specified by `type`.
*/
export async function fetchExtend<T extends keyof FetchResponseType>(path:string, type:T, option?:FetchInit){
const {origin, pathname} = /^http(s|):\/\//i.test(path) ? new URL(path) : new URL(path, location.href);
Expand Down Expand Up @@ -52,6 +51,8 @@ export async function fetchExtend<T extends keyof FetchResponseType>(path:string
case "buffer": return <FetchResponseType[T]>await response.arrayBuffer();
case "blob": return <FetchResponseType[T]>await response.blob();
case "ok": return <FetchResponseType[T]>response.ok;
case "code": return <FetchResponseType[T]>response.status;
case "header": return <FetchResponseType[T]>response.headers;
case "response": return <FetchResponseType[T]>response;
default: throw new Error();
}
Expand Down
49 changes: 20 additions & 29 deletions src/minipack.ts
Original file line number Diff line number Diff line change
@@ -1,45 +1,36 @@
const size = <const>{
import {deriveHash} from "./_utility.ts";
import {ucEncode, ucDecode, hexEncode} from "./text.ts";

const sizeOf = <const>{
hash: 32,
name: 1,
body: 4
};

const sizeTotal = Object.values(size).reduce((a, c) => a + c, 0);

async function byte2hash(data:Uint8Array){
return new Uint8Array(await crypto.subtle.digest("SHA-256", data));
}

function text2byte(data:string){
return new TextEncoder().encode(data);
}

function byte2text(data:Uint8Array){
return new TextDecoder().decode(data);
}
const sizeTotal = Object.values(sizeOf).reduce((a, c) => a + c, 0);

/**
* Convert from array of name and byte to "minipack" archive format.
* @param files array of pair of name and byte array.
* @return byte array in "minipack" format.
* @see https://deno.land/x/simple_utility
* @param files Array of name and byte.
*/
export async function minipackEncode(files:[string, Uint8Array][]){
const archive = new Uint8Array(files.reduce((a, [k, v]) => a + sizeTotal + text2byte(k).byteLength + v.byteLength, 0));
const archive = new Uint8Array(files.reduce((a, [k, v]) => a + sizeTotal + ucEncode(k).byteLength + v.byteLength, 0));

let offset = 0;

for(const [k, v] of files){
const name = text2byte(k);
const name = ucEncode(k);
const body = v;

archive.set(await byte2hash(body), offset);
offset += size.hash;
archive.set(await deriveHash(body), offset);
offset += sizeOf.hash;

new DataView(archive.buffer, offset).setUint8(0, name.byteLength);
offset += size.name;
offset += sizeOf.name;

new DataView(archive.buffer, offset).setUint32(0, body.byteLength);
offset += size.body;
offset += sizeOf.body;

archive.set(name, offset);
offset += name.byteLength;
Expand All @@ -52,29 +43,29 @@ export async function minipackEncode(files:[string, Uint8Array][]){
}

/**
* Convert from binary in "minipack" archive format to array of name and byte.
* @param archive byte array in "minipack" format.
* @return array of pair of name and byte array.
* @see https://deno.land/x/simple_utility
* @param archive The byte.
*/
export async function minipackDecode(archive:Uint8Array){
const files:[string, Uint8Array][] = [];

let offset = 0;

while(offset < archive.byteLength){
const hash = archive.subarray(offset, offset += size.hash);
const hash = archive.subarray(offset, offset += sizeOf.hash);

const ns = new DataView(archive.buffer, offset).getUint8(0);
offset += size.name;
offset += sizeOf.name;

const bs = new DataView(archive.buffer, offset).getUint32(0);
offset += size.body;
offset += sizeOf.body;

const name = byte2text(archive.subarray(offset, offset += ns));
const name = ucDecode(archive.subarray(offset, offset += ns));

const body = archive.subarray(offset, offset += bs);

if(hash.toString() !== (await byte2hash(body)).toString()){
if(hexEncode(hash) !== hexEncode(await deriveHash(body))){
throw new Error();
}

Expand Down
Loading

0 comments on commit 3abcedf

Please sign in to comment.