Filen S3

A package to start a S3 server for a Filen account.

Installation

1. Install using NPM

npm install @filen/s3@latest

2. Initialize the server and query it using aws-sdk

import { FilenSDK } from "@filen/sdk"
import path from "path"
import os from "os"
import { S3Server } from "@filen/s3"
import { S3 } from "aws-sdk"

// Initialize a SDK instance (optional)
const filen = new FilenSDK({
	metadataCache: true,
	connectToSocket: true,
	tmpPath: path.join(os.tmpdir(), "filen-sdk")
})

await filen.login({
	email: "[email protected]",
	password: "supersecret123",
	twoFactorCode: "123456"
})

const hostname = "127.0.0.1"
const port = 1700
const https = false
const endpoint = `${https ? "https" : "http"}://${hostname === "127.0.0.1" ? "local.s3.filen.io" : hostname}:${port}`

const server = new S3Server({
	hostname,
	port,
	https,
	user: {
		accessKeyId: "admin",
		secretKeyId: "admin",
		sdk: filen
	}
})

const s3 = new S3({
	accessKeyId: "admin",
	secretAccessKey: "admin",
	endpoint,
	s3ForcePathStyle: true, // Needed
	region: "filen" // Needed
})

// Start the server
await server.start()

console.log(`S3 server started on ${endpoint}`)

// List objects
await s3
	.listObjectsV2({
		Bucket: "filen",
		Prefix: ""
	})
	.promise()

3. Initialize the server in cluster mode

import { FilenSDK } from "@filen/sdk"
import path from "path"
import os from "os"
import { S3ServerCluster } from "@filen/s3"
import { S3 } from "aws-sdk"

// Initialize a SDK instance (optional)
const filen = new FilenSDK({
	metadataCache: true,
	connectToSocket: true,
	tmpPath: path.join(os.tmpdir(), "filen-sdk")
})

await filen.login({
	email: "[email protected]",
	password: "supersecret123",
	twoFactorCode: "123456"
})

const hostname = "127.0.0.1"
const port = 1700
const https = false
const endpoint = `${https ? "https" : "http"}://${hostname === "127.0.0.1" ? "local.s3.filen.io" : hostname}:${port}`

const server = new S3ServerCluster({
	hostname,
	port,
	https,
	user: {
		accessKeyId: "admin",
		secretKeyId: "admin",
		sdk: filen
	},
	threads: 16 // Number of threads to spawn. Defaults to CPU core count if omitted.
})

// Start the cluster
await server.start()

console.log(`S3 server cluster started on ${endpoint}`)

S3 Compatibility

Only methods listed here are currently implemented. Due to the underlying storage most methods are impossible to implement, though we try to implement all "fundamental" needed methods. Top level directories in your cloud are seen as buckets.

Method	100% Compatible	Info
ListObjects	🟥	Only supports Prefix parameter. Depth is always 0 if delimiter is "/", otherwise 10. EncodingType is always URL. There are no Markers. The server always responds with all keys matching the Prefix + Depth. *
ListObjectsV2	🟥	Only supports Prefix parameter. Depth is always 0 if delimiter is "/", otherwise 10. EncodingType is always URL. There are no ContinuationTokens. The server always responds with all keys matching the Prefix + Depth. *
ListBuckets	✅
CreateBucket	✅
DeleteBucket	✅
GetBucketLocation	✅
HeadBucket	🟥	Only returns "x-amz-bucket-region" header.
HeadObject	✅	*
GetObject	✅	*
DeleteObject	✅	Also supports deleting directories.
DeleteObjects	✅	Also supports deleting directories.
PutObject	🟥	Only returns ETag header. *
CopyObject	🟥	Only returns ETag and LastModified as the CopyObjectResult. *

Multipart uploads are not supported. putObject requests are fully buffered in memory. Make sure to not overwhelm your system memory limit.

Presigned URLs are not yet supported.

* An objects ETag is always its UUID. Since Filen is fully end-to-end encrypted there is no way to know the real MD5 file hash.

License

Distributed under the AGPL-3.0 License. See LICENSE for more information.