Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release v0.15.0 #3320

Merged
merged 47 commits into from
Dec 13, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
47 commits
Select commit Hold shift + click to select a range
27146cb
test(i): Fix the nodes ID/index bug in tests (#3162)
shahzadlone Oct 21, 2024
bb7843c
bot: Update dependencies (bulk dependabot PRs) 21-10-2024 (#3168)
github-actions[bot] Oct 21, 2024
7799b31
refactor(i): Lint casts & move panics under `must` funcs (#3134)
shahzadlone Oct 21, 2024
b268c0b
ci: Freeze goreleaser version and fix amd64 path (#3170)
fredcarle Oct 21, 2024
c1fcde0
fix(i): SetReplicator and DeleteReplicator params (#3175)
nasdf Oct 23, 2024
bb0917e
fix(i): Panic on aggregate alias (#3174)
nasdf Oct 24, 2024
d95c51f
feat: Add node identity (#3125)
islamaliev Oct 25, 2024
12669e4
fix: Add Authorization header to CORS allowed headers (#3178)
fredcarle Oct 25, 2024
fee0d7e
test: Allow soft-referencing of Cids in tests (#3176)
AndrewSisley Oct 28, 2024
3a3baac
bot: Update dependencies (bulk dependabot PRs) 2024-10-28 (#3188)
github-actions[bot] Oct 28, 2024
b4b2bf2
refactor: Remove indirection from crdt packages (#3192)
AndrewSisley Oct 29, 2024
7c7c030
refactor: Breakup core/keys.go file (#3198)
AndrewSisley Oct 30, 2024
2bd9857
bot: Update dependencies (bulk dependabot PRs) 04-11-2024 (#3207)
github-actions[bot] Nov 4, 2024
2b0b862
refactor: Simplify merkle/crdt code (#3200)
AndrewSisley Nov 4, 2024
c8fd3b1
feat: Filter alias target (#3201)
nasdf Nov 6, 2024
6d6c9f2
refactor: Rework core.Spans (#3210)
AndrewSisley Nov 6, 2024
2f53878
feat: Add ACP to pubsub KMS (#3206)
islamaliev Nov 7, 2024
85bbdc0
feat: Add support for branchable collections (#3216)
AndrewSisley Nov 8, 2024
a89524a
test(i): Skip test when detecting changes (#3234)
AndrewSisley Nov 11, 2024
dae2820
feat: Order alias target (#3217)
nasdf Nov 11, 2024
332f9d6
bot: Update dependencies (bulk dependabot PRs) 11-11-2024 (#3235)
github-actions[bot] Nov 11, 2024
909c4af
feat: Support for descending fields CLI index creation (#3237)
ChrisBQu Nov 14, 2024
198454b
refactor: Consolidate node-related fields into a struct (#3232)
islamaliev Nov 14, 2024
8e24ee4
fix(i): Remove flakyness of AES decrypt test (#3241)
fredcarle Nov 15, 2024
27893cf
feat(i): Handle collection commits over P2P (#3247)
AndrewSisley Nov 18, 2024
c45b07e
feat: Add support for cid-only time travel queries (#3256)
AndrewSisley Nov 20, 2024
fa0d92b
feat: Add support for branchable collection time-traveling (#3260)
AndrewSisley Nov 20, 2024
3fa579e
fix: Prevent over span (#3258)
fredcarle Nov 20, 2024
cdc5738
docs: Update discord link (#3231)
emmanuel-ferdman Nov 21, 2024
6cacff1
refactor: Add unified JSON interface (#3265)
islamaliev Nov 21, 2024
a61dcd7
fix: Resolve CORS errors in OpenAPI tab of Playground (#3263)
ChrisBQu Nov 22, 2024
ad2fcae
ci: Fix the gql mutation running in all tests (#3267)
shahzadlone Nov 25, 2024
e2c657e
bot: Update dependencies (bulk dependabot PRs) 25-11-2024 (#3273)
github-actions[bot] Nov 25, 2024
c7b8b93
feat: Add ability to add/delete relationship for all actors (#3254)
shahzadlone Nov 26, 2024
8509211
docs(i): Replace broken DQL link in README (#3274)
amarkpark Nov 27, 2024
6cb3289
bot: Update dependencies (bulk dependabot PRs) 03-12-2024 (#3288)
github-actions[bot] Dec 2, 2024
2776f1e
feat: Aggregate order alias targeting (#3293)
nasdf Dec 5, 2024
e4599e8
feat: Aggregate filter alias targeting (#3252)
nasdf Dec 5, 2024
89f9f41
fix: Add support for operationName and variables in HTTP GET (#3292)
ChrisBQu Dec 5, 2024
b4a3eba
fix: Make requests with no identity work with "*" target (#3278)
shahzadlone Dec 6, 2024
1ad9fa9
bot: Update dependencies (bulk dependabot PRs) 08-12-2024 (#3296)
github-actions[bot] Dec 8, 2024
edd0ce7
bot: Update dependencies (bulk dependabot PRs) 09-12-2024 (#3307)
github-actions[bot] Dec 9, 2024
be3f381
fix: Adjust OpenAPI index POST example request body (#3268)
ChrisBQu Dec 9, 2024
d6900b7
feat: Error if purge request made with dev mode disabled (#3295)
ChrisBQu Dec 9, 2024
648924e
bot: Update dependencies (bulk dependabot PRs) 10-12-2024 (#3312)
github-actions[bot] Dec 9, 2024
e1502c5
fix(i): Aggregate filter returns one result (#3316)
nasdf Dec 11, 2024
fecc50f
Release v0.15.0
fredcarle Dec 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
12 changes: 6 additions & 6 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ jobs:
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser-pro
version: latest
version: 2.3.2
args: release --clean --split ${{ env.flags }}
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand All @@ -67,7 +67,7 @@ jobs:
if: matrix.os == 'ubuntu-latest'
uses: actions/cache/save@v4
with:
path: dist/linux_amd64
path: dist/linux_amd64_v1
key: linux-${{ env.sha_short }}

- name: Save cache on MacOS
Expand All @@ -81,7 +81,7 @@ jobs:
if: matrix.os == 'windows-latest'
uses: actions/cache/save@v4
with:
path: dist/windows_amd64
path: dist/windows_amd64_v1
key: windows-${{ env.sha_short }}
enableCrossOsArchive: true

Expand Down Expand Up @@ -113,7 +113,7 @@ jobs:
id: restore-linux
uses: actions/cache/restore@v4
with:
path: dist/linux_amd64
path: dist/linux_amd64_v1
key: linux-${{ env.sha_short }}
fail-on-cache-miss: true

Expand All @@ -129,7 +129,7 @@ jobs:
id: restore-windows
uses: actions/cache/restore@v4
with:
path: dist/windows_amd64
path: dist/windows_amd64_v1
key: windows-${{ env.sha_short }}
fail-on-cache-miss: true
enableCrossOsArchive: true
Expand All @@ -147,7 +147,7 @@ jobs:
uses: goreleaser/goreleaser-action@v6
with:
distribution: goreleaser-pro
version: latest
version: 2.3.2
args: continue --merge
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
Expand Down
67 changes: 67 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,70 @@
<a name="v0.15.0"></a>
## [v0.15.0](https://github.com/sourcenetwork/defradb/compare/v0.14.0...v0.15.0)

> 2024-12-13

DefraDB v0.15 is a major pre-production release. Until the stable version 1.0 is reached, the SemVer minor patch number will denote notable releases, which will give the project freedom to experiment and explore potentially breaking changes.

To get a full outline of the changes, we invite you to review the official changelog below. This release does include a Breaking Change to existing v0.14.x databases. If you need help migrating an existing deployment, reach out at [[email protected]](mailto:[email protected]) or join our Discord at https://discord.gg/w7jYQVJ/.

### Features

* Add support for branchable collections ([#3216](https://github.com/sourcenetwork/defradb/issues/3216))
* Aggregate filter alias targeting ([#3252](https://github.com/sourcenetwork/defradb/issues/3252))
* Aggregate order alias targeting ([#3293](https://github.com/sourcenetwork/defradb/issues/3293))
* Add ability to add/delete relationship for all actors ([#3254](https://github.com/sourcenetwork/defradb/issues/3254))
* Add support for branchable collection time-traveling ([#3260](https://github.com/sourcenetwork/defradb/issues/3260))
* Add support for cid-only time travel queries ([#3256](https://github.com/sourcenetwork/defradb/issues/3256))
* Support for descending fields CLI index creation ([#3237](https://github.com/sourcenetwork/defradb/issues/3237))
* Order alias target ([#3217](https://github.com/sourcenetwork/defradb/issues/3217))
* Error if purge request made with dev mode disabled ([#3295](https://github.com/sourcenetwork/defradb/issues/3295))
* Add ACP to pubsub KMS ([#3206](https://github.com/sourcenetwork/defradb/issues/3206))
* Filter alias target ([#3201](https://github.com/sourcenetwork/defradb/issues/3201))
* Add node identity ([#3125](https://github.com/sourcenetwork/defradb/issues/3125))

### Fixes

* Adjust OpenAPI index POST example request body ([#3268](https://github.com/sourcenetwork/defradb/issues/3268))
* Make requests with no identity work with "*" target ([#3278](https://github.com/sourcenetwork/defradb/issues/3278))
* Add support for operationName and variables in HTTP GET ([#3292](https://github.com/sourcenetwork/defradb/issues/3292))
* Resolve CORS errors in OpenAPI tab of Playground ([#3263](https://github.com/sourcenetwork/defradb/issues/3263))
* Prevent over span ([#3258](https://github.com/sourcenetwork/defradb/issues/3258))
* Add Authorization header to CORS allowed headers ([#3178](https://github.com/sourcenetwork/defradb/issues/3178))

### Documentation

* Update discord link ([#3231](https://github.com/sourcenetwork/defradb/issues/3231))

### Refactoring

* Add unified JSON interface ([#3265](https://github.com/sourcenetwork/defradb/issues/3265))
* Consolidate node-related fields into a struct ([#3232](https://github.com/sourcenetwork/defradb/issues/3232))
* Rework core.Spans ([#3210](https://github.com/sourcenetwork/defradb/issues/3210))
* Simplify merkle/crdt code ([#3200](https://github.com/sourcenetwork/defradb/issues/3200))
* Breakup core/keys.go file ([#3198](https://github.com/sourcenetwork/defradb/issues/3198))
* Remove indirection from crdt packages ([#3192](https://github.com/sourcenetwork/defradb/issues/3192))

### Testing

* Allow soft-referencing of Cids in tests ([#3176](https://github.com/sourcenetwork/defradb/issues/3176))

### Continuous integration

* Fix the gql mutation running in all tests ([#3267](https://github.com/sourcenetwork/defradb/issues/3267))
* Freeze goreleaser version and fix amd64 path ([#3170](https://github.com/sourcenetwork/defradb/issues/3170))

### Bot

* Update dependencies (bulk dependabot PRs) 10-12-2024 ([#3312](https://github.com/sourcenetwork/defradb/issues/3312))
* Update dependencies (bulk dependabot PRs) 09-12-2024 ([#3307](https://github.com/sourcenetwork/defradb/issues/3307))
* Update dependencies (bulk dependabot PRs) 08-12-2024 ([#3296](https://github.com/sourcenetwork/defradb/issues/3296))
* Update dependencies (bulk dependabot PRs) 03-12-2024 ([#3288](https://github.com/sourcenetwork/defradb/issues/3288))
* Update dependencies (bulk dependabot PRs) 25-11-2024 ([#3273](https://github.com/sourcenetwork/defradb/issues/3273))
* Update dependencies (bulk dependabot PRs) 11-11-2024 ([#3235](https://github.com/sourcenetwork/defradb/issues/3235))
* Update dependencies (bulk dependabot PRs) 04-11-2024 ([#3207](https://github.com/sourcenetwork/defradb/issues/3207))
* Update dependencies (bulk dependabot PRs) 2024-10-28 ([#3188](https://github.com/sourcenetwork/defradb/issues/3188))
* Update dependencies (bulk dependabot PRs) 21-10-2024 ([#3168](https://github.com/sourcenetwork/defradb/issues/3168))

<a name="v0.14.0"></a>
## [v0.14.0](https://github.com/sourcenetwork/defradb/compare/v0.13.0...v0.14.0)

Expand Down
2 changes: 1 addition & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This document will guide you through the process of contributing to the project.

All contributions are appreciated, whether it's identifying problems, highlighting missing features, or contributing to the codebase in simple or complex ways.

You are encouraged to join the [Source Network Discord](discord.gg/w7jYQVJ) to discuss ideas, ask questions, and find inspiration for future developments.
You are encouraged to join the [Source Network Discord](https://discord.gg/w7jYQVJ) to discuss ideas, ask questions, and find inspiration for future developments.

## Getting started
To get started, clone the repository, build, and run it:
Expand Down
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -207,7 +207,7 @@ verify:

.PHONY: tidy
tidy:
go mod tidy -go=1.22
go mod tidy

.PHONY: clean
clean:
Expand Down
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@
</picture>
</p>

DefraDB is a user-centric database that prioritizes data ownership, personal privacy, and information security. Its data model, powered by the convergence of [MerkleCRDTs](https://arxiv.org/pdf/2004.00107.pdf) and the content-addressability of [IPLD](https://docs.ipld.io/), enables a multi-write-master architecture. It features [DQL](https://docs.source.network/references/query-specification/query-language-overview), a query language compatible with GraphQL but providing extra convenience. By leveraging peer-to-peer networking it can be deployed nimbly in novel topologies. Access control is determined by a relationship-based DSL, supporting document or field-level policies, secured by the SourceHub network. DefraDB is a core part of the [Source technologies](https://source.network/) that enable new paradigms of decentralized data and access-control management, user-centric apps, data trustworthiness, and much more.
DefraDB is a user-centric database that prioritizes data ownership, personal privacy, and information security. Its data model, powered by the convergence of [MerkleCRDTs](https://arxiv.org/pdf/2004.00107.pdf) and the content-addressability of [IPLD](https://docs.ipld.io/), enables a multi-write-master architecture. It features [DQL](https://docs.source.network/defradb/references/query-specification/query-language-overview), a query language compatible with GraphQL but providing extra convenience. By leveraging peer-to-peer networking it can be deployed nimbly in novel topologies. Access control is determined by a relationship-based DSL, supporting document or field-level policies, secured by the SourceHub network. DefraDB is a core part of the [Source technologies](https://source.network/) that enable new paradigms of decentralized data and access-control management, user-centric apps, data trustworthiness, and much more.

Read the documentation on [docs.source.network](https://docs.source.network/).

Expand Down Expand Up @@ -67,13 +67,16 @@ The following keys are loaded from the keyring on start:

- `peer-key` Ed25519 private key (required)
- `encryption-key` AES-128, AES-192, or AES-256 key (optional)
- `node-identity-key` Secp256k1 private key (optional). This key is used for node's identity.

A secret to unlock the keyring is required on start and must be provided via the `DEFRA_KEYRING_SECRET` environment variable. If a `.env` file is available in the working directory, the secret can be stored there or via a file at a path defined by the `--secret-file` flag.

The keys will be randomly generated on the inital start of the node if they are not found.
The keys will be randomly generated on the initial start of the node if they are not found.

Alternatively, to randomly generate the required keys, run the following command:

Node identity is an identity assigned to the node. It is used to exchange encryption keys with other nodes.

```
defradb keyring generate
```
Expand Down
41 changes: 41 additions & 0 deletions acp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -631,6 +631,27 @@ Result:
Error: document not found or not authorized to access
```

Sometimes we might want to give a specific access (i.e. form a relationship) not just with one identity, but with
any identity (includes even requests with no-identity).
In that case we can specify "*" instead of specifying an explicit `actor`:
```sh
defradb client acp relationship add \
--collection Users \
--docID bae-ff3ceb1c-b5c0-5e86-a024-dd1b16a4261c \
--relation reader \
--actor "*" \
--identity e3b722906ee4e56368f581cd8b18ab0f48af1ea53e635e3f7b8acd076676f6ac
```

Result:
```json
{
"ExistedAlready": false
}
```

**Note: specifying `*` does not overwrite any previous formed relationships, they will remain as is **

### Revoking Access To Private Documents

To revoke access to a document for an actor, we must delete the relationship between the
Expand Down Expand Up @@ -695,6 +716,26 @@ defradb client collection docIDs --identity 4d092126012ebaf56161716018a71630d994

**Result is empty from the above command**

We can also revoke the previously granted implicit relationship which gave all actors access using the "*" actor.
Similarly we can just specify "*" to revoke all access given to actors implicitly through this relationship:
```sh
defradb client acp relationship delete \
--collection Users \
--docID bae-ff3ceb1c-b5c0-5e86-a024-dd1b16a4261c \
--relation reader \
--actor "*" \
--identity e3b722906ee4e56368f581cd8b18ab0f48af1ea53e635e3f7b8acd076676f6ac
```

Result:
```json
{
"RecordFound": true
}
```

**Note: Deleting with`*` does not remove any explicitly formed relationships, they will remain as they were **

## DAC Usage HTTP:

### Authentication
Expand Down
36 changes: 34 additions & 2 deletions acp/acp_local.go
Original file line number Diff line number Diff line change
Expand Up @@ -254,9 +254,25 @@ func (l *ACPLocal) AddActorRelationship(

ctx = auth.InjectPrincipal(ctx, principal)

var newActorRelationship *types.Relationship
if targetActor == "*" {
newActorRelationship = types.NewAllActorsRelationship(
resourceName,
objectID,
relation,
)
} else {
newActorRelationship = types.NewActorRelationship(
resourceName,
objectID,
relation,
targetActor,
)
}

setRelationshipRequest := types.SetRelationshipRequest{
PolicyId: policyID,
Relationship: types.NewActorRelationship(resourceName, objectID, relation, targetActor),
Relationship: newActorRelationship,
CreationTime: creationTime,
}

Expand Down Expand Up @@ -285,9 +301,25 @@ func (l *ACPLocal) DeleteActorRelationship(

ctx = auth.InjectPrincipal(ctx, principal)

var newActorRelationship *types.Relationship
if targetActor == "*" {
newActorRelationship = types.NewAllActorsRelationship(
resourceName,
objectID,
relation,
)
} else {
newActorRelationship = types.NewActorRelationship(
resourceName,
objectID,
relation,
targetActor,
)
}

deleteRelationshipRequest := types.DeleteRelationshipRequest{
PolicyId: policyID,
Relationship: types.NewActorRelationship(resourceName, objectID, relation, targetActor),
Relationship: newActorRelationship,
}

deleteRelationshipResponse, err := l.engine.DeleteRelationship(ctx, &deleteRelationshipRequest)
Expand Down
64 changes: 42 additions & 22 deletions acp/acp_source_hub.go
Original file line number Diff line number Diff line change
Expand Up @@ -273,18 +273,28 @@ func (a *acpSourceHub) AddActorRelationship(
creationTime *protoTypes.Timestamp,
) (bool, error) {
msgSet := sourcehub.MsgSet{}

var newActorRelationship *acptypes.Relationship
if targetActor == "*" {
newActorRelationship = acptypes.NewAllActorsRelationship(
resourceName,
objectID,
relation,
)
} else {
newActorRelationship = acptypes.NewActorRelationship(
resourceName,
objectID,
relation,
targetActor,
)
}

cmdMapper := msgSet.WithBearerPolicyCmd(&acptypes.MsgBearerPolicyCmd{
Creator: a.signer.GetAccAddress(),
BearerToken: requester.BearerToken,
PolicyId: policyID,
Cmd: acptypes.NewSetRelationshipCmd(
acptypes.NewActorRelationship(
resourceName,
objectID,
relation,
targetActor,
),
),
Creator: a.signer.GetAccAddress(),
BearerToken: requester.BearerToken,
PolicyId: policyID,
Cmd: acptypes.NewSetRelationshipCmd(newActorRelationship),
CreationTime: creationTime,
})
tx, err := a.txBuilder.Build(ctx, a.signer, &msgSet)
Expand Down Expand Up @@ -323,18 +333,28 @@ func (a *acpSourceHub) DeleteActorRelationship(
creationTime *protoTypes.Timestamp,
) (bool, error) {
msgSet := sourcehub.MsgSet{}

var newActorRelationship *acptypes.Relationship
if targetActor == "*" {
newActorRelationship = acptypes.NewAllActorsRelationship(
resourceName,
objectID,
relation,
)
} else {
newActorRelationship = acptypes.NewActorRelationship(
resourceName,
objectID,
relation,
targetActor,
)
}

cmdMapper := msgSet.WithBearerPolicyCmd(&acptypes.MsgBearerPolicyCmd{
Creator: a.signer.GetAccAddress(),
BearerToken: requester.BearerToken,
PolicyId: policyID,
Cmd: acptypes.NewDeleteRelationshipCmd(
acptypes.NewActorRelationship(
resourceName,
objectID,
relation,
targetActor,
),
),
Creator: a.signer.GetAccAddress(),
BearerToken: requester.BearerToken,
PolicyId: policyID,
Cmd: acptypes.NewDeleteRelationshipCmd(newActorRelationship),
CreationTime: creationTime,
})

Expand Down
Loading
Loading