Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ability to do aggregation queries #420

Merged
merged 3 commits into from
Jul 19, 2024
Merged

Add ability to do aggregation queries #420

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
6d32fbe
Add ability to do aggregation queries
WolfgangSenff Jul 19, 2024
7631558
Fix typo in docs
WolfgangSenff Jul 19, 2024
46dc6f7
Fix whitespace changes
WolfgangSenff Jul 19, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
137 changes: 52 additions & 85 deletions addons/godot-firebase/firestore/firestore.gd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,39 +40,27 @@ const _MAX_POOLED_REQUEST_AGE = 30
## The code indicating the request Firestore is processing.
## See @[enum FirebaseFirestore.Requests] to get a full list of codes identifiers.
## @enum Requests
var request : int = -1

## Whether cache files can be used and generated.
## @default true
var persistence_enabled : bool = false

## Whether an internet connection can be used.
## @default true
var networking: bool = true : set = set_networking
var request: int = -1

## A Dictionary containing all authentication fields for the current logged user.
## @type Dictionary
var auth : Dictionary
var auth: Dictionary

var _config : Dictionary = {}
var _config: Dictionary = {}
var _cache_loc: String
var _encrypt_key := "5vg76n90345f7w390346" if Utilities.is_web() else OS.get_unique_id()


var _base_url : String = ""
var _extended_url : String = "projects/[PROJECT_ID]/databases/(default)/documents/"
var _query_suffix : String = ":runQuery"
var _base_url: String = ""
var _extended_url: String = "projects/[PROJECT_ID]/databases/(default)/documents/"
var _query_suffix: String = ":runQuery"
var _agg_query_suffix: String = ":runAggregationQuery"

#var _connect_check_node : HTTPRequest

var _request_list_node : HTTPRequest
var _requests_queue : Array = []
var _current_query : FirestoreQuery

var _offline: bool = false : set = _set_offline

func _ready() -> void:
pass
var _request_list_node: HTTPRequest
var _requests_queue: Array = []
var _current_query: FirestoreQuery

## Returns a reference collection by its [i]path[/i].
##
Expand All @@ -99,49 +87,69 @@ func collection(path : String) -> FirestoreCollection:
## Issue a query checked your Firestore database.
##
## [b]Note:[/b] a [code]FirestoreQuery[/code] object needs to be created to issue the query.
## This method will return a [code]FirestoreTask[/code] object, representing a reference to the request issued.
## If saved into a variable, the [code]FirestoreTask[/code] object can be used to yield checked the [code]result_query(result)[/code] signal, or the more generic [code]task_finished(result)[/code] signal.
##
## ex.
## [code]var query_task : FirestoreTask = Firebase.Firestore.query(FirestoreQuery.new())[/code]
## [code]await query_task.task_finished[/code]
## Since the emitted signal is holding an argument, it can be directly retrieved as a return variable from the [code]yield()[/code] function.
## When awaited, this function returns the resulting array from the query.
##
## ex.
## [code]var result : Array = await query_task.task_finished[/code]
## [code]var query_results = await Firebase.Firestore.query(FirestoreQuery.new())[/code]
##
## [b]Warning:[/b] It currently does not work offline!
##
## @args query
## @arg-types FirestoreQuery
## @return FirestoreTask
## @return Array[FirestoreDocument]
func query(query : FirestoreQuery) -> Array:
if query.aggregations.size() > 0:
Firebase._printerr("Aggregation query sent with normal query call: " + str(query))
return []

var task : FirestoreTask = FirestoreTask.new()
task.action = FirestoreTask.Task.TASK_QUERY
var body : Dictionary = { structuredQuery = query.query }
var url : String = _base_url + _extended_url + _query_suffix

var body: Dictionary = { structuredQuery = query.query }
var url: String = _base_url + _extended_url + _query_suffix
task.data = query
task._fields = JSON.stringify(body)
task._url = url
_pooled_request(task)
return await _handle_task_finished(task)


## Request a list of contents (documents and/or collections) inside a collection, specified by its [i]id[/i]. This method will return a [code]FirestoreTask[/code] object, representing a reference to the request issued. If saved into a variable, the [code]FirestoreTask[/code] object can be used to yield checked the [code]result_query(result)[/code] signal, or the more generic [code]task_finished(result)[/code] signal.
## [b]Note:[/b] [code]order_by[/code] does not work in offline mode.
## ex.
## [code]var query_task : FirestoreTask = Firebase.Firestore.query(FirestoreQuery.new())[/code]
## [code]await query_task.task_finished[/code]
## Since the emitted signal is holding an argument, it can be directly retrieved as a return variable from the [code]yield()[/code] function.

## Issue an aggregation query (sum, average, count) against your Firestore database;
## cheaper than a normal query and counting (for instance) values directly.
##
## [b]Note:[/b] a [code]FirestoreQuery[/code] object needs to be created to issue the query.
## When awaited, this function returns the result from the aggregation query.
##
## ex.
## [code]var result : Array = await query_task.task_finished[/code]
## [code]var query_results = await Firebase.Firestore.query(FirestoreQuery.new())[/code]
##
## [b]Warning:[/b] It currently does not work offline!
##
## @args query
## @arg-types FirestoreQuery
## @return Variant representing the array results of the aggregation query
func aggregation_query(query : FirestoreQuery) -> Variant:
if query.aggregations.size() == 0:
Firebase._printerr("Aggregation query sent with no aggregation values: " + str(query))
return 0

var task : FirestoreTask = FirestoreTask.new()
task.action = FirestoreTask.Task.TASK_AGG_QUERY

var body: Dictionary = { structuredAggregationQuery = { structuredQuery = query.query, aggregations = query.aggregations } }
var url: String = _base_url + _extended_url + _agg_query_suffix

task.data = query
task._fields = JSON.stringify(body)
task._url = url
_pooled_request(task)
var result = await _handle_task_finished(task)
return result

## Request a list of contents (documents and/or collections) inside a collection, specified by its [i]id[/i]. This method will return an Array[FirestoreDocument]
## @args collection_id, page_size, page_token, order_by
## @arg-types String, int, String, String
## @arg-defaults , 0, "", ""
## @return FirestoreTask
## @return Array[FirestoreDocument]
func list(path : String = "", page_size : int = 0, page_token : String = "", order_by : String = "") -> Array:
var task : FirestoreTask = FirestoreTask.new()
task.action = FirestoreTask.Task.TASK_LIST
Expand All @@ -160,38 +168,6 @@ func list(path : String = "", page_size : int = 0, page_token : String = "", ord
return await _handle_task_finished(task)


func set_networking(value: bool) -> void:
if value:
enable_networking()
else:
disable_networking()


func enable_networking() -> void:
if networking:
return
networking = true
_base_url = _base_url.replace("storeoffline", "firestore")
for coll in get_children():
if coll is FirestoreCollection:
coll._base_url = _base_url


func disable_networking() -> void:
if not networking:
return
networking = false
# Pointing to an invalid url should do the trick.
_base_url = _base_url.replace("firestore", "storeoffline")
for coll in get_children():
if coll is FirestoreCollection:
coll._base_url = _base_url


func _set_offline(value: bool) -> void:
return # Since caching is causing a lot of issues, I'm turning it off for now. We will revisit this in the future, once we have some time to investigate why the cache is being corrupted.


func _set_config(config_json : Dictionary) -> void:
_config = config_json
_cache_loc = _config["cacheLocation"]
Expand All @@ -213,10 +189,6 @@ func _check_emulating() -> void :
_base_url = "http://localhost:{port}/{version}/".format({ version = _API_VERSION, port = port })

func _pooled_request(task : FirestoreTask) -> void:
if _offline:
task._on_request_completed(HTTPRequest.RESULT_CANT_CONNECT, 404, PackedStringArray(), PackedByteArray())
return

if (auth == null or auth.is_empty()) and not Firebase.emulating:
Firebase._print("Unauthenticated request issued...")
Firebase.Auth.login_anonymous()
Expand Down Expand Up @@ -252,11 +224,6 @@ func _on_FirebaseAuth_token_refresh_succeeded(auth_result : Dictionary) -> void:
if coll is FirestoreCollection:
coll.auth = auth

func _on_connect_check_request_completed(result : int, _response_code, _headers, _body) -> void:
_set_offline(result != HTTPRequest.RESULT_SUCCESS)
#_connect_check_node.request(_base_url)


func _on_FirebaseAuth_logout() -> void:
auth = {}

Expand Down
39 changes: 28 additions & 11 deletions addons/godot-firebase/firestore/firestore_query.gd
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
## @meta-authors Nicoló 'fenix' Santilio
## @meta-authors Nicoló 'fenix' Santilio, Kyle Szklenski
## @meta-version 1.4
## A firestore query.
## Documentation TODO.
Expand All @@ -7,19 +7,19 @@ extends RefCounted
class_name FirestoreQuery

class Order:
var obj : Dictionary
var obj: Dictionary

class Cursor:
var values : Array
var before : bool
var values: Array
var before: bool

func _init(v : Array,b : bool):
values = v
before = b

signal query_result(query_result)

const TEMPLATE_QUERY : Dictionary = {
const TEMPLATE_QUERY: Dictionary = {
select = {},
from = [],
where = {},
Expand All @@ -30,11 +30,12 @@ const TEMPLATE_QUERY : Dictionary = {
limit = 0
}

var query : Dictionary = {}
var query: Dictionary = {}
var aggregations: Array[Dictionary] = []

enum OPERATOR {
# Standard operators
OPERATOR_NSPECIFIED,
OPERATOR_UNSPECIFIED,
LESS_THAN,
LESS_THAN_OR_EQUAL,
GREATER_THAN,
Expand Down Expand Up @@ -87,8 +88,6 @@ func from(collection_id : String, all_descendants : bool = true) -> FirestoreQue
query["from"] = [{collectionId = collection_id, allDescendants = all_descendants}]
return self



# @collections_array MUST be an Array of Arrays with this structure
# [ ["collection_id", true/false] ]
func from_many(collections_array : Array) -> FirestoreQuery:
Expand Down Expand Up @@ -159,8 +158,6 @@ func order_by_fields(order_field_list : Array) -> FirestoreQuery:
query["orderBy"] = order_list
return self



func start_at(value, before : bool) -> FirestoreQuery:
var cursor : Cursor = _cursor_object(value, before)
query["startAt"] = { values = cursor.values, before = cursor.before }
Expand Down Expand Up @@ -191,6 +188,26 @@ func limit(limit : int) -> FirestoreQuery:
return self


func aggregate() -> FirestoreAggregation:
return FirestoreAggregation.new(self)

class FirestoreAggregation extends RefCounted:
var _query: FirestoreQuery

func _init(query: FirestoreQuery) -> void:
_query = query

func sum(field: String) -> FirestoreQuery:
_query.aggregations.push_back({ sum = { field = { fieldPath = field }}})
return _query

func count(up_to: int) -> FirestoreQuery:
_query.aggregations.push_back({ count = { upTo = up_to }})
return _query

func average(field: String) -> FirestoreQuery:
_query.aggregations.push_back({ avg = { field = { fieldPath = field }}})
return _query

# UTILITIES ----------------------------------------

Expand Down
41 changes: 28 additions & 13 deletions addons/godot-firebase/firestore/firestore_task.gd
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ enum Task {
TASK_PATCH, ## A PATCH Request Task, processing a update() request
TASK_DELETE, ## A DELETE Request Task, processing a delete() request
TASK_QUERY, ## A POST Request Task, processing a query() request
TASK_AGG_QUERY, ## A POST Request Task, processing an aggregation_query() request
TASK_LIST, ## A POST Request Task, processing a list() request
TASK_COMMIT ## A POST Request Task that hits the write api
}
Expand All @@ -43,7 +44,8 @@ const TASK_MAP = {
Task.TASK_DELETE: "DELETE DOCUMENT",
Task.TASK_QUERY: "QUERY COLLECTION",
Task.TASK_LIST: "LIST DOCUMENTS",
Task.TASK_COMMIT: "COMMIT DOCUMENT"
Task.TASK_COMMIT: "COMMIT DOCUMENT",
Task.TASK_AGG_QUERY: "AGG QUERY COLLECTION"
}

## The code indicating the request Firestore is processing.
Expand All @@ -53,24 +55,23 @@ var action : int = -1 : set = set_action

## A variable, temporary holding the result of the request.
var data
var error : Dictionary
var document : FirestoreDocument
var error: Dictionary
var document: FirestoreDocument

var _response_headers : PackedStringArray = PackedStringArray()
var _response_code : int = 0
var _response_headers: PackedStringArray = PackedStringArray()
var _response_code: int = 0

var _method : int = -1
var _url : String = ""
var _fields : String = ""
var _headers : PackedStringArray = []
var _method: int = -1
var _url: String = ""
var _fields: String = ""
var _headers: PackedStringArray = []

func _on_request_completed(result : int, response_code : int, headers : PackedStringArray, body : PackedByteArray) -> void:
func _on_request_completed(result: int, response_code: int, headers: PackedStringArray, body: PackedByteArray) -> void:
var bod = body.get_string_from_utf8()
if bod != "":
bod = Utilities.get_json_data(bod)

var failed: bool = bod is Dictionary and bod.has("error") and response_code != HTTPClient.RESPONSE_OK

# Probably going to regret this...
if response_code == HTTPClient.RESPONSE_OK:
match action:
Expand All @@ -84,6 +85,18 @@ func _on_request_completed(result : int, response_code : int, headers : PackedSt
for doc in bod:
if doc.has('document'):
data.append(FirestoreDocument.new(doc.document))
Task.TASK_AGG_QUERY:
var agg_results = []
for agg_result in bod:
var idx = 0
var query_results = {}
for field_value in agg_result.result.aggregateFields.keys():
var agg = data.aggregations[idx]
var field = agg_result.result.aggregateFields[field_value]
query_results[agg.keys()[0]] = Utilities.from_firebase_type(field)
idx += 1
agg_results.push_back(query_results)
data = agg_results
Task.TASK_LIST:
data = []
if bod.has('documents'):
Expand Down Expand Up @@ -127,14 +140,16 @@ func set_action(value : int) -> void:
match action:
Task.TASK_GET, Task.TASK_LIST:
_method = HTTPClient.METHOD_GET
Task.TASK_POST, Task.TASK_QUERY:
Task.TASK_POST, Task.TASK_QUERY, Task.TASK_AGG_QUERY:
_method = HTTPClient.METHOD_POST
Task.TASK_PATCH:
_method = HTTPClient.METHOD_PATCH
Task.TASK_DELETE:
_method = HTTPClient.METHOD_DELETE
Task.TASK_COMMIT:
_method = HTTPClient.METHOD_POST
_:
assert(false)


func _merge_dict(dic_a : Dictionary, dic_b : Dictionary, nullify := false) -> Dictionary:
Expand Down
Loading