Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RPC Records list filtering #3700

Merged
merged 18 commits into from
Jul 26, 2024
Merged

RPC Records list filtering #3700

Show file tree
Hide file tree
Changes from 16 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
42235d2
add initial where building function and filters
mathemancer Jul 18, 2024
6abb681
add more filtering functions
mathemancer Jul 19, 2024
a127f18
add debugging output about what query ran to response
mathemancer Jul 19, 2024
08beaed
implement remaining filtering functions
mathemancer Jul 19, 2024
6294458
add test for each leaf filter function
mathemancer Jul 19, 2024
68ced5e
Merge branch 'develop' into records_list_filter
mathemancer Jul 22, 2024
b16958b
add composition test for filtering, fix composition
mathemancer Jul 22, 2024
2a152b0
add more complex composition example
mathemancer Jul 22, 2024
0380f32
fix records list test with new type
mathemancer Jul 22, 2024
15c61d1
fix stupid syntax mistake :hankey: :hankey: :hankey:
mathemancer Jul 22, 2024
c90664e
Merge branch 'develop' into records_list_filter
mathemancer Jul 22, 2024
1155d7f
change column_id -> attnum for conformity with other RPC functions
mathemancer Jul 22, 2024
45490f9
add new classes to docs
mathemancer Jul 22, 2024
bb78a2e
update filtering logic as per convo with Sean
mathemancer Jul 25, 2024
79631b0
generalize terminology to avoid 'filter' where appropriate
mathemancer Jul 25, 2024
2cadd69
Merge branch 'develop' into records_list_filter
mathemancer Jul 25, 2024
570c87b
use explicit type casting for json containment
mathemancer Jul 26, 2024
8447a14
Merge branch 'develop' into records_list_filter
mathemancer Jul 26, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 52 additions & 4 deletions db/sql/00_msar.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3303,6 +3303,52 @@ SELECT val;
$$ LANGUAGE SQL RETURNS NULL ON NULL INPUT;


DROP TABLE IF EXISTS msar.expr_templates;
CREATE TABLE msar.expr_templates (expr_key text PRIMARY KEY, expr_template text);
INSERT INTO msar.expr_templates VALUES
-- basic composition operators
('and', '(%s) AND (%s)'),
('or', '(%s) OR (%s)'),
-- general comparison operators
('equal', '(%s) = (%s)'),
('lesser', '(%s) < (%s)'),
('greater', '(%s) > (%s)'),
('lesser_or_equal', '(%s) <= (%s)'),
('greater_or_equal', '(%s) >= (%s)'),
('null', '(%s) IS NULL'),
('not_null', '(%s) IS NOT NULL'),
-- string specific filters
('contains_case_insensitive', 'strpos(lower(%s), lower(%s))::boolean'),
('starts_with_case_insensitive', 'starts_with(lower(%s), lower(%s))'),
('contains', 'strpos((%s), (%s))::boolean'),
('starts_with', 'starts_with((%s), (%s))'),
-- json(b) filters and expressions
('json_array_length', 'jsonb_array_length((%s)::jsonb)'),
('json_array_contains', '(%s) @> (%s)'),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if we should be explicitly typecasting to jsonb here, as this would allow columns storing a json blob in a text column to be typecasted and return a result. However, if that is the intended behavior, it would be good to add typecasting to json_array_contains as well.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main goal was to allow this to be used on both json and jsonb. But, your point stands. I'll add typecasting to json_array_contains.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That makes sense.

-- URI part getters
('uri_scheme', 'mathesar_types.uri_scheme(%s)'),
('uri_authority', 'mathesar_types.uri_authority(%s)'),
-- Email part getters
('email_domain', 'mathesar_types.email_domain_name(%s)')
;

CREATE OR REPLACE FUNCTION msar.build_expr(rel_id oid, tree jsonb) RETURNS text AS $$
SELECT CASE tree ->> 'type'
WHEN 'literal' THEN format('%L', tree ->> 'value')
WHEN 'attnum' THEN format('%I', msar.get_column_name(rel_id, (tree ->> 'value')::smallint))
ELSE
format(max(expr_template), VARIADIC array_agg(msar.build_expr(rel_id, inner_tree)))
END
FROM jsonb_array_elements(tree -> 'args') inner_tree, msar.expr_templates
WHERE tree ->> 'type' = expr_key
$$ LANGUAGE SQL RETURNS NULL ON NULL INPUT;


CREATE OR REPLACE FUNCTION msar.build_where_clause(rel_id oid, tree jsonb) RETURNS text AS $$
SELECT 'WHERE ' || msar.build_expr(rel_id, tree);
$$ LANGUAGE SQL RETURNS NULL ON NULL INPUT;


CREATE OR REPLACE FUNCTION
msar.sanitize_direction(direction text) RETURNS text AS $$/*
*/
Expand Down Expand Up @@ -3426,13 +3472,14 @@ BEGIN
EXECUTE format(
$q$
WITH count_cte AS (
SELECT count(1) AS count FROM %2$I.%3$I
SELECT count(1) AS count FROM %2$I.%3$I %7$s
), results_cte AS (
SELECT %1$s FROM %2$I.%3$I %6$s LIMIT %4$L OFFSET %5$L
SELECT %1$s FROM %2$I.%3$I %7$s %6$s LIMIT %4$L OFFSET %5$L
)
SELECT jsonb_build_object(
'results', jsonb_agg(row_to_json(results_cte.*)),
'count', max(count_cte.count)
'count', max(count_cte.count),
'query', $iq$SELECT %1$s FROM %2$I.%3$I %7$s %6$s LIMIT %4$L OFFSET %5$L$iq$
)
FROM results_cte, count_cte
$q$,
Expand All @@ -3441,7 +3488,8 @@ BEGIN
msar.get_relation_name(tab_id),
limit_,
offset_,
msar.build_order_by_expr(tab_id, order_)
msar.build_order_by_expr(tab_id, order_),
msar.build_where_clause(tab_id, filter_)
) INTO records;
RETURN records;
END;
Expand Down
Loading
Loading