Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[dataquery] New Data Query Module #8907

Merged
merged 144 commits into from
Dec 11, 2023
Merged
Show file tree
Hide file tree
Changes from 117 commits
Commits
Show all changes
144 commits
Select commit Hold shift + click to select a range
a4f32a6
[Query Interface] Add Module->getQueryEngine()
driusan Dec 7, 2022
3ee979f
Add NullQueryEngine
driusan Dec 7, 2022
f0d30da
Add Candidate Query Engine
driusan Dec 7, 2022
f97e3f9
Add SQLQueryEngine class
driusan Dec 7, 2022
c2b8efb
Fix some phan errors
driusan Dec 7, 2022
93b2428
Fix make checkstatic
driusan Dec 19, 2022
0284488
Move SQL related functions from CandidateQueryEngine to SQLQueryEngine
driusan Dec 19, 2022
a49507a
Improve documentation for SQLQueryEngine
driusan Dec 19, 2022
b7b1877
Add getQueryEngine to module
driusan Dec 19, 2022
29e788e
test signature must be compatible
driusan Jan 24, 2023
637ce4e
Remove reference to Module::factory, which no longer exists
driusan Aug 16, 2023
213f6ec
Remove call to non-existent Database::singleton
driusan Aug 16, 2023
ef6d120
Do not change return value of QueryEngine, causes errors with Instrum…
driusan Aug 16, 2023
8435986
[dataquery] Define new swagger schema for data query API
driusan Nov 7, 2022
1a1bb3b
Update schema after Xaviers review
driusan Dec 16, 2022
b9b148b
Create QueryObject
driusan Dec 16, 2022
1cc8e61
Change to POST, add QueryRunID to query run
driusan Dec 16, 2022
9f91a91
Split queries and queryruns into two different endpoints
driusan Dec 19, 2022
e905710
Add JSX for newest frontend for the dataquery module
driusan Dec 14, 2022
9729d4a
Update URL to correct module
driusan Dec 14, 2022
fbdc9b7
Add visitlist endpoint
driusan Dec 14, 2022
1966d1d
Fix loading of main page after latest API changes
driusan Dec 19, 2022
7990e9c
Take swagger from main branch
driusan Dec 19, 2022
2930554
[Query Interface] Add Module->getQueryEngine()
driusan Dec 7, 2022
b162f2e
Add NullQueryEngine
driusan Dec 7, 2022
61214d3
PHPCS
driusan Nov 7, 2022
525dac9
phan
driusan Nov 7, 2022
a78b112
PHPCS
driusan Dec 7, 2022
dedb20f
Fix some phan errors
driusan Dec 7, 2022
567f53f
Make QueryEngines getCandidateData be CandID => DataInstance instead …
driusan Dec 7, 2022
d1fff12
Suppress error caused by phan bug
driusan Dec 7, 2022
938e3c1
Replace getDataDictionary with getQueryEngine
driusan Dec 7, 2022
97780c6
Phan
driusan Dec 7, 2022
ed46321
[Query Interface] Add Module->getQueryEngine()
driusan Dec 7, 2022
6e9f48d
Add NullQueryEngine
driusan Dec 7, 2022
f72b8fa
Add Candidate Query Engine
driusan Dec 7, 2022
22ed795
Add SQLQueryEngine class
driusan Dec 7, 2022
6c8ea21
Fix some phan errors
driusan Dec 7, 2022
5971cd3
Fix make checkstatic
driusan Dec 19, 2022
44c7052
Move SQL related functions from CandidateQueryEngine to SQLQueryEngine
driusan Dec 19, 2022
ebff6bf
Improve documentation for SQLQueryEngine
driusan Dec 19, 2022
5e1ad6e
Add getQueryEngine to module
driusan Dec 19, 2022
01d0a64
test signature must be compatible
driusan Jan 24, 2023
b132620
Post to run
driusan Mar 10, 2023
cc156bc
Post request, not get, to run
driusan Mar 10, 2023
c7c1aca
Define missing function
driusan Mar 10, 2023
9a9999d
Some updates to first screen
driusan Mar 13, 2023
e184d68
Fix createRoot errors
driusan Jun 9, 2023
bdc4282
Fix crashing when sync fields is selected and candidate scoped variab…
driusan Jun 9, 2023
a4c5f6f
Fix share query button
driusan Jun 9, 2023
e8a156e
change share icon to globe
driusan Jun 9, 2023
85dc512
Fix star button
driusan Jun 9, 2023
fc1390e
Fix inline display
driusan Jun 9, 2023
45a1065
implement longitudinal view of data
driusan Jun 9, 2023
e75833c
Implement cross-sectional display of visits
driusan Jun 12, 2023
fb7936f
Remove console logs
driusan Jul 3, 2023
737a946
Include loading progress bar
driusan Jul 3, 2023
365642d
refactor to be more responsive for data organization
driusan Jul 3, 2023
26b6fb9
Change cancel to close
driusan Aug 8, 2023
f0c49a4
fix loading of visits when loading a saved query
driusan Aug 8, 2023
ae1cea1
check for empty fieldobj in getDictionary
driusan Aug 16, 2023
d30c787
Fix different display modes
driusan Aug 16, 2023
28d102c
Start adding PropTypes
driusan Aug 21, 2023
379b725
Take instrumentqueryengine from other branch
driusan Aug 21, 2023
50af8f3
Remove changes from alluserqueries.class.inc, take from DQTRestBacken…
driusan Aug 21, 2023
7bf79de
Remove changes from old DQT module, not sure why they were changed
driusan Aug 21, 2023
c37fe0c
Restore dictionary module from main branch
driusan Aug 21, 2023
de5f4e7
[JS] Add username/id to Loris helper object
driusan Aug 28, 2023
5488471
Convert module to typescript
driusan Aug 21, 2023
96b2f26
Fix query engines
driusan Sep 13, 2023
41895be
Readd dashboard widgets
driusan Sep 13, 2023
0be4e5f
Fix loading of query from url
driusan Sep 13, 2023
7804d60
Remove debug log, fix `npm run lint:js`
driusan Sep 13, 2023
96a424a
Add option to change header display format
driusan Sep 13, 2023
64aa38f
Slightly better progress meter on cross-sectional view
driusan Sep 14, 2023
97eef9e
[configuration] Escape values in config module
driusan Sep 28, 2023
001f233
Remove unnecessary no-op file changes from main
driusan Nov 13, 2023
b67e644
PHPCS
driusan Nov 13, 2023
a53f72d
Merge branch 'CandidateParametersQueryEngine' into NewestDQTFrontendR…
driusan Nov 13, 2023
e3c860a
phan
driusan Nov 13, 2023
a1340ce
Merge remote-tracking branch 'aces/main' into NewestDQTFrontendRebased2
driusan Nov 13, 2023
3d94cd9
Fix build after rebase
driusan Nov 13, 2023
31a5287
[Query Interface] Add Module->getQueryEngine()
driusan Dec 7, 2022
5b50f0a
Add NullQueryEngine
driusan Dec 7, 2022
05dece6
Add Candidate Query Engine
driusan Dec 7, 2022
1e45948
Add SQLQueryEngine class
driusan Dec 7, 2022
55b04e4
Fix some phan errors
driusan Dec 7, 2022
a089c22
Fix make checkstatic
driusan Dec 19, 2022
fd4ec79
Move SQL related functions from CandidateQueryEngine to SQLQueryEngine
driusan Dec 19, 2022
3293030
Improve documentation for SQLQueryEngine
driusan Dec 19, 2022
1d49d3b
Add getQueryEngine to module
driusan Dec 19, 2022
3dc8c9b
test signature must be compatible
driusan Jan 24, 2023
c47f628
Remove reference to Module::factory, which no longer exists
driusan Aug 16, 2023
fe8fd43
Remove call to non-existent Database::singleton
driusan Aug 16, 2023
1f845f9
Do not change return value of QueryEngine, causes errors with Instrum…
driusan Aug 16, 2023
1d02b12
Change subproject to cohort
driusan Nov 14, 2023
d4cc1e8
Update module search path in test, replace subproject with cohort in …
driusan Nov 14, 2023
79b3c3f
Capitalize table name
driusan Nov 14, 2023
ea03db4
Handle both traversable and arrays in assertion
driusan Nov 14, 2023
3109701
More test fixing
driusan Nov 14, 2023
de30750
phpcs
driusan Nov 15, 2023
360b064
fix duplicate call to iterator_to_array causing error
driusan Nov 15, 2023
85ea04f
Change substring search.
driusan Nov 15, 2023
1427b5f
Remove unnecessary assertion that is failing
driusan Nov 15, 2023
1b2ad6f
Fix substring search after subproject->cohort change, add debug for o…
driusan Nov 15, 2023
b25e024
fix test, handle session variables in newest return format
driusan Nov 15, 2023
d75e56c
Use sessionID as array key
driusan Nov 15, 2023
0a906d0
Cohort is also session scoped
driusan Nov 15, 2023
d27fdf0
put session tests in proper format in getCandidateData too
driusan Nov 15, 2023
cb86714
RegistrationSite is a Candidate variable, not session
driusan Nov 15, 2023
0c3e177
Merge branch 'CandidateParametersQueryEngine' into NewestDQTFrontendR…
driusan Nov 15, 2023
7d79038
Merge remote-tracking branch 'aces/main' into NewestDQTFrontendRebased2
driusan Nov 15, 2023
f73ee00
fix lint errors
driusan Nov 15, 2023
6c4d25d
Add Form.d.ts
driusan Nov 15, 2023
266f297
Remove unrelated change
driusan Nov 15, 2023
1179bf6
Merge remote-tracking branch 'aces/main' into NewestDQTFrontendRebased2
driusan Nov 20, 2023
c6d7e9e
Merge remote-tracking branch 'aces/main' into NewestDQTFrontendRebased2
driusan Nov 29, 2023
69f8547
Camille's review. Mostly removing no longer valid comments.
driusan Dec 1, 2023
f4864b8
Fix indentation
driusan Dec 1, 2023
de31f78
Remove unnecessary _
driusan Dec 1, 2023
49ddaa1
Rename Introductions to Instructions since it isnt the first thing on…
driusan Dec 1, 2023
8d12954
Use jsx/Panel instead of creating new Panel type for ExpansionPanels
driusan Dec 1, 2023
2b866ed
Remove iterator_to_array in instrumentqueryengine
driusan Dec 1, 2023
4dacd94
Center button
driusan Dec 1, 2023
2d49143
Fix unused import error
driusan Dec 1, 2023
35d861c
Use FilterableSelectGroup for operators
driusan Dec 1, 2023
fffcb7b
use FilterableSelectGroup for select values
driusan Dec 1, 2023
3570502
Use relative units
driusan Dec 1, 2023
f8582a3
Add dataquery_admin permission to schema and patch
driusan Dec 1, 2023
653f7b9
Fix primary key
driusan Dec 1, 2023
26b6aa5
Use DataTable instead of DynamicDataTable
driusan Dec 1, 2023
8d18465
Add border to Next Steps
driusan Dec 1, 2023
006b756
Fix import of candidate CSVs
driusan Dec 1, 2023
997c852
Add placeholder text in pin modal dialogs
driusan Dec 1, 2023
83cfc23
Remove console.log
driusan Dec 1, 2023
5f90864
Ensure admin query name contains more than whitespace
driusan Dec 4, 2023
dcbe25c
Hide downloadCSV if inline
driusan Dec 5, 2023
29ef909
Fix hide downloadCSV option
driusan Dec 5, 2023
296159f
Hide default column
driusan Dec 5, 2023
f271641
Group by all columns
driusan Dec 5, 2023
23d5262
Remove dead code
driusan Dec 6, 2023
d87fa66
Merge remote-tracking branch 'aces/main' into NewestDQTFrontendRebased2
driusan Dec 6, 2023
848f4e8
Fix download as CSV formatting
driusan Dec 6, 2023
c741b4b
Use the admin name in study queries
driusan Dec 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,9 @@ data_release:
instrument_manager:
target=instrument_manager npm run compile

dataquery:
target=dataquery npm run compile

login:
target=login npm run compile

Expand Down
55 changes: 55 additions & 0 deletions modules/dataquery/jsx/calcpayload.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
import {QueryTerm, QueryGroup} from './querydef';
import {
APIQueryObject,
APIQueryField,
APIQueryGroupField,
APIQueryCriteriaGroup,
} from './types';
/**
* Calculates the payload to submit to the search endpoint
* to run the query.
*
* @param {APIQueryField[]} fields - the fields to query
* @param {QueryGroup} filters - the root of the filters
* @returns {APIQueryObject} - The query to send to the API
*/
export function calcPayload(
fields: APIQueryField[],
filters: QueryGroup
): APIQueryObject {
const payload: APIQueryObject = {
type: 'candidates',
fields: fields.map((val: APIQueryField) => {
// console.log('payload ', val);
driusan marked this conversation as resolved.
Show resolved Hide resolved
const fieldpayload: APIQueryField = {
module: val.module,
category: val.category,
field: val.field,
};
// console.log('payload visits', val.visits);
driusan marked this conversation as resolved.
Show resolved Hide resolved
if (val.visits) {
fieldpayload.visits = val.visits;
// convert from React select to static string
// payload.visits = val.visits.map( (visitOption) => visitOption.value);
driusan marked this conversation as resolved.
Show resolved Hide resolved
}
return fieldpayload;
},
),
};
if (filters.group.length > 0) {
payload.criteria = {
operator: filters.operator,
group: filters.group.map( (val) => {
if (val instanceof QueryTerm) {
return val as APIQueryGroupField;
} else if (val instanceof QueryGroup) {
return val as APIQueryCriteriaGroup;
} else {
throw new Error('Invalid query');
}
}),
};
}
return payload;
}

122 changes: 122 additions & 0 deletions modules/dataquery/jsx/components/expansionpanels.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
import React, {useState} from 'react';

/**
* Component to render a single panel
*
* @param {object} props - The React Props
* @param {boolean} props.defaultOpen - Whether the default should default open
* @param {boolean} props.alwaysOpen - Whether the panel can not be collapsed
* @param {string} props.title - The panel title
* @param {React.ReactElement} props.content - The panel body
* @returns {React.ReactElement} - The rendered panel
*/
const Panel = (props: {
defaultOpen: boolean,
alwaysOpen: boolean,
title: string,
content: React.ReactElement,
}) => {
const [active, setActive] = useState(props.defaultOpen);

const styles = {
accordion: {
default: {
width: '100%',
padding: '18px',
driusan marked this conversation as resolved.
Show resolved Hide resolved
outline: 'none',
color: '#246EB6',
fontSize: '15px',
cursor: props.alwaysOpen ? 'default' : 'pointer',
textAlign: 'center' as const,
backgroundColor: '#fff',
border: '1px solid #246EB6',
transition: '0.4s',
},
active: {
color: '#fff',
textAlign: 'left' as const,
backgroundColor: '#246EB6',
},
},
panel: {
default: {
display: 'none',
padding: '20px 18px',
backgroundColor: '#fff',
border: '1px solid #246EB6',
overflow: 'hidden',
},
active: {
display: 'block',
margin: '0 0 10px 0',
},
},
};

/**
* Handle clicking on the header
*
* @returns {void}
*/
const handleExpansionClick = () => {
if (props.alwaysOpen) return;
setActive((active) => !active);
};

const styleAccordion: React.CSSProperties = {
...styles.accordion.default,
...(active ? styles.accordion.active : {}),
};

const stylePanel: React.CSSProperties = {
...styles.panel.default,
...(active ? styles.panel.active : {}),
};

return (
<div>
<button onClick={() => handleExpansionClick()}
style={styleAccordion}>
{props.title}
</button>
<div style={stylePanel}>
{props.content}
</div>
</div>
);
};

/**
* Render a series of expansion panels
*
* @param {object} props - React props
* @param {boolean?} props.alwaysOpen - If true, panels can not be toggled
* @param {object} props.panels - Array of individual panels
* @returns {React.ReactElement} - The panels
*/
const ExpansionPanels = (props: {
alwaysOpen?: boolean,
panels: {
title: string,
content: React.ReactElement,
defaultOpen?: boolean,
alwaysOpen: boolean,
}[]
}) => {
return (
<div className={'container-fluid'}
style={{margin: '0 auto', maxWidth: '900px'}}>
{ props.panels.map((panel, index) => (
<Panel
key={index}
title={panel.title}
content={panel.content}
alwaysOpen={panel.alwaysOpen}
defaultOpen={panel.defaultOpen || props.alwaysOpen || true}
/>
))}
</div>
);
};

export default ExpansionPanels;
90 changes: 90 additions & 0 deletions modules/dataquery/jsx/components/filterableselectgroup.tsx
Original file line number Diff line number Diff line change
@@ -0,0 +1,90 @@
import Select, {SingleValue} from 'react-select';

type SelectOption = {
label: string,
value: string,
module: string,
};

type SelectGroup = {
label: string,
options: SelectOption[],
};
/**
* Render a select with option groups that can be
* filtered
*
* @param {object} props - react props
* @param {function} props.onChange - Callback on selection change
* @param {string?} props.placeholder - An optional placeholder value when no elements are selected
* @param {object} props.groups - Groups to select the dropdown into
* @param {function} props.mapGroupName - A mapper from backend to frontend name for groups
* @returns {React.ReactElement} - The element
*/
function FilterableSelectGroup(props: {
onChange: (module: string, value: string) => void,
placeholder?: string,
groups: object,
mapGroupName?: (module: string) => string,
}) {
const groups: SelectGroup[] = [];
const placeholder = props.placeholder || 'Select a category';
for (const [module, subcategories]
of Object.entries(props.groups)) {
const options: SelectOption[] = [];
for (const [value, desc]
of Object.entries(subcategories) as unknown as [string, string]) {
options.push({
value: value,
label: desc,
module: module,
});
}

let label = module;
if (props.mapGroupName) {
label = props.mapGroupName(module);
}
groups.push({
label: label,
options: options,
});
}

/**
* Callback to call when the selection changes.
*
* @param {object} e - The click event callback
* @param {string} e.module - The module
* @param {string} e.value - The value
* @returns {void}
*/
const selected = (e: SingleValue<SelectGroup>) => {
// The callback should be (e: SelectOption) but typescript
// is convinced that it's a SingleValue<SelectGroup>.
// console.log(e) confirms that it has the same structure
// as SelectOption, so just convert it and explicitly
// cast it unsafely to make the compiler happy.
const val: SelectOption = e as unknown as SelectOption;
props.onChange(val.module, val.value);
};
return (
<div>
<Select options={groups} onChange={selected}
menuPortalTarget={document.body}
styles={{menuPortal:
/**
* Add a z-index to ensure the element stays visible
*
* @param {object} base - the base CSS
* @returns {object} - the new CSS with z-index added
*/
(base) => ({...base, zIndex: 9999})}
}
placeholder={placeholder}
/>
</div>
);
}

export default FilterableSelectGroup;
Loading
Loading