Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Javascript sql 2021 #2388

Merged
merged 27 commits into from
Oct 25, 2021
Merged

Javascript sql 2021 #2388

Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
8ca1031
add requests, async, module queries
NishuGoel Aug 30, 2021
36b0b45
Added ajax request per page percentile based query.
pankajparkar Sep 10, 2021
7774a0a
Added most requested content-types.
pankajparkar Sep 10, 2021
de19ca1
Formatted query.
pankajparkar Sep 11, 2021
bc6b137
Added query for iframe per page.
pankajparkar Sep 12, 2021
895f031
Percentage usage of different ajax APIs.
pankajparkar Sep 12, 2021
bf828ca
Added h1 vs h2 vs h3 script.
pankajparkar Sep 12, 2021
c87eea5
Added web_components_sepc sql.
pankajparkar Sep 12, 2021
0444c0f
Merge branch 'javascript-sql-2021' of https://github.com/NishuGoel/al…
max-ostapenko Sep 15, 2021
e4c7b30
linting
max-ostapenko Sep 15, 2021
5017cb6
consistent syntax to count number of rows
max-ostapenko Sep 16, 2021
812e8dc
frameworks, sourcemaps queries
NishuGoel Sep 24, 2021
b95883b
1p vs 3p
NishuGoel Sep 24, 2021
7a70e3e
adds unminified, preload/prefetch queries
NishuGoel Sep 24, 2021
d8808e3
Merge remote-tracking branch 'origin/main' into javascript-sql-2021
NishuGoel Sep 29, 2021
396e90e
PR feedback accomodate & fixes
NishuGoel Oct 5, 2021
2534371
link fix
NishuGoel Oct 5, 2021
04a4011
Feedback II update
NishuGoel Oct 6, 2021
387303c
Fixed review comment related to web_component_specs.
pankajparkar Oct 12, 2021
4642675
update content_type and webcomponent query
NishuGoel Oct 18, 2021
fec59a1
Javascript sql 2021 (#2386)
NishuGoel Oct 18, 2021
a48a2f2
Merge branch 'HTTPArchive:javascript-sql-2021' into javascript-sql-2021
NishuGoel Oct 18, 2021
1ac79f1
upstream fetch
NishuGoel Oct 18, 2021
a42124b
lint fixes
NishuGoel Oct 18, 2021
6025b6c
lint fix -2
NishuGoel Oct 18, 2021
11028b8
add review updates
NishuGoel Oct 20, 2021
d036f20
add trailing newline
NishuGoel Oct 20, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 10 additions & 9 deletions sql/2021/javascript/most_requested_content_type.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
SELECT
rviscomi marked this conversation as resolved.
Show resolved Hide resolved
resp_content_type,
COUNT(0) AS count,
SUM(COUNT(0)) OVER (PARTITION BY client) AS total,
COUNT(0) / SUM(COUNT(0)) OVER (PARTITION BY client) AS pct
FROM
`httparchive.almanac.requests`
WHERE
date = '2021-07-01'
client,
(select count(*) from `httparchive.almanac.requests` WHERE date = '2021-07-01' and resp_content_type like '%image%') as images,
(select count(*) from `httparchive.almanac.requests` WHERE date = '2021-07-01' and resp_content_type like '%font%') as font,
(select count(*) from `httparchive.almanac.requests` WHERE date = '2021-07-01' and resp_content_type like '%css%') as css,
(select count(*) from `httparchive.almanac.requests` WHERE date = '2021-07-01' and resp_content_type like '%javascript%') as javascript,
(select count(*) from `httparchive.almanac.requests` WHERE date = '2021-07-01' and resp_content_type like '%json%') as json,
count(*) as total
FROM
`httparchive.almanac.requests`
GROUP BY
client
ORDER BY
client DESC
client
75 changes: 48 additions & 27 deletions sql/2021/javascript/web_components_specs.sql
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,51 @@
SELECT
rviscomi marked this conversation as resolved.
Show resolved Hide resolved
client,
percentile,
APPROX_QUANTILES(custom_elements, 1000)[
OFFSET(percentile * 10)] AS custom_elements,
APPROX_QUANTILES(shadow_roots, 1000)[
OFFSET(percentile * 10)] AS shadow_roots,
APPROX_QUANTILES(template, 1000)[
OFFSET(percentile * 10)] AS template
FROM (
SELECT
_TABLE_SUFFIX AS client,
CAST(JSON_EXTRACT(JSON_EXTRACT_SCALAR(payload,
'$._javascript'),
'$.web_component_specs.custom_elements.length') AS INT64) AS custom_elements,
CAST(JSON_EXTRACT(JSON_EXTRACT_SCALAR(payload,
'$._javascript'),
'$.web_component_specs.shadow_roots.length') AS INT64) AS shadow_roots,
CAST(JSON_EXTRACT(JSON_EXTRACT_SCALAR(payload,
'$._javascript'),
'$.web_component_specs.template.length') AS INT64) AS template
FROM
`httparchive.pages.2021_07_01_*` ),
UNNEST([10, 25, 50, 75, 90, 100]) AS percentile
client,
percentile,
APPROX_QUANTILES(custom_elements,
1000)[OFFSET(percentile * 10)] AS custom_elements,
APPROX_QUANTILES(shadow_roots,
1000)[OFFSET(percentile * 10)] AS shadow_roots,
APPROX_QUANTILES(template,
1000)[OFFSET(percentile * 10)] AS template
FROM
(SELECT
_table_suffix AS client,
(CASE
WHEN
ARRAY_LENGTH(
JSON_QUERY_ARRAY(JSON_EXTRACT(JSON_EXTRACT_SCALAR(payload,
'$._javascript'
),
'$.web_component_specs.custom_elements'
))) > 1 THEN 1
ELSE 0
END) AS custom_elements,
(CASE
WHEN
ARRAY_LENGTH(
JSON_QUERY_ARRAY(JSON_EXTRACT(JSON_EXTRACT_SCALAR(payload,
'$._javascript'
),
'$.web_component_specs.shadow_roots'
))) > 1 THEN 1
ELSE 0
END) AS shadow_roots,
(CASE
WHEN
ARRAY_LENGTH(
JSON_QUERY_ARRAY(JSON_EXTRACT(JSON_EXTRACT_SCALAR(payload,
'$._javascript'
),
'$.web_component_specs.template'
))) > 1 THEN 1
ELSE 0
END) AS template
FROM
`httparchive.pages.2021_09_01_*` ),
UNNEST([10, 25, 50, 75, 90, 100])
GROUP BY
percentile,
client
percentile,
client
ORDER BY
percentile,
client
percentile,
client