Skip to content
This repository has been archived by the owner on Feb 20, 2023. It is now read-only.

Add Sqlancer Test Type and Installation Requirements #1500

Open
wants to merge 4 commits into
base: master
Choose a base branch
from

Conversation

dniu16
Copy link
Contributor

@dniu16 dniu16 commented Feb 26, 2021

This PR adds the sqlancer test type following the instruction here: https://github.com/cmu-db/noisepage/tree/master/script/testing. The test can be run using the command "PYTHONPATH=.. python3 -m script.testing.sqlancer" under the noisepage directory.

In addition, maven is installed in installation/package.sh since it is needed to build sqlancer.

@dniu16 dniu16 self-assigned this Feb 26, 2021
@dniu16 dniu16 added the feature Adds a requested feature label Feb 26, 2021
from ..util.constants import DIR_TMP

# git settings for sqlancer.
SQLANCER_GIT_URL = "https://github.com/dniu16/sqlancer.git"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@apavlo apavlo added the ready-for-ci Indicate that this build should be run through CI. label Mar 12, 2021
@noisepage-checks
Copy link

Major Decrease in Performance

STOP: this PR has a major negative performance impact

tps (%change) benchmark_type wal_device details
-2.05% tpcc RAM disk
Detailsmaster tps=8894.4, commit tps=8712.46, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=RAM disk, max_connection_threads=32
-1.06% tpcc None
Detailsmaster tps=9997.24, commit tps=9891.11, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=None, max_connection_threads=32
-2.83% tpcc HDD
Detailsmaster tps=491.73, commit tps=477.83, query_mode=extended, benchmark_type=tpcc, scale_factor=32.0000, terminals=32, client_time=60, weights={'Payment': 43, 'Delivery': 4, 'NewOrder': 45, 'StockLevel': 4, 'OrderStatus': 4}, wal_device=HDD, max_connection_threads=32
2.33% tatp RAM disk
Detailsmaster tps=3462.28, commit tps=3543.02, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=RAM disk, max_connection_threads=32
-3.25% tatp None
Detailsmaster tps=3799.59, commit tps=3676.15, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=None, max_connection_threads=32
-7.3% tatp HDD
Detailsmaster tps=351.98, commit tps=326.28, query_mode=extended, benchmark_type=tatp, scale_factor=1.0000, terminals=16, client_time=60, weights={'GetAccessData': 35, 'UpdateLocation': 14, 'GetNewDestination': 10, 'GetSubscriberData': 35, 'DeleteCallForwarding': 2, 'InsertCallForwarding': 2, 'UpdateSubscriberData': 2}, wal_device=HDD, max_connection_threads=32

@codecov
Copy link

codecov bot commented Mar 13, 2021

Codecov Report

Merging #1500 (7309ef8) into master (68c0069) will decrease coverage by 0.01%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #1500      +/-   ##
==========================================
- Coverage   81.79%   81.78%   -0.02%     
==========================================
  Files         706      706              
  Lines       50425    50425              
==========================================
- Hits        41246    41239       -7     
- Misses       9179     9186       +7     
Impacted Files Coverage Δ
src/storage/arrow_serializer.cpp 80.00% <0.00%> (-6.25%) ⬇️
src/include/metrics/metrics_store.h 61.66% <0.00%> (-1.67%) ⬇️
src/include/storage/index/bplustree.h 89.49% <0.00%> (-0.34%) ⬇️
src/storage/data_table.cpp 98.29% <0.00%> (+1.13%) ⬆️
src/include/storage/sql_table.h 100.00% <0.00%> (+2.32%) ⬆️
src/transaction/transaction_manager.cpp 95.23% <0.00%> (+2.72%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 68c0069...7309ef8. Read the comment docs.

@lmwnshn lmwnshn removed the ready-for-ci Indicate that this build should be run through CI. label Mar 14, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
feature Adds a requested feature
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants