Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Author: none **Files changed (172):** README.md (+6 -4 ~10) base/base.mk (+4 -4 ~8) base/makescripts/README.md (+0 -0 ~0) base/makescripts/did_tests_pass.sh (+0 -0 ~0) base/makescripts/send_deploy_email.py (+0 -0 ~0) base/ops/config/modules/modules/boto_kinesis.py (+101 -153 ~254) base/ops/config/modules/modules/virtualenv_mod.py (+494 -0 ~494) base/ops/config/modules/states/boto_kinesis.py (+79 -44 ~123) base/ops/config/modules/states/virtualenv_mod.py (+322 -0 ~322) base/ops/config/pillar/github_teams.sls (+4 -0 ~4) base/ops/config/pillar/groups.sls (+10 -11 ~21) base/ops/config/pillar/lyft_users.sls (+196 -24 ~220) base/ops/config/pillar/roles.sls (+22 -23 ~45) base/ops/config/pillar/services.sls (+42 -8 ~50) base/ops/config/states/base/manage_install.sls (+1 -5 ~6) base/ops/config/states/base/pam/access.conf (+0 -2 ~2) base/ops/config/states/base/pip/init.sls (+1 -13 ~14) base/ops/config/states/base/pip/pip.conf (+5 -3 ~8) base/ops/config/states/base/pip/pip_no_gemfury.conf (+0 -3 ~3) base/ops/config/states/base/pulldeploy/deploy.py (+24 -21 ~45) base/ops/config/states/base/pulldeploy/release.py (+10 -10 ~20) base/ops/config/states/base/statsd/init.sls (+10 -0 ~10) base/ops/config/states/base/statsd/piptools_requirements.txt (+25 -0 ~25) base/ops/config/states/base/statsd/requirements.in (+2 -0 ~2) base/ops/config/states/base/statsd/requirements.txt (+26 -1 ~27) base/ops/config/states/base/statsd/statsrelay-base.json (+2 -2 ~4) base/ops/config/states/base/users/humans.sls (+0 -1 ~1) base/ops/config/states/envoy2/init.sls (+3 -3 ~6) base/ops/config/states/lmdb/init.sls (+10 -0 ~10) base/ops/config/states/lmdb/piptools_requirements.txt (+25 -0 ~25) base/ops/config/states/lmdb/requirements.in (+3 -0 ~3) base/ops/config/states/lmdb/requirements.txt (+35 -2 ~37) base/ops/config/states/lyft-go/restart.sh (+2 -2 ~4) base/ops/config/states/lyft-kcl/init.sls (+10 -0 ~10) base/ops/config/states/lyft-kcl/piptools_requirements.txt (+25 -0 ~25) base/ops/config/states/lyft-kcl/requirements.in (+1 -0 ~1) base/ops/config/states/lyft-kcl/requirements.txt (+30 -0 ~30) base/ops/config/states/lyft-python/bandit/init.sls (+10 -0 ~10) base/ops/config/states/lyft-python/bandit/piptools_requirements.txt (+25 -0 ~25) base/ops/config/states/lyft-python/bandit/requirements.in (+2 -0 ~2) base/ops/config/states/lyft-python/bandit/requirements.txt (+33 -1 ~34) base/ops/config/states/lyft-python/init.sls (+28 -8 ~36) base/tests/test_pillars.py (+26 -0 ~26) base/tests/test_pulldeploy_release.py (+70 -0 ~70) base/tests/test_users.py (+1 -1 ~2) bootartifact/Dockerfile (+1 -1 ~2) buildnotify/Dockerfile (+2 -2 ~4) buildnotify/Makefile (+2 -2 ~4) buildnotify/buildnotify/deploy_message_processor.py (+95 -89 ~184) buildnotify/buildnotify/integration/test_validate_deploy_bot.py (+50 -0 ~50) buildnotify/buildnotify/integration/test_validate_staging.py (+31 -1 ~32) buildnotify/buildnotify/jenkins.py (+1 -1 ~2) buildnotify/buildnotify/routes.py (+10 -24 ~34) buildnotify/buildnotify/settings.py (+3 -2 ~5) buildnotify/buildnotify/submit_queue.py (+45 -57 ~102) buildnotify/buildnotify/tests/test_deploy_bot.py (+2 -47 ~49) buildnotify/buildnotify/tests/test_deploy_message_processor.py (+51 -0 ~51) buildnotify/buildnotify/tests/test_routes.py (+32 -25 ~57) buildnotify/buildnotify/tests/test_submit_queue.py (+14 -0 ~14) buildnotify/buildnotify/tests/test_v3_callbacks.py (+27 -4 ~31) buildnotify/buildnotify/tests/test_v3_jenkins_jobs.py (+20 -9 ~29) buildnotify/buildnotify/tests/test_v3_models.py (+2 -2 ~4) buildnotify/buildnotify/tests/test_v3_services.py (+17 -1 ~18) buildnotify/buildnotify/v3_callbacks.py (+26 -19 ~45) buildnotify/buildnotify/v3_jenkins_jobs.py (+11 -6 ~17) buildnotify/buildnotify/v3_models.py (+8 -5 ~13) buildnotify/buildnotify/v3_services.py (+11 -16 ~27) buildnotify/buildnotify/v3deploy_message_processor.py (+56 -2 ~58) buildnotify/manifest.yaml (+1 -0 ~1) buildnotify/ops/orca/buildnotify.sls (+9 -0 ~9) buildnotify/piptools_requirements.in (+5 -0 ~5) buildnotify/piptools_requirements.txt (+25 -0 ~25) buildnotify/requirements.in (+68 -0 ~68) buildnotify/requirements.txt (+62 -37 ~99) buildslave/piptools_requirements.in (+5 -0 ~5) buildslave/piptools_requirements.txt (+25 -0 ~25) buildslave/requirements.in (+50 -0 ~50) buildslave/requirements.txt (+47 -17 ~64) docs/graphite/README.md (+0 -73 ~73) docs/kinesis/runbook.md (+57 -9 ~66) docs/runbook.md (+0 -1 ~1) gateway/Dockerfile (+1 -1 ~2) grafana/Dockerfile (+1 -1 ~2) grafana/Makefile (+9 -1 ~10) grafana/manifest.yaml (+2 -0 ~2) grafana/ops/config/states/grafana/build.sls (+1 -2 ~3) grafana/ops/config/states/grafana/grafana.ini (+12 -0 ~12) graphite/.gitignore (+0 -1 ~1) graphite/Makefile (+0 -50 ~50) graphite/OWNERS (+0 -8 ~8) graphite/README.md (+0 -26 ~26) graphite/carbonate-sync-all.sh (+0 -40 ~40) graphite/carbonate-sync.sh (+0 -37 ~37) graphite/docs/runbook.md (+0 -184 ~184) graphite/onetime_scripts/dedicated_lc (+0 -15 ~15) graphite/onetime_scripts/grafana_cleanup.py (+0 -31 ~31) graphite/onetime_scripts/migrate_storage_schema.sh (+0 -14 ~14) graphite/onetime_scripts/netconsole_setup.sh (+0 -28 ~28) graphite/onetime_scripts/restore_from_duplicity.sh (+0 -76 ~76) graphite/onetime_scripts/upgrade_ixgbevf_driver.sh (+0 -30 ~30) graphite/ops/config/pillar/graphite.sls (+0 -2 ~2) graphite/ops/config/pillar/top.sls (+0 -7 ~7) graphite/ops/config/states/build.sls (+0 -89 ~89) graphite/ops/config/states/carbonzipper.json (+0 -6 ~6) graphite/ops/config/states/collectd/50-collectd-graphite-importer-checks.conf (+0 -3 ~3) graphite/ops/config/states/collectd/collectd-graphite-importer-checks.py (+0 -49 ~49) graphite/ops/config/states/conf/aggregation-rules.conf (+0 -60 ~60) graphite/ops/config/states/conf/apache2.htpasswd (+0 -12 ~12) graphite/ops/config/states/conf/blacklist-cache.conf (+0 -45 ~45) graphite/ops/config/states/conf/blacklist.conf (+0 -33 ~33) graphite/ops/config/states/conf/carbon.conf (+0 -262 ~262) graphite/ops/config/states/conf/carbonate.conf (+0 -35 ~35) graphite/ops/config/states/conf/carboncrelay-relay.conf (+0 -17 ~17) graphite/ops/config/states/conf/carboncrelay.conf (+0 -8 ~8) graphite/ops/config/states/conf/graphite-web-vhost.conf (+0 -111 ~111) graphite/ops/config/states/conf/local_settings.py (+0 -72 ~72) graphite/ops/config/states/conf/ports.conf (+0 -15 ~15) graphite/ops/config/states/conf/rewrite-rules.conf (+0 -93 ~93) graphite/ops/config/states/conf/storage-aggregation.conf (+0 -138 ~138) graphite/ops/config/states/conf/storage-schemas.conf (+0 -96 ~96) graphite/ops/config/states/conf/whitelist.conf (+0 -8 ~8) graphite/ops/config/states/development.sls (+0 -64 ~64) graphite/ops/config/states/graphite.sls (+0 -681 ~681) graphite/ops/config/states/memcached.conf (+0 -47 ~47) graphite/ops/config/states/top.sls (+0 -9 ~9) graphite/ops/orca/development.sls (+0 -7 ~7) graphite/ops/orca/graphite.sls (+0 -273 ~273) graphite/requirements.txt (+0 -4 ~4) graphite/tests/unit/blacklist.py (+0 -62 ~62) graphite/whisper-info.py (+0 -67 ~67) hacktools/README.md (+5 -8 ~13) hacktools/asg_resizer.py (+0 -54 ~54) hacktools/bssh (+0 -56 ~56) hacktools/fabfile.py (+0 -506 ~506) hacktools/fabutils.py (+0 -297 ~297) hacktools/fssh (+0 -56 ~56) hacktools/hack-telemetry-alarm.sh (+0 -9 ~9) hacktools/kcl.py (+0 -124 ~124) hacktools/kinesis_resize.py (+0 -227 ~227) hacktools/kinesis_settings_cleanup.py (+0 -96 ~96) hacktools/loadtest/README.md (+0 -59 ~59) hacktools/loadtest/loadtest.py (+0 -92 ~92) hacktools/loadtest/mongo_vpc_test.lua (+0 -31 ~31) hacktools/loadtest/randomseed.lua (+0 -13 ~13) hacktools/loadtest/regions_service_test.lua (+0 -19 ~19) hacktools/loadtest/regions_service_urls.txt (+0 -1000 ~1000) hacktools/loadtest/requirements.txt (+0 -4 ~4) hacktools/loadtest/urls.txt (+0 -1 ~1) hacktools/lock_asg_min_instances.py (+0 -128 ~128) hacktools/memoize.py (+0 -33 ~33) hacktools/put_iam_for_kinesis (+0 -84 ~84) hacktools/scan_jenkins_jobs.py (+0 -88 ~88) hacktools/sync-to-onebox-v3.sh (+0 -242 ~242) hacktools/unison-to-onebox-v3.sh (+0 -139 ~139) hacktools/update_library_pr.py (+0 -216 ~216) jenkins/ops/config/states/jenkins/templates/generate_service_update_branch.xml (+4 -0 ~4) jenkins/scripts/sha_check.py (+0 -0 ~0) makescripts (+1 -0 ~1) opstools/Dockerfile (+1 -1 ~2) opstools/manifest.yaml (+1 -1 ~2) orca/Dockerfile (+1 -1 ~2) orca/test/boto_kinesis_present.sls (+1 -0 ~1) packer/Dockerfile (+1 -1 ~2) statsd/Dockerfile (+1 -1 ~2) statsd/Makefile (+1 -1 ~2) statsd/manifest.yaml (+1 -1 ~2) statsd/ops/orca/statsd.sls (+30 -15 ~45) vpcproxy/Dockerfile (+1 -1 ~2) vpn/Dockerfile (+1 -1 ~2) vpn/docs/runbook.md (+31 -0 ~31) vpn/ops/config/states/vpn/pdns/recursor.lua (+6 -1 ~7) vpn/ops/config/states/vpn/sanewall.conf (+3 -1 ~4) **102 commits:** [Replace 'end_text' with 'master_status_warning' / don't render when nothing to deploy. (#10929)](lyft/ops_DEPRECATED@ca6a6bf) [graphite is deprecated (#10931)](lyft/ops_DEPRECATED@e3865a1) [Remove graphite runbooks (#10932)](lyft/ops_DEPRECATED@b6d5698) [A bit more deploy_message_processor cleanup (#10933)](lyft/ops_DEPRECATED@fe9c9df) [deploy_bot: Factor out the logic to generate the commits text and attachment. (#10934)](lyft/ops_DEPRECATED@7915679) [weekly opsbase update (#10939)](lyft/ops_DEPRECATED@53fa793) [weekly opsbase update (#10941)](lyft/ops_DEPRECATED@18979b1) [weekly opsbase update (#10942)](lyft/ops_DEPRECATED@05e5bfe) [weekly opsbase update (#10943)](lyft/ops_DEPRECATED@4d2b664) [Show list of commits for v3deploy. (#10938)](lyft/ops_DEPRECATED@4c02cd8) [weekly opsbase update (#10945)](lyft/ops_DEPRECATED@344edec) [weekly opsbase update (#10946)](lyft/ops_DEPRECATED@86c41d9) [weekly opsbase update (#10947)](lyft/ops_DEPRECATED@49f9b85) [Add cwoo as Analyst (#10930)](lyft/ops_DEPRECATED@c5cffbe) [grafana: add canary step (#10935)](lyft/ops_DEPRECATED@a4177f4) [Buildnotify edit comment to rebase (#10937)](lyft/ops_DEPRECATED@70d4495) [envoy canary 19b2593c0d34c94038b1bee8dfe1cc00c0584ef9 (#10953)](lyft/ops_DEPRECATED@8b92403) [On every salt run, clear the cache if it's greater than 200MB (#10949)](lyft/ops_DEPRECATED@3aad040) [Switch to wavefront's plugin for wavefront (#10900)](lyft/ops_DEPRECATED@4791f8a) [[SECHELP-492] Productionize access to AWS Batch for the ETA initiative. (#10951)](lyft/ops_DEPRECATED@782da6d) [[SEC-1395] Update VPN runbook with healthcheck information. (#10950)](lyft/ops_DEPRECATED@5e5ee10) [Enable output pass through for validate staging (#10956)](lyft/ops_DEPRECATED@0fcb95b) [[DS-925] give core-datastores team members ssh_devops (#10954)](lyft/ops_DEPRECATED@bade759) [Move logic for each service from routes to submit queue (#10936)](lyft/ops_DEPRECATED@46833fc) [[SEC-1381] Add all known GitHub bot team membership to github_teams pillar. (#10913)](lyft/ops_DEPRECATED@32b7135) [[SEC-1356] added dns split for onebox (#10857)](lyft/ops_DEPRECATED@81f1e4f) [weekly opsbase update (#10940)](lyft/ops_DEPRECATED@7fef03b) [weekly opsbase update (#10944)](lyft/ops_DEPRECATED@55f9c8d) [weekly opsbase update (#10948)](lyft/ops_DEPRECATED@56808fa) [Add data platform admin ui service (#10959)](lyft/ops_DEPRECATED@82611d6) [Add a unit test in base to ensure IAM pillars are YAML-parsable. (#10952)](lyft/ops_DEPRECATED@8158857) [add Spark port as accepted from vpn -> vpc (#10958)](lyft/ops_DEPRECATED@ac5e888) [Add deploy instruction to wait for base train (#10962)](lyft/ops_DEPRECATED@2b43c94) [have boto_kinesis use update_shard_count, and fail better (#10879)](lyft/ops_DEPRECATED@0a2c42a) [Update requirements.in the same way we update requirements.txt for etl/etlingest -> analyticsclient. (#10964)](lyft/ops_DEPRECATED@64008cc) [Revert "Only pypi.lyft.net for development and staging instances (#10 (#10972)](lyft/ops_DEPRECATED@fe022dc) [Buildnotify remove routes.get_services (#10976)](lyft/ops_DEPRECATED@d86eca3) [envoy prod 19b2593c0d34c94038b1bee8dfe1cc00c0584ef9 (#10977)](lyft/ops_DEPRECATED@eeb2573) [Revert "[SECHELP-492] Productionize access to AWS Batch for the ETA initiative. (#10951)" (#10978)](lyft/ops_DEPRECATED@073c720) [Revert "[DS-925] give core-datastores team members ssh_devops (#10954)" (#10979)](lyft/ops_DEPRECATED@70e4f6b) [Revert "Revert "[SECHELP-492] Productionize access to AWS Batch for the ETA initiative. (#10951)"" (#10980)](lyft/ops_DEPRECATED@a29f385) [Revert "Revert "[DS-925] give core-datastores team members ssh_devops (#10954)"" (#10981)](lyft/ops_DEPRECATED@4e7b010) [Buildnotify list_modified_services uses v3 services (#10971)](lyft/ops_DEPRECATED@02b02b9) [Fix orca (#10982)](lyft/ops_DEPRECATED@6852598) [Revert lyftpypi extra index (#10983)](lyft/ops_DEPRECATED@dcbeb1f) [[SECHELP-494] Add Jeiren to runtime (#10968)](lyft/ops_DEPRECATED@98ec7d4) [[SECHELP-491] Add azodhiates to i-s readonly (#10984)](lyft/ops_DEPRECATED@2c036b0) [Add infraevents to services.sls (#10973)](lyft/ops_DEPRECATED@2441238) [Revert "[SEC-1381] Add all known GitHub bot team membership to github (#10985)](lyft/ops_DEPRECATED@1abbf31) [Raise the coverage threshold for buildnotify from 67% to 74%. (#10821)](lyft/ops_DEPRECATED@a8871f8) [Buildnotify list_modified_services just uses v3_services (#10987)](lyft/ops_DEPRECATED@9338ea7) [Merges in Wavefront's upgrades (#10988)](lyft/ops_DEPRECATED@6a53ec6) [buildnotify: piptoolify (#10899)](lyft/ops_DEPRECATED@14d326d) [PROV-1278: Use service repo name when deploying (#10975)](lyft/ops_DEPRECATED@d3664f1) [Update @dafzal's slack name (#10955)](lyft/ops_DEPRECATED@c98ad35) [Jail statsd unit (#10907)](lyft/ops_DEPRECATED@837fa17) [Jail opstools unit (#10909)](lyft/ops_DEPRECATED@aa64507) [Add jnino to envoy-docs github team. (#10986)](lyft/ops_DEPRECATED@a808935) [Update buildnotify push logic for mobile repos (#10993)](lyft/ops_DEPRECATED@125e06e) [Remove humans from the root group, remove root from ssh access (#10992)](lyft/ops_DEPRECATED@466f7c8) [update escalation policy (#10994)](lyft/ops_DEPRECATED@437e6ad) [Test DeployMessageProcessor. (#10998)](lyft/ops_DEPRECATED@d827f44) [Buildnotify updating comment to rebase (#10996)](lyft/ops_DEPRECATED@fc37e47) [rainbow saddle update for buildnotify (#11000)](lyft/ops_DEPRECATED@bed0350) [Back out some wavefront plugin changes (#10999)](lyft/ops_DEPRECATED@dce9c2e) [add stats to services pillar (#10991)](lyft/ops_DEPRECATED@902712a) [Removing mobile image builds concurrent with V3 jenkins-mobile (#10995)](lyft/ops_DEPRECATED@3b0309b) [statsrelay: deprecate envoy-sfo metrics backhaul to iad (#10919)](lyft/ops_DEPRECATED@05ad7c6) [Set execute permissions on sha_check.py script (#11003)](lyft/ops_DEPRECATED@d492196) [update timecard escalation policy (#11004)](lyft/ops_DEPRECATED@9c59222) [Fix deploy.py (#11006)](lyft/ops_DEPRECATED@f3e3f2c) [remove duplicate hacktools (#11002)](lyft/ops_DEPRECATED@9cb87dc) [buildnotify: fix piptools reqs for py3 (#11007)](lyft/ops_DEPRECATED@c9dfc54) [piptoolify (#11009)](lyft/ops_DEPRECATED@8c572ea) [[SECHELP-496] Give Vishruta basic ssh access (#10960)](lyft/ops_DEPRECATED@e007337) [[SECHELP-498] Add lgreenwood to client-assets repo (#10961)](lyft/ops_DEPRECATED@c9dbf12) [Buildnotify check merge-base on all TEST_THIS_PLEASE comments (#11001)](lyft/ops_DEPRECATED@76206fd) [Envoy 919912712c36edc7d12011d22ba0f8a210d49d59 (#11008)](lyft/ops_DEPRECATED@14aa7a0) [Wavefront Bump (#11011)](lyft/ops_DEPRECATED@cc214d4) [Vendor virtualenv state (#11010)](lyft/ops_DEPRECATED@88e5713) [[SECHELP-457] Enable SSH tunnelling for looker user to hive/presto. (#10808)](lyft/ops_DEPRECATED@4e39b00) [grafana: disable alert execution (#11016)](lyft/ops_DEPRECATED@be19b79) [[buildnotify] Add logging to the validate staging step (#11015)](lyft/ops_DEPRECATED@c71edd2) [Allow jnode to assume buildnotify role (#11019)](lyft/ops_DEPRECATED@b9691d2) [Add marketdata service to the list (#11020)](lyft/ops_DEPRECATED@ee21e64) [[SECHELP-502] Add kgruye to new android-questions ro access group (#11013)](lyft/ops_DEPRECATED@c9043d7) [remove unused analytics events on deploys (#11018)](lyft/ops_DEPRECATED@866e978) [Add deploy bot validation (#11017)](lyft/ops_DEPRECATED@c58ce29) [Buildnotify uses v3_services to find services (#11021)](lyft/ops_DEPRECATED@560f97f) [Offboard :( (#11025)](lyft/ops_DEPRECATED@31581d6) [Install packages from multiple extra index urls (#11012)](lyft/ops_DEPRECATED@050c454) [Dedupe code to update owners check status. (#5091)](lyft/ops_DEPRECATED@f3c720a) [Buildnotify do not log error if job cannot continue (#11029)](lyft/ops_DEPRECATED@6a6d706) [grafana-plugin version update (#11027)](lyft/ops_DEPRECATED@ca53646) [stats piptools (#10974)](lyft/ops_DEPRECATED@c9c1819) [piptools kcl (#10967)](lyft/ops_DEPRECATED@2502f55) [bandit on piptools (#10966)](lyft/ops_DEPRECATED@79b7746) [piptools lmdb (#10965)](lyft/ops_DEPRECATED@ea7c728) [n00bz 2017-03-27 (#11033)](lyft/ops_DEPRECATED@c3fbb54) [Add silent option to curl command for Go services (#11035)](lyft/ops_DEPRECATED@bc7b501) [Grant pedwards access to readonlydb (#11038)](lyft/ops_DEPRECATED@253cdf0) [Move makescripts into base (#11041)](lyft/ops_DEPRECATED@fa2d0e1) NOT PASSING TESTS
- Loading branch information
1 parent
88d2bc5
commit 4e7aae8