Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{Packaging} Remove Jinja2, MarkupSafe and pycparser from requirements.txt #29422

Merged
merged 3 commits into from
Jul 21, 2024

Conversation

bebound
Copy link
Contributor

@bebound bebound commented Jul 19, 2024

Jinja2 and MarkupSafe are not used in CLI.
pycparser is a dependency of cffi.
We don't need to include them in requirements.txt

Resolve #29421

Copy link

azure-client-tools-bot-prd bot commented Jul 19, 2024

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.11
️✔️3.9
️✔️ams
️✔️latest
️✔️3.11
️✔️3.9
️✔️apim
️✔️latest
️✔️3.11
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.11
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️aro
️✔️latest
️✔️3.11
️✔️3.9
️✔️backup
️✔️latest
️✔️3.11
️✔️3.9
️✔️batch
️✔️latest
️✔️3.11
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.11
️✔️3.9
️✔️billing
️✔️latest
️✔️3.11
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.11
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.11
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.11
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.11
️✔️3.9
️✔️config
️✔️latest
️✔️3.11
️✔️3.9
️✔️configure
️✔️latest
️✔️3.11
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.11
️✔️3.9
️✔️container
️✔️latest
️✔️3.11
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.11
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.11
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️dla
️✔️latest
️✔️3.11
️✔️3.9
️✔️dls
️✔️latest
️✔️3.11
️✔️3.9
️✔️dms
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.11
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.11
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.11
️✔️3.9
️✔️find
️✔️latest
️✔️3.11
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.11
️✔️3.9
️✔️identity
️✔️latest
️✔️3.11
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.11
️✔️3.9
️✔️lab
️✔️latest
️✔️3.11
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.11
️✔️3.9
️✔️maps
️✔️latest
️✔️3.11
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.11
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.11
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.11
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.11
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.11
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.11
️✔️3.9
️✔️profile
️✔️latest
️✔️3.11
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.11
️✔️3.9
️✔️redis
️✔️latest
️✔️3.11
️✔️3.9
️✔️relay
️✔️latest
️✔️3.11
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️role
️✔️latest
️✔️3.11
️✔️3.9
️✔️search
️✔️latest
️✔️3.11
️✔️3.9
️✔️security
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.11
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.11
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.11
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.11
️✔️3.9
️✔️sql
️✔️latest
️✔️3.11
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.11
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.11
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9
️✔️util
️✔️latest
️✔️3.11
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.11
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.11
️✔️3.9
️✔️latest
️✔️3.11
️✔️3.9

Copy link

azure-client-tools-bot-prd bot commented Jul 19, 2024

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Jul 19, 2024

Packaging

@jiasli
Copy link
Member

jiasli commented Jul 19, 2024

I searched the whole repo and neither MarkupSafe nor pycparser is directly used. pipdeptree shows the same:

> pipdeptree --reverse --package pycparser
pycparser==2.19
└── cffi==1.15.1 [requires: pycparser]
    ├── azure-datalake-store==0.0.49 [requires: cffi]
    │   └── azure-cli==2.62.0 [requires: azure-datalake-store~=0.0.49]
    ├── bcrypt==3.2.0 [requires: cffi>=1.1]
    │   └── paramiko==3.4.0 [requires: bcrypt>=3.2]
    │       ├── azure-cli-core==2.62.0 [requires: paramiko>=2.0.8,<4.0.0]
    │       │   └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
    │       ├── fabric==3.2.2 [requires: paramiko>=2.4]
    │       │   └── azure-cli==2.62.0 [requires: fabric~=3.2.2]
    │       ├── scp==0.13.2 [requires: paramiko]
    │       │   └── azure-cli==2.62.0 [requires: scp~=0.13.2]
    │       └── sshtunnel==0.1.5 [requires: paramiko>=1.15.2]
    │           └── azure-cli==2.62.0 [requires: sshtunnel~=0.1.4]
    ├── cryptography==42.0.5 [requires: cffi>=1.12]
    │   ├── adal==1.2.7 [requires: cryptography>=1.1.0]
    │   │   ├── azure-datalake-store==0.0.49 [requires: adal>=0.4.2]
    │   │   │   └── azure-cli==2.62.0 [requires: azure-datalake-store~=0.0.49]
    │   │   └── msrestazure==0.6.4 [requires: adal>=0.6.0,<2.0.0]
    │   │       ├── azure-batch==14.2.0 [requires: msrestazure>=0.4.32,<2.0.0]
    │   │       │   └── azure-cli==2.62.0 [requires: azure-batch~=14.2.0]
    │   │       ├── azure-cli-core==2.62.0 [requires: msrestazure~=0.6.4]
    │   │       │   └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
    │   │       ├── azure-graphrbac==0.60.0 [requires: msrestazure>=0.4.32,<2.0.0]
    │   │       │   └── azure-cli==2.62.0 [requires: azure-graphrbac~=0.60.0]
    │   │       ├── azure-mgmt-datalake-store==0.5.0 [requires: msrestazure>=0.4.27,<2.0.0]
    │   │       ├── azure-mgmt-devtestlabs==4.0.0 [requires: msrestazure>=0.4.32,<2.0.0]
    │   │       │   └── azure-cli==2.62.0 [requires: azure-mgmt-devtestlabs~=4.0]
    │   │       ├── azure-mgmt-kusto==0.3.0 [requires: msrestazure>=0.4.32,<2.0.0]
    │   │       │   └── azure-cli==2.62.0 [requires: azure-mgmt-kusto~=0.3.0]
    │   │       └── azure-mgmt-managedservices==1.0.0 [requires: msrestazure>=0.4.32,<2.0.0]
    │   │           └── azure-cli==2.62.0 [requires: azure-mgmt-managedservices~=1.0]
    │   ├── azure-cli-core==2.62.0 [requires: cryptography]
    │   │   └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
    │   ├── azure-keyvault-keys==4.9.0b3 [requires: cryptography>=2.1.4]
    │   │   └── azure-cli==2.62.0 [requires: azure-keyvault-keys==4.9.0b3]
    │   ├── azure-multiapi-storage==1.2.0 [requires: cryptography>=2.1.4]
    │   │   ├── azdev==0.1.67 [requires: azure-multiapi-storage]
    │   │   └── azure-cli==2.62.0 [requires: azure-multiapi-storage~=1.2.0]
    │   ├── azure-storage-common==1.4.2 [requires: cryptography]
    │   │   └── azure-cli==2.62.0 [requires: azure-storage-common~=1.4]
    │   ├── msal==1.29.0 [requires: cryptography>=2.5,<45]
    │   │   ├── azure-cli-core==2.62.0 [requires: msal==1.29.0]
    │   │   │   └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
    │   │   └── msal-extensions==1.2.0 [requires: msal>=1.29,<2]
    │   │       └── azure-cli-core==2.62.0 [requires: msal-extensions==1.2.0]
    │   │           └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
    │   ├── paramiko==3.4.0 [requires: cryptography>=3.3]
    │   │   ├── azure-cli-core==2.62.0 [requires: paramiko>=2.0.8,<4.0.0]
    │   │   │   └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
    │   │   ├── fabric==3.2.2 [requires: paramiko>=2.4]
    │   │   │   └── azure-cli==2.62.0 [requires: fabric~=3.2.2]
    │   │   ├── scp==0.13.2 [requires: paramiko]
    │   │   │   └── azure-cli==2.62.0 [requires: scp~=0.13.2]
    │   │   └── sshtunnel==0.1.5 [requires: paramiko>=1.15.2]
    │   │       └── azure-cli==2.62.0 [requires: sshtunnel~=0.1.4]
    │   └── pyOpenSSL==24.0.0 [requires: cryptography>=41.0.5,<43]
    │       └── azure-cli-core==2.62.0 [requires: pyOpenSSL>=17.1.0]
    │           └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
    └── PyNaCl==1.5.0 [requires: cffi>=1.4.1]
        ├── azure-cli==2.62.0 [requires: PyNaCl~=1.5.0]
        ├── paramiko==3.4.0 [requires: PyNaCl>=1.5]
        │   ├── azure-cli-core==2.62.0 [requires: paramiko>=2.0.8,<4.0.0]
        │   │   └── azure-cli==2.62.0 [requires: azure-cli-core==2.62.0]
        │   ├── fabric==3.2.2 [requires: paramiko>=2.4]
        │   │   └── azure-cli==2.62.0 [requires: fabric~=3.2.2]
        │   ├── scp==0.13.2 [requires: paramiko]
        │   │   └── azure-cli==2.62.0 [requires: scp~=0.13.2]
        │   └── sshtunnel==0.1.5 [requires: paramiko>=1.15.2]
        │       └── azure-cli==2.62.0 [requires: sshtunnel~=0.1.4]
        └── PyGithub==1.55 [requires: PyNaCl>=1.4.0]
            └── azure-cli==2.62.0 [requires: PyGithub~=1.38]

> pipdeptree --reverse --package MarkupSafe
MarkupSafe==2.0.1
└── Jinja2==3.1.4 [requires: MarkupSafe>=2.0]
    ├── azdev==0.1.67 [requires: Jinja2]
    └── Sphinx==1.6.7 [requires: Jinja2>=2.3]
        └── azdev==0.1.67 [requires: Sphinx==1.6.7]

Maybe we can remove them?

They were initially added by #9785. pycparser was not updated since then and MarkupSafe was updated by #22602 once.

@bebound
Copy link
Contributor Author

bebound commented Jul 19, 2024

Yes. In this PR I wrote that jinja2 and markupsafe can be removed and I totally forget it. #27717

Future work
Jinja2 and MarkupSafe were introduced in requirements.txt in #9785 and updated in #22602, but CLI does not use it.(Some scripts rely on them). We should remove it from requirements.txt to reduce package size.

@bebound bebound changed the title {Packaging} Bump MarkupSafe to 2.1.5, pycparser to 2.22 {Packaging} Remove Jinja2 MarkupSafe pycparser from requirements.txt Jul 19, 2024
@bebound bebound changed the title {Packaging} Remove Jinja2 MarkupSafe pycparser from requirements.txt {Packaging} Remove Jinja2, MarkupSafe and pycparser from requirements.txt Jul 19, 2024
@bebound bebound marked this pull request as ready for review July 19, 2024 06:34
@bebound bebound requested review from kairu-ms and jsntcy as code owners July 19, 2024 07:51
@@ -29,6 +29,7 @@
logger = logging.getLogger(__name__)


@unittest.skip('alias extension is not working: https://github.com/Azure/azure-cli/issues/29422')
Copy link
Contributor Author

@bebound bebound Jul 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After removing markupsafe, the alias extension does not work anymore. Previously, the CLI's markupsafe was loaded; now it loads the outdated alias's markupsafe package.

self._run_cmd("alias create -n foo123 -c bar123", expect_failure=True)
fails with

Unable to load extension 'alias: cannot import name 'soft_unicode' from 'markupsafe' (C:\Users\kk\.azure\cliextensions\alias\markupsafe\__init__.py)'. Use --debug for more information.

az feedback drop is tracked in #29426

@bebound bebound merged commit 584ce57 into Azure:dev Jul 21, 2024
100 checks passed
@bebound bebound deleted the bump branch July 21, 2024 11:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot Packaging
Projects
None yet
Development

Successfully merging this pull request may close these issues.

CI fails because of setuptools change
3 participants