Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add authentication via Entra with a certificate #3064

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

artem-shelkovnikov
Copy link
Member

@artem-shelkovnikov artem-shelkovnikov commented Dec 30, 2024

Closes #3023

See: https://learn.microsoft.com/en-us/sharepoint/dev/sp-add-ins/retirement-announcement-for-azure-acs

Azure is retiring old ACS and our connector really needs it right now since we use REST API for Sharepoint. We can migrate some code to use Graph - for example, Graph API introduced new APIs to fetch page content: https://devblogs.microsoft.com/microsoft365dev/microsoft-graph-api-for-sharepoint-pages-is-now-generally-available/

Some other stuff, like List Item Attachments, might not be available. Migration to new API will require a bit more time though.

To buy some time this PR introduces certificate authentication to Sharepoint Online connector. It's possible to either authenticate via a secret (client_secret) or via a certificate (you'll need a certificate + private key).

How to create a certificate and a private key:

openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout azure_app.key -out azure_app.crt

azure_app.crt is a certificate, azure_app.key is a private key. Then you'll need to upload the certificate to the Azure App that's set up for your Sharepoint Online instance.

We have checked, and this works with new tenants - secret authentication is already not usable for new tenants since November 2024 will be fully retired as of April 2nd, 2026.

Checklists

Pre-Review Checklist

  • this PR does NOT contain credentials of any kind, such as API keys or username/passwords (double check config.yml.example)
  • this PR has a meaningful title
  • this PR links to all relevant github issues that it fixes or partially addresses
  • if there is no GH issue, please create it. Each PR should have a link to an issue
  • this PR has a thorough description
  • Covered the changes with automated tests
  • Tested the changes locally
  • Added a label for each target release version (example: v7.13.2, v7.14.0, v8.0.0)
  • Considered corresponding documentation changes
  • Contributed any configuration settings changes to the configuration reference
  • if you added or changed Rich Configurable Fields for a Native Connector, you made a corresponding PR in Kibana

Release Note

Introduced certificate authentication for Sharepoint Online connector to allow connector to work against tenants created after 1st of November 2025.

@artem-shelkovnikov
Copy link
Member Author

@leemthompo can I ask for help with documentation for this change? :)

Copy link
Member

@seanstory seanstory left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

few comments, but largely LGTM. I'm surprised it was that straightforward.

{"label": "Client Secret", "value": "secret"},
{"label": "Certificate", "value": "certificate"},
],
"value": "certificate",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will be a breaking change, as on upgrade, we'll add this new config to existing connectors, right? But existing connectors will be using secret? I think we should default to secret, but tooltip that certificate is preferred?

"label": "Content of certificate file",
"display": "textarea",
"sensitive": True,
"order": 5,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 5,
"order": 6,

"sensitive": True,
"order": 6,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 6,
"order": 7,

},
"site_collections": {
"display": "textarea",
"label": "Comma-separated list of sites",
"tooltip": "A comma-separated list of sites to ingest data from. If enumerating all sites, use * to include all available sites, or specify a list of site names. Otherwise, specify a list of site paths.",
"order": 5,
"order": 7,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 7,
"order": 8,

"type": "list",
"value": "*",
},
"enumerate_all_sites": {
"display": "toggle",
"label": "Enumerate all sites?",
"tooltip": "If enabled, sites will be fetched in bulk, then filtered down to the configured list of sites. This is efficient when syncing many sites. If disabled, each configured site will be fetched with an individual request. This is efficient when syncing fewer sites.",
"order": 6,
"order": 8,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 8,
"order": 9,

@@ -1235,7 +1356,7 @@ def get_default_configuration(cls):
"depends_on": [{"field": "use_document_level_security", "value": True}],
"display": "toggle",
"label": "Fetch drive item permissions",
"order": 10,
"order": 12,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 12,
"order": 13,

@@ -1244,7 +1365,7 @@ def get_default_configuration(cls):
"depends_on": [{"field": "use_document_level_security", "value": True}],
"display": "toggle",
"label": "Fetch unique page permissions",
"order": 11,
"order": 13,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 13,
"order": 14,

@@ -1253,7 +1374,7 @@ def get_default_configuration(cls):
"depends_on": [{"field": "use_document_level_security", "value": True}],
"display": "toggle",
"label": "Fetch unique list permissions",
"order": 12,
"order": 14,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 14,
"order": 15,

@@ -1262,7 +1383,7 @@ def get_default_configuration(cls):
"depends_on": [{"field": "use_document_level_security", "value": True}],
"display": "toggle",
"label": "Fetch unique list item permissions",
"order": 13,
"order": 15,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"order": 15,
"order": 16,

client_id,
certificate,
private_key,
f"https://{self._tenant_name}.sharepoint.com/.default",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this going to break non-path-based site collections?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Sharepoint Online: sync not working on new tenant due to Azure ACS retirement
2 participants