Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix UI Tests #1345

Merged
merged 16 commits into from
Jul 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/workflows/tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ jobs:
- name: Install CocoaPod dependencies
run: pod install
- name: Run Unit Tests
run: set -o pipefail && xcodebuild -workspace 'Braintree.xcworkspace' -sdk 'iphonesimulator' -configuration 'Debug' -scheme 'UnitTests' -destination 'name=iPhone 14,OS=17.2,platform=iOS Simulator' test | ./Pods/xcbeautify/xcbeautify
run: set -o pipefail && xcodebuild -workspace 'Braintree.xcworkspace' -sdk 'iphonesimulator' -configuration 'Debug' -scheme 'UnitTests' -destination 'name=iPhone 15,OS=17.2,platform=iOS Simulator' test | ./Pods/xcbeautify/xcbeautify
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pulled from the findings in PR #1328

ui_test_job:
name: UI
runs-on: macOS-14-xlarge
Expand All @@ -35,7 +35,7 @@ jobs:
- name: Install CocoaPod dependencies
run: pod install
- name: Run UI Tests
run: set -o pipefail && xcodebuild -workspace 'Braintree.xcworkspace' -sdk 'iphonesimulator' -configuration 'Release' -scheme 'UITests' -destination 'name=iPhone 14,OS=17.2,platform=iOS Simulator' test | ./Pods/xcbeautify/xcbeautify
run: set -o pipefail && xcodebuild -workspace 'Braintree.xcworkspace' -sdk 'iphonesimulator' -configuration 'Release' -scheme 'UITests' -destination 'name=iPhone 15,OS=17.2,platform=iOS Simulator' test | ./Pods/xcbeautify/xcbeautify
integration_test_job:
name: Integration
runs-on: macOS-14-xlarge
Expand All @@ -52,4 +52,4 @@ jobs:
- name: Install CocoaPod dependencies
run: pod install
- name: Run Integration Tests
run: set -o pipefail && xcodebuild -workspace 'Braintree.xcworkspace' -sdk 'iphonesimulator' -configuration 'Release' -scheme 'IntegrationTests' -destination 'name=iPhone 14,OS=17.2,platform=iOS Simulator' test | ./Pods/xcbeautify/xcbeautify
run: set -o pipefail && xcodebuild -workspace 'Braintree.xcworkspace' -sdk 'iphonesimulator' -configuration 'Release' -scheme 'IntegrationTests' -destination 'name=iPhone 15,OS=17.2,platform=iOS Simulator' test | ./Pods/xcbeautify/xcbeautify
2 changes: 2 additions & 0 deletions Demo/Demo.xcodeproj/project.pbxproj
Original file line number Diff line number Diff line change
Expand Up @@ -992,6 +992,7 @@
"$(inherited)",
"@executable_path/Frameworks",
"@loader_path/Frameworks",
"$(FRAMEWORK_SEARCH_PATHS)",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found a few threads in the GitHub actions issues that indicated that at times the CI runner cannot find the frameworks, so adding this to the demo app should increase the search path coverage in addition to the above paths.

);
PRODUCT_BUNDLE_IDENTIFIER = com.braintree.DemoUITests;
"PRODUCT_BUNDLE_IDENTIFIER[sdk=macosx*]" = "";
Expand All @@ -1015,6 +1016,7 @@
"$(inherited)",
"@executable_path/Frameworks",
"@loader_path/Frameworks",
"$(FRAMEWORK_SEARCH_PATHS)",
);
PRODUCT_BUNDLE_IDENTIFIER = com.braintree.DemoUITests;
"PRODUCT_BUNDLE_IDENTIFIER[sdk=macosx*]" = "";
Expand Down
5 changes: 5 additions & 0 deletions Demo/MockVenmo/SceneDelegate.swift
Original file line number Diff line number Diff line change
Expand Up @@ -7,5 +7,10 @@ class SceneDelegate: UIResponder, UIWindowSceneDelegate {
func scene(_ scene: UIScene, openURLContexts URLContexts: Set<UIOpenURLContext>) {
AppSwitcher.openVenmoURL = URLContexts.first?.url
}

func scene(_ scene: UIScene, willConnectTo session: UISceneSession, options connectionOptions: UIScene.ConnectionOptions) {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is pulled from the findings in PR #1290

let urlContexts = connectionOptions.urlContexts
AppSwitcher.openVenmoURL = urlContexts.first?.url
}
}

Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ class AmericanExpress_UITests: XCTestCase {
app.buttons["Valid card"].tap()
sleep(2)

XCTAssertTrue(app.buttons["45256433 Points, 316795.03 USD"].waitForExistence(timeout: 10))
XCTAssertTrue(app.buttons["45256433 Points, 316795.03 USD"].waitForExistence(timeout: 20))
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While there are several instances in this PR just bumping timeouts, several of these test were also hitting the same timeouts locally. If the test completes before time timeout is hit it'll move on to the next test so there is no real harm in bumping the timeouts for the recurring failing tests.

}

func testInsufficientPointsCard_receivesErrorMessage() {
Expand Down
4 changes: 2 additions & 2 deletions Demo/UI Tests/Helpers/BTUITest.swift
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
import XCTest

extension XCTestCase {
func waitForElementToAppear(_ element: XCUIElement, timeout: TimeInterval = 15) {
func waitForElementToAppear(_ element: XCUIElement, timeout: TimeInterval = 30) {
let existsPredicate = NSPredicate(format: "exists == true")

expectation(for: existsPredicate, evaluatedWith: element)

waitForExpectations(timeout: timeout)
}

func waitForElementToBeHittable(_ element: XCUIElement, timeout: TimeInterval = 15) {
func waitForElementToBeHittable(_ element: XCUIElement, timeout: TimeInterval = 30) {
let existsPredicate = NSPredicate(format: "exists == true && hittable == true")

expectation(for: existsPredicate, evaluatedWith: element)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ final class PayPalMessaging_Success_UITests: XCTestCase {
}

func testStart_withValidRequest_firesDelegates() {
XCTAssertTrue(app.buttons["DELEGATE: didAppear fired"].waitForExistence(timeout: 10))
XCTAssertTrue(app.buttons["DELEGATE: didAppear fired"].waitForExistence(timeout: 30))

let expectedButtonText = "PayPal - Pay monthly for purchases of $199-$10,000. Learn more"
waitForElementToBeHittable(app.buttons[expectedButtonText])
Expand All @@ -26,7 +26,7 @@ final class PayPalMessaging_Success_UITests: XCTestCase {
app.buttons["PayPal learn more modal close"].tap()
sleep(2)

XCTAssertTrue(app.buttons["DELEGATE: didSelect fired"].waitForExistence(timeout: 10))
XCTAssertTrue(app.buttons["DELEGATE: didSelect fired"].waitForExistence(timeout: 30))
}
}

Expand All @@ -46,6 +46,7 @@ final class PayPalMessaging_Failure_UITests: XCTestCase {

func testStart_withInvalidTokenizationKey_firesErrorDelegate() {
let expectedErrorText = "DELEGATE: onError fired with Could not find PayPal client ID in Braintree configuration."
XCTAssertTrue(app.buttons[expectedErrorText].waitForExistence(timeout: 10))

XCTAssertTrue(app.buttons[expectedErrorText].waitForExistence(timeout: 20))
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ class PayPal_Checkout_UITests: XCTestCase {

func testPayPal_checkout_cancelsSuccessfully_whenTappingCancelButtonOnPayPalSite() {
let webviewElementsQuery = app.webViews.element.otherElements
self.waitForElementToAppear(webviewElementsQuery.links["Cancel Sandbox Purchase"], timeout: 20)
self.waitForElementToAppear(webviewElementsQuery.links["Cancel Sandbox Purchase"])

webviewElementsQuery.links["Cancel Sandbox Purchase"].forceTapElement()

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -12,11 +12,10 @@ class ThreeDSecure_V2_UITests: XCTestCase {
app.launchArguments.append("-UITestHardcodedClientToken")
app.launchArguments.append("-Integration:ThreeDSecureViewController")
app.launch()

waitForElementToAppear(app.cardNumberTextField)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving this logic out of setup produced more stability in the tests both locally and in CI.

}

func testThreeDSecurePaymentFlowV2_frictionlessFlow_andTransacts() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001000", expirationDate: expirationDate)
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -25,6 +24,7 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_challengeFlow_andTransacts() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001091", expirationDate: expirationDate)
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -44,6 +44,7 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_noChallenge_andFails() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "5200000000001013", expirationDate: expirationDate)
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -52,6 +53,7 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_challengeFlow_andFails() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001109", expirationDate: expirationDate)
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -71,6 +73,7 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_acceptsPassword_failsToAuthenticateNonce_dueToCardinalError() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001125")
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -90,6 +93,7 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_returnsToApp_whenCancelTapped() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001091")
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -102,6 +106,7 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_bypassedAuthentication() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001083")
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -110,6 +115,7 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_lookupError() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001034")
app.tokenizeButton.tap()
sleep(2)
Expand All @@ -118,10 +124,11 @@ class ThreeDSecure_V2_UITests: XCTestCase {
}

func testThreeDSecurePaymentFlowV2_timeout() {
waitForElementToAppear(app.cardNumberTextField)
app.enterCardDetailsWith(cardNumber: "4000000000001075")
app.tokenizeButton.tap()
sleep(2)

waitForElementToAppear(app.liabilityCouldNotBeShiftedMessage, timeout:30)
waitForElementToAppear(app.liabilityCouldNotBeShiftedMessage, timeout:45)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test was failing because the Cardinal timeout for this card number is 30s, so I bumped it a bit beyond to ensure we always get a response

}
}
18 changes: 10 additions & 8 deletions Demo/UI Tests/Venmo UI Tests/Venmo_UITests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -17,53 +17,55 @@ class Venmo_UITests: XCTestCase {
demoApp.launchArguments.append("-ClientToken")
demoApp.launchArguments.append("-Integration:VenmoViewController")
demoApp.launch()

waitForElementToBeHittable(demoApp.buttons["Venmo"])
waitForElementToBeHittable(demoApp.buttons["Venmo (with ECD options)"])
}

func testTokenizeVenmo_whenSignInSuccessfulWithPaymentContext_returnsNonce() {
waitForElementToBeHittable(demoApp.buttons["Venmo"])
demoApp.buttons["Venmo"].tap()

waitForElementToBeHittable(mockVenmo.buttons["SUCCESS WITH PAYMENT CONTEXT"])
mockVenmo.buttons["SUCCESS WITH PAYMENT CONTEXT"].tap()

XCTAssertTrue(demoApp.buttons["Got a nonce. Tap to make a transaction."].waitForExistence(timeout: 15))
XCTAssertTrue(demoApp.buttons["Got a nonce. Tap to make a transaction."].waitForExistence(timeout: 30))
}

func testTokenizeVenmo_withECDOptions_whenSignInSuccessfulWithPaymentContext_returnsNonce() {
waitForElementToBeHittable(demoApp.buttons["Venmo (with ECD options)"])
demoApp.buttons["Venmo (with ECD options)"].tap()

waitForElementToBeHittable(mockVenmo.buttons["SUCCESS WITH PAYMENT CONTEXT"])
mockVenmo.buttons["SUCCESS WITH PAYMENT CONTEXT"].tap()

XCTAssertTrue(demoApp.buttons["Got a nonce. Tap to make a transaction."].waitForExistence(timeout: 15))
XCTAssertTrue(demoApp.buttons["Got a nonce. Tap to make a transaction."].waitForExistence(timeout: 30))
}

func testTokenizeVenmo_whenSignInSuccessfulWithoutPaymentContext_returnsNonce() {
waitForElementToBeHittable(demoApp.buttons["Venmo"])
demoApp.buttons["Venmo"].tap()

waitForElementToBeHittable(mockVenmo.buttons["SUCCESS WITHOUT PAYMENT CONTEXT"])
mockVenmo.buttons["SUCCESS WITHOUT PAYMENT CONTEXT"].tap()

XCTAssertTrue(demoApp.buttons["Got a nonce. Tap to make a transaction."].waitForExistence(timeout: 15))
XCTAssertTrue(demoApp.buttons["Got a nonce. Tap to make a transaction."].waitForExistence(timeout: 30))
}

func testTokenizeVenmo_whenErrorOccurs_returnsError() {
waitForElementToBeHittable(demoApp.buttons["Venmo"])
demoApp.buttons["Venmo"].tap()

waitForElementToBeHittable(mockVenmo.buttons["ERROR"])
mockVenmo.buttons["ERROR"].tap()

XCTAssertTrue(demoApp.buttons["An error occurred during the Venmo flow"].waitForExistence(timeout: 15))
XCTAssertTrue(demoApp.buttons["An error occurred during the Venmo flow"].waitForExistence(timeout: 30))
}

func testTokenizeVenmo_whenUserCancels_returnsCancel() {
waitForElementToBeHittable(demoApp.buttons["Venmo"])
demoApp.buttons["Venmo"].tap()

waitForElementToBeHittable(mockVenmo.buttons["Cancel"])
mockVenmo.buttons["Cancel"].tap()

XCTAssertTrue(demoApp.buttons["Canceled 🔰"].waitForExistence(timeout: 15))
XCTAssertTrue(demoApp.buttons["Canceled 🔰"].waitForExistence(timeout: 30))
}
}
Loading