Record Safari Technology Preview version

[foolip]

The raw report for the latest run has "browser_version": "12.1". This is also what shows up on wpt.fyi.

In the UI a much more detailed version string ("Release 65 (Safari 12.1, WebKit 13607.1.5.2)") is available:

Knowing the release number (65) and when it changes is important for comparing results over time.

Unfortunately, /usr/libexec/PlistBuddy -c Print /Applications/Safari\ Technology\ Preview.app/Contents/Info.plist doesn't include the "65", but it does include "13607.1.5.2" as CFBundleVersion.

Dict {
    CFBundleURLTypes = Array {
        Dict {
            CFBundleURLName = Web site URL
            CFBundleURLSchemes = Array {
                http
                https
            }
            LSIsAppleDefaultForScheme = false
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleURLName = Local file URL
            CFBundleURLSchemes = Array {
                file
            }
            LSHandlerRank = Alternate
        }
    }
    NSAppleScriptEnabled = Yes
    CFBundleInfoDictionaryVersion = 6.0
    DTPlatformVersion = GM
    CFBundleIconFile = technology-preview-compass
    CFBundleName = Safari Technology Preview
    DTSDKName = macosx10.13internal
    NSUserActivityTypes = Array {
        NSUserActivityTypeBrowsingWeb
    }
    NSServices = Array {
        Dict {
            NSMenuItem = Dict {
                default = Search With %WebSearchProvider@
            }
            NSSendTypes = Array {
                public.utf8-plain-text
            }
            NSPortName = Safari
            NSMessage = searchWithWebSearchProvider
            NSKeyEquivalent = Dict {
                default = L
            }
        }
        Dict {
            NSMenuItem = Dict {
                default = Add to Reading List
            }
            NSRequiredContext = Array {
                Dict {
                    NSTextContent = URL
                }
                Dict {
                    NSLinkSchemes = Array {
                        http
                        https
                    }
                }
            }
            NSSendTypes = Array {
                public.rtf
                public.utf8-plain-text
            }
            NSPortName = Safari
            NSMessage = addToReadingList
        }
    }
    NSPrincipalClass = BrowserApplication
    LSApplicationCategoryType = public.app-category.productivity
    CFBundleDocumentTypes = Array {
        Dict {
            CFBundleTypeExtensions = Array {
                css
            }
            CFBundleTypeMIMETypes = Array {
                text/css
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = CSS style sheet
            CFBundleTypeIconFile = css.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                pdf
            }
            CFBundleTypeMIMETypes = Array {
                application/pdf
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = PDF document
            CFBundleTypeIconFile = pdf.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                webarchive
            }
            CFBundleTypeMIMETypes = Array {
                application/x-webarchive
            }
            LSHandlerRank = Alternate
            LSIsAppleDefaultForType = true
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Web archive
            CFBundleTypeIconFile = webarchive.icns
            CFBundleTypeRole = Viewer
            ICExtension = ARCHIVE
        }
        Dict {
            CFBundleTypeExtensions = Array {
                webbookmark
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Safari bookmark
            CFBundleTypeIconFile = webbookmark.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                webhistory
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Safari history item
            CFBundleTypeIconFile = webhistory.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                webloc
            }
            CFBundleTypeOSTypes = Array {
                ilht
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Web internet location
            CFBundleTypeIconFile = webloc.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                download
            }
            LSTypeIsPackage = true
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Safari download
            CFBundleTypeIconFile = download10.icns
            CFBundleTypeRole = Editor
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                safariextz
            }
            CFBundleTypeMIMETypes = Array {
                application/x-safari-extension
            }
            LSTypeIsPackage = false
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Safari extension
            CFBundleTypeIconFile = safariextz.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Owner
        }
        Dict {
            CFBundleTypeExtensions = Array {
                gif
            }
            CFBundleTypeMIMETypes = Array {
                image/gif
            }
            CFBundleTypeOSTypes = Array {
                GIFf
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = GIF image
            CFBundleTypeIconFile = gif.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                html
                htm
                shtml
                jhtml
            }
            CFBundleTypeMIMETypes = Array {
                text/html
            }
            CFBundleTypeOSTypes = Array {
                HTML
            }
            LSHandlerRank = Alternate
            LSIsAppleDefaultForType = true
            CFBundleTypeName = HTML document
            NSDocumentClass = BrowserDocument
            CFBundleTypeIconFile = html.icns
            CFBundleTypeRole = Viewer
            ICExtension = HTML
        }
        Dict {
            CFBundleTypeExtensions = Array {
                js
            }
            CFBundleTypeMIMETypes = Array {
                application/x-javascript
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = JavaScript script
            CFBundleTypeIconFile = js.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                jpg
                jpeg
            }
            CFBundleTypeMIMETypes = Array {
                image/jpeg
            }
            CFBundleTypeOSTypes = Array {
                JPEG
            }
            LSHandlerRank = Alternate
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = JPEG image
            CFBundleTypeIconFile = jpeg.icns
            CFBundleTypeRole = Viewer
            ICExtension = JPEG
        }
        Dict {
            CFBundleTypeExtensions = Array {
                jp2
            }
            CFBundleTypeMIMETypes = Array {
                image/jp2
            }
            CFBundleTypeOSTypes = Array {
                jp2 
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = JPEG 2000 image
            CFBundleTypeIconFile = jp2.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                txt
                text
            }
            CFBundleTypeMIMETypes = Array {
                text/plain
            }
            CFBundleTypeOSTypes = Array {
                TEXT
            }
            LSHandlerRank = Alternate
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Plain text document
            CFBundleTypeIconFile = txt.icns
            CFBundleTypeRole = Viewer
            ICExtension = TXT
        }
        Dict {
            CFBundleTypeExtensions = Array {
                png
            }
            CFBundleTypeMIMETypes = Array {
                image/png
            }
            CFBundleTypeOSTypes = Array {
                PNGf
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = PNG image
            CFBundleTypeIconFile = png.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                tiff
                tif
            }
            CFBundleTypeMIMETypes = Array {
                image/tiff
            }
            CFBundleTypeOSTypes = Array {
                TIFF
            }
            LSHandlerRank = Alternate
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = TIFF image
            CFBundleTypeIconFile = tiff.icns
            CFBundleTypeRole = Viewer
            ICExtension = TIFF
        }
        Dict {
            CFBundleTypeExtensions = Array {
                url
            }
            CFBundleTypeOSTypes = Array {
                LINK
            }
            LSIsAppleDefaultForType = true
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Web site location
            CFBundleTypeIconFile = url.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                ico
            }
            CFBundleTypeMIMETypes = Array {
                image/x-icon
            }
            CFBundleTypeOSTypes = Array {
                ICO 
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = Windows icon image
            CFBundleTypeIconFile = ico.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
        Dict {
            CFBundleTypeExtensions = Array {
                xhtml
                xht
                xhtm
                xht
            }
            CFBundleTypeMIMETypes = Array {
                application/xhtml+xml
            }
            LSHandlerRank = Alternate
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = XHTML document
            CFBundleTypeIconFile = xhtml.icns
            CFBundleTypeRole = Viewer
            ICExtension = XHTML
        }
        Dict {
            CFBundleTypeExtensions = Array {
                xml
                xbl
                xsl
                xslt
            }
            CFBundleTypeMIMETypes = Array {
                application/xml
                text/xml
            }
            LSHandlerRank = Alternate
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = XML document
            CFBundleTypeIconFile = xml.icns
            CFBundleTypeRole = Viewer
            ICExtension = XML
        }
        Dict {
            CFBundleTypeExtensions = Array {
                svg
            }
            CFBundleTypeMIMETypes = Array {
                image/svg+xml
            }
            NSDocumentClass = BrowserDocument
            CFBundleTypeName = SVG document
            CFBundleTypeIconFile = svg.icns
            CFBundleTypeRole = Viewer
            LSHandlerRank = Alternate
        }
    }
    DTSDKBuild = 17A364
    CFBundleShortVersionString = 12.1
    CFBundleSupportedPlatforms = Array {
        MacOSX
    }
    BuildMachineOSBuild = 16B2657
    DTPlatformBuild = 9L95g
    CFBundlePackageType = APPL
    DTXcodeBuild = 9L95g
    CFBundleDevelopmentRegion = English
    UTExportedTypeDeclarations = Array {
        Dict {
            UTTypeConformsTo = Array {
                public.data
            }
            UTTypeIdentifier = com.apple.safari.bookmark
            UTTypeDescription = Safari bookmark
            UTTypeTagSpecification = Dict {
                public.filename-extension = Array {
                    webbookmark
                }
            }
        }
        Dict {
            UTTypeConformsTo = Array {
                public.data
            }
            UTTypeIdentifier = com.apple.safari.extension
            UTTypeDescription = Safari extension
            UTTypeTagSpecification = Dict {
                public.filename-extension = Array {
                    safariextz
                }
            }
        }
        Dict {
            UTTypeConformsTo = Array {
                public.data
            }
            UTTypeIdentifier = com.apple.safari.history
            UTTypeDescription = Safari history item
            UTTypeTagSpecification = Dict {
                public.filename-extension = Array {
                    webhistory
                }
            }
        }
    }
    CFBundleVersion = 13607.1.5.2
    CFBundleGetInfoString = 12.1, Copyright © 2003-2018 Apple Inc.
    NSSupportsSuddenTermination = true
    LSFileQuarantineEnabled = true
    NSSupportsAutomaticTermination = true
    OSAScriptingDefinition = Safari.sdef
    NSMainNibFile = MainMenu
    CFBundleIdentifier = com.apple.SafariTechnologyPreview
    HPDHelpProjectIdentifier = safari
    DTXcode = 0900
    CFBundleHelpBookName = com.apple.Safari.help
    CFBundleHelpBookFolder = Safari.help
    Application-Group = Array {
        dot-mac
        InternetAccounts
    }
    CFBundleExecutable = Safari Technology Preview
    NSLocationUsageDescription = Websites you visit may request your location.
    LSMinimumSystemVersion = 10.13.0
    CFBundleSignature = stp1
    DTCompiler = com.apple.compilers.llvm.clang.1_0
}

P.S. This issue is similar to web-platform-tests/wpt#13052 and web-platform-tests/wpt#13399, and the solution may be in wpt itself, but also perhaps not given that much of the Safari-specific setup is in this repo.

[foolip]

Another failed attempt:

/Applications/Safari\ Technology\ Preview.app/Contents/MacOS/safaridriver -p 9999
curl -d '{ "capabilities": {} }' http://localhost:9999/session
{"value":{"sessionId":"78FFD008-96EB-4C4E-B6B9-0BB7AA732F85","capabilities":{"browserVersion":"12.1","platformName":"macOS","acceptInsecureCerts":false,"setWindowRect":true,"browserName":"Safari Technology Preview"}}}

@burg, how would you advise people to get the STP release number if it wasn't installed from a known dmg? (Which I am currently doing, I believe @jugglinmike has told you that safaridriver doesn't work in STP 66, hence the 65 in this issue.)

[gsnedders]

@burg https://bugreport.apple.com/web/?problemID=43691655 is filed on the inability to get the STP release number programmatically FWIW

[burg]

Safari Technology Preview 67 and later has added the --version flag to safaridriver. Let me know if that doesn't solve your needs.

[foolip]

Wonderful, thanks @burg!

[foolip]

I don't see 67 on https://developer.apple.com/safari/download/ yet, do you know when it will be released, @burg? Thanks!

[jugglinmike]

You could also check the WebKit blog. Looks like version 67 hasn't been announced yet.

Also, Brian: the ATOM feed and the RSS feed are both out of date (they list the announcement of STP 62 as the latest post).

[foolip]

It's here: https://webkit.org/blog/8419/release-notes-for-safari-technology-preview-67/

I've sent Homebrew/homebrew-cask-versions#6387 for my own use, and verified that safaridriver --version outputs this:

Included with Safari Technology Preview (Release 67, 13607.1.9.0.1)

[foolip]

@burg, do you also know what that string will be in Safari stable? I'm asking because I'm updating ./wpt run to extract the string and would like to be proactive.

[burg]

Example:

bburg$ safaridriver --version
Included with Safari 12.1 (14607.1.11)

[burg]

The version string has been explicitly designed to be regex parsable. Let me know if you think this is problematic; I've used it internally with pytest to write Safari Technology Preview-specific tests.

[foolip]

Thanks @burg! web-platform-tests/wpt#13467 is where I'm writing the code to parse it, although I haven't finished it.

BTW, 12.1 doesn't show up in the string for STP, but it is the version that PlistBuddy will report. Do you consider part of the version, or just an internal detail?

[foolip]

12.1 is also in the UI screenshot I posted.

[burg]

Safari Technology Preview's Info.plist will list the next anticipated Safari release version, to signal that it is not Safari 12 (in this case). However I omit that from the version output as we typically don't reference that version string for Safari Technology Preview. There is no other programmatic way to determine the STP release from the Safari Technology Preview app.

[foolip]

Thanks for the explanation, then so think just stripping "Included with " in both cases to get "Safari Technology Preview (Release 67, 13607.1.9.0.1)" and "Safari 12.1 (14607.1.11)" is fine.

@jugglinmike, would you know why 12.1 shows up as the version for STP on wpt.fyi now? Did you use PlistBuddy?

@lukebjerring, will we need to adapt the wpt.fyi frontend to this as well? And can we change the version information retroactively to not have 12.1 ever match STP?

[jugglinmike]

When running Safari locally, the system infers the version based on files that are "near" the executable under test. The heuristic is implemented and explained in the read-browser-version.py script. Relevant comment:

# The Safari binary cannot be queried for this information. Because it may
# only be installed at the system level, this function operates on
# assumptions regarding files that are present relative to the binary.
# The XML-formatted `version.plist` file contains key/value pairs in the
# following structure:
#
#     <plist version="1.0">
#     <dict>
#         <key>EXAMPLE KEY</key>
#         <string>EXAMPLE VALUE</string>
#     </dict>
#     </plist>
#
# See:
# https://developer.apple.com/library/content/documentation/General/Reference/InfoPlistKeyReference/Articles/AboutInformationPropertyListFiles.html#//apple_ref/doc/uid/TP40009254-SW1

You might say we're our own PlistBuddy

[foolip]

Aha, well that explains it at least. And means that when wpt can get the version from safaridriver --version, that some tweaks around this to not override will also be needed.

@jugglinmike, do you know which exact STP versions have been used for which runs, so we could retroactively edit the version info?

[jugglinmike]

We've tested the latest version for almost every collection. The exception is we tested STP 65 until STP 67 was released because version 66 could not be automated. Relevant data is available in the build logs, so I can verify programtically when/if that's desirable.

[foolip]

Great, thank you! @lukebjerring, do you think we should just edit the version information in wpt.fyi's datastore, or will something bad happen if the information is out of sync with the reports? We could just leave it, but downside is that product=safari-12 can match the wrong thing for some shas, which is a problem I actually ran into.

[lukebjerring]

Yes; I'll defer to @Hexcles for a batch-update of mapping the (incorrect) data