GUI Scripting with AppleScript

GUI scripting can be a high-investment, high-reward endeavor. While often intimidating, it can give you some unique and powerful tools.

GUI Scripting with AppleScript

If you administer Apple computers, you're probably familiar with the adorable mess that is the softwareupdate binary. Since macOS Big Sur, Apple's command-line Software Update binary has just been a mess. Updates refusing to trigger, available updates not being detected by Macs, and errors occurring when you try to run an update. Resilient bunch we are, admins learn to adapt. Open-source projects like Nudge were thrust into the forefront, and we had to remediate problems by having users reinstall operating systems manually or sending emails to have folks run their Software Updates manually.

These methods of triggering Software Updates via the GUI tended to be a little more reliable but came with a slew of other problems. Walking users through multiple clicks can be more challenging than it has any right to be, and Apple's maddening design of the Software Update System Preferences pane makes it difficult for many users to distinguish the buttons to update vs upgrade. We need more powerful tools - and sometimes when you need to move forward, you have to look back.

GUI scripting can be an inelegant solution. Your code can be fragile, often requiring significant refactoring for different operating systems. With the advent of PPPCs, you also need to be very deliberate about what tools you use where. With some elbow grease and persistence, I was able to set up a one-click Applescript app (with accompanying PPPC profile) that would:

  • Open System Preferences
  • Open the Software Update preference pane
  • Wait for the softwareupdate process to check for available updates
  • Click on the appropriate UI button to run an update rather than an upgrade
  • Click "Install" or "Install Now" to proceed
  • If needed, click to accept the Apple terms and conditions of the update

I embedded this project into our Nudge json so that when we pinged our users, they could click one button and everything else would take care of itself (with the asterisk that users on Apple silicon computers would need to type out their password in a popup before the update would run). What makes GUI scripting so delicate is that you need to dissect the anatomy of the window you're working in to give very detailed commands on what piece of the interface you want to interact with.

Let's start with a simple command - if a button exists, click it:

tell application "System Events"
	tell process "System Preferences"
		if exists (button 1 of group 1 of window "Software Update") then
			click button 1 of group 1 of window "Software Update"
		end if
	end tell
AppleScript code example - click a button if it exists

In order to make this happen, we need to know the name of the application that owns the window we're interacting with. We then tell the "System Events" application, which acts as our intermediary, to tell the app to click the thing. In this example, we bundled the click inside an if statement, to prevent an error in case the button isn't present. You'll notice that the location of the button is pretty simple to understand, by reading the description backward:

  • Window "Software Update"
    • Group 1
      • Button 1

Finishing the path, the window "Software Update" in the above example is within the process named "System Preferences". This hierarchy was pretty simple, but not all of them will be that way. The above script works for macOS Monterey and Big Sur. In getting a similar script ready for macOS Ventura's redesigned System Settings app, things got a little more complicated. Here's the same command, targeting a specific button on the Software Update settings page:

tell application "System Events"
	tell process "System Settings"
    	if exists (button "Update Now" of scroll area 1 of group 1 of group 1 of group 2 of splitter group 1 of group 1 of window "Software Update" of application process "System Settings" of application "System Events") then
		click button "Update Now" of scroll area 1 of group 1 of group 1 of group 2 of splitter group 1 of group 1 of window "Software Update" of application process "System Settings" of application "System Events"
		end if
Applescript code example - Ventura-specific click

The hierarchy explodes with additional complexity. This is a great example of how significant the changes can be to perform similar tasks in multiple operating systems. Here's a breakdown of the hierarchy from the example above:

  • Application "System Events"
    • Application process "System Settings"
      • Window "Software Update"
        • Group 1
          • Splitter group 1
            • Group 2
              • Group 1
                • Group 1
                  • Scroll area 1
                    • Button "Update Now"

The rest of the script is fairly basic - what I really want to focus on here is how you can identify the pieces of the interface to interact with. To do this, I leaned on the script from this post:

tell application "System Events"
  tell front window of (first application process whose frontmost is true)
    set uiElems to entire contents
  end tell
end tell
Applescript: get UI elements of the front window

Running this script will ... probably return an error popup for you that Script Editor needs Accessibility permissions to act on other apps. Once you approve it manually or via a PPPC profile, you'll get an output that lists the UI elements in the window, one at a time. To be able to more elegantly target the window I wanted, I had to define it instead of just having the script run on whatever is in front:

tell application "System Events"
  tell front window of process "System Preferences"
    set uiElems to entire contents
  end tell
end tell
Applescript: get UI elements of the front window of System Preferences

Here's an example output from looking at the System Preferences → Software Update window when no update or upgrade is available:

{image 1 of window "Software Update" of application process "System Preferences" of application "System Events", static text "Software Update" of window "Software Update" of application process "System Preferences" of application "System Events", group 1 of window "Software Update" of application process "System Preferences" of application "System Events", static text "Your Mac is up to date — macOS Monterey 12.5" of group 1 of window "Software Update" of application process "System Preferences" of application "System Events", static text "Last checked: Today at 9:25 PM" of group 1 of window "Software Update" of application process "System Preferences" of application "System Events", group 2 of window "Software Update" of application process "System Preferences" of application "System Events", button 1 of group 2 of window "Software Update" of application process "System Preferences" of application "System Events", checkbox "Automatically keep my Mac up to date" of group 2 of window "Software Update" of application process "System Preferences" of application "System Events", button 2 of group 2 of window "Software Update" of application process "System Preferences" of application "System Events", button "Advanced…" of group 2 of window "Software Update" of application process "System Preferences" of application "System Events", toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", group 1 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", group 1 of group 1 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", button 1 of group 1 of group 1 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", button 2 of group 1 of group 1 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", group 2 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", group 1 of group 2 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", button 1 of group 1 of group 2 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", group 3 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", text field 1 of group 3 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", button 1 of text field 1 of group 3 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events", button 1 of window "Software Update" of application process "System Preferences" of application "System Events", button 2 of window "Software Update" of application process "System Preferences" of application "System Events", button 3 of window "Software Update" of application process "System Preferences" of application "System Events", static text "Software Update" of window "Software Update" of application process "System Preferences" of application "System Events"}

Not exactly elegant, but if you squint, you can start to see the comma-separated patterns. Each time, it's the same "right-to-left" reading of parent-to-child UI elements. Sometimes the elements are numbered (button 1, button 2), sometimes they can be called out by name (checkbox "Automatically keep my mac up to date"), but the flow will be the same:

  • button 2 of window "Software Update" of application process "System Preferences" of application "System Events"
  • button 1 of group 1 of group 2 of toolbar 1 of window "Software Update" of application process "System Preferences" of application "System Events"
  • checkbox "Automatically keep my Mac up to date" of group 2 of window "Software Update" of application process "System Preferences" of application "System Events"

With some trial and error, you can start identifying the items you want to click, and start building those up into your scripts. Once you've got the items called out, remember to set up a PPPC to make sure the app you create (or the binary that will be executing your script) has the permissions it needs to make the magic happen.