macOS Automator misbehaving

[macstatic]

I'm trying make Automator "learn" the way I use a specific application, as I do the same task every time, but being a little complicated it's hard to remember which menu to select etc.
I figured by telling Automator to follow my actions once it would later be a simple task to run that application.

I told Automator to "Record" my actions, but it doesn't seem to work as I had expected. For instance, my first task is to enter the BlueTooth system preference, then turn on BlueTooth. Somehow Automator doesn't understand this and gives me error messages when I try to play back for testing.
What's the correct procedure of "recording" an action?

 

[xUKHCx]

I've never been able to get the Record feature of Automator to work successfully.

What do you want to do as you may be able to do it via Apple script. For example the script to get check/decheck the bluetooth option is

tell application "System Preferences"
	activate
	set current pane to pane "com.apple.preferences.Bluetooth"
	tell application "System Events"
		tell process "System Preferences"
			click checkbox "On" of window "Bluetooth"
		end tell
	end tell
end tell

To run this copy and paste it into the application "Script Editor"

 

[macstatic]

Nope, didn't work.
When running it opened the BlueTooth preference window, then gave me an error message:

Applescript error
System events got an error.
NSReceiverEvaluationScriptError: 4
[OK]

Do you have to know some basic programming skills to use Applescript? I tried to create a new Applescript (File-New), pressed the "Record" button and opened the BlueTooth preference panel, turned on Bluetooth etc. and finally returned to Script editor where I pressed the "Stop" button.
I expected to see something similar to your code, but the window was completely empty! What gives?

And how can Applescript help me with the rest of my "actions" (accessing the menus and buttons of the application I'll be using) when Applescript won't even figure out turning on Bluetooth?

 

[xUKHCx]

Nope, didn't work.
When running it opened the BlueTooth preference window, then gave me an error message:

Applescript error
System events got an error.
NSReceiverEvaluationScriptError: 4
[OK]

That is an odd error, from searching for it online it seems to be a generic error given for when an element isn't found. What OS are you running? What does your Bluetooth Preferance pane look like

The script requires Assistive Devices to be turned on System Preferences - Universal Access - Enable Access for Assistive devices, if that is off it will cause problems but will usually through up a different error.

Do you have to know some basic programming skills to use Applescript? I tried to create a new Applescript (File-New), pressed the "Record" button and opened the BlueTooth preference panel, turned on Bluetooth etc. and finally returned to Script editor where I pressed the "Stop" button.
I expected to see something similar to your code, but the window was completely empty! What gives?

A bit of basic programming is needed, apple script's goal is to bring powerful tools into a readable script format. It can be difficult and rather odd at times but you can do quite a lot with it. I haven't written anything in a long time so the above maybe able to be completed in a simpler way.

And how can Applescript help me with the rest of my "actions" (accessing the menus and buttons of the application I'll be using) when Applescript won't even figure out turning on Bluetooth?

It can do a lot but I can't answer that without knowing what exactly you want to do with it. It can enable bluetooth as the above script works fine on my computers running Leopard. When we know what you want to do we can look at the best solution as there may be a downloadable script from http://automatorworld.com/ that does what you want to do.

 

[prostuff1]

Nope, didn't work.
When running it opened the BlueTooth preference window, then gave me an error message:

Applescript error
System events got an error.
NSReceiverEvaluationScriptError: 4
[OK]

Do you have to know some basic programming skills to use Applescript? I tried to create a new Applescript (File-New), pressed the "Record" button and opened the BlueTooth preference panel, turned on Bluetooth etc. and finally returned to Script editor where I pressed the "Stop" button.
I expected to see something similar to your code, but the window was completely empty! What gives?

And how can Applescript help me with the rest of my "actions" (accessing the menus and buttons of the application I'll be using) when Applescript won't even figure out turning on Bluetooth?

Like xUKHCx said, getting the record feature to work is very very hit and miss (and mostly miss). If you can tell us what you are trying to do we might be able to help a little more. Also, do some googleing for appleScript examples and you should find some examples.

Also, the above script works just fine for me on 10.5.7

 

[macstatic]

The computer/person I'm trying to help out has a recent Intel Mac running the latest version of Leopard. Myself, I'm still running MacOS Tiger (10.4.11) on a PowerMac G4. I can create the script on either computer.
I tried the script out on my G4, which is where I received the error message. I haven't tried it out on the Mac running Leopard yet.
And yes -I do have "Assistive devices" turned on.
This is what my Bluetooth preference window in MacOS 10.4.11 looks like:

I also have a Bluetooth icon in my menu-bar, but I assumed that I would run into problems if the script runs in case the user has removed the Bluetooth status icon in the menu-bar so it won't find it. After all, the Bluetooth preference window is available regardless personal configuration

The task I want to automate is to use a mobile-phone application called PhoneAgent which will be used to import SMS text messages from a mobile phone, "print" them to PDF and finally save the PDF file in a specific folder on the Mac hard drive (asking the user to fill in the file's name first, or even better: have the name automatically named by date such as "SMS messages 7-6-2009", using the current date).
The Bluetooth thing should be in the first part of the script so the user doesn't have to remember to turn it on first (if it's not already turned on that is). Near the end of the script (when the PDF file with the text-messages have been stored) Bluetooth on the Mac should be turned off, and a message should pop up, alerting the user something like "SMS transfer complete. You can now turn Bluetooth on your phone off." with an "OK" button to press when done reading, which quits the Phoneagent application as well.

By the way, why do Macs come with both Applescript and Automator?
Do they do different sort of things, or are they just different tools for doing the same kind of tasks?

 

[prostuff1]

By the way, why do Macs come with both Applescript and Automator?
Do they do different sort of things, or are they just different tools for doing the same kind of tasks?

Automator uses appleScript to an extent. You can also use AppleScript from within Automator. Automator is supposed to be an easy way for novices to do simple task, and for that purpose it works pretty well. I have a couple of Automator tasks that resize images, rename images, and create text documents.

Anyway, the AppleScript you describe in not a small one. I am not overly familiar with AppleScript so I would have to do some research on how to do all the things you ask.

 

[mysterytramp]

Your Bluetooth looks different than mine (Leopard). I'd try the script on the other computer.

If it fails check out something called "UI Inspector" inside your Applescript folder. Run it and a panel appears that changes as the mouse hovers over different user interface elements, providing the internal names you can use those names in your script.

mt

 

[macstatic]

I tried it on the Leopard equipped Mac and the script works fine there! Thanks
I couldn't find any "UI inspector" inside the Applescript folder though.

If I run the script a second time it turns Bluetooth off, so it obviously just toggles the Bluetooth on/off switch.
To make it completely fool-proof, is there a way it checks the on/off status so that it isn't turned off by the script if Bluetooth was already on to begin with?

Next, how do I make Applescript "record" what I'm about to do in PhoneAgent?
Step by step, this is what I do when transferring text messages from the phone to the Mac:

1) turn on Bluetooth if not already on
2) pop up a dialog on the Mac: "Please enable Bluetooth on your mobile phone"
3) Start the PhoneAgent application
4) click on the "SMS" icon (inside PhoneAgent). SMS messages are then automatically transferred from the phone to the SMS window after a few seconds
5) Press CMD-A (mark all SMS messages in the SMS window)
6) Click the "tools" icon drop-down button
7) choose "Print" from the button drop-down menu
8) (the "Print" dialog window appears) press the "PDF" button
9) From the PDF button drop-down menu choose "Save as PDF"
10) Wait for the user to enter a suitable filename (or have the script automatically name the file with the current date, then press the "Save" button
11) Quit PhoneAgent
12) Turn BlueTooth off
13) pop up a dialog window with the message "Transfer now complete. Please disable Bluetooth on your phone"
14) end of script

 

[xUKHCx]

Sorry for the long post.



I downloaded PhoneAgent and it doesn't seem to be applescript/automater friendly at all. The floating window is the tricky point and I am not sure if it is possible to use these tools to access buttons on it. Can you get to the SMS window that you require another way?

Is this only for one person or for wider use?
Also would you mind installing something to help achieve this?

The bluetooth preference pane is workable however not very user friendly as it will cause windows to open up and close. There is a small command line utility that can enable bluetooth in the background. This should be quicker and more robust.

The code for checking bluetooth and turning it on/leaving it on

set os_version to do shell script "sw_vers -productVersion" -- finds out what OS the user is using
set os_version to get (characters 1 through -3 of os_version) as string -- makes it more friendly format for the if statements later on


if os_version is in "10.5" then -- checks for Leopard
	
	tell application "System Preferences"
		activate
		set current pane to pane "com.apple.preferences.Bluetooth"
		tell application "System Events"
			tell process "System Preferences"
				set bluetoothstatus to get value of checkbox "On" of window "Bluetooth" -- determines if bluetooth is on
			end tell
		end tell
		
		if bluetoothstatus is equal to 0 then -- if bluetooth is off, turns it on
			
			tell application "System Events"
				tell process "System Preferences"
					click checkbox "On" of window "Bluetooth" -- turns bluetooth on
				end tell
			end tell
		end if
		
	end tell
end if


if os_version is in "10.4" then -- If it is Tiger need to find the values for the buttons.
	--do something
end if

tell application "System Preferences" to quit
display dialog "Please enable Bluetooth on your mobile phone"
tell application "PhoneAgent" to activate

With the command line utility installed the code is

set btstatus to do shell script "/usr/local/bin/blueutil status | awk '{print $2}'"

if btstatus is equal to "off" then
	do shell script "/usr/local/bin/blueutil on"
end if

tell application "System Preferences" to quit
display dialog "Please enable Bluetooth on your mobile phone"
tell application "PhoneAgent" to activate

1-3 Covered above
4 - May not be possible
5 - Depends but could be possible with the below code

tell application "PhoneAgent"
	activate
	tell application "System Events"
		key code 0 using command down -- simulates pressing A with command down
	end tell
end tell

6-7 - Is this just cmd+p?

If so

tell application "PhoneAgent"
	activate
	tell application "System Events"
		key code 35 using command down -- simulates pressing P with command down
	end tell
end tell

8-9 Might be possible depends on the application naming scheme and how it appears

10 - Not sure if it will allow entering text into the box but could be achievable to do automatically

11 - 14 Easy to do.

As you can see it still depends on getting a few of the big issues resolved. I think I have a suitable Sony Ericsson Phone somewhere about. If so I'll try and get it up and running, but it may not be possible to fully automate the entire process.

 

[macstatic]

Thanks for your quick reply.
Yes, the script is basically for one person, though I could have good use for it myself.

What distinguishes an "Applescript friendly application" from one that isn't? Should the accessable functions have keyboard shortcuts? I was looking for that myself, but couldn't find any for the required features/functions. Maybe I'll contact the developer and ask.

Great to hear that there's a better way to turn on/off Bluetooth. I'll check it out later.

6 and 7) Saving to PDF: no, it's not the same as CMD-p (print) unfortunately.

When you try Phoneagent out yourself you have to keep a few things in mind:
a) set up a "serial port" in the Bluetooth setting -follow the "Bluetooth read me" files found inside the application folder when installed.
b) In PhoneAgent's preferences window:
- (upper left hand corner): choose your phone with the serial port settings
- (upper left hand corner): enable "Connect at startup"

The next time you run the application it will automatically connect as long as you have Bluetooth turned on. And if you click on the "SMS" icon in the application it'll automatically transfer your SMS messages (I'm using the unregistered version).

 

[xUKHCx]

Thanks for your quick reply.
Yes, the script is basically for one person, though I could have good use for it myself.

As you are using Tiger and I don't have it I can only give suggestions on how to adapt it to get it to work (assuming you want to stick with the system Preferences way of turning blueooth on/off).

Instead of

tell application "System Events"
			tell process "System Preferences"
				click checkbox "On" of window "Bluetooth" -- turns bluetooth on
			end tell
		end tell

try

tell application "System Events"
			tell process "System Preferences"
				click button "Turn Bluetooth On" of window "Bluetooth" -- turns bluetooth on
			end tell
		end tell

What distinguishes an "Applescript friendly application" from one that isn't? Should the accessable functions have keyboard shortcuts? I was looking for that myself, but couldn't find any for the required features/functions. Maybe I'll contact the developer and ask.

The issue with this application is that it doesn't seem to use the window function but rather creates everything in dialog boxes which means little tricks and standard scripting behaviour will not work. Also a major issue is that the floating window which holds the SMS button isn't recognised by UI element browsers so I am not sure if it is possible to even access that button.

Here is the script in a workable format. It does everything automatically except press the SMS button so what I have done is put a dialog in asking the user to do it and press the OK button when they have done so. Not ideal but for now may have to do.

Also there are lots of delay's put in which could do with tweaking. It is quite a clunky script all around, could be cleaned up but it is working (for me at least).

I would suggest making sure that all your SMS' are backed up before testing this (as well as everything else).

set os_version to do shell script "sw_vers -productVersion" -- finds out what OS the user is using, only needed if you want to have it used on different OSs that may have different buttons, ala bluetooth
set os_version to get (characters 1 through -3 of os_version) as string -- makes it more friendly format for the if statements later on


if os_version is in "10.5" then -- checks for Leopard
	
	tell application "System Preferences"
		activate
		set current pane to pane "com.apple.preferences.Bluetooth"
		tell application "System Events"
			tell process "System Preferences"
				set bluetoothstatus to get value of checkbox "On" of window "Bluetooth" -- determines if bluetooth is on
			end tell
		end tell
		
		if bluetoothstatus is equal to 0 then -- if bluetooth is off, turns it on
			
			tell application "System Events"
				tell process "System Preferences"
					click checkbox "On" of window "Bluetooth" -- turns bluetooth on
				end tell
			end tell
		end if
		
	end tell
end if


if os_version is in "10.4" then -- If it is Tiger need to find the values for the buttons.
	--do something
end if


tell application "System Preferences" to quit

delay 0.5

display dialog "Please enable Bluetooth on your mobile phone"


tell application "PhoneAgent" to activate

delay 3.0

display dialog "Please press enter when you have activated the SMS window"

tell application "PhoneAgent"
	activate
	tell application "System Events"
		key code 0 using command down -- simulates pressing A with command down
	end tell
end tell


tell application "PhoneAgent"
	activate
	tell application "System Events"
		tell process "PhoneAgent"
			key code 0 using command down -- simulates pressing A with command down
			click button 4 of window "SMS"
			delay 0.5
			key code 35 -- press P to access the Print menu
			delay 0.5
			key code 36 -- press enter to activate the print menu
			delay 0.5
			click menu button "PDF" of window "Print"
			delay 0.5
			click menu item "Save as PDF…" of menu 1 of menu button "PDF" of window "Print"
			delay 0.5
			
			set curTime to (do shell script "date \"+%H:%M:%S\"") -- gets the time
			set curDate to (do shell script "date \"+%Y-%m-%d\"") -- gets the date in reverse format YYYY-MM-DD
			
			set savestring to "SMS " & curDate & " " & curTime as string -- creates the save name
			
			tell application "PhoneAgent"
				tell application "System Events"
					tell process "PhoneAgent"
						set value of text field 1 of window "Save" to savestring -- enters the file name to be saved
						click button "Save" of window "Save" -- presses save
						
					end tell
				end tell
			end tell
			
			
		end tell
		
	end tell
end tell
delay 10

tell application "PhoneAgent"
	
	quit
end tell
display dialog "Transfer now complete. Please disable Bluetooth on your phone"


tell application "System Preferences"
	activate
	set current pane to pane "com.apple.preferences.Bluetooth"
	tell application "System Events"
		tell process "System Preferences"
			set bluetoothstatus to get value of checkbox "On" of window "Bluetooth" -- determines if bluetooth is on
		end tell
	end tell
	
	if bluetoothstatus is equal to 1 then -- if bluetooth is on, turns it off
		
		tell application "System Events"
			tell process "System Preferences"
				click checkbox "On" of window "Bluetooth" -- turns bluetooth off
			end tell
		end tell
	end if
	
end tell

 

[macstatic]

That's awesome -it works!
I manually switched on Bluetooth (on the Mac), but will try the suggestions given here later for the script to do it.

A minor issue:
The script editor window (or script in my case as I saved it as a script application to try out) icon in the dock jumps up and down to get attention.
When I click it the PhoneAgent window disappears while the "Please press Enter when you have activated the SMS window" dialog pops up.
This can be quite confusing to the novice Mac user.

If I get in touch with the people behind PhoneAgent what can I suggest they do in order to make it easier to script (at least the parts we're working on)?

Another minor issue: When completed saving as a PDF file (via the "Print" window) I get an error message saying "NSReceiverEvaluationScriptError: 4". Not sure what this is, but it may be some incompatibility with MacOS Tiger which is where I tried it.
As far as I can see the date should be automatically added to the filename, but in my case it only opens the "Save" dialog window with the filename "SMS" in it, and I have to manually press the "Save" button.
I'll give it a go in MacOS Leopard.
(PS: working properly in Leopard is what's important, so unless it's an easy job to make it compatible with both Tiger and Leopard we can skip Tiger).

Other than that the PDF file looks great, so it basically works but needs a little fine tuning.

 

[xUKHCx]

That's awesome -it works!
I manually switched on Bluetooth (on the Mac), but will try the suggestions given here later for the script to do it.

A minor issue:
The script editor window (or script in my case as I saved it as a script application to try out) icon in the dock jumps up and down to get attention.
When I click it the PhoneAgent window disappears while the "Please press Enter when you have activated the SMS window" dialog pops up.
This can be quite confusing to the novice Mac user.

This is part of the PhoneAgent problem, if it was a normal window it will fade slightly and go to the background just like most other applications. However as PhoneAgent uses a floating window for the main display this window disappears when it is not in focus.

I have changed the script a little to try and come up with a more user friendly version however it operates slightly differently.

The first script I made above asks the user to press the SMS button and confirm in the script.

This script tells the user to press the button and then checks to see if they have. My issue with this method is that it may take a longer time than I have allowed to download the SMSs from the phone.

I've added lots of comments so hopefully you can follow what is going on. If you would prefer it to go back to the previous method then it is just a simple swap.

(*set os_version to do shell script "sw_vers -productVersion" -- finds out what OS the user is using
set os_version to get (characters 1 through -3 of os_version) as string -- makes it more friendly format for the if statements later on


if os_version is in "10.5" then -- checks for Leopard
*)

tell application "System Preferences" -- this section turns bluetooth on at the beginning of the script
	activate
	set current pane to pane "com.apple.preferences.Bluetooth" --opens the bluetooth preferences pane
	tell application "System Events"
		tell process "System Preferences"
			set bluetoothstatus to get value of checkbox "On" of window "Bluetooth" -- determines if bluetooth is on
		end tell
	end tell
	
	if bluetoothstatus is equal to 0 then -- if bluetooth is off, turns it on
		
		tell application "System Events"
			tell process "System Preferences"
				click checkbox "On" of window "Bluetooth" -- turns bluetooth on
			end tell
		end tell
	end if
	
end tell
--end if

(*
if os_version is in "10.4" then -- If it is Tiger need to find the values for the buttons.
	--do something
end if
*)

tell application "System Preferences" to quit

delay 0.5

tell application "SystemUIServer" -- alerts the user to press the SMS button
	activate
	display dialog "Please enable Bluetooth on your mobile phone and when the Phone Agent window appears press the SMS button"
end tell


tell application "PhoneAgent" to activate -- starts PhoneAgent

tell application "PhoneAgent" -- this section checks for the SMS window and if not present askes the user to press the button
	activate
	tell application "System Events"
		tell process "PhoneAgent"
			activate
			set windowname to "test"
			set tcount to 1
			repeat until windowname is equal to "SMS" -- this checkes for the SMS window and repeats until it is present
				try
					set windowname to get name of window "sms"
				end try
				delay 2 -- delay is here to prevent it using up lots of CPU time and to allow for the user to press the buttons without harassing them  too much
				set tcount to tcount + 1 -- counts so that it doesn't bug the user every time
				if tcount is equal to 10 then -- enters this section when the count is equal to the number, can change this number for fine tunning
					tell application "SystemUIServer" -- If the user hasn't pressed the SMS button this dialog will appear and ask them to, and then the PhoneAgent window will reappear
						activate
						display dialog "Please Press then SMS button when the PhoneAgent windows appears" giving up after 5
					end tell
					tell application "PhoneAgent" to activate
				end if
			end repeat
		end tell
	end tell
end tell
delay 10 -- allows time for the SMSs to be downloaded


tell application "PhoneAgent" -- this section selects the SMSs and prints them
	activate
	tell application "System Events"
		tell process "PhoneAgent"
			key code 0 using command down -- simulates pressing A with command down,  in order to select all in the window
			click button 4 of window "SMS"
			delay 0.5
			key code 35 -- press P to access the Print menu
			delay 0.5
			key code 36 -- press enter to activate the print menu
			delay 0.5
			click menu button "PDF" of window "Print" -- press the PDF button
			delay 0.5
			click menu item "Save as PDF…" of menu 1 of menu button "PDF" of window "Print"
			delay 0.5
			
			set curTime to (do shell script "date \"+%H:%M:%S\"") -- gets the time
			set curDate to (do shell script "date \"+%Y-%m-%d\"") -- gets the date in reverse format YYYY-MM-DD
			
			set savestring to "SMS " & curDate & " " & curTime as string -- creates the save name
			
			tell application "PhoneAgent" -- sets the File name for the PDF that is being produced
				tell application "System Events"
					tell process "PhoneAgent"
						set value of text field 1 of window "Save" to savestring -- enters the file name to be saved
						click button "Save" of window "Save" -- presses save
						
					end tell
				end tell
			end tell
			
			
		end tell
		
	end tell
end tell
delay 10 -- allows for the PDF to be saved, can be shortened for tweaking

tell application "PhoneAgent" -- quits PhoneAgent
	quit
end tell

tell application "System Preferences" -- this section turns the bluetooth off at the end
	activate
	set current pane to pane "com.apple.preferences.Bluetooth" --opens the bluetooth preferences pane
	tell application "System Events"
		tell process "System Preferences"
			set bluetoothstatus to get value of checkbox "On" of window "Bluetooth" -- determines if bluetooth is on
		end tell
	end tell
	
	if bluetoothstatus is equal to 1 then -- if bluetooth is on, turns it off
		
		tell application "System Events"
			tell process "System Preferences"
				click checkbox "On" of window "Bluetooth" -- turns bluetooth off
			end tell
		end tell
	end if
	quit
	
end tell

tell application "SystemUIServer" -- tells the user that the script is finished and so can turn off bluetooth from their phone
	activate
	display dialog "Transfer now complete. Please disable Bluetooth on your phone" giving up after 5 -- the "giving up after" option closes the dialog box after the set number of seconds, can be tweaked.
end tell

If I get in touch with the people behind PhoneAgent what can I suggest they do in order to make it easier to script (at least the parts we're working on)?

A small change that wouldn't be hard for them to implement would be to add the option to select the item via a menu / shortcut.

Another minor issue: When completed saving as a PDF file (via the "Print" window) I get an error message saying "NSReceiverEvaluationScriptError: 4". Not sure what this is, but it may be some incompatibility with MacOS Tiger which is where I tried it.
As far as I can see the date should be automatically added to the filename, but in my case it only opens the "Save" dialog window with the filename "SMS" in it, and I have to manually press the "Save" button.
I'll give it a go in MacOS Leopard.
(PS: working properly in Leopard is what's important, so unless it's an easy job to make it compatible with both Tiger and Leopard we can skip Tiger).

It is the same error as in the first instance we noted above. Basically the script relies on certain buttons being called the same names and being in the same order. I don't have access to a machine with Tiger on so can't check. You can try adapting it by using UIElementInspector, download from here (direct link to a .zip file from the apple website). Open up the .zip and near the bottom is the compiled application UiElementInspector. This displays a window which when you hover your mouse over items tells you the location/names of the items.

 

[macstatic]

I've been quiet a while because I've emailed the makers of Phoneagent asking if they're willing to make keyboard shortcuts and menu items out of the functions that now only work with a mouse-click.
I'll get back here as things evolve.