Jun 3, 2007

Automator - Cocoa based action with AppleScript integration

I've built a bunch of Automator actions lately. Most of these have been simple AppleScript actions, with a few Shell based actions thrown in. I'll probably talk about some of those in future postings.

Right now I want to talk about one particular action I built that incorporated a combination of AppleScript and Objective-C code. There are a couple of ways to build such a “hybrid” action as Apple's documentation calls it. One way is to use primarily AppleScript, with added Objective-C classes that are called from the AppleScript. The other way is to primarily use Objective-C, with small amounts of AppleScript as necessary.

The reason I went down this road is that I was building a search action for DEVONagent. DEVONagent is a very cool program that allows you to search using multiple search engines, following links along the way, and aggregating the results. It then extracts the common key topics from the web pages it searches and builds clustering diagrams based on the results. Here's an example based on a search for “Apple WWDC”

DEVONagent showing search results

DEVONagent comes with some useful actions out of the box, but it does not come with an action to initiate a search(!), so I decided to write one. One of the main features of the program is its use of “search sets” and plugins to allow searches to be customized. That search set list can be dynamic - users can develop their own custom search sets for specialized sites or for any reason whatsoever.

Therefore I wanted my UI for my action to reflect the current list of search sets available in the program. So, I needed to be able to query DEVONagent for its list of search sets, and then populate a popup menu in the action's interface with the current list. Then, I wanted the selected search set to be used to perform the search.

To me, querying the list of search sets, and executing the search sound like jobs for AppleScript, and populating a user interface popup menu sounds like a job for Cocoa bindings and Objective-C. I believe that this type of action is possible to write completely in AppleScript, but at this point my Obj-C is much stronger than my Applescript so I decided to call AppleScript from my Cocoa based action.

The first step for me was to take a look at DEVONagent's scripting dictionary to see if it could do what I wanted it to do. The program has a pretty extensive scripting dictionary and I quickly found what I needed using the Script Editor application. Here's the application dictionary showing search sets:

DEVONagent "application" object dictionary

And here's the small script I wrote to query the app for the list of search sets:

AppleScript to list search sets

Executing a search was similarly easy. Here's the search command in the dictionary:

And here's the script to run a search. I chose to search for “Apple WWDC” for which I've already shown the results.

Now, I have to build the action. Starting XCode, I'll choose “Cocoa Automator action” from the project list and call it “Perform Search With Specified Text” The name is long because the project name is what shows up in Automator and it needs to be descriptive. This action is intended to use text fed in as input from the previous action as the search terms and search using whatever search set the user selects.

The project defines a class for the action that is a subclass of AMBundleAction. There is also a nib file, which presents the interface for the action. Let's start with our Objective-C code. The class predefines a method called -(id)runWithInput:(id) fromAction:(AMAction *)anAction error:(NSDictionary **)errorInfo. We've added one getter method to return the list of search sets so we can bind the popup in the GUI to something. Here's the header for the class:

#import <Cocoa/Cocoa.h>
#import <Automator/AMBundleAction.h>

@interface Perform_Search_with_Specified_Text : AMBundleAction
- (NSArray *)searchSets
- (id)runWithInput:(id)input
fromAction:(AMAction *)anAction
error:(NSDictionary **)errorInfo;


Here's the implementation:

#import “Perform Search with Specified Text.h“
#import “DevonAgent.h“

@implementation Perform_Search_with_Specified_Text

static DevonAgent *devonAgent = nil;

I've created a DevonAgent class to encapsulate all interaction with the application. I suspect that at some time in the future the need to write this class will go away.

- (void)awakeFromBundle{
if( devonAgent == nil ){
devonAgent = [[DevonAgent alloc] init];
[[self parameters] setValue:[devonAgent defaultSearchSet]
[self parametersUpdated];

-awakeFromBundle is called when this class is loaded, similarly to how -awakeFromNib is called when a class is instantiated out of the nib. This is where we connect to the DevonAgent class. We also set the initial selectedSearchSet parameter so that the user interface will reflect the selection. You call -parametersUpdated to tell the user interface to update its configuration based on the parameters.

- (NSArray *)searchSets{
return [devonAgent searchSets];

When the interface needs the search sets, we go ask for it from the app.

- (id)runWithInput:(id)input
fromAction:(AMAction *)anAction
error:(NSDictionary **)errorInfo
NSString *selectedSearchSet =
[[self parameters] valueForKey:@“selectedSearchSet“];

At this point we grab our parameters and get the “selectedSearchSet” parameter. I'll talk about how the parameters get set in a little while. Suffice it to say at this point that there are parameters and that these are set by selecting controls in the GUI and they are populated by Cocoa Bindings in the nib file.

[devonAgent performSearchFor:input
return input;


Now, runWithInput:fromAction:error is called when the action is run. We simply grab the currently selected search set out of the parameters for this action and launch the search. We're not returning the results of the search in this action. When DEVONagent runs, we can tell it to run a script when the action finishes (it can take quite a while for these searches to run, depending on the settings) and at that point we'd want to launch the next phase of the workflow.

This part of the action is pretty straighforward. Here's the DevonAgent class interface:

#import <Cocoa/Cocoa.h>

@interface DevonAgent : NSObject {
- (NSString *)defaultSearchSet;
- (NSMutableArray *)searchSets;
- (void)performSearchFor:(NSString *)search
usingSearchSet:(NSString *)searchSet
error:(NSDictionary **)errorInfo;

And the implementation:

@implementation DevonAgent

- (NSString *)defaultSearchSet{
return @“Internet (Fast Scan)“;

This method is used to make sure that there is a selected search set when the interface is first populated. This is a good default.

- (NSMutableArray *)searchSets{
NSMutableArray *searchSets =
[[[NSMutableArray alloc] init] autorelease];
NSString *script =
@“tell application \“DEVONagent\“ to return search sets“;
NSAppleScript *getSearchSets =
[[[NSAppleScript alloc] initWithSource:script]
NSDictionary *error;
NSAppleEventDescriptor *result =
[getSearchSets executeAndReturnError:&error];
int i;
int items = [result numberOfItems];
//Descriptor list index is one based!
for( i = 1; i <= items; i++ ){
NSString *setName =
[[result descriptorAtIndex:i] stringValue];
[searchSets addObject:setName];
return searchSets;

The searchSets method executes the AppleScript code:

tell application 'DEVONagent' to return search sets

This is equivalent to but shorter than:

tell application 'DEVONagent'
return search sets
end tell

The result of executing an AppleScript program is an NSEventDescriptor object which contains the results of the script. In this case, it contains the list of names of all the current search sets. Because this will get called as soon as the action is dragged out into the workflow, you shouldn't be surprised to see DEVONagent launch itself at that time. One thing that is important to remember is that the index in a NSAppleEventDescriptor list starts at one, not zero as you might expect.

- (void)performSearchFor:(NSString *)search
usingSearchSet:(NSString *)searchSet
error:(NSDictionary **)errorInfo{
NSString *script;
NSString *searchTemplate =
@“tell application \“DEVONagent\“ to search \“%@\“\
using set \“%@\““;
script = [NSString stringWithFormat:searchTemplate,
search, searchSet];
NSAppleScript *as =
[[NSAppleScript alloc] initWithSource:script];
[as executeAndReturnError:errorInfo];
[as release];


This method executes the search.

On to the nib. When you open the MainMenu.nib file for your action you will see a view with nothing but a placeholder string. The nib file also contains a controller object called “Parameters.” This controller is used to pass parameters from the interface to the script. In the case of the Cocoa action, the AMBundleAction class has a -parameters method that returns an NSMutableDictionary containing the parameters defined in the nib.

The first thing I'll do here is to build my view. It's pretty simple, a string and an NSPopUpButton.

We will bind the "contentValues" of the NSPopUpButton to the “searchSets” key of the file's owner, which is our action class. This will make sure that the list reflects the search sets in the application. We bind the “selectedValue” to the Parameters object controller selection.selectedSearchSet to make sure that the parameters get the updated value.

That's all that's necessary for the nib file.

There's only two more steps. First, we have to edit the info.plist and infoPlist.strings to define our inputs and outputs and to fill in the description window in Automator. To do that, we will select Project > Edit Active Target “Perform Search with Specified Text” and select the Properties tab. Here are my settings for each of the panes of this dialog box that I modified:

I don't edit the description here in this dialog box since the localized ones in infoPlist.strings seem to override them. Here's what the action looks like in Automator once all the documentation has been filled in.

The last step is testing. One of the great things about developing automator actions is that they are very easy to debug. Whether you are developing in Objective-C or Cocoa, the XCode debugger will launch automator for you, load your action into the program, and then when you run your workflow you can debug the action just like any other program.

Well, that's how I built a Cocoa based Automator action that uses AppleScript to communicate with one specific commercial app. The action uses AppleScript to extract information from a program and to control its operations. Cocoa and Cocoa bindings are used to populate a popup list with dynamic information from the app. I hope that you found this interesting and informative. Until next time, happy coding.


has said...

Good read. Automator programming is one of those things I keep meaning to learn, but have never gotten round to; perhaps when version 2.0 comes out. (Incidentally, have you thought of doing an article for somewhere like O'ReillyNet or MacTech? I'm sure you could.)

One slight problem I did notice - you need to be careful when doing code generation stuff like this:

NSString *searchTemplate =
@“tell application \“DEVONagent\“ to search \“%@\“ using set \“%@\““;
script = [NSString stringWithFormat:searchTemplate,
search, searchSet];

If the search or searchSet strings contain any unescaped backslashes or double quotes, things could get nasty (typically, the script will simply fail to compile, although in theory you could get arbitrary code executed).

One solution would be to do your own escaping: convert these strings to NSMutableStrings and replace any occurrences of @"\\" with @"\\\\" and @"\"" with @"\\\"". For a simple task like this one, this ought to be adequate.

Another option would be to pass the strings to the AppleScript as arguments to the 'run' command, which basically involves building an 'aevtoapp' Apple event using NSAppleEventDescriptor and packing a list of arguments as its direct parameter. Stick an 'on run {arg1, arg2, ...} ... end run' handler in your script, and you're good to go. (That said, this approach is usually only worth doing for large or user-defined scripts, as it's not much simpler than just sending an event directly to the target application.)

A third option is to bypass AppleScript and speak directly to the Apple Event Manager, either via its C API (e.g. see the AEBuild* suite of functions, which are not too hard to use), or via an ObjC bridge. Apple have already announced their own bridge for Leopard, though I've not used it myself so can't say how good it is. Or you could use the ObjC version of my appscript bridge - it's not quite finished but is already very usable and it also works on 10.3 and 10.4.


Paul Franceus said...


Thanks for the comments. Yeah, the security/robustness issues are certainly important. I probably should have added some code to handle those sorts of things, although the issues are perhaps lesser in this case than they would be if this was running in a public web site or something.

Adding the run handler is an excellent idea. I'm still just getting my feet wet with AppleScript and so any pointers are welcome.

I was aware of the Leopard bridge (thus my comment about writing the DevonAgent class not being necessary in the future) but I hadn't seen your appscript bridge until I had already published the article. I'll take a look at it and perhaps post a follow up.

I think Automator 2.0 is going to be much more useful than the original - the addition of things like variables and looping will make it a much more easily useful and powerful tool. I'm really looking forward to it.

As far as the article for O'Reilly, I tried to go that route before I started the blog, but then figured I'd just publish it myself. I'd love to do it, perhaps I'll contact them again.

Thanks for the comments!


has said...

"Adding the run handler is an excellent idea. I'm still just getting my feet wet with AppleScript and so any pointers are welcome."

Here's a basic example (with apologies for lame formatting; Blogger won't allow pre/code tags in comments):

#import <Foundation/Foundation.h>
#import "Appscript/Appscript.h"

int main (int argc, const char * argv[]) {
NSAutoreleasePool * pool = [[NSAutoreleasePool alloc] init];

// Create AEMApplication object for building AEMEvents
AEMApplication *app = [[AEMApplication alloc] init];

// Create AEMCodecs object for unpacking script results
AEMCodecs *codecs = [[AEMCodecs alloc] init];

// Build parameter list for script
NSArray *params = [NSArray arrayWithObjects: @"foo", @"bar", nil];

// Create 'run' AEMEvent
AEMEvent *evt = [app eventWithEventClass:'aevt' eventID:'oapp'];

// Add parameter list to AEMEvent
[evt setParameter:params forKeyword:'----'];

// Get an NSAppleEventDescriptor from AEMEvent
NSAppleEventDescriptor *evtDesc = [evt appleEventDescriptor];

// Compile (or load) AppleScript
NSAppleScript *scpt = [[NSAppleScript alloc] initWithSource:
@"on run {arg1, arg2}\n return arg1 & arg2\n end"];

// Send NSAppleEventDescriptor to AppleScript
NSDictionary *error;
NSAppleEventDescriptor *resDesc = [scpt executeAppleEvent:evtDesc error:&error];

// Unpack script result
id res = [codecs unpack:resDesc];
NSLog(@"Result = %@", res); // Result = foobar

[scpt release];
[codecs release];
[app release];
[pool release];
return 0;

I've used aem (part of appscript) for convenience, but you can pack and unpack your own NSAppleEventDescriptors if you prefer; it's just a few more lines of code. I've also omitted error handling code for brevity, but I'm sure you can add that if you need it.

Like I say, this approach is more useful for large or user-supplied scripts - for just a single command, it's not much more effort to create an Apple event that you can send directly to the application. Or even less if you use appscript, e.g.:

DAApplication *dag = [[DAApplication alloc] initWithName:@"DEVONagent.app"];

DASearchCommand *cmd = [[dag search:searchStr] usingSet:searchSet];

id res = [cmd send];
if (res) {
NSLog(@"Result: %@", res);
} else {
NSLog(@"Error: %i %@", [res errorNumber], [res errorString]);

[dag release];

(Not complete or tested since I don't have DEVONagent myself, but you get the idea.)