Mar 28, 2008

Possible Interface Builder bug with QCView in Leopard


A friend of mine was trying to embed a Quartz Composer composition into a Cocoa app, and he was having problems getting it to work properly. The composition responded to mouse events and it worked just fine within QC, but for some reason not within a QCView. I took a look at the project and he had done everything right, as far as I could tell; the "ForwardAllEvents" checkbox was checked. 

So, I decided to perform a little experiment. I added a view controller to the application with the QCView as an outlet.

Lo and behold, the eventForwardingMask was set to 1, not NSAnyEventMask as it should have been. The solution then presented itself: Set this value correctly in the -awakeFromNib and now the application worked properly
Here's the header file:

#import <cocoa/cocoa.h>
#import <quartzcomposer/qcview.h>

@interface MyViewController : NSObject {
IBOutlet QCView *myView;
}

@end

And here's the implementation:

#import "MyViewController.h"

@implementation MyViewController

- (void)awakeFromNib
{
NSLog(@"%d", [myView eventForwardingMask]);
[myView setEventForwardingMask:NSAnyEventMask];
NSLog(@"%d", [myView eventForwardingMask]);
}

@end

I'm going to file a bugreporter bug right away.


Jul 23, 2007

Breaking news.

Greetings to all. Wow, it's been quite a while since I've blogged and much has happened since then. I still plan on updating this with some more AppleScript stuff and some other things I've been getting interested in, but I've just been too busy. Stay tuned for more.

The most important news is that I've accepted a position at Google. Pretty exciting for me to be moving from my remote village here in Baltimore to the heart of the Silicon Valley. How this all happened is that I got an email from a Google recruiter through my profile on linkedin.com (I highly recommend this site to everyone), and figured, sure, what the heck, I'll talk to you. After two phone interviews I got a rejection email while I was at WWDC. A week later, I got a call from another recruiter saying that they wanted to bring me in for a live interview. Flew out and had a day of discussion and answering lots of questions and here I sit, offer in hand. I'm going to start there in the end of September. I figure that I've been with my current company for a long long time and it's time for something different. East coast to west coast, Government contractor to commercial search engine giant. Should be fun.

Anyway, it's been a bit of a busy couple of months.

Jun 6, 2007

Heading to WWDC.

Well, I'm very excited this year to be off to WWDC again. Blogging this year has been a pleasure and I hope that I can meet some of the people that have made this a worthwhile exercise for me.

Thanks,

Paul Franceus

Jun 3, 2007

Automator - Cocoa based action with AppleScript integration

I've built a bunch of Automator actions lately. Most of these have been simple AppleScript actions, with a few Shell based actions thrown in. I'll probably talk about some of those in future postings.

Right now I want to talk about one particular action I built that incorporated a combination of AppleScript and Objective-C code. There are a couple of ways to build such a “hybrid” action as Apple's documentation calls it. One way is to use primarily AppleScript, with added Objective-C classes that are called from the AppleScript. The other way is to primarily use Objective-C, with small amounts of AppleScript as necessary.

The reason I went down this road is that I was building a search action for DEVONagent. DEVONagent is a very cool program that allows you to search using multiple search engines, following links along the way, and aggregating the results. It then extracts the common key topics from the web pages it searches and builds clustering diagrams based on the results. Here's an example based on a search for “Apple WWDC”

DEVONagent showing search results

DEVONagent comes with some useful actions out of the box, but it does not come with an action to initiate a search(!), so I decided to write one. One of the main features of the program is its use of “search sets” and plugins to allow searches to be customized. That search set list can be dynamic - users can develop their own custom search sets for specialized sites or for any reason whatsoever.

Therefore I wanted my UI for my action to reflect the current list of search sets available in the program. So, I needed to be able to query DEVONagent for its list of search sets, and then populate a popup menu in the action's interface with the current list. Then, I wanted the selected search set to be used to perform the search.

To me, querying the list of search sets, and executing the search sound like jobs for AppleScript, and populating a user interface popup menu sounds like a job for Cocoa bindings and Objective-C. I believe that this type of action is possible to write completely in AppleScript, but at this point my Obj-C is much stronger than my Applescript so I decided to call AppleScript from my Cocoa based action.

The first step for me was to take a look at DEVONagent's scripting dictionary to see if it could do what I wanted it to do. The program has a pretty extensive scripting dictionary and I quickly found what I needed using the Script Editor application. Here's the application dictionary showing search sets:

DEVONagent "application" object dictionary

And here's the small script I wrote to query the app for the list of search sets:

AppleScript to list search sets

Executing a search was similarly easy. Here's the search command in the dictionary:

And here's the script to run a search. I chose to search for “Apple WWDC” for which I've already shown the results.

Now, I have to build the action. Starting XCode, I'll choose “Cocoa Automator action” from the project list and call it “Perform Search With Specified Text” The name is long because the project name is what shows up in Automator and it needs to be descriptive. This action is intended to use text fed in as input from the previous action as the search terms and search using whatever search set the user selects.

The project defines a class for the action that is a subclass of AMBundleAction. There is also a nib file, which presents the interface for the action. Let's start with our Objective-C code. The class predefines a method called -(id)runWithInput:(id) fromAction:(AMAction *)anAction error:(NSDictionary **)errorInfo. We've added one getter method to return the list of search sets so we can bind the popup in the GUI to something. Here's the header for the class:

#import <Cocoa/Cocoa.h>
#import <Automator/AMBundleAction.h>

@interface Perform_Search_with_Specified_Text : AMBundleAction
{
}
- (NSArray *)searchSets
- (id)runWithInput:(id)input
fromAction:(AMAction *)anAction
error:(NSDictionary **)errorInfo;

@end

Here's the implementation:

#import “Perform Search with Specified Text.h“
#import “DevonAgent.h“

@implementation Perform_Search_with_Specified_Text

static DevonAgent *devonAgent = nil;

I've created a DevonAgent class to encapsulate all interaction with the application. I suspect that at some time in the future the need to write this class will go away.

- (void)awakeFromBundle{
if( devonAgent == nil ){
devonAgent = [[DevonAgent alloc] init];
}
[[self parameters] setValue:[devonAgent defaultSearchSet]
forKey:@“selectedSearchSet“];
[self parametersUpdated];
}

-awakeFromBundle is called when this class is loaded, similarly to how -awakeFromNib is called when a class is instantiated out of the nib. This is where we connect to the DevonAgent class. We also set the initial selectedSearchSet parameter so that the user interface will reflect the selection. You call -parametersUpdated to tell the user interface to update its configuration based on the parameters.

- (NSArray *)searchSets{
return [devonAgent searchSets];
}

When the interface needs the search sets, we go ask for it from the app.

- (id)runWithInput:(id)input
fromAction:(AMAction *)anAction
error:(NSDictionary **)errorInfo
{
NSString *selectedSearchSet =
[[self parameters] valueForKey:@“selectedSearchSet“];


At this point we grab our parameters and get the “selectedSearchSet” parameter. I'll talk about how the parameters get set in a little while. Suffice it to say at this point that there are parameters and that these are set by selecting controls in the GUI and they are populated by Cocoa Bindings in the nib file.

[devonAgent performSearchFor:input
usingSearchSet:selectedSearchSet
error:errorInfo];
return input;
}

@end

Now, runWithInput:fromAction:error is called when the action is run. We simply grab the currently selected search set out of the parameters for this action and launch the search. We're not returning the results of the search in this action. When DEVONagent runs, we can tell it to run a script when the action finishes (it can take quite a while for these searches to run, depending on the settings) and at that point we'd want to launch the next phase of the workflow.

This part of the action is pretty straighforward. Here's the DevonAgent class interface:

#import <Cocoa/Cocoa.h>

@interface DevonAgent : NSObject {
}
- (NSString *)defaultSearchSet;
- (NSMutableArray *)searchSets;
- (void)performSearchFor:(NSString *)search
usingSearchSet:(NSString *)searchSet
error:(NSDictionary **)errorInfo;
@end

And the implementation:

@implementation DevonAgent

- (NSString *)defaultSearchSet{
return @“Internet (Fast Scan)“;
}


This method is used to make sure that there is a selected search set when the interface is first populated. This is a good default.


- (NSMutableArray *)searchSets{
NSMutableArray *searchSets =
[[[NSMutableArray alloc] init] autorelease];
NSString *script =
@“tell application \“DEVONagent\“ to return search sets“;
NSAppleScript *getSearchSets =
[[[NSAppleScript alloc] initWithSource:script]
autorelease];
NSDictionary *error;
NSAppleEventDescriptor *result =
[getSearchSets executeAndReturnError:&error];
int i;
int items = [result numberOfItems];
//Descriptor list index is one based!
for( i = 1; i <= items; i++ ){
NSString *setName =
[[result descriptorAtIndex:i] stringValue];
[searchSets addObject:setName];
}
return searchSets;
}


The searchSets method executes the AppleScript code:


tell application 'DEVONagent' to return search sets


This is equivalent to but shorter than:


tell application 'DEVONagent'
return search sets
end tell



The result of executing an AppleScript program is an NSEventDescriptor object which contains the results of the script. In this case, it contains the list of names of all the current search sets. Because this will get called as soon as the action is dragged out into the workflow, you shouldn't be surprised to see DEVONagent launch itself at that time. One thing that is important to remember is that the index in a NSAppleEventDescriptor list starts at one, not zero as you might expect.


- (void)performSearchFor:(NSString *)search
usingSearchSet:(NSString *)searchSet
error:(NSDictionary **)errorInfo{
NSString *script;
NSString *searchTemplate =
@“tell application \“DEVONagent\“ to search \“%@\“\
using set \“%@\““;
script = [NSString stringWithFormat:searchTemplate,
search, searchSet];
NSAppleScript *as =
[[NSAppleScript alloc] initWithSource:script];
[as executeAndReturnError:errorInfo];
[as release];
}

@end


This method executes the search.

On to the nib. When you open the MainMenu.nib file for your action you will see a view with nothing but a placeholder string. The nib file also contains a controller object called “Parameters.” This controller is used to pass parameters from the interface to the script. In the case of the Cocoa action, the AMBundleAction class has a -parameters method that returns an NSMutableDictionary containing the parameters defined in the nib.

The first thing I'll do here is to build my view. It's pretty simple, a string and an NSPopUpButton.

We will bind the "contentValues" of the NSPopUpButton to the “searchSets” key of the file's owner, which is our action class. This will make sure that the list reflects the search sets in the application. We bind the “selectedValue” to the Parameters object controller selection.selectedSearchSet to make sure that the parameters get the updated value.

That's all that's necessary for the nib file.

There's only two more steps. First, we have to edit the info.plist and infoPlist.strings to define our inputs and outputs and to fill in the description window in Automator. To do that, we will select Project > Edit Active Target “Perform Search with Specified Text” and select the Properties tab. Here are my settings for each of the panes of this dialog box that I modified:



I don't edit the description here in this dialog box since the localized ones in infoPlist.strings seem to override them. Here's what the action looks like in Automator once all the documentation has been filled in.

The last step is testing. One of the great things about developing automator actions is that they are very easy to debug. Whether you are developing in Objective-C or Cocoa, the XCode debugger will launch automator for you, load your action into the program, and then when you run your workflow you can debug the action just like any other program.

Well, that's how I built a Cocoa based Automator action that uses AppleScript to communicate with one specific commercial app. The action uses AppleScript to extract information from a program and to control its operations. Cocoa and Cocoa bindings are used to populate a popup list with dynamic information from the app. I hope that you found this interesting and informative. Until next time, happy coding.

May 17, 2007

Heading down the road with Automator and AppleScript

Lately, I've been working with a large customer to prototype the use of Macs to see if they are a viable alternative to the Windows based platform that they have standardized on in the past. You and I know this to be the case, but they need some convincing. One of the most compelling things for the users at this customer appears to be the use of Automator to automate their individual workflows.

First of all, the users I am dealing with are not programmers, but somehow when they see Automator they immediately get how it could help them get their jobs done. The problem I've had is that a lot of what these folks do involves querying multiple web sites and passing data from the results of one query to the next.

Certainly, any specific workflow could be implemented in code by a developer, but what I really want to do is to figure out a way to provide them with the tools they need to be able to build their own custom workflows.

So, this has gotten me to finally get off my tail and learn AppleScript and Automator. As a developer, it seemed to me that Automator was nothing more than a flashy toy that would be difficult to get any real work done. But perhaps the truth was that I, with years of C, C++, Java, JavaScript, Objective-C, Perl and Python and my programmer's brain, am not the real target audience for this tool. And AppleScript has always seemed to be a very weird language with strange syntax and poor documentation. Heck some of the best programmers I know have stayed away.

Well, necessity is a good teacher, and so lately I have been developing a lot of Automator actions to fill in the gaps I've seen in the built-in actions and third party additions. I've also become interested in how I can use GUI scripting to enable users to more easily help themselves.

Over the next several posts, I'll share some of my actions, talk about finally getting my head around AppleScript, and head down the GUI scripting road. Perhaps some of you will share some of your own experiences too. It would be great to hear from you.

Here are some good links I've found regarding Automator and AppleScript:

Automator:

http://developer.apple.com/macosx/automator.html
http://automator.us
http://automatorworld.com
http://automatoractions.com

AppleScript:

http://applescriptsourcebook.com/
http://developer.apple.com/applescript/
http://macscripter.net/
AppleScript: The Definitive Guide, 2nd Edition

May 12, 2007

Bicycle commuting around Baltimore

I'm a pretty avid cyclist, and I enjoy commuting to work on my bike. For the past couple of years, I’ve been attempting to compile a list of routes that I use to get around the Baltimore area on my bike. I’d rather avoid driving my bike somewhere to ride, so I’m always looking for ways to jump on my bike at my back door. It’s a little more difficult to find good routes for a city dweller like myself, so I'll be posting them here from time to time in case anyone is interested.


The first 18 miles of this route represent my standard commuting route for the past couple of years. It starts out on the Gwynns Falls Trail, heads out Frederick Rd towards Catonsville, Once through Catonsville, I head south on Hilltop Rd, across the Patapsco river and up Ilchester Rd towards Columbia. The big challenge of this ride comes around mile 11. Heading out, you have to climb an 18 percent grade up the first part of Ilchester. The way home has a similar climb up Hilltop. A more scenic and easier climb up Bonnie Branch Rd. is also an alternative. Normally I’ll ride home the same way, but on weekends I often take the loop route home. The only major difficulty is dealing with the I 95 interchange off of 175, which is much lighter in traffic on the weekend. Mileage to work is about 18, total milage is around 41.

Apr 29, 2007

Cocoa Application with custom Core Image filter 6: Embedding a Quartz Composer composition directly in an app.

In the last episode, we built a Cocoa application to filter an image using Convolution. This application followed a more traditional path to developing a Cocoa application. We took an image from our view, handed it to the model to be processed and sent it back to the view for display. With the advent of Quartz Composer, this application can be implemented in an entirely different way that opens up all kinds of interesting possibilities for interactive graphics applications. I'm sure all of you have seen that Quartz Compositions can respond to external inputs. Quartz Composer can take input from keyboard and mouse, spotlight image searches, folders of images, and even RSS feeds. Apple's RSS screensaver is a prime example of a Quartz Composition that uses RSS input.

An other way of feeding input and getting output from a composition is by publishing input and output parameters from your compositions. In the most simple sense, these will allow Quartz Composer itself to present a simple UI to allow you to interact with your composition. For example, in my test.qtz composition, I publish the inputImage and all the coefficients so that QC can provide a simple user interface.

More interesting than this, however, is the use of a composition embedded within a QCView and accessing its parameters using Cocoa bindings technology and the QCPatchController. I've provided a link to this project so you can see how I did it: Convolver.zip

The first thing you need to do to use these in your projects is to add the Quartz Composer palette to Interface Builder. To do this:

Open Interface Builder Preferences:

IB preferences showing palettes tab

Click the Add... button:

selecting the QuartzComposer palette

You will get this new palette in Interface Builder.

the Quartz Composer palette

Next, we need to copy the test.qtz composition into the bundle for our application. It doesn't really matter where you put it. For our purposes, we just need to make sure that it is packaged with the app when it ships. After that we need to open the composition and make sure that we publish all the inputs that we care about so that the QCPatchController can access them. In order to do this, you have to right-click on whatever patches in you composition have external inputs or outputs and check off each one you wish to publish. As far as I've been able to tell, there are no shortcuts to this, you have to select each one individually.

Publishing inputs from the compostion

Whatever you name the inputs or outputs at this point is the key that will be used to access that parameter later.

Now that we have a properly set up composition, we have to modify the original project to use the QCView and QCPatchController instead of old-school target/action. First thing to do is to replace the outputImage NSImageView on the right of the main window of the app with a QCView. Then, drag a QCPatchController into the nib file. I set the erase color of the QCView to transparent to avoid the large black square on the right side of the application when it starts.

Next., we'll go ahead and set up the bindings for the project. The QCView's patch binding is bound to the patch key on the QCPatchController.


We set the value binding of the NSImageView to the patch of the QCPatchController with the src.value key path.


Finally, each cell of the NSMatrix gets bound to the correct value in the composition:


The final piece of this setup is to load the composition into the QCPatchController


Now, you can actually test the working application from within IB. If you choose File->Test Interface and drop an image on the left image view you will see the convolved output image displayed on the right. Of course, this is exploring the barest minimum of the possibly utility of Quartz Compositions and QCView.

This has greatly reduced the amount of Objective-C code we need to write to build this application. The entire Convolver class that converted the image to a CIImage, called the filter and reconverted the result back into an NSImage is not needed any more. I still want to be able to select File->Open... to drop down a sheet and open the image, so that's really the only thing I need the ConvolutionController to do at this point. There turns out to be one small catch that I'll point out when I get to it.

Here's the new header for the ConvolverController class:

/* ConvolverController */

#import <Cocoa/Cocoa.h>
#import <QuartzComposer/QuartzComposer.h>
#import <QuartzCore/QuartzCore.h>/span>


@interface ConvolverController : NSObject
{
IBOutlet QCView *resultImage;
IBOutlet NSImageView *sourceImage;
IBOutlet NSWindow *window;
}
- (IBAction)openImage:(id)sender;
@end
Some of you may look at this and wonder why I need a pointer to the resultImage any more since the bindings should take care of it. That's where the catch comes in, as you'll see below.

This is the source code for the class:

#import “ConvolverController.h“


@implementation ConvolverController

- (IBAction)openImage:(id)sender
{
NSOpenPanel *panel = [NSOpenPanel openPanel];
[panel beginSheetForDirectory: nil
file:nil
types:
[NSImage imageFileTypes]

modalForWindow: window
modalDelegate:self
didEndSelector:
@selector(openPanelDidEnd:returnCode:contextInfo:)

contextInfo:nil];
}

- (void)openPanelDidEnd:(NSOpenPanel *)panel
returnCode:(int)returnCode
contextInfo:(void *)contextInfo{

NSArray *files = [panel filenames];
NSString *filename = [files objectAtIndex:0];
NSImage *image = [[[NSImage alloc]
initByReferencingFile:filename]
autorelease];

[sourceImage setImage: image];
[resultImage setValue: image forInputKey: @“src“];
}
@end

This is normal sheet handling code, just like in the previous project, but a little simpler. The catch comes in when I have to call [resultImage setValue: image forInputKey: @“src“] even though you might think that the bindings for NSImageView should automatically be updated, they are not. Apparently it's an already filed bug and this one line of code provides a simple workaround.

So, that ends the journey of building a Cocoa app with a custom Core Image filter. Hope it was useful to you. This was a very simple example, there's so much more that can be done with these amazing technologies. Until next time, happy coding!