The Launchd Nightmare

My Mac application, MailSteward, has always had a feature allowing the user to schedule the app to archive their email in a database. To do this MailSteward uses an old standby in the UNIX system, called crontab, which has been around since the beginning of time, i.e., the creation of UNIX.

Now, running the beta of Mojave, MacOS 10.14, it doesn’t work anymore. So I am switching over to launchd, the Apple system for scheduling daemons and agents. What a nightmare! The crontab is simple and elegant. Launchd is incredibly complicated and has some of the worst documentation I have ever seen.

If you google something like “macOS cocoa schedule application with launchd”, you will get a list of links to very frustrated developers. I never was able to find a clear explanation of how the Hell to do it. There were a few quite confident how tos, but they used old commands that Apple has since deprecated and which don’t work anymore.

So I had to use the traditional computer science technique, trial and error. It only took a couple of days. Here is the code it takes to set up a simple schedule in launchd:

– (IBAction)schedule:(id)sender {
int rc;
NSRange aRange;
NSTask *task1;
NSTask *task2;
NSTask *task3;
NSTask *task4;
NSPipe *stdOutPipe = [ NSPipe pipe ];
NSFileHandle *stdOutReadHandle = [ stdOutPipe fileHandleForReading ];
NSMutableData *idData;
NSMutableArray *args = [ NSMutableArray array ];
NSFileManager *manager = [NSFileManager defaultManager];
NSMutableString *idString = [NSMutableString string];
NSMutableString *guiString = [NSMutableString string];
NSMutableString *dictAll = [NSMutableString string];
NSMutableString *dictWeekday = [NSMutableString string];
NSMutableString *junk = [NSMutableString string];
NSString *plistPath = [[NSBundle mainBundle] pathForResource:@”ms.schedule” ofType:@”plist”];
NSMutableString *plistFile = [NSMutableString stringWithContentsOfFile:plistPath encoding:NSUTF8StringEncoding error:NULL];
[dictAll setString:@”\nHour\n8\nMinute\n45\n\n”];
[dictWeekday setString:@”\nHour\n8\nMinute\n45\nWeekday\n1\n\n”];

[self savePrefButton:self];
[self getPrefs];

task1 = [[NSTask alloc] init];
[args removeAllObjects];
[args addObject:@”-u”];
[ task1 setCurrentDirectoryPath: @”.” ];
[ task1 setLaunchPath:@”/usr/bin/id”];
[ task1 setArguments: args ];
[ task1 setStandardOutput: stdOutPipe ];
[ task1 launch ];
[ task1 waitUntilExit ];
idData = [ [ NSMutableData alloc ] initWithData :[ stdOutReadHandle readDataToEndOfFile ] ];
[stdOutReadHandle closeFile];
idString = [[ NSMutableString alloc ] initWithData: idData encoding: [ NSString defaultCStringEncoding ]];
[task1 release];

if ( [schedAll state] ) {
aRange = [plistFile rangeOfString:@”StartCalendarInterval“];
aRange.location += 1 + aRange.length;
[plistFile insertString:dictAll atIndex:aRange.location];
[junk setString:@”Hour\n“];
[junk appendString:[NSString stringWithFormat:@”%d”,schedHour]];
rc = [plistFile replaceOccurrencesOfString:@”Hour\n8″ withString:junk options:NSBackwardsSearch range:NSMakeRange(0, [plistFile length])];
[junk setString:@”Minute\n“];
[junk appendString:[NSString stringWithFormat:@”%d”,schedMin]];
rc = [plistFile replaceOccurrencesOfString:@”Minute\n45″ withString:junk options:NSBackwardsSearch range:NSMakeRange(0, [plistFile length])];
//NSLog(@”plistFile = %@”,plistFile);
} else {

}
[junk setString:@”~/Library/LaunchAgents/mailsteward.schedule.plist”];
[junk setString:[junk stringByExpandingTildeInPath]];
if ( [manager fileExistsAtPath:junk] ) {
[manager removeItemAtPath:junk error:nil];
}
[manager createFileAtPath:junk contents:[NSData dataWithBytes:[plistFile UTF8String] length:[plistFile length]] attributes:nil];

task4 = [[NSTask alloc] init];
[guiString setString:@”gui/”];
[guiString appendString:idString];
[args removeAllObjects];
[args addObject:@”bootout”];
[args addObject:guiString];
[args addObject:junk];
[task4 setLaunchPath:launchctlCommand];
[task4 setArguments:args];
[task4 launch];
[task4 waitUntilExit];
[task4 release];

task2 = [[NSTask alloc] init];
[guiString setString:@”gui/”];
[guiString appendString:idString];
[args removeAllObjects];
[args addObject:@”bootstrap”];
[args addObject:guiString];
[args addObject:junk];
[task2 setLaunchPath:launchctlCommand];
[task2 setArguments:args];
[task2 launch];
[task2 waitUntilExit];
[task2 release];

[guiString appendString:@”/mailsteward.schedule”];
task3 = [[NSTask alloc] init];
[args removeAllObjects];
[args addObject:@”enable”];
[args addObject:guiString];
[task3 setLaunchPath:launchctlCommand];
[task3 setArguments:args];
[task3 launch];
[task3 waitUntilExit];
[task3 release];

}

True Love

I was reminded today of the time when I first met and fell in love with the C programming language, and the UNIX operating system, which is written in C and is an extension of the C philosophy. Make no mistake, C and UNIX are not just computer programs. They are guides to how to think.

Developed at Bell Labs in the 70s by Dennis Ritchie, Ken Thompson, and others, Unix has become the universal operating system. MacOS is based on it. Every web site you visit probably runs on Unix. It is difficult to imagine a time in the future when Unix will not be the basis of computing at every level, as it is now, from toasters to server farms. It ranks up there with the printing press for a reality-changing human invention.

I got out my old copy of the slim volume, The C Programming Language by Dennis Ritchie and Brian Kernighan. It’s a second edition from 1988. I think I’ll read it again. The writing is like the C language itself, clean, elegant, inevitable. Reading this book is therapy. Your mind comes out the other side reorganized.

From the copyright page:

“This book was typeset (pic|tbl:rqn|troff -ms) in Times Roman and Courier by the authors, using an Autologic APS-5 phototypesetter and a DEC VAX 8550 running the 9th Edition of the UNIX operating system.”

AAPL – $300 a Share

it is just beginning to dawn on the financial punditry, but will probably take another six months to a year to really sink in. Apple is a software and services ecosystem. The hardware, though fabulously profitable, is just a means to an end. The ecosystem is the business. When this becomes generally known, Apple stock will be worth at least $300 a share.

if you own the mobile market, the tablet market, and the laptop market, and all of the software and services market for all of those customers, then you don’t need to be the most wildass innovative company on the planet. You just need to keep doing what you’re doing, better and better.

Mac Code Signing Hell

Recently I got a text message from a friend and a long-time customer saying that when he launched MailSteward, an error message appeared saying that “MailSteward is an application from an unidentified developer”.

MailSteward is a Mac application that has been developed, maintained, upgraded, and successfully distributed on the internet for the past twelve years. Now he had to bypass the OS X Gatekeeper security in order to launch it.

This was news to me.

I have been using the latest Xcode for development. I have the box checked to “Automatically manage signing”. When I archive to produce a signed copy of the latest version of MailSteward, it compiles and links without any error messages. I have assumed that all was well.

Apparently MailSteward has not been properly signed since the introduction of MacOS Sierra (10.12), which may account for a sudden, significant drop in sales since Apple released Sierra last Fall.

I immediately ran the codesign utility in a terminal window to check the validity of MailSteward’s signing. It passed with flying colors.

I then ran the spctl utility just to make sure, but it returned “rejected”. I ran it in verbose mode, but “rejected” was all it would say.

I spent the rest of the day googling for suggestions, trying different options in Xcode and repeatedly running spctl — being rejected over and over. I finally hit on the magic combination of parameters that persuaded spctl to grudgingly yield the elusive “accepted” response.

The first thing I did was uncheck the “Automatically manage signing” box. Then I set out to choose the right combination of Provisional Profile, Team, and Signing Certificate. Don’t ask me what those things are. I have never been able to understand any of the Apple documentation explaining them.

The right combo must be chosen, not only for the application itself, but also for any embedded projects within the main project, such as one I use in MailSteward called MailCore.

Once I managed to get MailSteward accepted by spctl, I then had to do the same for MailSteward Lite and MailSteward Pro, each of them, though sharing a common code base with MailSteward, required different settings in order to be rendered acceptable.

Code signing Mac applications is one of the great mysteries, like the virgin birth and string theory. Release from code signing Hell is only possible through grace.

C’mon Phil

Looks like Phil Schiller is cleaning the Augean stables over at the Mac App Store. Approvals are happening in one day. Subscriptions are coming soon.

Could it possibly be that powerful applications, you know, apps that can actually do something, such as BBEdit, MailSteward, and many others, could maybe be allowed in the store, even though they are not sandboxed? Assuming of course that the developer is properly vetted.

After all, Apple gives itself a pass for its own applications.

Why does Apple not want to be involved in the market for Macintosh applications that are actually able to do something? It is insanity. I’m counting on Phil.

Amazon is a Software Company

I’ve always thought of Amazon as a web site design company. Of course they have warehouses and deals with everybody, but all of that is just infrastructure behind the web site. The Amazon web site is, and always has been, the best commercial web site in the world. Nobody else is close, except maybe B & H Photo. Everything works. It is constantly improving. It is the point of the spear.

Jeff Bezos understood from the very beginning that it was all about the design and implementation of the web site. At first the only product was books, but the real product was how easy it was to find books, and find out about them, and see what others thought about the books, and to buy them. Amazon is a software company, specifically a user interface and database software company.

It’s the Ecosystem, Stupid

Those who analyze Apple stock seem to assume that Apple sells a bunch of unrelated hardware and software products. They assume that if the competition, for example, comes out with a better smart phone at a lower price, then it’s all over for Apple. Thus Apple is valued as if it is always teetering on the edge of bankruptcy. They must constantly create mind-blowing new products. Fail to do so even once, and they are out of business.

This is incredibly stupid. Apple’s product is the entire unified ecosystem created by the combination of all of its hardware and software. Apple’s ecosystem is like the Hotel California. You can check out any time you like, but you can never leave. Once sucked in, you will purchase new Apple products annually for the rest of your life, and your children will do the same after you are gone. The number of people living in the Apple world only increases. The potential market is every man, woman, and child on planet Earth.

I have a MacBook Pro. My wife has a MacBook Pro. We both have iPhones. I have an iPad. She has an iPad mini. We don’t have Apple watches yet, but we probably will by next year. We also have an Apple Time Capsule and an Apple TV, and a couple of 27″ Apple monitors. We’re not going anywhere.

So far this year, we have bought new iPhones, the iPad mini, and her new MacBook Pro, somewhere around $4,000 worth. And we just ordered the new Apple TV. Next year, we’ll probably get the watches, and it’ll be time to finally upgrade my old iPad 2. Another $1,300 or so. Oh yeah, and we’ll probably get new phones as well. And so it goes. Forever. And I’m not even talking about cars.

Apple is one of the most, if not the most, secure bets of any company in the world, and it is treated by the analysts as one of the shakiest investments anywhere. I actually read an article making the point that if Apple sells too many iPhones, it will be bad news, because they will be moving that much closer to market saturation.

Apple announces earnings this evening. I sure hope they didn’t sell too many iPhones.

Apple Mail account names

The Mail application in El Capitan made a few changes. It moved all the email files and folders out of the V2 directory into the V3 directory. It also got rid of the Accounts.plist file that previously contained the display names for all of the Mail email accounts. And it changed the names of the email account folders in /Users/[your home folder]/Library/Mail/v3 from meaningful labels to meaningless hexidecimal character strings.

So I sent this tech support query to Apple:

I am the developer of MailSteward, an email management system that archives email into a relational database. Before El Capitan, I was able to access the email account display names from a file named Accounts.plist. That file no longer exists and I have been unable to find any source for those display names. As a result MailSteward is now using the folder names of the accounts in the V3 directory. However, with new accounts in El Capitan, these folder names are just strings of hexadecimal characters. Is there any way I can find the display names to associate with these hex account ids?

Apple replied:

…Our engineers have reviewed your request and have concluded that there is no supported way to achieve the desired functionality given the currently shipping system configureations…

It took awhile, but I finally figured out a way to provide meaningful display names for the Apple Mail email accounts listed in the MailSteward settings. Here is a piece of the objective C code that does the job, after first finding an email file in either the inbox or sent folders, from which I can extract an email address:

if ( [myFilePath containsString:@”Sent”] ) {
if ( [emailBlob containsString:@”Return-path: “] ) {
aRange = [emailBlob rangeOfString:@”Return-path: “];
aRange.location += 13;
} else {
aRange = [emailBlob rangeOfString:@”From: “];
aRange.location += 6;
}
} else {
if ( [emailBlob containsString:@”Envelope-to: “] ) {
aRange = [emailBlob rangeOfString:@”Envelope-to: “];
aRange.location += 13;
} else {
aRange = [emailBlob rangeOfString:@”To: “];
aRange.location += 4;
}
}
if ( aRange.length && [myFilePath hasSuffix:@”.emlx”] ) {
aRange.length = [emailBlob length] – aRange.location;
[myAcctName setString:[emailBlob substringWithRange:aRange]];
aRange.length = [myAcctName rangeOfString:@”\n”].location;
aRange.location = 0;
[myAcctName setString:[myAcctName substringWithRange:aRange]];
[myAcctName replaceOccurrencesOfString:@”\%40″ withString:@”@” options:0 range:NSMakeRange(0, [myAcctName length])];
}
if ( [myFilePath containsString:@”[Gmail]”] ) {
[myAcctName setString:@”Gmail”];
}

The Mac App Store Blues

We here at Pubblog have a number of products on the Mac App Store. It’s been great. It’s a great idea. Thank you Apple for providing this universal, integrated, easy to use, safe venue for independent software developers.

We also applaud The new Mountain Lion requirements for a developer certificate, which means that everyone, including Apple, knows who you are, and if your app does anything bad, it can be purged from everyone’s computer. This is an excellent idea, and one we are happy to comply with.

However, we have serious problems with the new, as of June 1st, sandboxing rules on the Mac App Store.

One of our products is called FileMyFiles. It has been on the Mac App Store pretty much from the beginning. One of its features is the ability to remove an application along with all of the files related to that application. Of course you must have Administrator privileges in order for it to work. This feature was rejected by the Mac App Store, so the App Store version does not have it. However the slightly more expensive version sold on the FileMyFiles web site still does include this feature.

There is also a feature for making file name and folder name changes to a collection of files and folders. Recently a bug that affected changing spaces to other characters, such as underscores and periods, was reported by a user. The bug was quickly fixed and tested, and a new version was loaded on the Pubblog Store, and submitted to the Mac App Store.

A week later, this bug fix to the App Store version was rejected, because of yet another FileMyFiles feature. The feature from which File My Files derives its name enables a user to organize all of the files in a particular folder, into separate subfolders according to file type. The user chooses the folder to organize, and the program then creates whatever subfolders are needed and moves the files into the appropriate subfolder. You can also unfile your files, which will put them all back where they came from and remove the subfolders that were created.

The bug-fixed app was rejected because the user does not specifically ask, via a standard Save As dialog, to create each of the file type subfolders, or to remove said folders. This, even though the user has already specifically requested that File My Files perform its tasks on a particular folder that the user has chosen.

This is a rule that is part of the new sandboxing requirements. It seems they have been put into effect, not only for new apps, but for existing App Store bug fix releases. However, all previous bug fixes have been accepted, and the current version of FileMyFiles with the unfixed bug has been allowed to remain in the store so far.

If this continues to be the policy, we will be forced to remove FileMyFiles from the App Store rather than have a version out there with a bug we are not allowed to fix.

Another app, MailSteward, uses the SQLite database engine, which creates a journal file as a database is being updated. Since the user has not authorized the creation of this file, even though it is in the same location as the user has chosen for the database file, it violates the sandboxing rules. So that app too cannot exist on the app store.

A new unreleased app downloads thousands of files into a folder chosen by the user. The sandboxing rules dictate that the user must authorize the downloading of each of the thousands of files, which would make the app useless. So this app also cannot be in the app store.

In short it looks like the new sandboxing rules will eliminate the majority of utility type programs which have anything to do with the file system. Or the app must be crippled and/or given a dumbed down user interface, for which the developer, not Apple, will be blamed.

I do not understand how these Mac App Store policies serve the customer, or Apple, or the developer, in any way.

Make Xcode 3.2.6 Create Universal Binary Mac Applications

Before Xcode 3.2.6, the default build configuration for an application was a universal binary that ran on both PPC and Intel Macs. Xcode 3.2.6, however, has a default of Intel only, and there is no universal binary option in the drop-down in the Build tab of the Project Settings.

When I did a new release of MailSteward recently, using Xcode 3.2.6 for the first time, the result, unbeknownst to me, was a 32 bit Intel only build, not a universal binary build. I didn’t expect Xcode to secretly change my project settings, so I didn’t notice what had happened.

Here is how to fix it. Select Other… from the Architectures drop-down in the Build tab of your Project Settings. You can then add your own architecture, as in this screen shot:

null

Your Architectures Setting will now look like this:

null

Then clean all targets and do a new build and the result will be a universal binary application.