Still Alive…

I’m still alive, but life has taken a big change (for the better).  A few months ago I started working for Apple as a software engineer.  It’s been a very hectic time for me (a new job and moving to a new city take a lot of time).  Not much more to say but I’m very happy how things are working out.

I’m trying to figure out what to do with this blog.  I’m not going to be able to post any more code things for the time being, that’s obvious (and that means that my iCloud tutorial is not going to be finished).  Likewise, I’m not about to make any comments about the tech industry, Apple or its competitors.  I don’t want to drop this blog entirely, though.

I love this industry, and I’m very excited about what’s going on.  It’s still the early days, everyone…

Acorn: Filter to save icons for Dropbox usage

I’m adding Dropbox support to my storyboarding application, and I use Acorn from Flying Meat Software to do most of my graphic work.  A while back Gus Mueller (Mr. Flying Meat) posted a plugin to save Acorn native images in both retina display format and in “regular” format.  To submit an app to Dropbox you should provide them with 16×16, 64×64, and 128×128 icons.  What I did was take Gus’s plugin and created one to save an Acorn native image in these formats for Dropbox.

As Gus shared his code, so I share my little swizzle on his work: (file: Save@Dropbox.jstalk)

// This script requires Acorn 3.0 or later, and goes in your
// ~/Library/Application Support/Acorn/Plug-Ins/ folder

function main(ciimage, doc, layer) {
  // make sure the file is saved, and make sure it's saved
  // as an Acorn file.
  if (![doc fileURL] || !([[doc fileURL] pathExtension] == "acorn")) {
    return;
  }

  var path1 = [[[doc fileURL] path] stringByDeletingPathExtension] + "-16.png"
  var path2 = [[[doc fileURL] path] stringByDeletingPathExtension] + "-64.png"
  var path3 = [[[doc fileURL] path] stringByDeletingPathExtension] + "-128.png"

  // save our 128/128 first.
  [doc scaleImageToWidth:128];
  var opts = {'uti': 'public.png', 'file': path3};
  [doc webExportWithOptions:opts];

  // scale our image to 64x64 size and save it.
  [doc scaleImageToWidth:64]
  var opts = {'uti': 'public.png', 'file': path2};
  [doc webExportWithOptions:opts];

  // Now save our 16x16 image.
  [doc scaleImageToWidth:16]
  var opts = {'uti': 'public.png', 'file': path1};
  [doc webExportWithOptions:opts];

  // undo the scale
  [doc undo];
}

Another point on “The Church of Market Share”

If you haven’t read Gruber’s post titled The Church of Market Share you should now before I continue.

There.  Finished? Good.

I agree with everything that Gruber wrote in this piece.  In fact, I’d like to dig a little into history to reinforce what he said.

Consider mobile app development in the early 2000′s.  The leading platform in terms of market share was Symbian (Nokia mostly, although some Ericsson and later Sony/Ericsson).  They were selling like hotcakes. In fact, the only competition that they had were a brick-sized and shaped phone from Qualcomm running Palm OS, another Palm OS phone from Samsung, the phone add-in from Handspring (again, Palm OS), and maybe the earliest Windows Mobile phones from a couple of vendors. Again, none of these could hold a candle to Symbian in terms of market share. In fact, the only competition really was between Symbian phones and non-phone Personal Digital Assistants (PDA’s), such as the Palm V, Handspring Visor and the Compaq iPaq, but even there, the device sales numbers were roundoff error compared to Symbian phones.  (Blackberry was just starting to take off so they don’t factor into this.)

So where did the developers migrate to?  Palm OS.  Pocket PC.  We weren’t against Symbian development, but the few hardy souls that ventured into those waters were very disappointed with respect to the number of sales.

Now realize that this was a pre-App Store environment.  Developers would either try to distribute their apps themselves, but, more often, we would use Electronic Sales Distributors (ESD’s) like PalmGear and Handango.  Users would have to know about these sites so that they could go and buy apps.  Buying apps was also a painful process, with an email going to the developer from the ESD and the developer sending the customer an activation code that they would have to enter by hand onto the screen.  Sometimes the email from the ESD to the developer got lost; sometimes the email from the developer to the customer; sometimes the customer had trouble getting the code in (if they could find the registration screen in the first place).

Getting back on point, most developers were building for Palm (especially for consumer-oriented apps) and Pocket PC/Windows CE (mostly for enterprise).  Some were making good money, but most were just scratching out a living.  Periodically, some new developer would ask one of us if it was worth developing for Symbian, but it never was a long discussion.

(I’m sure there were a few developers who had successful Symbian applications, but it wasn’t anywhere near the numbers that Palm and Pocket PC/Windows CE developers were seeing.)

Food for thought when people start talking about market share.

Sorry for the iCloud tutorial delay

I became very busy with life and work after my presentation at CocoaHeads NY, so I set this aside.  I’m now getting back into the swing of things, and I’m now getting back to this tutorial

Sorry for the delay.

(BTW, if you had noticed, the source code for the initial non-iCloud version of my tutorial program had disappeared.  During building the iCloud version I refactored the code significantly – to the point that the non-iCloud version of NotePlus was a different program.  I’ve now retrofitted the non-iCloud version so that it is in line with my updated code.  It’s better, trust me.)

iCloud 3: Data and Metadata, and an introduction to NotePlus

This is part 3 in my iCloud tutorial series – the previous article is iCloud 2: Setting up an application for iCloud

Before I start digging into the code, I thought I’d write about how devices communicate with iCloud – the types of data and metadata that are shared bet we the iCloud infrastructure and the various platforms.  There’s not a lot of information on this, so I thought it would be helpful to include this into the tutorial.

The types of data that are stored in iCloud

There are two types of data that you store in iCloud

  • Key-value objects (property lists)
  • Documents – files and directories

The following figure illustrates these types of data.

iCloud synchronization (Source: Apple.com)

Key-value storage

Key-value storage is designed for holding small amounts of data which you access through key-value bindings.  Think of this as a property list or dictionary – you ask for an object using a key and get the object (or nil if no object for that key exists).  A good use of this is for storing application settings for sharing between devices.

This is obviously not meant for storing a lot of data.  In fact, Apple limits it to 64 KB per application.  That’s kilobytes, not gigabytes.

Document Storage

Document storage is where most of the interesting storage is going on in iCloud.  Typically you store files that your user generates in the application.  For example, in NotePlus, this is the notes and audio files that are created when the user creates a note.

Let’s consider a single file that is in iCloud.  There are two components associated with this file:

  • Metadata: information about the file, such as its name, versioning information and the number of bytes of data in it;
  • Data: the actual contents of the file

The idea is that metadata is very small in relation to the overall data for the file, and that an application can easily contain all the metadata for all the files stored in iCloud.  Any changes to the metadata of a file on a device are pushed back to the iCloud servers. When you create a file on a device, iCloud will also pull that file to the server.

Conversely, on a device pulling a file is managed by the operating system and the application in tandem. You should assume that your application will request a file that is not on the device and then it will be pulled down (there are scenarios where the OS will pull down files, but this has not been documented by Apple). This is good in terms of managing the amount of space you need for an application; the cost is that now opening a file is an asynchronous operation with a noticeable lag.  This latency may be long enough that you cannot just put everything on hold until you have finished the operation – for example, if you are doing a file open and read in the middle of a multi-player game the lag might be long enough that your character gets killed while waiting.  Not good. The good thing is that your application will be able to determine if a file is local or not, and you, as the developer, can handle this accordingly.

Data and efficiency

iCloud initially uploads the entire file from a device.  When the user makes a change to the file, then iCloud sends only the changes back to the server to create a new version of the document.  The following illustrate this action:

 

The device wants to push a new version of a file to iCloud, ...

 

Changes in the file are identified

... Changes in the file are identified, ...

 

A new version of the file is created

... the changes are uploaded and a new version of the file is created.

 

NotePlus

(Note: The source code for the non-iCloud version of NotePlus is available by clicking this link.)

I’m not going to change gears and talk a little more about the NotePlus application. When I wrote NotePlus I tried to follow the Model-View-Controller (MVC) design pattern as best I could.  I have several data objects that represent the text of a note (NpNoteText) and the audio associated with the note (NpNoteAudio).  A class, NpNoteManager, provides an interface to get a list of note names and to manage notes by providing interfaces to add and remove them from the system.

The master (note list) screen for NotePlus (NpMasterViewController)

The NotePlus app is a Universal Master-Detail application.  On the iPad this is represented as a split-view controller; on the iPhone this is done by relying on the navigation controller mechanism to navigate to a note.  The two main screen objects are the master controller (NpMasterViewController) which displays the list of all the notes, and NpDetailViewController, which provides the interface for editing a note and recording and playing the audio associated with the note.  On the iPad one or both controller may be visible.

NotePlus Note Screen

Editing a note in NotePlus (NpDetailViewController)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

On NpMasterViewController’s navigation bar are buttons to create a new note and to edit the notes (which only supports deleting notes).  A user may also delete a note by swiping from left to right on the note in the list.  The advantage of metadata is obvious in an application like NotePlus.  Populating the list inNpMasterViewController only needs the metadata, not the full note information.

NpDetailController is the note editor.  This class contains a UITextView for editing the text of the note and a UIToolbar that has buttons for recording and playing audio.  It supports both landscape and portrait orientations.

One more data class I have is NpNoteConfiguration.  This is a simple class that keeps track of the last note the user opened. This is used mostly for remembering the user’s state when the user quits the application and returns.

Where do we go from here

Now that I have introduced the NotePlus application, I will start to update it to incorporate iCloud into the application.  The steps I will need to take are:

  • Determine if iCloud is available or not;
  • Access the metadata for all the files (for the master screen);
  • Load a note from iCloud storage (including resolving conflicts);
  • Save a note and synchronize it with iCloud.

Again, the source code for the initial version of NotePlus is available here.

Until next time!

Next: Determining if iCloud is available.

iCloud 2: Setting up an application for iCloud

This is part 2 in my iCloud tutorial series – the previous article is iCloud 1: An introduction to iCloud.

In this installment I’ll discuss how to set up an application to use iCloud. In this post I discuss the steps you need to do to set up iCloud support for your application (I do create an application in this posting, it just doesn’t do any of the “iCloud goodness” yet).

The things you need to do for setting up an application to use iCloud are:

  • Setting up an app ID in the Apple Provisioning Portal and configuring it to support iCloud;
  • Setting up the iCloud entitlements in your application;
  • Obtaining a provisioning profile for the application.

What is confusing is that these steps take place in different locations – the first is done using the provisioning portal on the developer web site using a web browser, the other is done in XCode (Note: I am using XCode 4.2.1 for this tutorial – you need to be using XCode 4.2 or later due to the need to support iOS 5 for iCloud).

In this tutorial I am going to focus on iOS devices.  You can build and test a lot of your application there, but Apple currently does not allow the simulator to support iCloud.  This means the only way you can test iCloud functionality is to run it on a real device, which means the device needs to be registered with the provisioning portal and enabled for development use.  I will not go over how to use the provisioning portal – if you need help on this the best place to go is the Apple iOS Dev Center at http://developer.apple.com/ios.

For this tutorial I will use an application that I have developed calle NotePlus. This is a simple note taking application that also allows you to record an audio file to go along with each note.  I have made the source code for the initial version of NotePlus available here (Note that this is a ZIP archive that contains all the source code for this application).

NotePlus is a universal application.  I’m doing this because the easiest to see if iCloud is working by testing with two devices, and I assume that most people will either have an iPad and an iPhone or iPod Touch tied to the same Apple ID or they’ll have an iPhone and an iPod touch tied to the same Apple ID.

The importance of the bundle identifier

Make sure that you note the value of the Bundle Identifier for NotePlus – this is the suffix for the App ID we use to identify this appellation (Note that my bundle identifier is com.beret.NotePlus – you will use a different bundle identifier based on your personal or company Apple Developer Portal setup). This is what we will use to enable iCloud for this application in the provisioning portal.

Setting up NotePlus for iCloud in the Provisioning Portal

iCloud is enabled on an app-by-app basis in the provisioning portal.  The current default value is that it is not configured, which makes sense when you think about it. I will configure the NotePlus application in the provisioning portal and configure iCloud during the process.

The steps are:

  • Go to the App ID section of the provisioning portal and click on the “New App ID” button;
  • Fill out the form for the new App ID, ensuring that you put the correct bundle identifier in the right place on the form, then click the “Submit” button.
New Application ID

New App ID form for the NotePlus app

  • Once you click submit you will see the App ID has been added to the list of App ID’s that you have created.  It should look like something in the file below.  You are not done, however.  If you look at the column for iCloud you will see that there is a yellow ball with the text “Configurable” next to it. Click on the blue Configure link to the right of the App ID entry so that we can configure this.
NotePlus App ID created in the provisioning profile

NotePlus App ID in the provisioning profile

  • When the configuration dialog opens in the browser, click on the “Enable for iCloud” checkbox (Note that you will get a dialog box that will inform you that all provisioning profiles that support this application will support iCloud).  Once you click through the information dialog click on the “Done” button.
Configure the NotePlus app ID for iCloud

Configuring the NotePlus App ID for iCloud

  • Now you are back at the list of App ID’s in the provisioning profile.  Note now that the NotePlus application has a green ball and the text “Enabled” in the iCloud column. Our job is done here, so we can go back to XCode and do the second part.
NotePlus is now iCloud enabled

NotePlus is now iCloud enabled

Getting a Provisioning Profile for NotePlus

Now that you have set up NotePlus for iCloud, you need to get a provisioning profile to use.  You can either generate one using XCode (in the Organizer) or you can create a new profile in the Provisioning Portal.  All this is done using the standard techniques for obtaining a provisioning profile.

Setting up entitlements for NotePlus

Once you have completed the above we need to set up the entitlements for NotePlus to allow it to use iCloud. Apple uses this as part of mapping the application to the iCloud storage, and it also allows the developer to manage the type of data that the developer wants to synchronize.  The developer can specify one or more directories/folders to enable for iCloud storage and also enable key-value storage.

Why would you want to manage this from an application standpoint?

  • iCloud storage is not an infinite resource.  Users get 5 GB of storage for free but this storage is shared between all their applications (excluding storage for items bought through the iTunes music storage).  This means that your application and other applications are trying to carve out a piece of this storage space for themselves.  Since this is obviously not coordinated, at some point in the future the user may run out of space in the cloud.  The options the user has are to either buy more storage space from Apple or to free up some space.  As an application developer you need to be mindful of what you store in iCloud.
  • iCloud does allow you to share data between applications (just not between users). You can specify the bundle ID’s for other applications you wish to use to share data.

Once you’ve unpacked the NotePlus application open it in XCode.  Once it has loaded select “NotePlus” in the “Targets” portion of the middle area of the screen.  You should see something like the following:

XCode project summary

NotePlus target summary information in XCode

Be sure to change the value of the “Identifier” field in the right-most section from com.beret.NotePlus to whatever bundle ID you have defined for your application in the provisioning portal.

Now scroll down the right-most section to the “Entitlements” section.  This is where you enable your entitlements.  You should see the following on your screen:

The Entitlements section for the NotePlus application before enabling iCloud

Now click on the checkbox at the top of the Entitlements section (the one labelled Enable Entitlements).  You should now see the following (only with your bundle ID in the various sections instead of com.beret.NotePlus - you did change it, didn’t you?):

After enabling the Entitlements for NotePlus

Notice that the act of checking the Enable Entitlement checkbox fills in the information for iCloud, including the iCloud key-value store information and the iCloud containers.  You can use the iCloud containers to add access to shared containers (where you can share data with other applications).  For this tutorial series I’m going to keep it simple and just use the defaults.

At this point, we’ve done all the infrastructure setup to enable NotePlus to use iCloud.  Of course, just enabling it doesn’t make data sharing happen – we need to add logic to the code to actually access the iCloud data.  That will happen in the next several tutorial postings.  ‘Til then…

Next: Data and Metadata

iCloud 1: An introduction to iCloud

Before talking about how to use iCloud, it helps to understand what it is.  Let’s start with a description from Apple’s iCloud web site:

“iCloud stores your music, photos, documents, and more and wirelessly pushes them to all your devices. Automatic, effortless, and seamless — it just works.”

To make it simpler: iCloud is synchronization.  It syncs your data between your iPhone and the cloud.  It syncs your data between your iPhone and your Mac (there is also some limited support for Windows). It is incorporated into iOS and OS X Lion in such a way that, when combined with other enhancements (such as document versioning) you can get access to the latest, most up-to-date version of your data.  The image below illustrates this flow.

iCloud

iCloud Synchronization (Source: Apple.com)

Note that in the above description I emphasized the word “your”.  As it is currently implemented, it is only data that you create or consume on your devices. Data access and storage is based on your Apple ID.  There is no mechanism for associating multiple Apple ID’s with your data.

Before iCloud, applications normally accessed data objects (files/documents/databases) that were stored locally on the device.  Sharing data between devices was typically done through an explicit user action, such as copy the file to and from a file server or web site.

What iCloud does is provide a series of interfaces that developers can use to integrate synchronizing this data with Apple’s iCloud servers.  The synchronization is a part of the operating system (OS) that developers can access using these application programming interfaces (API’s).  Some of the features of these interfaces include:

  • File upload/download;
  • Data streaming (for database objects);
  • Conflict resolution (there are changes in both the document on the server and the document locally that need to be reconciled).

Apple has done a really good job at integrating iCloud in such a way as to minimize the amount of change you need to make to your code, but there are some challenges that developers need to keep in mind.  For example, opening a document that is stored locally on the file system is fairly simple – you can easily search the file system for the file, and opening it is so fast that you can typically wait for the file to open.  With iCloud, you now have to deal with network latency issues.  Opening a file now might entail downloading the document from the iCloud servers.  You now are at the mercy of the speed of the network and the size of the document.

iCloud versus Dropbox

A lot of developers are incorporating services like Dropbox into their applications to enable synchronization and sharing of information (there are other services, such as Box.net, that some use as well, but Dropbox seems to be the most popular as of this writing, so that is where I will focus).

Dropbox provides file-level syncing between devices and the Dropbox cloud. Applications can take advantage of the API that Dropbox provides for uploading/downloading files.  There is no intelligent cache management – it just downloads the files to the device and stores it if there is enough room.  On some systems Dropbox also hooks into the operating system to provide an intelligent folder that mirrors the layout of data in the Dropbox cloud.

Another advantage of Dropbox – it supports multiple operating systems (OS X, iOS, Windows, Linux) and provides API’s for several different languages (Objective-C, Ruby, Python).  There are also some third-party API’s that provide support for C#/.Net and Node.js (among others).

Finally, Dropbox supports sharing files and folders between users.  This is done using the Dropbox login ID.  There are no limits to the number of users with whom you can share data.

The cost of Dropbox is that you need to learn another API, and the amount of storage that you get for free is 2 GB.  iCloud gives you 5 GB of storage, over and above any purchases you make in iTunes (any music/apps/books that you buy via iTunes do not count against your 5 GB storage limit).  Both services allow you to purchase additional storage for a fee.

Any data shared between Dropbox users counts against the quota of each Dropbox users.  For example, if user A shares a folder with users B and C, and C puts 2 GB of data in that folder, then both A and B are at their 2 GB limit, even if they haven’t put a single byte of data into their Dropbox storage.

Which to use?  It all depends.  If you have an existing appellation that stores data in local storage and want to take advantage of cloud backup and synchronization, then iCloud is the simplest way to get this.  If you need to share data with multiple users, then you should use Dropbox (or one of its competitors).

Next: Enabling an XCode application to use iCloud

Announcing an iCloud Tutorial

I’m going to be speaking at CocoaHeads NY next month (December 5th) (as one of a couple of speakers).  I’m planning on doing an introduction to iCloud for developers.  What I’d like to do is work on pieces of this presentation here so that I could get some feedback during this.

I am currently in the finishing up stage of an application that I am building – a storyboarding application for the iPad (StoryBuild).  I am incorporating iCloud into this app partly as a learning experience and partly as a mechanism to sync the storyboards between different iPads (eventually I plan to port this application to other devices, but the iPad was my inspiration for this app).  Some of the topics I plan on covering in this tutorial are (in no particular order):

  • How to set up an application to use iCloud
  • Using NSFileManager and friends with iCloud
  • UIDocument and iCloud
  • Core Data and iCloud
  • Gotchas and failing gracefully

I only have a 30 minute presentation slot, and this is an ambitious list.  While I plan on referencing code I’m using in StoryBuild, I plan on having a separate project that will be available to download for you to try.  As to the format, I’m planning on following the same format I used in my Introduction to Cocos2D presentation from 360|iDev Austin in November, 2010.

Re: A Letter to the Developer Community

(Reference: http://wildchocolate.tumblr.com/post/12555879965/a-letter-to-the-developer-community)

Please read the above first before reading the following:

—-

That this kind of stuff happens at conferences is both not surprising and totally unacceptable.  We live in an age where we should be above this kind of crap.  Unfortunately, between the above and postings like this from a Rails conference a couple of years ago I guess we still have a long way to go.

9/11 ten years later

I remember that morning so well. I had a doctor appointment, so instead of going into work like I normally would, I left the house later. My appointment was sometime after 9 in the morning, so I left some amount of time early.

Now our house is located in a town about a half hour north of NYC. If I stand at the end of my driveway I can see the Hudson River. My car was parked in the driveway, and as I was about to get in I heard an airplane flying by. I looked up and saw it flying on a southward path along the Hudson. I now know that that was Flight 11, the first plane to hit the World Trade Center.

I was driving to my doctors and listening to the traffic. Almost immediately there was a commotion about a plane hitting the north tower. At that point everyone assumed that it was a small plane that hit it by accident. I kept flipping through the stations and was hearing a lot of confusing comments about what happened.

Sometime after 9 a traffic report helicopter was reporting about a plane hitting the south tower. At that point I heard this exchange:

Radio Announcer: Excuse me, don’t you mean the north tower?
Traffic Reporter: No, a second plane hit the south tower. We know because we barely got out of the way before it hit.

At that point I knew we were under attack by terrorists. Honestly, I was scared.

Once I got to the doctor’s office I asked if anyone knew what was going on at the World Trace Center. This was the first they had heard of this, and two women working in the office started to cry because they had boyfriends or spouses that worked there. I barely remember the doctor appointment itself.

After that, I started to drive to work, but then it hit me – the bridge I take (the Tappan Zee Bridge) is closest bridge to New York City over the Hudson. Perhaps the other bridges will be taken out or worse. I was talking with my boss on my cell phone and told him I was turning around and going home. Of course he understood.

While I was driving the first tower collapsed. I was in something like shock at this point. I knew I had to get home to my wife.

Once I got home I stood in the living room and watched as the second tower fell.

That afternoon I spoke with a coworker of mine, a Post Doc from Germany. He had found out that one of his best friends from Germany was travelling with his pregnant wife on one of the planes. It was their first visit to the US.

When my kids came home from school we told them. They told us that the school went into lockdown mode. They said that every once in a while the main office would call their rooms asking for one of the children to be escorted out. My daughter told me she saw some people crying in the hallway. My son, who was younger, said that they thought it was a bear or a gunman outside of their school, sinc the schools were not telling them what was going on. They saw the video playback of the towers collapsing and they just couldn’t believe it.

I forget if it was the next day or the day after when I finally drove into work. There was a spot on the Tappan Zee bridge, near the Westchester side, where you could see the towers standing. I used to do a quick turn to look at them every day. Now all I saw was the cloud billowing up towards the sky. I knew the buildings were gone, but it was still so surreal.

These days I still think about the people who lost their lives in that incident, and how lucky we have been to not have had another, and how grateful I am for all the people who have prevented these kinds of things from happening here. I also think of the people who have lost their lives in other incidents of terrorism around the world, and of the wars that have been fought since then. It has changed our world view, just like the raid on Pearl Harbor did back in 1941.

There are a couple more thoughts I’ve had relating to this:

  • In America, we need to get beyond the idea that Muslims are terrorists. It doesn’t work that way. There are Muslims who are terrorists, true, but there are also Christians who are terrorists. And atheists, and Hindus, and whatever belief system you can find. It’s not the religion that defines a terrorist; rather, it is often the perversion of a religious system that is used to justify these heinous acts.  Add in a charismatic leader and people who feel helpless and you have a recipe for disaster.
  • I agree with Bruce Schneier – the only two things that have made a difference in dealing with terrorists on airplanes are (1) locks on the cockpit doors and (2) passengers are now more conscious and watchful for people acting suspiciously and are willing to take action (the brave passengers on Flight 93 have served as role models for all of us). The other things that have been tried (like the new full-body scanners) are just security theater that do little to truly make us safe. I do think there is a role for the TSA in helping to find terrorists, it’s just that they haven’t figured it out yet, so they go with the easy “solutions”.