Still Alive…

I’m still alive, but life has taken a big change (for the better).  A few months ago I started working for Apple as a software engineer.  It’s been a very hectic time for me (a new job and moving to a new city take a lot of time).  Not much more to say but I’m very happy how things are working out.

I’m trying to figure out what to do with this blog.  I’m not going to be able to post any more code things for the time being, that’s obvious (and that means that my iCloud tutorial is not going to be finished).  Likewise, I’m not about to make any comments about the tech industry, Apple or its competitors.  I don’t want to drop this blog entirely, though.

I love this industry, and I’m very excited about what’s going on.  It’s still the early days, everyone…

Sorry for the iCloud tutorial delay

I became very busy with life and work after my presentation at CocoaHeads NY, so I set this aside.  I’m now getting back into the swing of things, and I’m now getting back to this tutorial

Sorry for the delay.

(BTW, if you had noticed, the source code for the initial non-iCloud version of my tutorial program had disappeared.  During building the iCloud version I refactored the code significantly – to the point that the non-iCloud version of NotePlus was a different program.  I’ve now retrofitted the non-iCloud version so that it is in line with my updated code.  It’s better, trust me.)

iCloud 3: Data and Metadata, and an introduction to NotePlus

This is part 3 in my iCloud tutorial series – the previous article is iCloud 2: Setting up an application for iCloud

Before I start digging into the code, I thought I’d write about how devices communicate with iCloud – the types of data and metadata that are shared bet we the iCloud infrastructure and the various platforms.  There’s not a lot of information on this, so I thought it would be helpful to include this into the tutorial.

The types of data that are stored in iCloud

There are two types of data that you store in iCloud

  • Key-value objects (property lists)
  • Documents – files and directories

The following figure illustrates these types of data.

iCloud synchronization (Source: Apple.com)

Key-value storage

Key-value storage is designed for holding small amounts of data which you access through key-value bindings.  Think of this as a property list or dictionary – you ask for an object using a key and get the object (or nil if no object for that key exists).  A good use of this is for storing application settings for sharing between devices.

This is obviously not meant for storing a lot of data.  In fact, Apple limits it to 64 KB per application.  That’s kilobytes, not gigabytes.

Document Storage

Document storage is where most of the interesting storage is going on in iCloud.  Typically you store files that your user generates in the application.  For example, in NotePlus, this is the notes and audio files that are created when the user creates a note.

Let’s consider a single file that is in iCloud.  There are two components associated with this file:

  • Metadata: information about the file, such as its name, versioning information and the number of bytes of data in it;
  • Data: the actual contents of the file

The idea is that metadata is very small in relation to the overall data for the file, and that an application can easily contain all the metadata for all the files stored in iCloud.  Any changes to the metadata of a file on a device are pushed back to the iCloud servers. When you create a file on a device, iCloud will also pull that file to the server.

Conversely, on a device pulling a file is managed by the operating system and the application in tandem. You should assume that your application will request a file that is not on the device and then it will be pulled down (there are scenarios where the OS will pull down files, but this has not been documented by Apple). This is good in terms of managing the amount of space you need for an application; the cost is that now opening a file is an asynchronous operation with a noticeable lag.  This latency may be long enough that you cannot just put everything on hold until you have finished the operation – for example, if you are doing a file open and read in the middle of a multi-player game the lag might be long enough that your character gets killed while waiting.  Not good. The good thing is that your application will be able to determine if a file is local or not, and you, as the developer, can handle this accordingly.

Data and efficiency

iCloud initially uploads the entire file from a device.  When the user makes a change to the file, then iCloud sends only the changes back to the server to create a new version of the document.  The following illustrate this action:

 

The device wants to push a new version of a file to iCloud, ...

 

Changes in the file are identified

... Changes in the file are identified, ...

 

A new version of the file is created

... the changes are uploaded and a new version of the file is created.

 

NotePlus

(Note: The source code for the non-iCloud version of NotePlus is available by clicking this link.)

I’m not going to change gears and talk a little more about the NotePlus application. When I wrote NotePlus I tried to follow the Model-View-Controller (MVC) design pattern as best I could.  I have several data objects that represent the text of a note (NpNoteText) and the audio associated with the note (NpNoteAudio).  A class, NpNoteManager, provides an interface to get a list of note names and to manage notes by providing interfaces to add and remove them from the system.

The master (note list) screen for NotePlus (NpMasterViewController)

The NotePlus app is a Universal Master-Detail application.  On the iPad this is represented as a split-view controller; on the iPhone this is done by relying on the navigation controller mechanism to navigate to a note.  The two main screen objects are the master controller (NpMasterViewController) which displays the list of all the notes, and NpDetailViewController, which provides the interface for editing a note and recording and playing the audio associated with the note.  On the iPad one or both controller may be visible.

NotePlus Note Screen

Editing a note in NotePlus (NpDetailViewController)

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

On NpMasterViewController’s navigation bar are buttons to create a new note and to edit the notes (which only supports deleting notes).  A user may also delete a note by swiping from left to right on the note in the list.  The advantage of metadata is obvious in an application like NotePlus.  Populating the list inNpMasterViewController only needs the metadata, not the full note information.

NpDetailController is the note editor.  This class contains a UITextView for editing the text of the note and a UIToolbar that has buttons for recording and playing audio.  It supports both landscape and portrait orientations.

One more data class I have is NpNoteConfiguration.  This is a simple class that keeps track of the last note the user opened. This is used mostly for remembering the user’s state when the user quits the application and returns.

Where do we go from here

Now that I have introduced the NotePlus application, I will start to update it to incorporate iCloud into the application.  The steps I will need to take are:

  • Determine if iCloud is available or not;
  • Access the metadata for all the files (for the master screen);
  • Load a note from iCloud storage (including resolving conflicts);
  • Save a note and synchronize it with iCloud.

Again, the source code for the initial version of NotePlus is available here.

Until next time!

Next: Determining if iCloud is available.

iCloud 2: Setting up an application for iCloud

This is part 2 in my iCloud tutorial series – the previous article is iCloud 1: An introduction to iCloud.

In this installment I’ll discuss how to set up an application to use iCloud. In this post I discuss the steps you need to do to set up iCloud support for your application (I do create an application in this posting, it just doesn’t do any of the “iCloud goodness” yet).

The things you need to do for setting up an application to use iCloud are:

  • Setting up an app ID in the Apple Provisioning Portal and configuring it to support iCloud;
  • Setting up the iCloud entitlements in your application;
  • Obtaining a provisioning profile for the application.

What is confusing is that these steps take place in different locations – the first is done using the provisioning portal on the developer web site using a web browser, the other is done in XCode (Note: I am using XCode 4.2.1 for this tutorial – you need to be using XCode 4.2 or later due to the need to support iOS 5 for iCloud).

In this tutorial I am going to focus on iOS devices.  You can build and test a lot of your application there, but Apple currently does not allow the simulator to support iCloud.  This means the only way you can test iCloud functionality is to run it on a real device, which means the device needs to be registered with the provisioning portal and enabled for development use.  I will not go over how to use the provisioning portal – if you need help on this the best place to go is the Apple iOS Dev Center at http://developer.apple.com/ios.

For this tutorial I will use an application that I have developed calle NotePlus. This is a simple note taking application that also allows you to record an audio file to go along with each note.  I have made the source code for the initial version of NotePlus available here (Note that this is a ZIP archive that contains all the source code for this application).

NotePlus is a universal application.  I’m doing this because the easiest to see if iCloud is working by testing with two devices, and I assume that most people will either have an iPad and an iPhone or iPod Touch tied to the same Apple ID or they’ll have an iPhone and an iPod touch tied to the same Apple ID.

The importance of the bundle identifier

Make sure that you note the value of the Bundle Identifier for NotePlus – this is the suffix for the App ID we use to identify this appellation (Note that my bundle identifier is com.beret.NotePlus – you will use a different bundle identifier based on your personal or company Apple Developer Portal setup). This is what we will use to enable iCloud for this application in the provisioning portal.

Setting up NotePlus for iCloud in the Provisioning Portal

iCloud is enabled on an app-by-app basis in the provisioning portal.  The current default value is that it is not configured, which makes sense when you think about it. I will configure the NotePlus application in the provisioning portal and configure iCloud during the process.

The steps are:

  • Go to the App ID section of the provisioning portal and click on the “New App ID” button;
  • Fill out the form for the new App ID, ensuring that you put the correct bundle identifier in the right place on the form, then click the “Submit” button.
New Application ID

New App ID form for the NotePlus app

  • Once you click submit you will see the App ID has been added to the list of App ID’s that you have created.  It should look like something in the file below.  You are not done, however.  If you look at the column for iCloud you will see that there is a yellow ball with the text “Configurable” next to it. Click on the blue Configure link to the right of the App ID entry so that we can configure this.
NotePlus App ID created in the provisioning profile

NotePlus App ID in the provisioning profile

  • When the configuration dialog opens in the browser, click on the “Enable for iCloud” checkbox (Note that you will get a dialog box that will inform you that all provisioning profiles that support this application will support iCloud).  Once you click through the information dialog click on the “Done” button.
Configure the NotePlus app ID for iCloud

Configuring the NotePlus App ID for iCloud

  • Now you are back at the list of App ID’s in the provisioning profile.  Note now that the NotePlus application has a green ball and the text “Enabled” in the iCloud column. Our job is done here, so we can go back to XCode and do the second part.
NotePlus is now iCloud enabled

NotePlus is now iCloud enabled

Getting a Provisioning Profile for NotePlus

Now that you have set up NotePlus for iCloud, you need to get a provisioning profile to use.  You can either generate one using XCode (in the Organizer) or you can create a new profile in the Provisioning Portal.  All this is done using the standard techniques for obtaining a provisioning profile.

Setting up entitlements for NotePlus

Once you have completed the above we need to set up the entitlements for NotePlus to allow it to use iCloud. Apple uses this as part of mapping the application to the iCloud storage, and it also allows the developer to manage the type of data that the developer wants to synchronize.  The developer can specify one or more directories/folders to enable for iCloud storage and also enable key-value storage.

Why would you want to manage this from an application standpoint?

  • iCloud storage is not an infinite resource.  Users get 5 GB of storage for free but this storage is shared between all their applications (excluding storage for items bought through the iTunes music storage).  This means that your application and other applications are trying to carve out a piece of this storage space for themselves.  Since this is obviously not coordinated, at some point in the future the user may run out of space in the cloud.  The options the user has are to either buy more storage space from Apple or to free up some space.  As an application developer you need to be mindful of what you store in iCloud.
  • iCloud does allow you to share data between applications (just not between users). You can specify the bundle ID’s for other applications you wish to use to share data.

Once you’ve unpacked the NotePlus application open it in XCode.  Once it has loaded select “NotePlus” in the “Targets” portion of the middle area of the screen.  You should see something like the following:

XCode project summary

NotePlus target summary information in XCode

Be sure to change the value of the “Identifier” field in the right-most section from com.beret.NotePlus to whatever bundle ID you have defined for your application in the provisioning portal.

Now scroll down the right-most section to the “Entitlements” section.  This is where you enable your entitlements.  You should see the following on your screen:

The Entitlements section for the NotePlus application before enabling iCloud

Now click on the checkbox at the top of the Entitlements section (the one labelled Enable Entitlements).  You should now see the following (only with your bundle ID in the various sections instead of com.beret.NotePlus – you did change it, didn’t you?):

After enabling the Entitlements for NotePlus

Notice that the act of checking the Enable Entitlement checkbox fills in the information for iCloud, including the iCloud key-value store information and the iCloud containers.  You can use the iCloud containers to add access to shared containers (where you can share data with other applications).  For this tutorial series I’m going to keep it simple and just use the defaults.

At this point, we’ve done all the infrastructure setup to enable NotePlus to use iCloud.  Of course, just enabling it doesn’t make data sharing happen – we need to add logic to the code to actually access the iCloud data.  That will happen in the next several tutorial postings.  ‘Til then…

Next: Data and Metadata

iCloud 1: An introduction to iCloud

Before talking about how to use iCloud, it helps to understand what it is.  Let’s start with a description from Apple’s iCloud web site:

“iCloud stores your music, photos, documents, and more and wirelessly pushes them to all your devices. Automatic, effortless, and seamless — it just works.”

To make it simpler: iCloud is synchronization.  It syncs your data between your iPhone and the cloud.  It syncs your data between your iPhone and your Mac (there is also some limited support for Windows). It is incorporated into iOS and OS X Lion in such a way that, when combined with other enhancements (such as document versioning) you can get access to the latest, most up-to-date version of your data.  The image below illustrates this flow.

iCloud

iCloud Synchronization (Source: Apple.com)

Note that in the above description I emphasized the word “your”.  As it is currently implemented, it is only data that you create or consume on your devices. Data access and storage is based on your Apple ID.  There is no mechanism for associating multiple Apple ID’s with your data.

Before iCloud, applications normally accessed data objects (files/documents/databases) that were stored locally on the device.  Sharing data between devices was typically done through an explicit user action, such as copy the file to and from a file server or web site.

What iCloud does is provide a series of interfaces that developers can use to integrate synchronizing this data with Apple’s iCloud servers.  The synchronization is a part of the operating system (OS) that developers can access using these application programming interfaces (API’s).  Some of the features of these interfaces include:

  • File upload/download;
  • Data streaming (for database objects);
  • Conflict resolution (there are changes in both the document on the server and the document locally that need to be reconciled).

Apple has done a really good job at integrating iCloud in such a way as to minimize the amount of change you need to make to your code, but there are some challenges that developers need to keep in mind.  For example, opening a document that is stored locally on the file system is fairly simple – you can easily search the file system for the file, and opening it is so fast that you can typically wait for the file to open.  With iCloud, you now have to deal with network latency issues.  Opening a file now might entail downloading the document from the iCloud servers.  You now are at the mercy of the speed of the network and the size of the document.

iCloud versus Dropbox

A lot of developers are incorporating services like Dropbox into their applications to enable synchronization and sharing of information (there are other services, such as Box.net, that some use as well, but Dropbox seems to be the most popular as of this writing, so that is where I will focus).

Dropbox provides file-level syncing between devices and the Dropbox cloud. Applications can take advantage of the API that Dropbox provides for uploading/downloading files.  There is no intelligent cache management – it just downloads the files to the device and stores it if there is enough room.  On some systems Dropbox also hooks into the operating system to provide an intelligent folder that mirrors the layout of data in the Dropbox cloud.

Another advantage of Dropbox – it supports multiple operating systems (OS X, iOS, Windows, Linux) and provides API’s for several different languages (Objective-C, Ruby, Python).  There are also some third-party API’s that provide support for C#/.Net and Node.js (among others).

Finally, Dropbox supports sharing files and folders between users.  This is done using the Dropbox login ID.  There are no limits to the number of users with whom you can share data.

The cost of Dropbox is that you need to learn another API, and the amount of storage that you get for free is 2 GB.  iCloud gives you 5 GB of storage, over and above any purchases you make in iTunes (any music/apps/books that you buy via iTunes do not count against your 5 GB storage limit).  Both services allow you to purchase additional storage for a fee.

Any data shared between Dropbox users counts against the quota of each Dropbox users.  For example, if user A shares a folder with users B and C, and C puts 2 GB of data in that folder, then both A and B are at their 2 GB limit, even if they haven’t put a single byte of data into their Dropbox storage.

Which to use?  It all depends.  If you have an existing appellation that stores data in local storage and want to take advantage of cloud backup and synchronization, then iCloud is the simplest way to get this.  If you need to share data with multiple users, then you should use Dropbox (or one of its competitors).

Next: Enabling an XCode application to use iCloud

Announcing an iCloud Tutorial

I’m going to be speaking at CocoaHeads NY next month (December 5th) (as one of a couple of speakers).  I’m planning on doing an introduction to iCloud for developers.  What I’d like to do is work on pieces of this presentation here so that I could get some feedback during this.

I am currently in the finishing up stage of an application that I am building – a storyboarding application for the iPad (StoryBuild).  I am incorporating iCloud into this app partly as a learning experience and partly as a mechanism to sync the storyboards between different iPads (eventually I plan to port this application to other devices, but the iPad was my inspiration for this app).  Some of the topics I plan on covering in this tutorial are (in no particular order):

  • How to set up an application to use iCloud
  • Using NSFileManager and friends with iCloud
  • UIDocument and iCloud
  • Core Data and iCloud
  • Gotchas and failing gracefully

I only have a 30 minute presentation slot, and this is an ambitious list.  While I plan on referencing code I’m using in StoryBuild, I plan on having a separate project that will be available to download for you to try.  As to the format, I’m planning on following the same format I used in my Introduction to Cocos2D presentation from 360|iDev Austin in November, 2010.